As u_mulder says in the comments, your webserver (Apache, Nginx, etc) isn't configured to process PHP files. The reason you're getting that output is because your browser is rendering the the below as a single opening HTML tag:
<?php
echo "<p>
and then displaying the subsequent characters:
Hello world!
"; ?>
as text. If you View Source in your browser, you'll see the full contents of your file.
To fix this, you'll need to enable PHP processing in your webserver's configuration - this will be different depending on which server software you are using, but you should be able to easily find a guide.