2

I have asked a question on this before, but again ran into problems and just cannot solve it. I have a internal proxy server and a content server. The code on the proxy server is this. (Some comments may be wrong but leaving here to convey what is my understanding):

<?php
session_start();
$data_server_url = "http://my_data_server_url/";
$i_var_prefix="i_var_";

$process_headers_separately=0;
//$process_headers_separately=1;
// WARNING! Has problems with GZIPPED DATA!
// AVOID/REMOVE OPTION ALLTOGETHER
// (Set to 1 if you want to catch received headers
// and send explicit headers to clients)
//-----------------------------------------

// Other important request dependent 'SERVER' variables.
if(isset($_SERVER['HTTPS']))
{ $_POST["${i_var_prefix}_HTTPS"]=$_SERVER['HTTPS']; };

if(isset($_SERVER['REMOTE_ADDR']))
{ $_POST["${i_var_prefix}_REMOTE_ADDR"]=$_SERVER['REMOTE_ADDR']; };

$request_uri="";
if(isset($_SERVER['REQUEST_URI'])) { $request_uri = $_SERVER['REQUEST_URI']; };
$curl_url="${data_server_url}${request_uri}";

$field_array= array(
      'Accept' => 'HTTP_ACCEPT',
      'Accept-Charset' => 'HTTP_ACCEPT_CHARSET',
      'Accept-Encoding' => 'HTTP_ACCEPT_ENCODING',
      'Accept-Language' => 'HTTP_ACCEPT_LANGUAGE',
      'Connection' => 'HTTP_CONNECTION',
      'Host' => 'HTTP_HOST',
      'Referer' => 'HTTP_REFERER',
      'User-Agent' => 'HTTP_USER_AGENT'
      );

$curl_request_headers=array();

foreach ($field_array as $key => $value) {
   if(isset($_SERVER["$value"])) {
      $server_value=$_SERVER["$value"];
      $curl_request_headers[]="$key: $server_value";
   }
};
//------
session_write_close();

//Open connection
$curl_handle = curl_init();
curl_setopt($curl_handle,CURLOPT_COOKIE,session_name()."=".session_id().";");
//Set the url, number of POST vars, POST data
curl_setopt($curl_handle, CURLOPT_URL, $curl_url);
curl_setopt($curl_handle, CURLOPT_POST, count($_POST));
curl_setopt($curl_handle, CURLOPT_POSTFIELDS, $_POST);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl_handle, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($curl_handle, CURLOPT_HEADER, $process_headers_separately);
curl_setopt($curl_handle, CURLOPT_HTTPHEADER, $curl_request_headers);
curl_setopt($curl_handle, CURLOPT_ENCODING, "identity");


//Execute post
$result = curl_exec($curl_handle);

//Close connection
curl_close($curl_handle);

if ($process_headers_separately) {
   list($headers,$content)=explode("\r\n\r\n",$result,2);
   foreach (explode("\r\n",$headers) as $hdr) {
      header($hdr);
   }
   echo $content;
} else {
   echo $result;
}    
?>

Problem 1: With the current code, even if the Content-Type returned by the data_server is text/plain, the content-type seen by the client is text/html. For example, see http://sarcastic-quotes.com/robots.txt This request goes to the file above. I have checked that the data server is actually returning Content-Type as text/plain. But through the proxy, client sees content-type in response headers as text/html.

Problem 2: Note the use of variable process_headers_separately. If I set it to 1, then the browser tries to download a gzip file instead of displaying the contents (no matter what content-type the data server returns). Thus, there is some logical bug in that code flow.

I just want the above code to work as an internal proxy that seamlessly acts as a bridge between my data server and the client. Any thoughts would be appreciated, I am really confused with how to correctly handle the headers above.

regards, JP

I have found the cause of the problem when process_headers_separately=1 (browser downloading file instead of displaying). But it is SO strange and am unable to solve it. Problem: If I uncomment the lines if(isset($_SERVER['REMOTE_ADDR'])) { $_POST["${i_var_prefix}_REMOTE_ADDR"]=$_SERVER['REMOTE_ADDR']; }; then things start working fine! Strange!

Its gotta be some crazy whitespace problem like Neil suggested.

Anyways, trying to fix this - I have spent 3 days (2 earlier and 1 now) due to this crazy bug :(. Thanks to Neil and RF for helping me on this.

Improve this question
8
  • RabidFire's approach of curl_getinfo works in that atleast the content type is correctly transffered. However, I am trying the 'relay all' approach. In that, no matter what I try, the browser is always trying to download the file. It appears that the whole header and content stuff is compressed again by the proxy server, and some sort of double compression is taking place. If I uncompress the downloaded file, the uncompressed file has the headers as readable text and actual page content as gzipped data. Commented Dec 22, 2010 at 10:50
  • If your browser's trying to download the file, then either your Content-Type header is set to something the browser can't display, like application/octet-stream, or the Content-Disposition header is set incorrectly. Compressed data not being decompressed, with the right headers, should at least just show gobbledegook. Commented Dec 22, 2010 at 11:02
  • Can you edit the question to reflect any changes you've made to the code? I'll have another squiz at it. Commented Dec 22, 2010 at 11:05
  • I have put the new code, with combined changes from your and RF's suggestions. thanks Commented Dec 22, 2010 at 11:18
  • Make sure there's no mismatch between the Content-Encoding response header from CURL and the output it actually serves. CURL will try to decompress content if it thinks it can, so you'll need to recompress it using gzcompress() or gzdeflate(). Apart from compressing, is your content type issue still around? Also, try replacing $curl_request_headers with a treatment of apache_request_headers(). Sometimes, when you use mod_rewrite, you end up with stuff like $_SERVER['HTTP_REDIRECT_ACCEPT_ENCODING'] instead of what you're looking for. Commented Dec 22, 2010 at 11:36

2 Answers 2

Reset to default
2

This all looks pretty kosher. The only discrepancy I can see is that SOME servers only separate headers with \n, rather than \r\n. But that's probably not the issue.

Can you try writing var_export(explode("\r\n",$headers),true) to a file and see what comes out?

Re your gzip issue, make sure there's no trailing whitespace after your closing ?>, or even safer, change echo $result to die($result). Obviously if you don't pass on the CURL response's headers, Apache will make up its own, so you're going to need that CURLOPT_HEADER flag.

Is CURL decompressing the response for you? If so, you may need to intervene with the Content-Encoding and Content-Length response headers.

n.

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Thanks Neil. The splitting is happening correctly with \r\n, that I checked. This whitespace thing is tricky and dangerous - I never bothered about it! But in the present file, there is not whitespace. I don't thing curl is decompressing the response because I set CURLOPT_ENCODING as "identity".
Does "die" count as an error or it is perfectly 'neat' practice to use die?
No, it won't log anything or throw any exceptions. I use it for a lot of AJAX responder stuff, and yes, with gzcompress as well. die() is perfect for a "final output" operation; it basically means "send this stuff and then pull the plug". Supporting gzip can only be a good thing, so I'd recommend taking the time to iron it out. Your headers won't be compressed, only your response body, and you can use gzuncompress() and gzinflate() to process response bodies further if you need to.
curl_getinfo() is more intended for internal processing than relaying. You're on the right track with parsing the raw response, because you don't need to translate anything; you can just relay it all back to the client. In fact, you can do the same on the way in too; rather than picking through $_SERVER, since you're on apache, as long as you're not using CGI, you can just use apache_request_headers() instead.
CURLOPT_ENCODING interferes with the Accept-Encoding request header. Since you're passing your request's Accept-Encoding on to the server, you may want to omit this.
2

You could use the following lines of code:

//Execute post
$result = curl_exec($curl_handle);
$result_info = curl_getinfo($curl_handle);

And then:

} else {
    if (isset($result_info['content_type'])) {
        header("Content-Type: {$result_info['content_type']}");
    }
    echo $result;
}

Basically, your proxy server is the one that's returning the data. So, that's where you would have to set the Content-Type. By default, this would be html. But if you check if the CURL response had a specific Content-Type and manage to set that, it'll do the trick! :)

Let us know if that works for you.

I kinda understand what you're trying to do now. So here's how to get the headers in your proxy's response:

If you set the CURLOPT_HEADER option to 1, you will receive the headers in your output which will let you do this:

// Grab the headers from the result
if ($process_headers_separately) {
    list($header_string, $content) = explode("\r\n\r\n", $result, 2);
    // OR :
    // $header_string = substr($result, 0, $result_info['header_size']);
    $headers = http_parse_headers($header_string);
    foreach ($headers as $key => $value) {
        header("$key: $value");
    }
}

I suggest using the http_parse_headers function because of the reasons mentioned by the poster of the first answer. I'm not sure if this will solve your gzipping problem. Let us know.

Improve this answer

13 Comments

aha. curl_getinfo looks neat. pardon my lack of knowledge on this - but can there be multiple headers in a response? I mean just like result_info['content_type'], should I look for other specific fields in result_info?
Well, nothing relevant to your current problem: response status, last-modified, timing information, etc. curl_getinfo returns a number of things: php.net/manual/en/function.curl-getinfo.php - you could get only a single piece like so: $content_type = curl_getinfo($curl_handle, CURLINFO_CONTENT_TYPE)
The latter is correct. Bare in mind that you may miss other important headers, like those relating to caching etc.
Sorry, but, yeah. I know it's a bit trickier because of compression and whatnot, but it's a much broader-sweeping solution that takes you from "relay content-type" to "relay all headers", which is what you really need.
@JP19 - Following up on Neil's words, you could use some of the user contributed functions on this page: php.net/http_parse_headers :)
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.