Web server in Python 2.7: Browser not displaying page

Question

I am building a web server in python using the select() function - I/O multiplexing. I am able to connect to multiple clients which in my case are web browsers (safari, chrome, firefox) and accept each clients HTTP 1.1 GET requests. Once i receive the request I return the html page content to the the browser where the html page is displayed.

The problem i am getting is when i try to keep the connection open for a while. I realized that i am not able to display anything in the browser until i close the connection using fd.close().

Here is the function i am using to accept and respond to the browser request. The problem is after i use fd.sendall(), i dont want to close the connection but the page wont display until i do. Please help! Any help or suggestion is appreciated..

def handleConnectedSocket():
    try:
        recvIsComplete = False
        rcvdStr = ''

        line1 = "HTTP/1.1 200 OK\r\n"
        line2 = "Server: Apache/1.3.12 (Unix)\r\n"
        line3 = "Content-Type: text/html\r\n" # Alternately, "Content-Type: image/jpg\r\n"
        line4 = "\r\n"

        line1PageNotFound = "HTTP/1.1 404 Not Found\r\n"
        ConnectionClose = "Connection: close\r\n"

        while not recvIsComplete:
            rcvdStr = fd.recv( 1024 )

            if rcvdStr!= "" :

# look for the string that contains the html page
                recvIsComplete = True
                RequestedFile = ""
                start = rcvdStr.find('/') + 1 
                end = rcvdStr.find(' ', start)
                RequestedFile = rcvdStr[start:end] #requested page in the form of xyz.html

                try:
                    FiletoRead = file(RequestedFile , 'r')
                except:
                    FiletoRead = file('PageNotFound.html' , 'r')
                    response = FiletoRead.read()
                    request_dict[fd].append(line1PageNotFound + line2 + ConnectionClose + line4) 
                    fd.sendall( line1PageNotFound + line2 + line3 + ConnectionClose + line4 + response )
#                    fd.close()   <--- DONT WANT TO USE THIS
                else:    
                    response = FiletoRead.read()
                    request_dict[fd].append(line1 + line2 + line3 + ConnectionClose + line4 + response)
                    fd.sendall(line1 + line2 + line3 + line4 + response)
#                    fd.close()   <--- DONT WANT TO USE THIS
            else:
                recvIsComplete = True
#Remove messages from dictionary
                del request_dict[fd]    
                fd.close()

The client (browser) request is in HTTP 1.1 form as shown:

GET /Test.html HTTP/1.1
Host: 127.0.0.1:22222
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8) AppleWebKit/536.25 (KHTML, like    Gecko) Version/6.0 Safari/536.25
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: keep-alive

Your indentation is a little mixed up (look at your while not recvIsComplete part) - also file (open is recommended instead) was deprecated so I don't think you need a python-3.x tag, so I've removed it — Jon Clements
– Jon Clements, Commented Feb 6, 2013 at 2:19
i added python 3.x because i think this problem is not only python 2.7 related. I am looking for suggestions and ANY advice on how i can fix this. Even a python 3.x person can suggest a solution. — CarbonD1225
– CarbonD1225, Commented Feb 6, 2013 at 2:22
Okies- then no version specific tags are required at all then ;) — Jon Clements
– Jon Clements, Commented Feb 6, 2013 at 2:23
Is this a learning exercise - or are you deliberately ignoring tried and tested frameworks (or just Python's own SimpleHTTPServer) for some reason? — Jon Clements
– Jon Clements, Commented Feb 6, 2013 at 2:28

icktoofay · Accepted Answer · 2013-02-06 03:43:01Z

1

Connection: close indicates to the browser that you'll tell it when you're done sending data by closing the connection. Since you don't want to do that, you'll probably want to use a different value for Connection, like Keep-Alive. If you use that, though, then you'll also need to send Content-Length or do something else so the browser knows when you're done sending it data.

Even if you're not using Keep-Alive, Content-Length is a good thing to send, because it allows the browser to know the current progress in downloading the page. If you have a big file you're sending and don't send Content-Length, the browser can't, say, show a progress bar. Content-Length enables that.

So how do you send a Content-Length header? Count up the number of bytes of data you'll send. Turn that into a string and use that as the value. It's that simple. For example:

# Assuming data is a byte string.
# (If you're dealing with a Unicode string, encode it first.)
content_length_header = "Content-Length: {0}\r\n".format(len(data))

Here's some code that's working for me:

#!/usr/bin/env python3
import time
import socket

data = b'''\
HTTP/1.1 200 OK\r\n\
Connection: keep-alive\r\n\
Content-Type: text/html\r\n\
Content-Length: 6\r\n\
\r\n\
Hello!\
'''


def main(server_address=('0.0.0.0', 8000)):
    server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
    server.bind(server_address)
    server.listen(5)
    while True:
        try:
            client, client_address = server.accept()
            handle_request(client, client_address)
        except KeyboardInterrupt:
            break


def handle_request(client, address):
    with client:
        client.sendall(data)
        time.sleep(5)  # Keep the socket open for a bit longer.
        client.shutdown(socket.SHUT_RDWR)


if __name__ == '__main__':
    main()

edited Feb 6, 2013 at 3:43

answered Feb 6, 2013 at 2:29

icktoofay

130k23 gold badges261 silver badges239 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

CarbonD1225 Over a year ago

hmmm. i can try to use Connection: Keep-live . Can you give me an example on how to use the content-length? and how to calculate the value?

Francis Avila Over a year ago

You need to learn a bit about HTTP, particuarly how to determine or declare message length

CarbonD1225 Over a year ago

thanks @icktoofay. But it still doesn't seem to display anything. I am sending the following as a response: HTTP/1.1 200 OK\r\nServer: Apache/1.3.12 (Unix)\r\nContent-Length: 207\r\nConnection: Keep-Alive\r\n\r\n <html>.... along with the HTML body. Any suggestions?

icktoofay Over a year ago

@user1319424: Unfortunately, no. I tried writing a little server like that and tried it with Chrome, Firefox, and Opera, and it worked fine on each. I'll add my code to my answer in a moment.

CarbonD1225 Over a year ago

hey that finally worked! i was doing something stupid else where... thanks a lot! i have marked it as the answer :)

Collectives™ on Stack Overflow

Web server in Python 2.7: Browser not displaying page

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related