Introduction to asyncio

@saghul
Introduction to asyncio
Saúl Ibarra Corretgé
PyLadies Amsterdam - 20th March 2014

Sockets 101
import socket
server = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM)
server.bind(('127.0.0.1', 1234))
server.listen(128)
print("Server listening on: {}".format(server.getsockname()))
client, addr = server.accept()
print("Client connected: {}".format(addr))
while True:
data = client.recv(4096)
if not data:
print("Client has disconnected")
break
client.send(data)
server.close()

Scaling Up!
import socket
import _thread
def handle_client(client, addr):
print("Client connected: {}".format(addr))
while True:
data = client.recv(4096)
if not data:
print("Client has disconnected")
break
client.send(data.upper())
server = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM)
server.bind(('127.0.0.1', 1234))
server.listen(128)
print("Server listening on: {}".format(server.getsockname()))
while True:
client, addr = server.accept()
_thread.start_new_thread(handle_client, (client, addr))

Scaling Up! (really?)
Threads are too expensive
Also, context switching
Use an event loop instead!

The Async Way (TM)
Single thread
Block waiting for sockets to be ready to read or
write
Perform i/o operations and call the callbacks!
Repeat
(Windows is not like this)

Why asyncio?
asyncore and asynchat are not enough
Fresh new implementation of Asynchronous I/O
Python >= 3.3
Trollius: backport for Python >= 2.6
Use new language features: yield from
Designed to interoperate with other frameworks

Components
Event loop, policy
Coroutines, Futures, Tasks
Transports, Protocols

asyncio 101
import asyncio
loop = asyncio.get_event_loop()
@asyncio.coroutine
def infinity():
while True:
print("hello!")
yield from asyncio.sleep(1)
loop.run_until_complete(infinity())

Coroutines, futures & tasks
Coroutine
generator function, can receive values
decorated with @coroutine
Future
promise of a result or an error
Task
Future which runs a coroutine

Futures
Similar to Futures from PEP-3148
concurrent.futures.Future
API (almost) identical:
f.set_result(); r = f.result()
f.set_exception(e); e = f.exception()
f.done(); f.cancel(); f.cancelled()
f.add_done_callback(x); f.remove_done_callback(x)

Futures + Coroutines
yield from works with Futures!
f = Future()
Someone will set the result or exception
r = yield from f
Waits until done and returns f.result()
Usually returned by functions

Undoing callbacks
@asyncio.coroutine
def sync_looking_function(*args):
fut = asyncio.Future()
def cb(result, error):
if error is not None:
fut.set_result(result)
else:
fut.set_exception(Exception(error))
async_function(cb, *args)
return (yield from fut)

Tasks
Unicorns covered in fairy dust
It’s a coroutine wrapped in a Future
WAT
Inherits from Future
Works with yield from!
r = yield from Task(coro(...))

Tasks vs coroutines
A coroutine doesn’t “advance” without a
scheduling mechanism
Tasks can advance on their own
The event loop is the scheduler!
Magic!

Echo Server
import asyncio
class EchoProtocol(asyncio.Protocol):
def connection_made(self, transport):
print('Client connected')
self.transport = transport
def data_received(self, data):
print('Received data:',data)
self.transport.write(data)
def connection_lost(self, exc):
print('Connection closed', exc)
f = loop.create_server(lambda: EchoProtocol(), 'localhost', 1234)
server = loop.run_until_complete(f)
print('Server started')
loop.run_forever()

Echo Server Reloaded
import asyncio
clients = {} # task -> (reader, writer)
def accept_client(client_reader, client_writer):
task = asyncio.Task(handle_client(client_reader, client_writer))
clients[task] = (client_reader, client_writer)
def client_done(task):
del clients[task]
task.add_done_callback(client_done)
@asyncio.coroutine
def handle_client(client_reader, client_writer):
while True:
data = (yield from client_reader.readline())
client_writer.write(data)
f = asyncio.start_server(accept_client, '127.0.0.1', 1234)
loop.run_forever()

HTTP Server
import asyncio
import aiohttp
import aiohttp.server
class HttpServer(aiohttp.server.ServerHttpProtocol):
@asyncio.coroutine
def handle_request(self, message, payload):
print('method = {!r}; path = {!r}; version = {!r}'.format(
message.method, message.path, message.version))
response = aiohttp.Response(self.transport, 200, close=True)
response.add_header('Content-type', 'text/plain')
response.send_headers()
response.write(b'Hello world!rn')
response.write_eof()
f = loop.create_server(lambda: HttpServer(), 'localhost', 1234)
loop.run_forever()

Redis
import asyncio
import asyncio_redis
@asyncio.coroutine
def subscriber(channels):
# Create connection
connection = yield from asyncio_redis.Connection.create(host='localhost', port=6379)
# Create subscriber.
subscriber = yield from connection.start_subscribe()
# Subscribe to channel.
yield from subscriber.subscribe(channels)
# Wait for incoming events.
while True:
reply = yield from subscriber.next_published()
print('Received: ', repr(reply.value), 'on channel', reply.channel)
loop.run_until_complete(subscriber(['my-channel']))

More?
We just scratched the surface!
Read PEP-3156 (it’s an easy read, I promise!)
Checkout the documentation
Checkout the third-party libraries
Go implement something cool!
“I hear and I forget. I see and I remember.
I do and I understand.” - Confucius

Questions?
@saghul
bettercallsaghul.com

Introduction to asyncio

More Related Content

What's hot

Viewers also liked

Similar to Introduction to asyncio

More from Saúl Ibarra Corretgé

Recently uploaded

Introduction to asyncio