Introduction to NATS with Python3
SF Python Meetup / June 2019
Waldemar Quevedo / @wallyqs
+
About me
● Waldemar Quevedo / @wallyqs
● Software Engineer at
● NATS core maintainer
● Using NATS based systems since 2012
● Author of Practical NATS (Apress, 2018)
The original book
about NATS
Available at:
https://www.apress.com/us/book/9781484235690
NATS v2 is out! 🎉
Released on Monday (June 10th, 2019)
Biggest release of the project since it started.
New Docs!
https://nats-io.github.io/docs/
Agenda
● Introduction to the NATS project
● Getting started with NATS using Python3
● What is new in NATS v2
Introduction to NATS
NATS is a simple,
production proven,
cloud-native
messaging system.
CNCF Landscape
Joined CNCF as
an incubation
project in 2018
https://landscape.cncf.io
CNCF Landscape
Joined CNCF as
an incubation
project in 2018
https://landscape.cncf.io
CNCF Landscape
https://landscape.cncf.io
Growing Community: NATS End Users
About NATS
● Created by Derek Collison in 2011 for CloudFoundry
● DNA: Performance, simplicity, security, and availability
● TCP/IP based with a very simple plain text protocol
| PUB | SUB | UNSUB | CONNECT | INFO | MSG | -ERR | +OK | PING | PONG |
Delivery modes
At least once (Streaming)
A message will always be delivered, but in certain cases may be
delivered more than once
https://github.com/nats-io/nats-streaming-server
At most once (Core)
No guarantee of delivery - messages can be lost - applications
must detect and handle lost messages
https://github.com/nats-io/nats-server
The NATS Servers
At least once (Streaming)
At most once (Core)
https://github.com/nats-io/nats-server
https://github.com/nats-io/nats-streaming-server
NATS Python Clients
At least once (Streaming)
At most once (Core)
https://github.com/nats-io/nats.py
https://github.com/nats-io/stan.py
The NATS Server
The NATS Server (aka. core NATS)
For NATS v2, release main project repo name has changed
Before:
https://github.com/nats-io/gnatsd
Now:
https://github.com/nats-io/nats-server
⚠
The NATS Server (aka. core NATS)
For NATS v2, release main project repo name has changed
Before:
https://github.com/nats-io/gnatsd
Now:
https://github.com/nats-io/nats-server
⚠
The NATS Server (aka. core NATS)
● Written in Go
● Extremely high performance
● TLS support
● Authorization and Authentication
● Full-mesh one hop clustering for HA
● Auto discovery via gossip
Full mesh NATS Cluster
NATS
Client
NATS
NATSNATS
NATS
Client
Full mesh NATS Cluster
NATS
Client
NATS
NATSNATS
NATS
Client
Full mesh NATS Cluster
NATS
Client
NATS
NATSNATS
NATS
Client
NATS
Client
NATS
NATSNATS
NATS
Client
Full mesh NATS Cluster
NATS Streaming (STAN)
NATS Streaming (aka. STAN)
● Supports persistence of messages / ‘Message replay’
● Protocol based on NATS Request/Reply
● Multiple store options:
a. File based with Raft based replicated log for clustering
b. Backed by a relational database (mysql / postgres)
NATS Streaming (aka. STAN)
NATS
Streaming
Client
NATS
NATSNATS
NATS
Streaming
Server
‘cluster-A’
NATS
Streaming
Server
‘cluster-A’
NATS
Streaming
Server
‘cluster-A’
Getting started with NATS in Python
Running the NATS Server
Binary releases available at:
https://github.com/nats-io/nats-server/releases
Official Docker image:
docker run nats
For Kubernetes support check:
https://github.com/nats-io/nats-operator
The Python NATS Clients
The clients repositories have also changed:
Before:
https://github.com/nats-io/asyncio-nats
https://github.com/nats-io/asyncio-nats-streaming
Now:
https://github.com/nats-io/nats.py
https://github.com/nats-io/stan.py
⚠
The Python NATS Clients
● The official NATS Python3 client is asyncio based:
https://github.com/nats-io/nats.py
● Python2 client also available and is based in Tornado:
https://github.com/nats-io/nats.py2
● The STAN client (stan.py) uses nats.py:
https://github.com/nats-io/stan.py
Note: There is only Python3 support for STAN/NATS Streaming
The Python NATS Clients
● The official NATS Python3 client is asyncio based:
pip install asyncio-nats-client
● Python2 client also available and is based in Tornado:
pip install nats-client
● The STAN client (stan.py) uses nats.py:
pip install asyncio-nats-streaming
Note: There is only Python3 support for STAN/NATS Streaming
Performance of nats.py
Performance of nats.go
Core of NATS:
3 Simple Patterns
Messaging Patterns
✓ Publish/Subscribe (1:N)
✓ Request/Reply (1:1)
✓ Load Balanced Queue Subscribers (1:1)
Messaging Patterns
✓ Publish/Subscribe (Streams)
✓ Request/Reply (Services)
✓ Load Balanced Queue Subscribers
(Streams or Services)
Publish/Subscribe (1:N)
PUB foo
NATS
Client
NATS
Client
NATS
Client
NATS
Client
SUB foo
Publish/Subscribe (1:N)
Publish/Subscribe (1:N)
Publish/Subscribe (1:N)
Publish/Subscribe (1:N)
Request/Response (1:1)
NATS
Client
NATS
Client
Request/Response (1:1)
Request/Response (1:1)
Request/Response (1:1)
Request/Response (1:1)
Request/Response (1:1)
PUB foo
NATS
Client
NATS
Client
NATS
Client
NATS
Client
SUB foo
Request/Response (1:1)
PUB foo
NATS
Client
NATS
Client
NATS
Client
NATS
Client
SUB foo
First message would be lowest latency response
Request/Response (1:N)
PUB foo
NATS
Client
NATS
Client
NATS
Client
NATS
Client
SUB foo
Scatter & Gather multiple replies
Request/Response (1:N)
Request/Response (1:N)
Load Balanced Queues
SUB foo workers
PUB foo
NATS
Client
NATS
Client
NATS
Client
NATS
ClientRandomly distributed
Load Balanced Queues
SUB foo workers
PUB foo
NATS
Client
NATS
Client
NATS
Client
NATS
ClientRandomly distributed
Load Balanced Queues
SUB foo workers
PUB foo
NATS
Client
NATS
Client
NATS
Client
NATS
ClientRandomly distributed
Load Balanced Queues
SUB foo workers
PUB foo
NATS
Client
NATS
Client
NATS
Client
NATS
ClientRandomly distributed
Load Balanced Queues
SUB foo workers
PUB foo
NATS
Client
NATS
Client
NATS
Client
NATS
ClientRandomly distributed
Load Balanced Queues
Load Balanced Queues
Wildcards
SUB foo workers
PUB foo
NATS
Client
NATS
Client
NATS
Client
NATS
ClientNATS
Client
SUB >
Wildcards
SUB foo workers
PUB foo
NATS
Client
NATS
Client
NATS
Client
NATS
ClientNATS
Client
SUB >
Wildcards
SUB foo workers
PUB foo.bar
NATS
Client
NATS
Client
NATS
Client
NATS
ClientNATS
Client
SUB >
Wildcards
SUB foo workers
PUB foo.bar
NATS
Client
NATS
Client
NATS
Client
NATS
ClientNATS
Client
SUB >
Wildcards
SUB foo.bar workers
PUB foo.bar
NATS
Client
NATS
Client
NATS
Client
NATS
ClientNATS
Client
SUB > NATS
Client
SUB foo.*
Wildcards
PUB foo.bar
NATS
Client
NATS
Client
NATS
Client
NATS
ClientNATS
Client
SUB > NATS
Client
SUB foo.*
SUB foo.bar workers
Wildcards
PUB foo.bar
NATS
Client
NATS
Client
NATS
Client
NATS
ClientNATS
Client
SUB > NATS
Client
SUB foo.*
SUB foo.bar workers
Tip: Do not assume the
audience of a message!
PUB foo.bar
NATS
Client
NATS
Client
NATS
Client
NATS
ClientNATS
Client
SUB > NATS
Client
SUB foo.*
SUB foo.bar workers
Messages Replay with NATS Streaming
Messages Replay with NATS Streaming
Messages Replay with NATS Streaming
The NATS v2 Release
What’s in NATS v2
Expands the security and
project capabilities of the server
to become a core component
used to build a global
communication network.
https://synadia.com/ngs
What’s in NATS v2
● Gateways, Super clusters & Leafnodes
● New clustering protocol
● Accounts isolation
○ Like containers for messaging
● NKEYS (ed25519 based keys)
● Decentralized authorization with JWTs
● TLS certs SAN based auth
Docs! https://nats-io.github.io/docs/
What’s in NATS v2
● Gateways, Super clusters & Leafnodes
● New clustering protocol
● Accounts isolation
○ Like containers for messaging
● NKEYS (ed25519 based keys)
● Decentralized authorization with JWTs
● TLS certs SAN based auth
Docs! https://nats-io.github.io/docs/
Services
Export
Import
Streams
Export
Import
NATS Super Cluster
NATS Super Clusters
NATS Super Cluster
NATS Super Cluster
NGS: A NATS based global communication network
telnet connect.ngs.global 4222
Trying 34.241.195.68...
Connected to euwest1.aws.ngs.global.
Escape character is '^]'.
INFO
{"server_id":"NBZCGYFTHLCMZC5FWCJXQHVK7VUXSDRYEALBY6GFYCGEBZHGJLWHZR7J",
"version":"2.0.0-RC5","nonce":"kBpF7HCm6rb_TW0",
"cluster":"aws-euwest1","connect_urls":["34.241.195.68:4222","34.243.238.95:4222","34.249.1
87.230:4222"]}
NGS: A NATS based global communication network
telnet connect.ngs.global 4222
Trying 34.241.195.68...
Connected to euwest1.aws.ngs.global.
Escape character is '^]'.
INFO
{"server_id":"NBZCGYFTHLCMZC5FWCJXQHVK7VUXSDRYEALBY6GFYCGEBZHGJLWHZR7J",
"version":"2.0.0-RC5","nonce":"kBpF7HCm6rb_TW0",
"cluster":"aws-euwest1","connect_urls":["34.241.195.68:4222","34.243.238.95:4222","34.249.1
87.230:4222"]}
NATS Super Cluster
NATS Super Cluster
NATS Super Cluster + Leafnodes
NATS Super Cluster + Leafnodes
NATS Super Cluster + Leafnodes
Questions?
Thanks!
github.com/nats-io / @nats_io
https://nats.io

SF Python Meetup - Introduction to NATS Messaging with Python3