Divide and Conquer – Microservices with Node.js

Divide and conquer
Christian Steiner / pixelio.de

Basti
• Sebastian Springer
• from Munich
• work @ MaibornWolff GmbH
• https://github.com/sspringer82
• @basti_springer
• JavaScript Developer

Agenda
HTTP Microservice with Express
• Routing
• Structure
• Database
• Logging
• Tests
• Container
Commandbased Microservices with
Seneca
• Patterns
• Plugins
• Queueing
Rainer Sturm / pixelio.de

Microservices
Small
Scaleable
Resilient
Independent Testable
Focused

JavaScript is nearly everywhere.
There are a lot of OSS Modules.
Node.js was built for the web
Node.js supports most of the protocols, systems and
databases.
The platform is lightweight.

Example
A microservice that delivers articles for a web shop.
M. Hermsdorf / pixelio.de

Webshop
Client API
Articles
Payment
Users
DB
DB
DB
Auth
Logger

List of articles
Router
Controller
Model
Datenbank
Logger

Packages
S. Hofschlaeger / pixelio.de

Express.js
Express is a lightweight (< 2MB) web application framework
for Node.js.
Express supports HTTP and HTTPS in version 1 and 2.
Express handles routing within a web application for you.
Additionally it provides you with a plugin interface
(Middleware).

Installation
yarn init
yarn add express

Structure of a Microservice
.
├── db
│   ├── config.js
│   └── db.sqlite3
├── package.json
├── spec
└── src
   ├── config.js
   ├── controller.js
   ├── index.js
   ├── logger.js
   ├── model.js
   └── router.js

Entry point
const express = require('express');
const router = require('./router');
const app = express();
router(app);
app.listen(8080, () => {
console.log('Service is listening to http://localhost:
8080');
});

Entry point
In this file all global packages are included and the app is
configured.
Additional features should be put in separate files.

Router
const controller = require('./controller');
module.exports = (app) => {
app.get('/article', controller.getAll);
app.get('/article/:id', controller.getOne);
app.post('/article', controller.create);
app.put('/article', controller.update);
app.delete('/article', controller.remove);
}

Router
The router defines all available routes of a microservice.
A route consists of a HTTP method and a path. By adding
variables to routes they become more flexible.
To keep the code of your router readable, you should put the
routing callbacks in a separate file.

Controller
const model = require('./model');
module.exports = {
getAll(req, res) {
res.json(
model.getAll()
);
},
getOne(req, res) {},
create(req, res) {},
update(req, res) {},
remove(req, res) {}
}

Controller
The controller holds your routing callback functions. It
extracts information from the requests and hands the
information over to the model.
To access variables of your routes, you can use the
req.params object.
To deal with information within the request body, you should
install the body-parser middleware. It introduces the req.body
object which represents the message body.

Model
The model contains the business logic of your application. It
also controls access to the database.
Usually every microservice has its own database, which
increases independence between services.

Model
Node.js supports all common databases such as OracleDB,
MySQL, Redis, MongoDB.
To access your database, you have to install a database driver
ﬁrst.
yarn add sqlite3
To further simplify the access, you can use various ORMs or
ODMs
yarn add orm

ORM
An ORM handles CRUD operations for you. It already
implements all common use cases. You just have to
conﬁgure it with the structure of your database.
The ORM also takes care of some security features such as
escaping.

ORM
const express = require('express');
const orm = require('orm');
const router = require('./router');
const dbConfig = require('./db/config.js');
const {dbPath} = require('./config.js');
const app = express();
app.use(orm.express(dbPath, dbConfig));
router(app);
app.listen(8080, () => {
console.log('Service is listening to http://localhost:
8080');
});

ORM
module.exports = {
define(db, models, next) {
models.articles = db.define('articles', {
id: Number,
title: String,
price: Number
});
next();
}
}

Model
Besides encapsulating database communication your model
takes care of additional tasks such as validation of user input
and various calculations.
Most of these operations are asynchronous. To provide a
clean API you should think about using promises, async
functions or streams instead of plain callbacks.

async getAll(req, res) {
try {
const articles = await model.getAll(req);
res.json(articles);
} catch (e) {
res.status(500).send(e);
}
}
getAll(req) {
return new Promise((resolve, reject) => {
req.models.articles.all((err, results) => {
if (err) {
reject(err);
} else {
resolve(results);
}
});
});
}
controller
model

Logging
Tim Reckmann / pixelio.de

Logging
Within your microservice errors might occur at any time. Of
course you should be prepared to catch and handle them.
But you should also keep a log of errors somewhere.
A logger should not be a ﬁxed part of a certain microservice.
logging is a shared service which is available for all services
in your application.
Centralised logging provides some advantages over local
logging such as scalability and an improved maintainability.

Logging
You can use log4js for remote logging in your application.
This library provides a plugin interface for external appenders
which support various log targets such as ﬁles or logstash.
For remote logging you could use the logstash appender and
a centralised logstash server.

Logging
const log4js = require('log4js');
log4js.configure({
appenders: [{
"host": "127.0.0.1",
"port": 10001,
"type": "logstashUDP",
"logType": "database",
"layout": {
"type": "pattern",
"pattern": "%m"
},
"category": "database"
}],
categories: { default: { appenders: ['database'], level:
'error' } }
});
module.exports = log4js;

Logging
const log4js = require('./logger');
module.exports = {
getAll(req) {
return new Promise((resolve, reject) => {
req.models.articles.all((err, results) => {
if (err) {
reject(err);
log4js.getLogger('database').error(err);
} else {
resolve(results);
}
});
});
}
}

Tests
Dieter Schütz / pixelio.de

Tests
Quality and stability are two very important aspects of
microservices. To ensure this, you need the ability to
automatically test your services.
There are two levels of tests for your microservice application:
unittests for smaller units of code and integration tests for
external interfaces.
For unittests you can use Jasmine as a framework. With
Frisby.js you can test your interfaces.

Unittests
yarn add jasmine
node_modules/.bin/jasmine init

Unittests
const model = require('../model');
describe('model', () => {
it('should handle a database error correctly', () => {
const req = {
models: {
articles: {
all: (cb) => {cb('error', null);}
}
}
}
model.getAll(req).catch((e) => {
expect(e).toBe('error');
});
})
});

Unittests
To execute your tests, just issue the command npx jasmine.
You have two variants for organising your tests. You can either
store them in a separate directory, usually named “spec” or
you put your tests where your source code is located.

Mockery
In your microservice application you have to deal with a lot of
dependencies. They are resolved via the node module
system. Mockery is a tool that helps you with dealing with
dependencies. Mockery replaces libraries with test doubles.
yarn add -D mockery

Mockery
const mockery = require('mockery');
beforeEach(() => {
mockery.enable();
const fsMock = {
stat: function (path, cb) {...}
};
mockery.registerMock('fs', fsMock);
});
afterEach(() => {
mockery.disable();
});

Integration tests
Frisby.js is a library for testing REST interfaces. Frisby is an
extension for Jasmine. So you don’t have to learn an additonal
technology.
In order to run Frisby.js, you have to install jasmine-node.
yarn add -D frisby jasmine-node

Integration tests
require('jasmine-core');
var frisby = require('frisby');
frisby.create('Get all the articles')
.get('http://localhost:8080/article')
.expectStatus(200)
.expectHeaderContains('content-type', 'application/json')
.expectJSON('0', {
id: function (val) { expect(val).toBe(1);},
title: 'Mannesmann Schlosserhammer',
price: 7
})
.toss();

PM2
Node.js is single threaded. Basically that’s not a problem
because of its nonblocking I/O nature. To optimally use all
available resources, you can use the child_process module.
A more convenient way of locally scaling your application is
using PM2.
yarn add pm2

PM2
pm2 start app.js
pm2 start app.js -i 4
pm2 reload all
pm2 scale <app-name> 10
pm2 list
pm2 stop
pm2 delete

Docker
All your services run in independent, self contained
containers.
You can start as many instances of a certain container as you
need.
To extend the features of your application, you simply add
additional services by starting containers.

Dockerﬁle
FROM node:7.10
# Create app directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
# Install app dependencies
COPY package.json /usr/src/app/
RUN yarn install
# Bundle app source
COPY . /usr/src/app
EXPOSE 8080
CMD [ "yarn", "start" ]

Docker
docker build -t basti/microservice .
docker run -p 8080:8080 -d basti/microservice

API Gateway
Jürgen Reitböck / pixelio.de

API Gateway
Each service of your application has to focus on its purpose.
In order to accomplish this, you have to centralise certain
services. A typical example for a central service is
authentication.
An API Gateway forwards authorised requests to the services
of your application, receives the answers and forwards them to
the client.

Seneca
Seneca follows a completely different approach. All services
of your application communicate via messages and become
independent of the transport layer.

Service Definition
const seneca = require('seneca')();
seneca.add({role: 'math', cmd: 'sum'}, controller.getAll);
The first argument of the add method is the pattern, which
describes the service. You are free to define the pattern as
you want. A common best practice is to define a role and a
command. The second argument is an action, a function that
handles incoming requests.

Service Handler
async getAll(msg, reply) {
try {
const articles = await model.getAll(req);
reply(null, JSON.stringify(articles));
} catch (e) {
reply(e);
}
}
Similar to express a service handler receives the
representation of a request. It also gets a reply function. To
create a response, you call the reply function with an error
object and the response body.

Service Call
seneca.act({role: 'article', cmd: 'get'}, (err, result) => {
if (err) {
return console.error(err);
}
console.log(result);
});
With seneca.act you consume a microservice. You supply the
object representation of a message and a callback function.
With the object, you trigger the service. As soon as the answer
of the service is available, the callback is executed.

Patterns
Helmut / pixelio.deHelmut / pixelio.de

Patterns
You can create multiple patterns of the same type. If a
service call matches multiple patterns, the most speciﬁc is
used.
You can use this behaviour to implement versioning of
interfaces.

Plugins
A plugin is a collection of patterns. There are multiple
sources of plugins: built-in, your own plugins and 3rd party
plugins.
Plugins are used for logging or debugging.

Plugins
You organise your patterns with the use method.
The name of the function is used for logging purposes.
You can pass options to your plugin to further conﬁgure it.
The init pattern is used instead of a constructor.
function articles(options) {
this.add({role:'article',cmd:'get'}, controller.getAll);
}
seneca.use(articles);

Plugins
this.wrap({role:'article'}, controller.verify);
}
seneca.use(articles);
The wrap method deﬁnes features that are used by multiple
patterns. With this.prior(msg, respond) the original service
can be called.

Client/Server
cre8tive / pixelio.de

Server
this.wrap({role:'article'}, controller.verify);
}
require('seneca')()
.use(articles)
.listen(8080)
Just like in the ordinary web server the listen method binds
the server to the port 8080. Your service can then be called by
the browser or another server:
http://localhost:8080/act?role=article&cmd=get

Client
require('seneca')()
.client(8080)
.act({role: 'article', cmd: 'get'}, console.log);
The client method lets you connect to a seneca microservice.

Change the transport
// client
seneca.client({
type: 'tcp',
port: 8080
});
// server
seneca.listen({
type: 'tcp',
port: 8080
});
By providing the type ‘tcp’ the TCP protocol is used instead of
HTTP as a communication protocol.

Integration in Express
const SenecaWeb = require('seneca-web');
const Express = require('express');
const router = new Express.Router();
const app = Express();
const senecaWebConfig = {
context: router,
adapter: require('seneca-web-adapter-express'),
options: { parseBody: false }
}
app.use( require('body-parser').json() )
.use( router )
.listen(8080);
seneca.use(SenecaWeb, senecaWebConfig )
.use('api')
.client( { type:'tcp', pin:'role:article' } );

Integration in Express
With this conﬁguration seneca adds routes to your Express
application.
You have to adopt your seneca patterns a little bit. All routes
deﬁned with seneca.act('role:web', {routes: routes}) are added
as Express routes.
Via the path-pattern a corresponding matching is done.

Communication via a
queue
S. Hofschlaeger / pixelio.de

Queue
You can use various 3rd party plugins to communicate via a
message queue instead of sending messages over network.
The main advantage of using a Queue is decoupling client
and server. Your system becomes more robust.

Queue
yarn add seneca-servicebus-transport

Queue
require('seneca')()
.use('seneca-servicebus-transport')
.use(articles)
.listen({
type: 'servicebus',
connection_string: '...'
});
require('seneca')()
.use('seneca-servicebus-transport')
.client({
type: 'servicebus',
connection_string: '...'
})
.act({role: 'article', cmd: 'get'}, console.log);
Server
Client

Microservices, the silver
bullet?
Microservices might be a good solution for certain problems
but deﬁnitely not for all problems in web development.
Microservices introduce an increased complexity to your
application. There is a lot of Node.js packages dealing with
this problem. Some of these packages are outdated or of bad
quality.
Take a look at GitHub statistics and npmjs.com if it ﬁts your
needs.

Questions
Rainer Sturm / pixelio.de

CONTACT
Sebastian Springer
sebastian.springer@maibornwolff.de
MaibornWolff GmbH
Theresienhöhe 13
80339 München
@basti_springer
https://github.com/sspringer82

Divide and Conquer – Microservices with Node.js

More Related Content

What's hot

Similar to Divide and Conquer – Microservices with Node.js

More from Sebastian Springer

Recently uploaded

Divide and Conquer – Microservices with Node.js