1
FIWARE NGSI:
Managing Context Information
at large scale
Fermín Galán Márquez
Technological Expert. Orion Context Broker Development Lead
fermin.galanmarquez@telefonica.com
• Context Management in FIWARE
• Orion Context Broker
• Creating and pulling data
• Pushing data and notifications
• Batch operations
• Advanced functionality
3
Outline
Being “Smart” requires first being “Aware”
• Implementing a Smart Application requires gathering and
managing context information
• Context information refers to the values of attributes
characterizing entities relevant to the application
Bus
• Location
• No. passengers
• Driver
• Licence plate
Citizen
• Name-Surname
• Birthday
• Preferences
• Location
• ToDo list
Shop
• Location
• Business name
• Franchise
• offerings
Context Information
Application
4
Being “Smart” requires first being “Aware”
• Implementing a Smart Application requires gathering and
managing context information
• Context information refers to the values of attributes
characterizing entities relevant to the application
Boiler
• Manufacturer
• Last revision
• Product id
• temperature
Users
• Name-Surname
• Birthday
• Preferences
• Location
• ToDo list
Flowerpot
• Humidity
• Watering plan
Context Information
Application
5
Different sources of context need to be handle
• Context information may come from many sources:
– Existing systems
– Users, through mobile apps
– Sensor networks (IoT Devices)
• Source of info for a given entity.attribute may vary over
time
Place = “X”, temperature = 30º
What’s the current
temperature in place “X”?
Standard API
A sensor in a
pedestrian street
The Public Bus Transport
Management systemA person from his smartphone
It’s too hot!
Notify me the changes of
temperature in place “X”
6
A non-intrusive approach is required
• Capable to integrate with existing or future systems dealing with
management of municipal services without impact in their
architectures
• Info about attributes of one entity may come from different
systems, which work either as Context Producers or Context
Providers
• Applications rely on a single model adapting to systems of each city
Application/Service
Standard API
System A System B
Context Producer Context Provider
attribute “location” attribute “driver”
7
FIWARE NGSI: “The SNMP for IoT”
• Capturing data from, or Acting upon, IoT devices becomes
as easy as to read/change the value of attributes linked to
context entities using a Context Broker
Context Broker
NGSI APINGSI API
GET <Oauth token>
/v2/entities/lamp1/attrs/presenceSensor
PUT <Oauth token>
/v2/entities/lamp1/attrs/status/value
“light on”
Setting up the value of attribute
“status” to “light on” triggers
execution of a function in the IoT
device that switches the lamp on
Issuing a get operation on the
“presenceSensor” attribute
enables the application to get
info about presence of people
near the lamp
8
Connecting to the Internet of Things
• Capturing data from, or Acting upon, IoT devices becomes
as easy as to read/change the value of attributes linked to
context entities using a Context Broker
Context Broker
GET <Oauth token>
/v2/entities/lamp1/attrs/humidity
PUT <Oauth token>
/v2/entities/lamp1/attrs/status/value
“watering”
Setting up the value of attribute
“status” to “watering” triggers
execution of a function in the IoT
device that waters the plant
Issuing a get operation on the
“humidity” attribute enables the
application to find out whether
the plant has to be watered
9
NGSI APINGSI API
Context Management in FIWARE
• The FIWARE Context Broker GE implements the OMA NGSI-
9/10 API: a simple yet powerful standard API for managing
Context information complying with the requirements of a
smart city
• The FIWARE NGSI API is Restful: any web/backend
programmer gets quickly used to it
Application/Service
Context Broker
NGSI API
Boiler
• Manufacturer
• Last revision
• Product id
• temperature
Users
• Name-Surname
• Birthday
• Preferences
• Location
• ToDo list
Flowerpot
• Humidity
• Watering plan
10
Orion Context Broker
• Main functions:
– Context management
– Context availability management (advanced topic)
• HTTP and REST-based
– JSON payload support
• Context in NGSI is based in an entity-attribute model:
Attributes
• Name
• Type
• Value
Entity
• EntityId
• EntityType
1 n
“has”
11
Two “flavors” of NGSI API
• NGSIv1
– Original NGSI RESTful binding of OMA-NGSI
– Implemented in 2013
– Uses the /v1 prefix in resource URL
• NGSIv2
– A revamped, simplified binding of OMA-NGSI
• Simple things must be easy
• Complex things should be possible
• Agile, implementation-driven approach
• Make it as developer-friendly as possible (RESTful, JSON, …)
– Enhanced functionality compared with NGSIv1 (eg. filtering)
– Stable, ready for production, version already available
• Current NGSIv2 version is Release Candidate 2016.10 http://telefonicaid.github.io/fiware-
orion/api/v2/stable
• New features coming (http://telefonicaid.github.io/fiware-orion/api/v2/stable)
– Uses the /v2 prefix in resource URL
• Introduction to NGSIv2
– https://docs.google.com/presentation/d/1_fv9dB5joCsOCHlb4Ld6A-
QmeIYhDzHgFHUWreGmvKU/edit#slide=id.g53c31d7074fd7bc7_0
12
NGSIv2 status (AKA the “NGSIv2 disclaimer”)
• NGSIv2 is in “release candidate” status
– By "release candidate" we mean that the specification is quite stable,
but changes may occur with regard to new release candidates or the
final version. In particular changes may be of two types:
• Extensions to the functionality currently specified by this
document. Note that in this case there isn't any risk of breaking
backward compatibility on existing software implementations.
• Slight modifications in the functionality currently specified by this
document, as a consequence of ongoing discussions. Backward
compatibility will be taken into account in this case, trying to
minimize the impact on existing software implementations. In
particular, only completely justified changes impacting backward
compatibility will be allowed and "matter of taste" changes will not
be allowed.
13
So… when should I use NGSIv1 or NGSIv2?
• In general, it is always preferable to use NGSIv2
• However, you would need to use NGSIv1 if
– You need register/discovery operations (context management
availability functionality)
• Not yet implemented in NGSIv2 (in roadmap)
– Zero tolerance to changes in software interacting with Orion
• Even if you use NGSIv1, you can still use NGSIv2 advanced
functionality
– See “Considerations on NGSIv1 and NGSIv2 coexistence”
section at Orion manual
• For a NGSIv1-based version of this presentation have a
look to
– http://bit.ly/fiware-orion-ngsiv1
14
Orion Context Broker in a nutshell
Orion Context Broker
Context
Producers
Context
Consumers
subscriptions
update
query
notify
notify
update
update
DB
1026
1026
15
GET <cb_host>:1026/version
{
"orion" : {
"version" : "1.6.0",
"uptime" : "7 d, 21 h, 33 m, 39 s",
"git_hash" : "aee96414cc3594bba161afb400f69d101978b39c",
"compile_time" : "Mon Dec 5 08:38:58 CET 2016",
"compiled_by" : "fermin",
"compiled_in" : "centollo"
}
}
16
Orion Context Broker – check health
Orion Context Broker Basic Operations
Entities
• GET /v2/entities
• Retrieve all entities
• POST /v2/entities
• Creates an entity
• GET /v2/entities/{entityID}
• Retrieves an entity
• [PUT|PATCH|POST] /v2/entities/{entityID}
• Updates an entity (different “flavors”)
• DELETE /v2/entities/{entityID}
• Deletes an entity
17
Orion Context Broker Basic Operations
Attributes
• GET /v2/entities/{entityID}/attrs/{attrName}
• Retrieves an attribute’s data
• PUT /v2/entities/{entityID}/attrs/{attrName}
• Updates an attribute’s data
• DELETE /v2/entities/{entityID}/attrs/{attrName}
• Deletes an attribute
• GET /v2/entities/{entityID}/attrs/{attrName}/value
• Retrieves an attribute’s value
• PUT /v2/entities/{entityID}/attrs/{attrName}/value
• Updates an attribute’s value
18
Context Broker operations: create & pull data
• Context Producers publish data/context elements by invoking the update
operations on a Context Broker.
• Context Consumers can retrieve data/context elements by invoking the query
operations on a Context Broker
Context Consumer
query
Context Producer
update
Context Broker
19
Quick Usage Example: Car Create
201 Created
20
POST <cb_host>:1026/v2/entities
Content-Type: application/json
...
{
"id": "Car1",
"type": "Car",
"speed": {
"type": "Float",
"value": 98
}
}
Quick Usage Example: Car Speed Update (1)
PUT <cb_host>:1026/v2/entities/Car1/attrs/speed
Content-Type: application/json
...
{
"type": "Float",
"value": 110
}
204 No Content
…
21
In the case of id ambiguity, you can use
"?type=Car" to specify entity type
Quick Usage Example: Car Speed Query (1)
200 OK
Content-Type: application/json
...
{
"type": "Float",
"value": 110,
"metadata": {}
}
22
You can get all the attributes of the entity using the
entity URL:
GET/v2/entities/Car1/attrs
GET <cb_host>:1026/v2/entities/Car1/attrs/speed
Quick Usage Example: Car Speed Update (2)
PUT <cb_host>:1026/v2/entities/Car1/attrs/speed/value
Content-Type: text/plain
...
115
204 No Content
…
23
Quick Usage Example: Car Speed Query (2)
24
200 OK
Content-Type: text/plain
...
115.000000
GET <cb_host>:1026/v2/entities/Car1/attrs/speed/value
Accept: text/plain
201 Created
...
Quick Usage Example: Room Create (1)
POST <cb_host>:1026/v2/entities
Content-Type: application/json
...
{
"id": "Room1",
"type": "Room",
"temperature": {
"type": "Float",
"value": 24
},
"pressure": {
"type": "Integer",
"value": 718
}
}
25
204 No Content
…
Quick Usage Example: Room Update (1)
PATCH <cb_host>:1026/v2/entities/Room1/attrs
Content-Type: application/json
...
{
"temperature“: {
"type": "Float",
"value": 25
},
"pressure": {
"type": "Integer",
"value": 720
}
}
26
Quick Usage Example: Room Query (1)
27
200 OK
Content-Type: application/json
...
{
"pressure": {
"type": "Integer",
"value": 720,
"metadata": {}
},
"temperature": {
"type": "Float",
"value": 25,
"metadata": {}
}
}
GET <cb_host>:1026/v2/entities/Room1/attrs
Quick Usage Example: Room Query (2)
28
200 OK
Content-Type: application/json
...
{
"pressure": 720,
"temperature": 25
}
GET <cb_host>:1026/v2/entities/Room1/attrs?options=keyValues
201 Created
...
Quick Usage Example: Room Create (2)
POST <cb_host>:1026/v2/entities
Content-Type: application/json
...
{
"id": "Room2",
"type": "Room",
"temperature": {
"type": "Float",
"value": 29
},
"pressure": {
"type": "Integer",
"value": 730
}
}
29
Quick Usage Example: Filters (1)
30
200 OK
Content-Type: application/json
...
[
{
"id": "Room2",
"pressure": 730,
"temperature": 29,
"type": "Room"
}
]
GET <cb_host>:1026/v2/entities?options=keyValues&q=temperature>27
Quick Usage Example: Filters (2)
31
200 OK
Content-Type: application/json
...
[
{
"id": "Room1",
"pressure": 720,
"temperature": 25,
"type": "Room"
}
]
GET <cb_host>:1026/v2/entities?options=keyValues&q=pressure==715..725
The full description of the Simple
Query Language for filtering can be
found in the NGSIv2 Specification
document
Context Broker operations: push data
• Context Consumers can subscribe to receive context information that satisfy
certain conditions using the subscribe operation. Such subscriptions may
have an expiration time.
• The Context Broker notifies updates on context information to subscribed
Context Consumers by invoking the notify operation they export
subId = subscribeContext (consumer, expr, expiration)
Context Consumer
notify (subId, data/context)
Context Broker
Application
32
Quick Usage Example: Subscription
POST <cb_host>:1026/v2/subscriptions
Content-Type: application/json
…
{
"subject": {
"entities": [
{
"id": "Room1",
"type": "Room"
}
],
"condition": {
"attrs": [ "temperature" ]
}
},
"notification": {
"http": {
"url": "http://<host>:<port>/publish"
},
"attrs": [ "temperature" ]
},
"expires": "2026-04-05T14:00:00.00Z"
}
201 Created
Location: /v2/subscriptions/51c0ac9ed714fb3b37d7d5a8
...
33
25
19
Quick Usage Example: Notification
34
POST /publish HTTP/1.1
Content-type: application/json; charset=utf-8
Ngsiv2-AttrsFormat: normalized
…
{
"subscriptionId": "574d720dbef222abb860534a",
"data": [
{
"id": "Room1",
"type": "Room",
"temperature": {
"type": "Float",
"value": 19,
"metadata": {}
}
}
]
}
Quick Usage Example: Notification
35
List existing subscriptions
36
200 OK
Content-Type: application/json
…
[{
"id": " 51c0ac9ed714fb3b37d7d5a8 ",
"expires": "2026-04-05T14:00:00.00Z",
"status": "active",
"subject": {
"entities": [{
"id": "Room1",
"type": "Room"
}],
"condition": {
"attrs": ["temperature"]
}
},
"notification": {
"timesSent": 3,
"lastNotification": "2016-05-31T11:19:32.00Z",
"lastSuccess": "2016-05-31T11:19:32.00Z",
"attrs": ["temperature"],
"attrsFormat": "normalized",
"http": {
"url": "http://localhost:1028/publish"
}
}
}]
The full description of the
subscription object (including all
its fields) can be found in the
NGSIv2 Specification
GET <cb_host>:1026/v2/subscriptions
Orion Context Broker batch operations
37
• Batch query and batch update
• They are equivalent in functionality to previously described RESTful
operations
• All them use POST as verb and the /v2/op URL prefix, including
operation parameters in the JSON payload
• They implement extra functionality that cannot be achieved with
RESTful operations, e.g. to create several entities with the same
operation
• They are not a substitute but a complement to RESTful operations
201 Created
...
Batch Operation Example: Create Several Rooms
POST <cb_host>:1026/v2/op/update
Conten-Type: application/json
...
{
"actionType": "APPEND",
"entities": [
{
"type": "Room",
"id": "Room3",
"temperature": {
"value": 21.2,
"type": "Float"
},
"pressure": {
"value": 722,
"type": "Integer"
}
},
…
38
…
{
"type": "Room",
"id": "Room4",
"temperature": {
"value": 31.8,
"type": "Float"
},
"pressure": {
"value": 712,
"type": "Integer"
}
}
]
}
How to get Orion? (Virtual Machines)
39
• FIWARE Lab image
– Image: orion-psb-image-R<x>.<y>
• VirtualBox image
– http://bit.ly/fiware-orion024-vbox (it’s big!)
– User/pass:
• fiware/fiware
• root/fiware
• Hint: update Orion package once the VM is deployed
How to get Orion? (Docker containers)
40
• Assuming docker is installed in your system
• Documentation in https://github.com/telefonicaid/fiware-
orion/tree/develop/docker
• Quick guide
git clone https://github.com/telefonicaid/fiware-orion.git
cd fiware-orion/docker
sudo docker-compose up
• That’s all!
– curl localhost:1026/version
• Have a look to the FIWARE Reference Tutorial
application
– git clone https://github.com/Fiware/tutorials.TourGuide-
App.git
– cd tutorials.TourGuide-App/
– docker-compose up orion
– curl localhost:1026/version
• Self-explanatory README.md at root directory
• Open a Postman session and rock and roll
– Postman collection:
https://github.com/Fiware/tutorials.TourGuide-
App/blob/develop/contrib/CampusParty2016.postman_coll
ection
41
Would you like to play with this?
42
Pagination
Metadata
Compound attribute/metadata values
Type browsing
Geo-location
Query filters
DateTime support
Custom notifications
Notification status
Attribute/metadata filtering
Special attribute/metadata
Registrations & context providers
Multitenancy
Orion advanced functionality
Creating & pulling data
Pushing data
Subscriptions & Notifications
Batch operations
• Entities may have a location
• Queries/subscriptions may use the location as
search criteria
• Specific session on this tomorrow 12:45-
13:30
– NGSI: geoqueries and Carto integration, Fermín
Galán & Francisco Romero (Data Team)
43
Geo-location
• Pagination helps clients organize query and
discovery requests with a large number of
responses.
• Three URI parameters:
– limit
• Number of elements per page (default: 20, max: 1000)
– offset
• Number of elements to skip (default: 0)
– count (option)
• Returns total elements (default: not return)
44
Pagination
• Example, querying the first 100 entries:
– GET <orion_host>:1026/v2/entities?limit=100&options=count
• The first 100 elements are returned, along with the
following header in the response:
– Fiware-Total-Count: 322
• Now we now there are 322 entities, we can keep querying
the broker for them:
– GET <orion_host>:1026/v2/entities?offset=100&limit=100
– GET <orion_host>:1026/v2/entities?offset=200&limit=100
– GET <orion_host>:1026/v2/entities?offset=300&limit=100
45
Pagination
• By default, results are ordered by entity creation date
• This behavior can be overridden using orderBy URI parameter
– A comma-separated list of attributes. Results are ordered by the first
attribute. On ties, the results are ordered by the second attribute and
so on. A "!" before the attribute name means that the order is
reversed.
• Example: get the first 10 entities ordered by temp in ascending
order, then humidity in descending order
GET <orion_host>:1026/v2/entities?limit=20&offset=0&orderBy=temp,!humidity
• dateCreated and dateModified can be used to ordering by
entity creation and modification date, respectively
46
Pagination
• Users may attach metadata to attributes
• Reserved metadata: ID, location, dateCreated, dateModified, previousValue,
actionType
• Examples:
47
…
"temperature": {
"type": "Float",
"value": 26.5,
"metadata": {
{
"accuracy": {
"type": "Float",
"value": 0.9
}
}
}
…
…
"temperature": {
"type": "Float",
"value": 26.5,
"metadata": {
{
“average": {
"type": "Float",
"value": 22.4
}
}
}
…
Metadata
Complete NGSI Model
Attributes
• Name
• Type
• Value
Entity
• EntityId
• EntityType
1 n
“has”
Metadata
• Name
• Type
• Value1 n
“has”
48
• Attributes and metadata can have a structured
value. Vectors and key-value maps are
supported.
• It maps directly to JSON's objects and arrays.
49
Compound Attribute/Metadata Values
• Example: we have
a car whose four
wheels' pressure
we want to
represent as a
compound
attribute for a car
entity. We would
create the car
entity like this:
{
"type": "Car",
"id": "Car1",
"tirePressure": {
"type": "kPa",
"value": {
"frontRight": "120",
"frontLeft": "110",
"backRight": "115",
"backLeft": "130"
}
}
}
50
Compound Attribute/Metadata Values
Type Browsing
• GET /v2/types
• Retrieve a list of all entity types currently in Orion,
including their corresponding attributes and entities
count
• GET /v2/types/{typeID}
• Retrieve attributes and entities count associated to an
entity type
PRO TIP
GET /v2/contextTypes?options=values
Retrieves just a list of all entity types without any extra info
51
• For the GET /v2/entities operation
• By entity type
• By entity id list
• By entity id pattern (regex)
• By entity type pattern (regex)
• By geographical location
– Described in detail in previous slides
• Filters can be used simultaneously (i.e. like AND condition)
52
GET <cb_host>:1026/v2/entities?type=Room
GET <cb_host>:1026/v2/entities?id=Room1,Room2
GET <cb_host>:1026/v2/entities?idPattern=^Room[2-5]
GET <cb_host>:1026/v2/entities?typePattern=T[ABC]
Query filters
• By attribute value (q)
• By metadata value (mq)
• See full details about q and mq query language in NGSIv2 specification
53
GET <cb_host>:1026/v2/entities?q=temperature>25
GET <cb_host>:1026/v2/entities?q=tirePressure.frontRight >130
attribute name
attribute sub-key (for compound attribute values only)
GET <cb_host>:1026/v2/entities?mq=temperature.avg>25
GET <cb_host>:1026/v2/entities?mq=tirePressure.accuracy.frontRight >90
metadata sub-key (for compound
metadata values only)
attribute name
metadata name
Query filters
54
POST <cb_host>:1026/v2/subscriptions
…
{
"subject": {
"entities": [
{
"id": “Car5",
"type": “Car"
},
{
"idPattern": “^Room[2-5]",
"type": "Room"
},
{
"id": “D37",
"typePattern": "Type[ABC]"
},
],
"condition": {
"attrs": [ "temperature" ],
"expression": {
"q": "temperature>40",
"mq": "humidity.avg==80..90",
"georel": "near;maxDistance:100000",
"geometry": "point",
"coords": "40.418889,-3.691944"
}
}
},
…
}
• Filters can be also used in
subscriptions
– id
– type
– id pattern
– type pattern
– attribute values
– metadata value
– geographical location
Query filters
Datetime support
• Orion implements date support
– Based on ISO ISO8601 format, including partial
representations and timezones
• See https://fiware-
orion.readthedocs.io/en/master/user/ngsiv2_implementati
on_notes/index.html#datetime-support for syntax details
– Use reserved attribute type DateTime to express a date
– Date-based filters are supported
55
Datetime support
• Attribute value arithmetic filters can be used with dates as if they
were numbers
• Entity dateModified and dateCreated special attributes, to get
entity creation and last modification timestamps
– They are shown in query responses using
attrs=dateModified,dateCreated
• Entity dateModified and dateCreated special metadata, to get
attribute creation and last modification timestamps
– They are shown in query responses using
metadata=dateModified,dateCreated
56
POST /v2/entities
…
{
"id": "John",
"birthDate": {
"type": "DateTime",
"value": "1979-10-14T07:21:24.238Z"
}
}
GET /v2/entities?q=birthDate<1985-01-01T00:00:00
Example: create entity John,
with birthDate attribute using
type DateTime
• Apart from the standard formats defined in the
previous slides NGSIv2 allows to re-define all the
notification aspects
• httpInfo is used instead of http, with the
following subfields
– URL query parameters
– HTTP method
– HTTP headers
– Payload (not necessarily JSON!)
• A simple macro substitution language based on ${..}
syntax can be used to “fill the gaps” with entity data (id,
type or attribute values)
– Exception: this cannot be used in HTTP method field
57
Custom notifications
58
…
"httpCustom": {
"url": "http://foo.com/entity/${id}",
"headers": {
"Content-Type": "text/plain"
},
"method": "PUT",
"qs": {
"type": "${type}"
},
"payload": "The temperature is ${temp} degrees"
}
…
PUT http://foo.com/entity/DC_S1-D41?type=Room
Content-Type: text/plain
Content-Length: 31
The temperature is 23.4 degrees
PUT /v2/entities/DC_S1-D41/attrs/temp/value?type=Room
…
23.4
Custom notification configuration
update
notificaiton
Custom notifications
• Status failed means that last
attempt to notify failed
– E.g. the endpoint is not reachable
• Detailed information in the
notifications element
– timesSent: total number of
notifications attempts (both
successful and failed)
– lastSuccess: last time that
notification was successfully sent
– lastFailure: last time that
notification was tried and failed
– lastNotification: last time the
notification was sent (either success
or failure)
• Corollary: lastNotification value is the
same than either lastFailure or
lastSuccess
59
200 OK
Content-Type: application/json
…
[{
"id": " 51c0ac9ed714fb3b37d7d5a8 ",
"expires": "2026-04-05T14:00:00.00Z",
"status": "failed",
"subject": { … },
"notification": {
"timesSent": 3,
"lastNotification": "2016-05-31T11:19:32.00Z",
"lastSuccess": "2016-05-31T10:07:32.00Z",
"lastFailure": "2016-05-31T11:19:32.00Z",
…
}
}]
Notification status
• By default all attribute are included in query
responses or notifications
• The attrs field (as parameter in GET operations
and as notification sub-field in subscriptions)
can be used to specify a filtering list
• The attrs field can be also used to explicitly
include some special attributes (not included by
default)
– dateCreated, dateModified: described in previous
slide
• The “*” can be used as an alias of “all the
(regular) attributes”
60
Attribute filtering and special attributes
• Examples
– Include only attributes temp and lum
• In queries: GET /v2/entities?attrs=temp,lum
• In subscriptions: "attrs": [ "temp", "lum" ]
– Include dateCreated and not any other attribute
• In queries: GET /v2/entities?attrs=dateCreated
• In subscriptions: "attrs": [ "dateCreated" ]
– Include dateModified and all the other (regular)
attributes
• In queries: GET /v2/entities?attrs=dateModified,*
• In subscriptions: "attrs": [ "dateModified", "*" ]
– Include all attributes (same effect that not using attrs,
not very interesting)
• In queries: GET /v2/entities?attrs=*
• In subscriptions: "attrs": [ "*" ]
61
Attribute filtering and special attributes
• By default all attribute metadata are included in query
responses and notifications
• The metadata field (as parameter in GET operations and
as notification sub-field in subscriptions) can be used to
specify a filtering list
• The metadata field can be also used to explicitly include
some special metadata (not included by default)
– dateCreated, dateModified: described in previous slide
– actionType: which value is the action type corresponding to
the update triggering the notification: “update”, “append” or
“delete”
– previousValue: which provides the value of the attribute
previous to processing the update that triggers the notification
• The “*” can be used as an alias of “all the (regular)
metadata”
62
Metadata filtering and special attributes
• Examples
– Include only metadata MD1 and MD2
• In queries: GET /v2/entities?metadata=MD1,MD2
• In subscriptions: "metadata": [ "MD1", "MD2" ]
– Include previousValue and not any other metadata
• In queries: GET /v2/entities?metadata=previousValue
• In subscriptions: "attrs": [ "previousValue" ]
– Include actionType and all the other (regular) metadata
• In queries: GET /v2/entities?metadata=actionType,*
• In subscriptions: "attrs": [ "actionType", "*" ]
– Include all metadatata (same effect that not using
metadata, not very interesting)
• In queries: GET /v2/entities?metadata=*
• In subscriptions: "metadata": [ "*" ]
63
Metadata filtering and special attributes
• Uncached queries and updates
64
Application
ContextBroker ContextProvider
1. registerContext(provider= )
db
2. query 3. query
4. data5. data
Context
Consumer
Registration & Context Providers
POST <cb_host>:1026/v1/registry/registerContext
…
{
"contextRegistrations": [
{
"entities": [
{
"type": "Car",
"isPattern": "false",
"id": "Car1"
},
"attributes": [
{
"name": "speed",
"type": "float",
"isDomain": "false"
}
],
"providingApplication": "http://contextprovider.com/Cars"
}
],
"duration": "P1M"
}
200 OK
...
{
"duration" : "P1M",
"registrationId" : "52a744b011f5816465943d58"
}
65
Context management availability functionality not
yet specified in NGSIv2. Thus, a NGSIv1 operation is
used to create the registration.
Registration & Context Providers
66
GET <cb_host>:1026/v2/entities/Car1/attrs
ContextBroker ContextProvider
db
query
data
200 OK
Content-Type: application/json
...
{
"type": "Float",
"value": 110,
"metadata": {}
}
Registration & Context Providers
• Simple multitenant model based on
logical database separation.
• It eases tenant-based authorization
provided by other components.
• Just use an additional HTTP header
called "Fiware-Service", whose value
is the tenant name. Example:
Fiware-Service: Tenant1
Context
Broker
Tenant1
Tenant2
…
67
Multitenancy
• A service path is a hierarchical scope assigned to an entity
at creation time (with POST /v2/entities).
68
Service Paths
• In order to use a service path we put in a new HTTP header
called "Fiware-ServicePath". For example:
Fiware-ServicePath: /Madrid/Gardens/ParqueNorte/Parterre1
• Properties:
– A query on a service path will look only into the specified node
– Use "ParentNode/#" to include all child nodes
– Queries without Fiware-ServicePath resolve to "/#"
– Entities will fall in the "/" node by default
ParqueNorte
Parterre2Parterre1
69
Service Paths
• Properties (continued):
– You can OR a query using a comma (,)
operator in the header
• For example, to query all street lights that are either in
ParqueSur or in ParqueOeste you would use:
ServicePath: Madrid/Gardens/ParqueSur,
Madrid/Gardens/ParqueOeste
• You can OR up to 10 different scopes.
– Maximum scope levels: 10
• Scope1/Scope2/.../Scope10
– You can have the same element IDs in
different scopes (be careful with this!)
– You can't change scope once the element is
created
– One entity can belong to only one scope
– It works not only with queries, but also with
subscriptions/notifications
– It works not only in NGSI10, but also with
registrations/discoveries (NGSI9)
ParqueNorte
Parterre1
light1
light1
A B
A or B
70
Service Paths
• Interesting NGSI&Orion-related stuff during
FIWARE Summit
– Wednesday 14th 12:45-13:30: “NGSI: geoqueries
and Carto integration”, Fermín Galán & Francisco
Romero (Data Team)
– Wednesday 14th, 15:45-16:30: “Creating context
historic using Cygnus”, Francisco Romero (Data team)
– Wednesday 14th 18:15-18:45 “NGSIv2-Overview-
for-Developers-That-Already-Know-NGSIv1”, Fermín
Galán (Orion Context Broker Development Lead)
– Thursday 15th 12:15-13:00, “Hands-on FIWARE
Context Provider Simulator Tutorial”, German Toro
(Data team)
71
Where (and when ) to go after this talk?
• The easy way
– This presentation: google for “fermingalan slideshare” and search the one
named “Managing Context Information at large scale”
– Orion User Manual: google for “Orion FIWARE manual” and use the first hit
– Orion Catalogue page: google for “Orion FIWARE catalogue” and use the first
hit
• References
– NGSIv2 Specification
• http://fiware.github.io/specifications/ngsiv2/stable
• http://fiware.github.io/specifications/ngsiv2/latest
– NGSIv2 for NGSIv1 developers
• http://bit.ly/ngsiv2-vs-ngsiv1
– This presentation
• http://www.slideshare.net/fermingalan/fiware-managing-context-information-at-large-
scale
– Orion Catalogue:
• http://catalogue.fiware.org/enablers/publishsubscribe-context-broker-orion-context-
broker
– Orion support trhough StackOverflow
• Ask your questions using the “fiware-orion” tag
• Look for existing questions at http://stackoverflow.com/questions/tagged/fiware-orion
72
Would you like to know more?
Thank you!
http://fiware.org
Follow @FIWARE on Twitter
Integration with sensor networks
• The backend IoT Device Management GE enables creation and
configuration of NGSI IoT Agents that connect to sensor networks
• Each NGSI IoT Agent can behave as Context Consumers or Context
Providers, or both
FIWARE Context Broker
IoT
Agent-1
IoT
Agent-2
IoT
Agent-n
IoT Agent
Manager
create/monitor
FIWARE Backend IoT
Device Management
OMA NGSI API (northbound interface)
(southbound interfaces)
MQTTETSI M2M IETF CoAP
74
• Federation of infrastructures (private/public regions)
• Automated GE deploymentCloud
• Complete Context Management Platform
• Integration of Data and Media ContentData
•Easy plug&play of devices using multiple protocols
•Automated Measurements/Action Context updatesIoT
•Visualization of data (operation dashboards)
•Publication of data sets/servicesApps
•Easy support of UIs with advanced web-based 3D and AR
capabilities
•Visual representation of context information.
Web UI
•Advanced networking capabilities (SDN) and Middleware
•Interface to robotsI2ND
•Security Monitoring
•Built-in Identity/Access/Privacy ManagementSecurity
Context Management in FIWARE
75
FI-WARE Context/Data Management Platform
Context/Data Management Platform
Applications
OMA NGSI-9/10
Processing/Analysis
Algorithms
Gathered data is
injected for
processing/analysis
Distributed
Context
Sources Complex Event
Processing
(PROTON)
BigData
(COSMOS)
Processed data is
injected for
processing/analysi
s
Data generated either by CEP
or BigData is published
Gathered data injected
for CEP-like processing
Direct
bigdata
injection
Programming of
rules
76
• Used by /v2/op/update (batch operation)
• Conventional actionTypes
– APPEND: append (or update if the attribute already
exists)
– UPDATE: update
– DELETE: delete
• Special actionTypes
– APPEND_STRICT: strict append (returns error if some of
the attributes to add already exists)
– REPLACE: delete all the entity attributes, next append
the ones in the update request
Special update action types
77

FIWARE NGSI: Managing Context Information at Large Scale

  • 1.
  • 2.
    FIWARE NGSI: Managing ContextInformation at large scale Fermín Galán Márquez Technological Expert. Orion Context Broker Development Lead fermin.galanmarquez@telefonica.com
  • 3.
    • Context Managementin FIWARE • Orion Context Broker • Creating and pulling data • Pushing data and notifications • Batch operations • Advanced functionality 3 Outline
  • 4.
    Being “Smart” requiresfirst being “Aware” • Implementing a Smart Application requires gathering and managing context information • Context information refers to the values of attributes characterizing entities relevant to the application Bus • Location • No. passengers • Driver • Licence plate Citizen • Name-Surname • Birthday • Preferences • Location • ToDo list Shop • Location • Business name • Franchise • offerings Context Information Application 4
  • 5.
    Being “Smart” requiresfirst being “Aware” • Implementing a Smart Application requires gathering and managing context information • Context information refers to the values of attributes characterizing entities relevant to the application Boiler • Manufacturer • Last revision • Product id • temperature Users • Name-Surname • Birthday • Preferences • Location • ToDo list Flowerpot • Humidity • Watering plan Context Information Application 5
  • 6.
    Different sources ofcontext need to be handle • Context information may come from many sources: – Existing systems – Users, through mobile apps – Sensor networks (IoT Devices) • Source of info for a given entity.attribute may vary over time Place = “X”, temperature = 30º What’s the current temperature in place “X”? Standard API A sensor in a pedestrian street The Public Bus Transport Management systemA person from his smartphone It’s too hot! Notify me the changes of temperature in place “X” 6
  • 7.
    A non-intrusive approachis required • Capable to integrate with existing or future systems dealing with management of municipal services without impact in their architectures • Info about attributes of one entity may come from different systems, which work either as Context Producers or Context Providers • Applications rely on a single model adapting to systems of each city Application/Service Standard API System A System B Context Producer Context Provider attribute “location” attribute “driver” 7
  • 8.
    FIWARE NGSI: “TheSNMP for IoT” • Capturing data from, or Acting upon, IoT devices becomes as easy as to read/change the value of attributes linked to context entities using a Context Broker Context Broker NGSI APINGSI API GET <Oauth token> /v2/entities/lamp1/attrs/presenceSensor PUT <Oauth token> /v2/entities/lamp1/attrs/status/value “light on” Setting up the value of attribute “status” to “light on” triggers execution of a function in the IoT device that switches the lamp on Issuing a get operation on the “presenceSensor” attribute enables the application to get info about presence of people near the lamp 8
  • 9.
    Connecting to theInternet of Things • Capturing data from, or Acting upon, IoT devices becomes as easy as to read/change the value of attributes linked to context entities using a Context Broker Context Broker GET <Oauth token> /v2/entities/lamp1/attrs/humidity PUT <Oauth token> /v2/entities/lamp1/attrs/status/value “watering” Setting up the value of attribute “status” to “watering” triggers execution of a function in the IoT device that waters the plant Issuing a get operation on the “humidity” attribute enables the application to find out whether the plant has to be watered 9 NGSI APINGSI API
  • 10.
    Context Management inFIWARE • The FIWARE Context Broker GE implements the OMA NGSI- 9/10 API: a simple yet powerful standard API for managing Context information complying with the requirements of a smart city • The FIWARE NGSI API is Restful: any web/backend programmer gets quickly used to it Application/Service Context Broker NGSI API Boiler • Manufacturer • Last revision • Product id • temperature Users • Name-Surname • Birthday • Preferences • Location • ToDo list Flowerpot • Humidity • Watering plan 10
  • 11.
    Orion Context Broker •Main functions: – Context management – Context availability management (advanced topic) • HTTP and REST-based – JSON payload support • Context in NGSI is based in an entity-attribute model: Attributes • Name • Type • Value Entity • EntityId • EntityType 1 n “has” 11
  • 12.
    Two “flavors” ofNGSI API • NGSIv1 – Original NGSI RESTful binding of OMA-NGSI – Implemented in 2013 – Uses the /v1 prefix in resource URL • NGSIv2 – A revamped, simplified binding of OMA-NGSI • Simple things must be easy • Complex things should be possible • Agile, implementation-driven approach • Make it as developer-friendly as possible (RESTful, JSON, …) – Enhanced functionality compared with NGSIv1 (eg. filtering) – Stable, ready for production, version already available • Current NGSIv2 version is Release Candidate 2016.10 http://telefonicaid.github.io/fiware- orion/api/v2/stable • New features coming (http://telefonicaid.github.io/fiware-orion/api/v2/stable) – Uses the /v2 prefix in resource URL • Introduction to NGSIv2 – https://docs.google.com/presentation/d/1_fv9dB5joCsOCHlb4Ld6A- QmeIYhDzHgFHUWreGmvKU/edit#slide=id.g53c31d7074fd7bc7_0 12
  • 13.
    NGSIv2 status (AKAthe “NGSIv2 disclaimer”) • NGSIv2 is in “release candidate” status – By "release candidate" we mean that the specification is quite stable, but changes may occur with regard to new release candidates or the final version. In particular changes may be of two types: • Extensions to the functionality currently specified by this document. Note that in this case there isn't any risk of breaking backward compatibility on existing software implementations. • Slight modifications in the functionality currently specified by this document, as a consequence of ongoing discussions. Backward compatibility will be taken into account in this case, trying to minimize the impact on existing software implementations. In particular, only completely justified changes impacting backward compatibility will be allowed and "matter of taste" changes will not be allowed. 13
  • 14.
    So… when shouldI use NGSIv1 or NGSIv2? • In general, it is always preferable to use NGSIv2 • However, you would need to use NGSIv1 if – You need register/discovery operations (context management availability functionality) • Not yet implemented in NGSIv2 (in roadmap) – Zero tolerance to changes in software interacting with Orion • Even if you use NGSIv1, you can still use NGSIv2 advanced functionality – See “Considerations on NGSIv1 and NGSIv2 coexistence” section at Orion manual • For a NGSIv1-based version of this presentation have a look to – http://bit.ly/fiware-orion-ngsiv1 14
  • 15.
    Orion Context Brokerin a nutshell Orion Context Broker Context Producers Context Consumers subscriptions update query notify notify update update DB 1026 1026 15
  • 16.
    GET <cb_host>:1026/version { "orion" :{ "version" : "1.6.0", "uptime" : "7 d, 21 h, 33 m, 39 s", "git_hash" : "aee96414cc3594bba161afb400f69d101978b39c", "compile_time" : "Mon Dec 5 08:38:58 CET 2016", "compiled_by" : "fermin", "compiled_in" : "centollo" } } 16 Orion Context Broker – check health
  • 17.
    Orion Context BrokerBasic Operations Entities • GET /v2/entities • Retrieve all entities • POST /v2/entities • Creates an entity • GET /v2/entities/{entityID} • Retrieves an entity • [PUT|PATCH|POST] /v2/entities/{entityID} • Updates an entity (different “flavors”) • DELETE /v2/entities/{entityID} • Deletes an entity 17
  • 18.
    Orion Context BrokerBasic Operations Attributes • GET /v2/entities/{entityID}/attrs/{attrName} • Retrieves an attribute’s data • PUT /v2/entities/{entityID}/attrs/{attrName} • Updates an attribute’s data • DELETE /v2/entities/{entityID}/attrs/{attrName} • Deletes an attribute • GET /v2/entities/{entityID}/attrs/{attrName}/value • Retrieves an attribute’s value • PUT /v2/entities/{entityID}/attrs/{attrName}/value • Updates an attribute’s value 18
  • 19.
    Context Broker operations:create & pull data • Context Producers publish data/context elements by invoking the update operations on a Context Broker. • Context Consumers can retrieve data/context elements by invoking the query operations on a Context Broker Context Consumer query Context Producer update Context Broker 19
  • 20.
    Quick Usage Example:Car Create 201 Created 20 POST <cb_host>:1026/v2/entities Content-Type: application/json ... { "id": "Car1", "type": "Car", "speed": { "type": "Float", "value": 98 } }
  • 21.
    Quick Usage Example:Car Speed Update (1) PUT <cb_host>:1026/v2/entities/Car1/attrs/speed Content-Type: application/json ... { "type": "Float", "value": 110 } 204 No Content … 21 In the case of id ambiguity, you can use "?type=Car" to specify entity type
  • 22.
    Quick Usage Example:Car Speed Query (1) 200 OK Content-Type: application/json ... { "type": "Float", "value": 110, "metadata": {} } 22 You can get all the attributes of the entity using the entity URL: GET/v2/entities/Car1/attrs GET <cb_host>:1026/v2/entities/Car1/attrs/speed
  • 23.
    Quick Usage Example:Car Speed Update (2) PUT <cb_host>:1026/v2/entities/Car1/attrs/speed/value Content-Type: text/plain ... 115 204 No Content … 23
  • 24.
    Quick Usage Example:Car Speed Query (2) 24 200 OK Content-Type: text/plain ... 115.000000 GET <cb_host>:1026/v2/entities/Car1/attrs/speed/value Accept: text/plain
  • 25.
    201 Created ... Quick UsageExample: Room Create (1) POST <cb_host>:1026/v2/entities Content-Type: application/json ... { "id": "Room1", "type": "Room", "temperature": { "type": "Float", "value": 24 }, "pressure": { "type": "Integer", "value": 718 } } 25
  • 26.
    204 No Content … QuickUsage Example: Room Update (1) PATCH <cb_host>:1026/v2/entities/Room1/attrs Content-Type: application/json ... { "temperature“: { "type": "Float", "value": 25 }, "pressure": { "type": "Integer", "value": 720 } } 26
  • 27.
    Quick Usage Example:Room Query (1) 27 200 OK Content-Type: application/json ... { "pressure": { "type": "Integer", "value": 720, "metadata": {} }, "temperature": { "type": "Float", "value": 25, "metadata": {} } } GET <cb_host>:1026/v2/entities/Room1/attrs
  • 28.
    Quick Usage Example:Room Query (2) 28 200 OK Content-Type: application/json ... { "pressure": 720, "temperature": 25 } GET <cb_host>:1026/v2/entities/Room1/attrs?options=keyValues
  • 29.
    201 Created ... Quick UsageExample: Room Create (2) POST <cb_host>:1026/v2/entities Content-Type: application/json ... { "id": "Room2", "type": "Room", "temperature": { "type": "Float", "value": 29 }, "pressure": { "type": "Integer", "value": 730 } } 29
  • 30.
    Quick Usage Example:Filters (1) 30 200 OK Content-Type: application/json ... [ { "id": "Room2", "pressure": 730, "temperature": 29, "type": "Room" } ] GET <cb_host>:1026/v2/entities?options=keyValues&q=temperature>27
  • 31.
    Quick Usage Example:Filters (2) 31 200 OK Content-Type: application/json ... [ { "id": "Room1", "pressure": 720, "temperature": 25, "type": "Room" } ] GET <cb_host>:1026/v2/entities?options=keyValues&q=pressure==715..725 The full description of the Simple Query Language for filtering can be found in the NGSIv2 Specification document
  • 32.
    Context Broker operations:push data • Context Consumers can subscribe to receive context information that satisfy certain conditions using the subscribe operation. Such subscriptions may have an expiration time. • The Context Broker notifies updates on context information to subscribed Context Consumers by invoking the notify operation they export subId = subscribeContext (consumer, expr, expiration) Context Consumer notify (subId, data/context) Context Broker Application 32
  • 33.
    Quick Usage Example:Subscription POST <cb_host>:1026/v2/subscriptions Content-Type: application/json … { "subject": { "entities": [ { "id": "Room1", "type": "Room" } ], "condition": { "attrs": [ "temperature" ] } }, "notification": { "http": { "url": "http://<host>:<port>/publish" }, "attrs": [ "temperature" ] }, "expires": "2026-04-05T14:00:00.00Z" } 201 Created Location: /v2/subscriptions/51c0ac9ed714fb3b37d7d5a8 ... 33
  • 34.
  • 35.
    POST /publish HTTP/1.1 Content-type:application/json; charset=utf-8 Ngsiv2-AttrsFormat: normalized … { "subscriptionId": "574d720dbef222abb860534a", "data": [ { "id": "Room1", "type": "Room", "temperature": { "type": "Float", "value": 19, "metadata": {} } } ] } Quick Usage Example: Notification 35
  • 36.
    List existing subscriptions 36 200OK Content-Type: application/json … [{ "id": " 51c0ac9ed714fb3b37d7d5a8 ", "expires": "2026-04-05T14:00:00.00Z", "status": "active", "subject": { "entities": [{ "id": "Room1", "type": "Room" }], "condition": { "attrs": ["temperature"] } }, "notification": { "timesSent": 3, "lastNotification": "2016-05-31T11:19:32.00Z", "lastSuccess": "2016-05-31T11:19:32.00Z", "attrs": ["temperature"], "attrsFormat": "normalized", "http": { "url": "http://localhost:1028/publish" } } }] The full description of the subscription object (including all its fields) can be found in the NGSIv2 Specification GET <cb_host>:1026/v2/subscriptions
  • 37.
    Orion Context Brokerbatch operations 37 • Batch query and batch update • They are equivalent in functionality to previously described RESTful operations • All them use POST as verb and the /v2/op URL prefix, including operation parameters in the JSON payload • They implement extra functionality that cannot be achieved with RESTful operations, e.g. to create several entities with the same operation • They are not a substitute but a complement to RESTful operations
  • 38.
    201 Created ... Batch OperationExample: Create Several Rooms POST <cb_host>:1026/v2/op/update Conten-Type: application/json ... { "actionType": "APPEND", "entities": [ { "type": "Room", "id": "Room3", "temperature": { "value": 21.2, "type": "Float" }, "pressure": { "value": 722, "type": "Integer" } }, … 38 … { "type": "Room", "id": "Room4", "temperature": { "value": 31.8, "type": "Float" }, "pressure": { "value": 712, "type": "Integer" } } ] }
  • 39.
    How to getOrion? (Virtual Machines) 39 • FIWARE Lab image – Image: orion-psb-image-R<x>.<y> • VirtualBox image – http://bit.ly/fiware-orion024-vbox (it’s big!) – User/pass: • fiware/fiware • root/fiware • Hint: update Orion package once the VM is deployed
  • 40.
    How to getOrion? (Docker containers) 40 • Assuming docker is installed in your system • Documentation in https://github.com/telefonicaid/fiware- orion/tree/develop/docker • Quick guide git clone https://github.com/telefonicaid/fiware-orion.git cd fiware-orion/docker sudo docker-compose up • That’s all! – curl localhost:1026/version
  • 41.
    • Have alook to the FIWARE Reference Tutorial application – git clone https://github.com/Fiware/tutorials.TourGuide- App.git – cd tutorials.TourGuide-App/ – docker-compose up orion – curl localhost:1026/version • Self-explanatory README.md at root directory • Open a Postman session and rock and roll – Postman collection: https://github.com/Fiware/tutorials.TourGuide- App/blob/develop/contrib/CampusParty2016.postman_coll ection 41 Would you like to play with this?
  • 42.
    42 Pagination Metadata Compound attribute/metadata values Typebrowsing Geo-location Query filters DateTime support Custom notifications Notification status Attribute/metadata filtering Special attribute/metadata Registrations & context providers Multitenancy Orion advanced functionality Creating & pulling data Pushing data Subscriptions & Notifications Batch operations
  • 43.
    • Entities mayhave a location • Queries/subscriptions may use the location as search criteria • Specific session on this tomorrow 12:45- 13:30 – NGSI: geoqueries and Carto integration, Fermín Galán & Francisco Romero (Data Team) 43 Geo-location
  • 44.
    • Pagination helpsclients organize query and discovery requests with a large number of responses. • Three URI parameters: – limit • Number of elements per page (default: 20, max: 1000) – offset • Number of elements to skip (default: 0) – count (option) • Returns total elements (default: not return) 44 Pagination
  • 45.
    • Example, queryingthe first 100 entries: – GET <orion_host>:1026/v2/entities?limit=100&options=count • The first 100 elements are returned, along with the following header in the response: – Fiware-Total-Count: 322 • Now we now there are 322 entities, we can keep querying the broker for them: – GET <orion_host>:1026/v2/entities?offset=100&limit=100 – GET <orion_host>:1026/v2/entities?offset=200&limit=100 – GET <orion_host>:1026/v2/entities?offset=300&limit=100 45 Pagination
  • 46.
    • By default,results are ordered by entity creation date • This behavior can be overridden using orderBy URI parameter – A comma-separated list of attributes. Results are ordered by the first attribute. On ties, the results are ordered by the second attribute and so on. A "!" before the attribute name means that the order is reversed. • Example: get the first 10 entities ordered by temp in ascending order, then humidity in descending order GET <orion_host>:1026/v2/entities?limit=20&offset=0&orderBy=temp,!humidity • dateCreated and dateModified can be used to ordering by entity creation and modification date, respectively 46 Pagination
  • 47.
    • Users mayattach metadata to attributes • Reserved metadata: ID, location, dateCreated, dateModified, previousValue, actionType • Examples: 47 … "temperature": { "type": "Float", "value": 26.5, "metadata": { { "accuracy": { "type": "Float", "value": 0.9 } } } … … "temperature": { "type": "Float", "value": 26.5, "metadata": { { “average": { "type": "Float", "value": 22.4 } } } … Metadata
  • 48.
    Complete NGSI Model Attributes •Name • Type • Value Entity • EntityId • EntityType 1 n “has” Metadata • Name • Type • Value1 n “has” 48
  • 49.
    • Attributes andmetadata can have a structured value. Vectors and key-value maps are supported. • It maps directly to JSON's objects and arrays. 49 Compound Attribute/Metadata Values
  • 50.
    • Example: wehave a car whose four wheels' pressure we want to represent as a compound attribute for a car entity. We would create the car entity like this: { "type": "Car", "id": "Car1", "tirePressure": { "type": "kPa", "value": { "frontRight": "120", "frontLeft": "110", "backRight": "115", "backLeft": "130" } } } 50 Compound Attribute/Metadata Values
  • 51.
    Type Browsing • GET/v2/types • Retrieve a list of all entity types currently in Orion, including their corresponding attributes and entities count • GET /v2/types/{typeID} • Retrieve attributes and entities count associated to an entity type PRO TIP GET /v2/contextTypes?options=values Retrieves just a list of all entity types without any extra info 51
  • 52.
    • For theGET /v2/entities operation • By entity type • By entity id list • By entity id pattern (regex) • By entity type pattern (regex) • By geographical location – Described in detail in previous slides • Filters can be used simultaneously (i.e. like AND condition) 52 GET <cb_host>:1026/v2/entities?type=Room GET <cb_host>:1026/v2/entities?id=Room1,Room2 GET <cb_host>:1026/v2/entities?idPattern=^Room[2-5] GET <cb_host>:1026/v2/entities?typePattern=T[ABC] Query filters
  • 53.
    • By attributevalue (q) • By metadata value (mq) • See full details about q and mq query language in NGSIv2 specification 53 GET <cb_host>:1026/v2/entities?q=temperature>25 GET <cb_host>:1026/v2/entities?q=tirePressure.frontRight >130 attribute name attribute sub-key (for compound attribute values only) GET <cb_host>:1026/v2/entities?mq=temperature.avg>25 GET <cb_host>:1026/v2/entities?mq=tirePressure.accuracy.frontRight >90 metadata sub-key (for compound metadata values only) attribute name metadata name Query filters
  • 54.
    54 POST <cb_host>:1026/v2/subscriptions … { "subject": { "entities":[ { "id": “Car5", "type": “Car" }, { "idPattern": “^Room[2-5]", "type": "Room" }, { "id": “D37", "typePattern": "Type[ABC]" }, ], "condition": { "attrs": [ "temperature" ], "expression": { "q": "temperature>40", "mq": "humidity.avg==80..90", "georel": "near;maxDistance:100000", "geometry": "point", "coords": "40.418889,-3.691944" } } }, … } • Filters can be also used in subscriptions – id – type – id pattern – type pattern – attribute values – metadata value – geographical location Query filters
  • 55.
    Datetime support • Orionimplements date support – Based on ISO ISO8601 format, including partial representations and timezones • See https://fiware- orion.readthedocs.io/en/master/user/ngsiv2_implementati on_notes/index.html#datetime-support for syntax details – Use reserved attribute type DateTime to express a date – Date-based filters are supported 55
  • 56.
    Datetime support • Attributevalue arithmetic filters can be used with dates as if they were numbers • Entity dateModified and dateCreated special attributes, to get entity creation and last modification timestamps – They are shown in query responses using attrs=dateModified,dateCreated • Entity dateModified and dateCreated special metadata, to get attribute creation and last modification timestamps – They are shown in query responses using metadata=dateModified,dateCreated 56 POST /v2/entities … { "id": "John", "birthDate": { "type": "DateTime", "value": "1979-10-14T07:21:24.238Z" } } GET /v2/entities?q=birthDate<1985-01-01T00:00:00 Example: create entity John, with birthDate attribute using type DateTime
  • 57.
    • Apart fromthe standard formats defined in the previous slides NGSIv2 allows to re-define all the notification aspects • httpInfo is used instead of http, with the following subfields – URL query parameters – HTTP method – HTTP headers – Payload (not necessarily JSON!) • A simple macro substitution language based on ${..} syntax can be used to “fill the gaps” with entity data (id, type or attribute values) – Exception: this cannot be used in HTTP method field 57 Custom notifications
  • 58.
    58 … "httpCustom": { "url": "http://foo.com/entity/${id}", "headers":{ "Content-Type": "text/plain" }, "method": "PUT", "qs": { "type": "${type}" }, "payload": "The temperature is ${temp} degrees" } … PUT http://foo.com/entity/DC_S1-D41?type=Room Content-Type: text/plain Content-Length: 31 The temperature is 23.4 degrees PUT /v2/entities/DC_S1-D41/attrs/temp/value?type=Room … 23.4 Custom notification configuration update notificaiton Custom notifications
  • 59.
    • Status failedmeans that last attempt to notify failed – E.g. the endpoint is not reachable • Detailed information in the notifications element – timesSent: total number of notifications attempts (both successful and failed) – lastSuccess: last time that notification was successfully sent – lastFailure: last time that notification was tried and failed – lastNotification: last time the notification was sent (either success or failure) • Corollary: lastNotification value is the same than either lastFailure or lastSuccess 59 200 OK Content-Type: application/json … [{ "id": " 51c0ac9ed714fb3b37d7d5a8 ", "expires": "2026-04-05T14:00:00.00Z", "status": "failed", "subject": { … }, "notification": { "timesSent": 3, "lastNotification": "2016-05-31T11:19:32.00Z", "lastSuccess": "2016-05-31T10:07:32.00Z", "lastFailure": "2016-05-31T11:19:32.00Z", … } }] Notification status
  • 60.
    • By defaultall attribute are included in query responses or notifications • The attrs field (as parameter in GET operations and as notification sub-field in subscriptions) can be used to specify a filtering list • The attrs field can be also used to explicitly include some special attributes (not included by default) – dateCreated, dateModified: described in previous slide • The “*” can be used as an alias of “all the (regular) attributes” 60 Attribute filtering and special attributes
  • 61.
    • Examples – Includeonly attributes temp and lum • In queries: GET /v2/entities?attrs=temp,lum • In subscriptions: "attrs": [ "temp", "lum" ] – Include dateCreated and not any other attribute • In queries: GET /v2/entities?attrs=dateCreated • In subscriptions: "attrs": [ "dateCreated" ] – Include dateModified and all the other (regular) attributes • In queries: GET /v2/entities?attrs=dateModified,* • In subscriptions: "attrs": [ "dateModified", "*" ] – Include all attributes (same effect that not using attrs, not very interesting) • In queries: GET /v2/entities?attrs=* • In subscriptions: "attrs": [ "*" ] 61 Attribute filtering and special attributes
  • 62.
    • By defaultall attribute metadata are included in query responses and notifications • The metadata field (as parameter in GET operations and as notification sub-field in subscriptions) can be used to specify a filtering list • The metadata field can be also used to explicitly include some special metadata (not included by default) – dateCreated, dateModified: described in previous slide – actionType: which value is the action type corresponding to the update triggering the notification: “update”, “append” or “delete” – previousValue: which provides the value of the attribute previous to processing the update that triggers the notification • The “*” can be used as an alias of “all the (regular) metadata” 62 Metadata filtering and special attributes
  • 63.
    • Examples – Includeonly metadata MD1 and MD2 • In queries: GET /v2/entities?metadata=MD1,MD2 • In subscriptions: "metadata": [ "MD1", "MD2" ] – Include previousValue and not any other metadata • In queries: GET /v2/entities?metadata=previousValue • In subscriptions: "attrs": [ "previousValue" ] – Include actionType and all the other (regular) metadata • In queries: GET /v2/entities?metadata=actionType,* • In subscriptions: "attrs": [ "actionType", "*" ] – Include all metadatata (same effect that not using metadata, not very interesting) • In queries: GET /v2/entities?metadata=* • In subscriptions: "metadata": [ "*" ] 63 Metadata filtering and special attributes
  • 64.
    • Uncached queriesand updates 64 Application ContextBroker ContextProvider 1. registerContext(provider= ) db 2. query 3. query 4. data5. data Context Consumer Registration & Context Providers
  • 65.
    POST <cb_host>:1026/v1/registry/registerContext … { "contextRegistrations": [ { "entities":[ { "type": "Car", "isPattern": "false", "id": "Car1" }, "attributes": [ { "name": "speed", "type": "float", "isDomain": "false" } ], "providingApplication": "http://contextprovider.com/Cars" } ], "duration": "P1M" } 200 OK ... { "duration" : "P1M", "registrationId" : "52a744b011f5816465943d58" } 65 Context management availability functionality not yet specified in NGSIv2. Thus, a NGSIv1 operation is used to create the registration. Registration & Context Providers
  • 66.
    66 GET <cb_host>:1026/v2/entities/Car1/attrs ContextBroker ContextProvider db query data 200OK Content-Type: application/json ... { "type": "Float", "value": 110, "metadata": {} } Registration & Context Providers
  • 67.
    • Simple multitenantmodel based on logical database separation. • It eases tenant-based authorization provided by other components. • Just use an additional HTTP header called "Fiware-Service", whose value is the tenant name. Example: Fiware-Service: Tenant1 Context Broker Tenant1 Tenant2 … 67 Multitenancy
  • 68.
    • A servicepath is a hierarchical scope assigned to an entity at creation time (with POST /v2/entities). 68 Service Paths
  • 69.
    • In orderto use a service path we put in a new HTTP header called "Fiware-ServicePath". For example: Fiware-ServicePath: /Madrid/Gardens/ParqueNorte/Parterre1 • Properties: – A query on a service path will look only into the specified node – Use "ParentNode/#" to include all child nodes – Queries without Fiware-ServicePath resolve to "/#" – Entities will fall in the "/" node by default ParqueNorte Parterre2Parterre1 69 Service Paths
  • 70.
    • Properties (continued): –You can OR a query using a comma (,) operator in the header • For example, to query all street lights that are either in ParqueSur or in ParqueOeste you would use: ServicePath: Madrid/Gardens/ParqueSur, Madrid/Gardens/ParqueOeste • You can OR up to 10 different scopes. – Maximum scope levels: 10 • Scope1/Scope2/.../Scope10 – You can have the same element IDs in different scopes (be careful with this!) – You can't change scope once the element is created – One entity can belong to only one scope – It works not only with queries, but also with subscriptions/notifications – It works not only in NGSI10, but also with registrations/discoveries (NGSI9) ParqueNorte Parterre1 light1 light1 A B A or B 70 Service Paths
  • 71.
    • Interesting NGSI&Orion-relatedstuff during FIWARE Summit – Wednesday 14th 12:45-13:30: “NGSI: geoqueries and Carto integration”, Fermín Galán & Francisco Romero (Data Team) – Wednesday 14th, 15:45-16:30: “Creating context historic using Cygnus”, Francisco Romero (Data team) – Wednesday 14th 18:15-18:45 “NGSIv2-Overview- for-Developers-That-Already-Know-NGSIv1”, Fermín Galán (Orion Context Broker Development Lead) – Thursday 15th 12:15-13:00, “Hands-on FIWARE Context Provider Simulator Tutorial”, German Toro (Data team) 71 Where (and when ) to go after this talk?
  • 72.
    • The easyway – This presentation: google for “fermingalan slideshare” and search the one named “Managing Context Information at large scale” – Orion User Manual: google for “Orion FIWARE manual” and use the first hit – Orion Catalogue page: google for “Orion FIWARE catalogue” and use the first hit • References – NGSIv2 Specification • http://fiware.github.io/specifications/ngsiv2/stable • http://fiware.github.io/specifications/ngsiv2/latest – NGSIv2 for NGSIv1 developers • http://bit.ly/ngsiv2-vs-ngsiv1 – This presentation • http://www.slideshare.net/fermingalan/fiware-managing-context-information-at-large- scale – Orion Catalogue: • http://catalogue.fiware.org/enablers/publishsubscribe-context-broker-orion-context- broker – Orion support trhough StackOverflow • Ask your questions using the “fiware-orion” tag • Look for existing questions at http://stackoverflow.com/questions/tagged/fiware-orion 72 Would you like to know more?
  • 73.
  • 74.
    Integration with sensornetworks • The backend IoT Device Management GE enables creation and configuration of NGSI IoT Agents that connect to sensor networks • Each NGSI IoT Agent can behave as Context Consumers or Context Providers, or both FIWARE Context Broker IoT Agent-1 IoT Agent-2 IoT Agent-n IoT Agent Manager create/monitor FIWARE Backend IoT Device Management OMA NGSI API (northbound interface) (southbound interfaces) MQTTETSI M2M IETF CoAP 74
  • 75.
    • Federation ofinfrastructures (private/public regions) • Automated GE deploymentCloud • Complete Context Management Platform • Integration of Data and Media ContentData •Easy plug&play of devices using multiple protocols •Automated Measurements/Action Context updatesIoT •Visualization of data (operation dashboards) •Publication of data sets/servicesApps •Easy support of UIs with advanced web-based 3D and AR capabilities •Visual representation of context information. Web UI •Advanced networking capabilities (SDN) and Middleware •Interface to robotsI2ND •Security Monitoring •Built-in Identity/Access/Privacy ManagementSecurity Context Management in FIWARE 75
  • 76.
    FI-WARE Context/Data ManagementPlatform Context/Data Management Platform Applications OMA NGSI-9/10 Processing/Analysis Algorithms Gathered data is injected for processing/analysis Distributed Context Sources Complex Event Processing (PROTON) BigData (COSMOS) Processed data is injected for processing/analysi s Data generated either by CEP or BigData is published Gathered data injected for CEP-like processing Direct bigdata injection Programming of rules 76
  • 77.
    • Used by/v2/op/update (batch operation) • Conventional actionTypes – APPEND: append (or update if the attribute already exists) – UPDATE: update – DELETE: delete • Special actionTypes – APPEND_STRICT: strict append (returns error if some of the attributes to add already exists) – REPLACE: delete all the entity attributes, next append the ones in the update request Special update action types 77

Editor's Notes

  • #16 Orion Context Broker is an implementation of a context information broker with persistent storage It implements OMA NGSI9/10 specification NGSI9 is about context information availability (i.e. sources of context information) management NGSI10 is about context information itself