Building
Learning
Agents
Jeff Smith
@jeffksmithjr
JohnD.one
Bio
Bio
Bio
johnd.one
Designing Agents
Agents
Software Agents
Software Agents
Mmmm…
Lunch.
Software Agents
Sensors
Actuators
Function
Intelligent Agents
Intelligent Agents
Maybe
Later
Intelligent Agents
Sensors
Actuators
Knowledge
Function
Learning Agents
Learning Agents
Not that
One!
Learning Agents
Sensors
Actuators
Knowledge
Learning
Function
Reactive Machine Learning
Reactive
Reactive Systems
Responsive
Resilient Elastic
Message-Driven
Reactive Strategies
Replication Containment Supervision
Reactive Machine Learning
Reactive Machine Learning
Uncertain DataInfinite Data
Laziness
Pure
Functions
Immutable
Facts
Possible
Worlds
Reactive Learning Agents
Learning Agents
Sensors
Actuators
Knowledge
Learning
Function
Reactive Systems
Responsive
Resilient Elastic
Message-Driven
Reactive
+ Learning AgentsReactive
Source of AI Components
Source of AI Components
Yours
Source of AI Components
Other People’s Yours
Sensors
Sensors
•Email/Messaging
•Phone cameras & sensors
•Wearables
•Speaker mIcs
Sensors
•Email/Messaging
•Phone cameras & sensors
•Wearables
•Speaker mIcs
•Bespoke Data Collection
Systems
Reactive Data Collection
Reactive Data Collection
Reactive Data Collection
Reactive Data Collection
Reactive Data Collection
Reactive Data Collection
Reactive Data Collection
•Replication via Distributed databases
•Event-sourcing/immutable fact databases
•Uncertainty-aware data models
•AP Databases
https://skillsmatter.com/skillscasts/7038-lightning-talks-2
Actuators
Actuators
•Email/Messaging
•Notifications
•Commerce APIs
•Speaker devices
Actuators
•Email/Messaging
•Notifications
•Commerce APIs
•Speaker devices
•Self-driving Cars
•External API Calls
•Internal API Calls
Reactive Action
Sensors
Actuators
Knowledge
Learning
Function
Reactive Action
Sensors
Actuators
Knowledge
Learning
Function
Reactive Action
Sensors
Actuators
Knowledge
Learning
Function
Reactive Action
Sensors
Actuators
Knowledge
Learning
Function
Knowledge
Knowledge
•External Datasets
•Libraries
•Knowledge as a Service
Knowledge
•External Datasets
•Libraries
•Knowledge as a Service
•Your Training corpus
•Libraries
•Institutional knowledge
technical Detour
Erlang & the BEAM
•Functional Language
•Concurrency-oriented
•Actor Model
•Over 30 years old!
Elixir
•Functional Language
•Concurrency-oriented
•Homoiconic Syntax
•Developer tooling
Reactive Knowledge Systems
Adapted from https://github.com/mitchellhenke/circuit_breaker
Reactive Knowledge Systems
Reactive Knowledge Systems
Reactive Knowledge Systems
Reactive Knowledge Systems
Reactive Knowledge Systems
Learning
Learning
•Machine Learning
as a Service APIs
•Open Source Libraries
Learning
•Machine Learning
as a Service APIs
•Open Source Libraries
•Your novel Algorithms
•Organizational Learning
Reactive Learning
Reactive Learning
Reactive Learning
Raw
Data
Features Model
Reactive Learning
Raw
Data
Features Model
Reactive Learning
Raw
Data
Features Model
Raw
Data
Features Model
Reactive Learning
Raw
Data
Features Model
Raw
Data
Features Model
Reactive Learning
Raw
Data
Features Model
Raw
Data
Features Model
Reactive Learning
Raw
Data
Features Model
Reactive Learning
Raw
Data
Features Model
https://youtu.be/B7XYlBaxglU
Agent Function
Agent Function
•Templates
Agent Function
•Templates •Decision Component
Reactive Agent Functions
Reactive Agent Functions
Reactive Agent Functions
Reactive Agent Functions
Reactive Agent Functions
Dialyzer
•Linter
•Type System
•Optional
•Incremental
Reactive Agent Functions
Dialyzer in Action
ml_system.ex:3: Function predict/1 has no local return
ml_system.ex:6: The call
'Elixir.MLSystem':call_model_b(feature@1::number()) will never
return since it differs in the 1st argument from the success
typing arguments: (binary())
ml_system.ex:22: Invalid type specification for function
'Elixir.MLSystem':call_model_b/1. The success typing is
(binary()) -> binary()
ml_system.ex:23: Function call_model_b/1 has no local return
Dialyzer in Action
ml_system.ex:24: The call
'Elixir.String':upcase(feature@1::number()) will never return
since the success typing is (binary()) -> bitstring() and the
contract is (t()) -> t()
ml_system.ex:38: Function ensemble/1 will never be called
Agents
Agents
•Templates
Agents
•Templates •Whole System
Model Registry
Public API
server Callbacks
Testing with Concuerror
Concuerror
•Testing tool
•Concurrency-oriented
•Explores Interleavings
Concuerror Output
Interleaving #1
----------------------------------------------------------------
-------------
Errors found:
* Blocked at a 'receive' (when all other processes have exited):
P.1 in gen_server.erl line 382
----------------------------------------------------------------
-------------
Model Registry
Model Registry
Model Registry
Testing Agents on the BEAM
Tool Tests
Dialyzer Types
QuickCheck Properties
Concuerror Interleavings
Agent Frameworks
Agent Frameworks
•API.AI (Google)
•Lex (Amazon)
•Microsoft Bot Framework
•Botkit Studio
•Recime
Demo
Recap
Learning Agents
Sensors
Actuators
Knowledge
Learning
Function
Reactive Systems
Responsive
Resilient Elastic
Message-Driven
Reactive Machine Learning
Uncertain DataInfinite Data
Laziness
Pure
Functions
Immutable
Facts
Possible
Worlds
Source of AI Components
Other People’s Yours
Elixir & the BEAM
Agent Frameworks
For later
Reactive Machine Learning
reactivemachinelearning.com
@jeffksmithjr
johnd.one
Building
Learning
Agents
Jeff Smith
@jeffksmithjr
JohnD.one

Building Learning Agents