Automating cell-based screening
with open source, robotics and AI
Ola Spjuth, Professor
Department of Pharmaceutical Biosciences and Science for Life Laboratory
Uppsala University
https://www.linkedin.com/in/olaspjuth/
l Processing Stream Processing
Dat a
osit ory
Query
request
response
Real- T ime
Analyt ics
Dat a Result s
Current fact finding
Analyze data in motion – before it is stored
ormation stored on disk
Our mission: Accelerating drug discovery
Applications
in
drug
discovery
Cell profiling
Automation
AI/ML
Accelerate drug discovery using AI,
automation and intelligent design
of experiments
• Predict safety concerns
• Predict drug mechanisms
• Screen for new drugs
• Precision medicine
Research objective:
Applications
in
drug
discovery
Cell profiling
Automation
AI/ML
Open source fully automated cell profiling lab
Cell Painting – high-content imaging Computational and Data Management Infrastructure
AI and Robotics
Morphological profiling experiments
Intelligent
design of
experiments
Vision: Autonomous
experimentation
Cell
profiling
AI/ML
Morphological
Profile
Cell segmentation
& Image analysis
Microscopy
imaging
Chemical perturbation
of cells
Experiments in
multiwell plates
Cell Painting Bray et al. Nat Protoc 2016
DNA Damage
Prediction
Deep Neural Network
Drug-treated cells
Kinase inhibitor
Kensert A, Harrison PJ, Spjuth O.
Transfer learning with deep convolutional neural network for classifying cellular
morphological changes. SLAS DISCOVERY: Advancing Life Sciences R&D. 24, 4 (2019)
Our objectives with automation
• Fully automated Cell Painting
• Robotized experiments
• Configurable, serviceable by us
• Cost-efficient, use off-the-shelf instruments
• Full control of all steps
• Automated analytics
• Standardized data capturing, preprocessing,
and AI modeling
• Closed-loop science
• Full control of all steps open up for online
decision making
AI and Robotics
Morphological profiling experiments
Intelligent
design of
experiments
Vision: Autonomous
experimentation
Automating cell profiling:
Building an open source robotized lab
Plate incubator
Washer
Dispenser
Robot arm
https://github.com/pharmbio/aros
Washer
incubation for 24-48h
at 37℃ and 5% CO2
Washer
(2X PBS)
Mitotracker staining
(live cells staining)
[dispenser peripump 1]
incubation for 20 min
at 37℃ and 5% CO2
Washer
(3X PBS)
Fixation
(4% PFA)
[dispenser Syringe A]
Seeded plates with compound
treatment
Washer
(3X PBS)
incubation for 20 min
in room temperature
Permeabilization
(0.1% Triton X-100)
[dispenser Syringe B]
Washer
(3X PBS)
incubation for 20 min
in room temperature
Post-fixation staining
(staining mixture of 5 dyes)
[dispenser peripump 2]
Washer
(5X PBS)
incubation for 20 min
in room temperature
Imaging store in 4°C
Optimizing and executing experiments
Detailed logs
Checkpoint('batch_1')
WaitForCheckpoint('batch_1') + 'plate_1_get_delay'
EnqueueOnMachine(incubator, RunMachine(incubator, 'get L1'))
RunMachine(robotarm, 'goto incubator')
WaitForMachineReady(incubator)
RunMachine(robotarm, 'get from incubator')
RunMachine(robotarm, 'lid off')
RunMachine(robotarm, 'to washer')
EnqueueOnMachine(washer, [
WaitForCheckpoint('batch_1') + 'plate_1_wash_delay',
RunMachine(washer, 'wash_2X_before_mito.LHC')])
EnqueueOnMachine(dispenser, [
Idle('mito_prime_delay', optimize='maximize'),
RunMachine(dispenser, 'mito_prime.LHC')])
WaitForMachineReady(washer)
RunMachine(robotarm, 'wash to disp')
WaitForMachineReady(disp)
EnqueueOnMachine(dispenser, RunMachine(dispenser, 'mito_40ul.LHC'))
WaitForMachineReady(disp)
Checkpoint('plate_1_incubation_1')
...
Protocols expressed in code.
Generates a protcol in a DSL.
Optimized using constraint programming.
https://github.com/pharmbio/aros
AI for experimental design
AI for optimizing
experiments
AI and robotics to automate cell profiling
AI for data analysis
AI to guide image
acquisition
AI for cell segmentaion
Microplate
Wells
1
384
Sites
3 456
Images
>15-25 K
(200 GB)
Cells
>300 K
Measurements
>400 M
x9/ 2h
Dealing with continuous large
scale data
GPU cluster
CPU server
Storage
Cloud
Online processing
Automating analysis
pipelines
Images AI modeling
QC
Morphology profiles
Computational
infrastructure
• Image and metadata storage in database
• Access through browser
• Quick image visualisation including:
o Timepoint
o Site
o Channel
• Direct link to analysis pipeline
Image Database + web viewer
https://imagedb.k8s-prod.pharmb.io/
Automating Analysis pipelines
Jupyter Notebooks for data analysis
• QC pipeline
• Data analysis
• Data visualisation
• Publish via github
How to scale up imaging?
• Minimize images captured and stored, also go towards live-cell imaging
• Get more and faster microscopes
Hongquan L. et al. “Squid: Simplifying Quantitative
Imaging Platform Development and Deployment.”
bioRxiv 2020.12.28.424613
https://squid-imaging.org
Data-driven image prioritization
Microscope farm Image streams
…
…
…
Feature
Extraction
Interestingness
function
Policy
Prioritization
How can we deal with large, continuous streams of images?
Designing cell profiling experiments
• Many options to consider:
• Which cell line(s)
• How many and which compounds
• Concentrations
• Replicates
• Exposure time(s)
• Total number of plates
• Positive controls, negative controls
• Can AI be used to optimize design of
experiments?
• Minimize experiment time and cost
• Maximize usefulness of resulting data
-Manuscript Draft- 13th
Dec, 2021, 15:32
Plate effect Border layout Random layout Effective layout
Figure 1: Examples of the distribution of 20 negative control in layouts for
384-well microplates. The colors indicate the intensity measured at each well.
Approach: Use constraint programming to design
effective plate layouts
• Optimal under set of constraints
• Declarative / flexible to adapt
Designing microplate experiments with AI
Rodríguez MAF, Carreras-Puigvert J, and Spjuth O. “Designing microplate layouts
using artificial intelligence.” bioRxiv. 2022.03.31.486595 (2022).
-Manuscri
(b) Relative IC50/ EC50
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
(d) Residuals
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
Screening
Dose-response
https://github.com/pharmbio/plaid
PLAID: Plate Layouts
using Artificial
Intelligence Design
From prediction to decision making:
What is the next, best experiment?
Automated lab AI models Data
Decision making
Screening for antiviral drugs by phenomics
Reversal of the SARS-CoV-2-induced phenotype in Vero E6 cells
Unpublished data
Rietdijk J, Tampere M, Pettke A, Georgieva P, Lapins M, Warpman Berglund U, Spjuth O, Puumalainen MR,
Carreras-Puigvert J. “A phenomics approach for antiviral drug discovery.” BMC Biology 19 (2021).
Can we improve using
drug combinations?
• Safety of chemicals and drugs assessed individually
• Combination therapy is common for many diseases
• How can we research combinations/mixtures of chemicals?
• Our approach:
• Cell Painting: Ideal tradeoff between speed/cost/information content
• AI/ML: Select and design next batch of combinations to test
• Automation: Iteratively carry out experiments and repeat
Ongoing research: Study combinations
of compounds
Mixtures and Cell Painting
AI and Robotics
Morphological profiling experiments
Cell lines /
Primary cells
Profiling of individual drugs with
known mechanisms
….
Drugs
Profiling and screening for
drug combinations
….
….
A discovery engine!
Iterative exploration of
individualized drug combinations
Primary
cells and
organoids
Individualized drug
combination
Patient
• Manage life cycle of AI
models, improve FAIR1
• Deploy and use apps and AI
models in production
• Effort at SciLifeLab Data
Center, Sweden
Working with ML models
1 Spjuth O, Frid J, and Hellander A.
The Machine Learning Life Cycle and the Cloud: Implications for Drug Discovery
Expert Opinion On Drug Discovery. 16, 9, 1071-1079. (2021).
DOI: 10.1080/17460441.2021.1932812 https://serve.scilifelab.se
Teaching next-generation scientists
Introduction to Lab Automation, 7.5 c
Learn to use automated pipetting, plate
handling, microscopy.
Big Data in Life Science, 5c
Compute infrastructure and analysis
methods to analyze large data.
Other courses (italic=online):
Pharmaceutical Bioinformatics, 7.5 c
Pharmaceutical Bioinformatics with Sequence Analysis, 7.5c
Applied Pharmaceutical Bioinformatics, 5c
Applied Pharmaceutical Structural Bioinformatics, 5c
Artificial Intelligence in drug
discovery, 7.5c
Online introductory course.
Acknowledgements
Funding:
Pharmb.io research group
Jordi Carreras-Puigvert
Wesley Schaal
Jonathan Alvarsson
Maris Lapins
Polina Georgieva
Malin Jarvius
Anders Larsson
Dan Rosén
Andreina Rodrigues
Christa Ringers
Staffan McShane
Amelie Wenz
Ebba Bergman
Phil Harrison
Jonne Rietdijk
David Holmberg
Akshai Sreenivasan
Martin Johansson
HASTE project
Carolina Wählby, UU
Andreas Hellander, UU
Salman Toor, UU
Håkan Wieslander, UU
Ankit Gupta, UU
Tianru Zhang, UU
Xiaobo Zhao, UU
Ben Blamey, UU
Alan Sabirsh, AstraZeneca
Ida-Maria Sintorn, Vironova
Karolinska Institutet
Päivi Östling
Brinton Seashore-Ludlow
Marianna Tampere
SciLifeLab Data Center
The Serve team
NTNU/Trondheim
Åsmund Flobak
Astrid Laegrid
Ulf Norinder
Ernst Ahlberg
Lars Carlsson
Fredrik Svensson
CBCS
Research group website: http://pharmb.io
Ola Spjuth
ola.spjuth@farmbio.uu.se
https://www.linkedin.com/in/olaspjuth/
Thank you

Automating cell-based screening with open source, robotics and AI

  • 1.
    Automating cell-based screening withopen source, robotics and AI Ola Spjuth, Professor Department of Pharmaceutical Biosciences and Science for Life Laboratory Uppsala University https://www.linkedin.com/in/olaspjuth/
  • 2.
    l Processing StreamProcessing Dat a osit ory Query request response Real- T ime Analyt ics Dat a Result s Current fact finding Analyze data in motion – before it is stored ormation stored on disk Our mission: Accelerating drug discovery
  • 3.
    Applications in drug discovery Cell profiling Automation AI/ML Accelerate drugdiscovery using AI, automation and intelligent design of experiments • Predict safety concerns • Predict drug mechanisms • Screen for new drugs • Precision medicine Research objective:
  • 4.
    Applications in drug discovery Cell profiling Automation AI/ML Open sourcefully automated cell profiling lab Cell Painting – high-content imaging Computational and Data Management Infrastructure AI and Robotics Morphological profiling experiments Intelligent design of experiments Vision: Autonomous experimentation
  • 5.
    Cell profiling AI/ML Morphological Profile Cell segmentation & Imageanalysis Microscopy imaging Chemical perturbation of cells Experiments in multiwell plates Cell Painting Bray et al. Nat Protoc 2016 DNA Damage Prediction Deep Neural Network Drug-treated cells Kinase inhibitor Kensert A, Harrison PJ, Spjuth O. Transfer learning with deep convolutional neural network for classifying cellular morphological changes. SLAS DISCOVERY: Advancing Life Sciences R&D. 24, 4 (2019)
  • 6.
    Our objectives withautomation • Fully automated Cell Painting • Robotized experiments • Configurable, serviceable by us • Cost-efficient, use off-the-shelf instruments • Full control of all steps • Automated analytics • Standardized data capturing, preprocessing, and AI modeling • Closed-loop science • Full control of all steps open up for online decision making AI and Robotics Morphological profiling experiments Intelligent design of experiments Vision: Autonomous experimentation
  • 7.
    Automating cell profiling: Buildingan open source robotized lab Plate incubator Washer Dispenser Robot arm https://github.com/pharmbio/aros Washer
  • 8.
    incubation for 24-48h at37℃ and 5% CO2 Washer (2X PBS) Mitotracker staining (live cells staining) [dispenser peripump 1] incubation for 20 min at 37℃ and 5% CO2 Washer (3X PBS) Fixation (4% PFA) [dispenser Syringe A] Seeded plates with compound treatment Washer (3X PBS) incubation for 20 min in room temperature Permeabilization (0.1% Triton X-100) [dispenser Syringe B] Washer (3X PBS) incubation for 20 min in room temperature Post-fixation staining (staining mixture of 5 dyes) [dispenser peripump 2] Washer (5X PBS) incubation for 20 min in room temperature Imaging store in 4°C Optimizing and executing experiments Detailed logs Checkpoint('batch_1') WaitForCheckpoint('batch_1') + 'plate_1_get_delay' EnqueueOnMachine(incubator, RunMachine(incubator, 'get L1')) RunMachine(robotarm, 'goto incubator') WaitForMachineReady(incubator) RunMachine(robotarm, 'get from incubator') RunMachine(robotarm, 'lid off') RunMachine(robotarm, 'to washer') EnqueueOnMachine(washer, [ WaitForCheckpoint('batch_1') + 'plate_1_wash_delay', RunMachine(washer, 'wash_2X_before_mito.LHC')]) EnqueueOnMachine(dispenser, [ Idle('mito_prime_delay', optimize='maximize'), RunMachine(dispenser, 'mito_prime.LHC')]) WaitForMachineReady(washer) RunMachine(robotarm, 'wash to disp') WaitForMachineReady(disp) EnqueueOnMachine(dispenser, RunMachine(dispenser, 'mito_40ul.LHC')) WaitForMachineReady(disp) Checkpoint('plate_1_incubation_1') ... Protocols expressed in code. Generates a protcol in a DSL. Optimized using constraint programming. https://github.com/pharmbio/aros
  • 9.
    AI for experimentaldesign AI for optimizing experiments AI and robotics to automate cell profiling AI for data analysis AI to guide image acquisition AI for cell segmentaion
  • 10.
    Microplate Wells 1 384 Sites 3 456 Images >15-25 K (200GB) Cells >300 K Measurements >400 M x9/ 2h
  • 11.
    Dealing with continuouslarge scale data GPU cluster CPU server Storage Cloud Online processing Automating analysis pipelines Images AI modeling QC Morphology profiles Computational infrastructure
  • 12.
    • Image andmetadata storage in database • Access through browser • Quick image visualisation including: o Timepoint o Site o Channel • Direct link to analysis pipeline Image Database + web viewer https://imagedb.k8s-prod.pharmb.io/
  • 13.
  • 14.
    Jupyter Notebooks fordata analysis • QC pipeline • Data analysis • Data visualisation • Publish via github
  • 15.
    How to scaleup imaging? • Minimize images captured and stored, also go towards live-cell imaging • Get more and faster microscopes Hongquan L. et al. “Squid: Simplifying Quantitative Imaging Platform Development and Deployment.” bioRxiv 2020.12.28.424613 https://squid-imaging.org
  • 16.
    Data-driven image prioritization Microscopefarm Image streams … … … Feature Extraction Interestingness function Policy Prioritization How can we deal with large, continuous streams of images?
  • 17.
    Designing cell profilingexperiments • Many options to consider: • Which cell line(s) • How many and which compounds • Concentrations • Replicates • Exposure time(s) • Total number of plates • Positive controls, negative controls • Can AI be used to optimize design of experiments? • Minimize experiment time and cost • Maximize usefulness of resulting data
  • 18.
    -Manuscript Draft- 13th Dec,2021, 15:32 Plate effect Border layout Random layout Effective layout Figure 1: Examples of the distribution of 20 negative control in layouts for 384-well microplates. The colors indicate the intensity measured at each well. Approach: Use constraint programming to design effective plate layouts • Optimal under set of constraints • Declarative / flexible to adapt Designing microplate experiments with AI Rodríguez MAF, Carreras-Puigvert J, and Spjuth O. “Designing microplate layouts using artificial intelligence.” bioRxiv. 2022.03.31.486595 (2022). -Manuscri (b) Relative IC50/ EC50 ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ (d) Residuals ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ ⇤ Screening Dose-response
  • 19.
  • 20.
    From prediction todecision making: What is the next, best experiment? Automated lab AI models Data Decision making
  • 21.
    Screening for antiviraldrugs by phenomics Reversal of the SARS-CoV-2-induced phenotype in Vero E6 cells Unpublished data Rietdijk J, Tampere M, Pettke A, Georgieva P, Lapins M, Warpman Berglund U, Spjuth O, Puumalainen MR, Carreras-Puigvert J. “A phenomics approach for antiviral drug discovery.” BMC Biology 19 (2021). Can we improve using drug combinations?
  • 22.
    • Safety ofchemicals and drugs assessed individually • Combination therapy is common for many diseases • How can we research combinations/mixtures of chemicals? • Our approach: • Cell Painting: Ideal tradeoff between speed/cost/information content • AI/ML: Select and design next batch of combinations to test • Automation: Iteratively carry out experiments and repeat Ongoing research: Study combinations of compounds
  • 23.
  • 24.
    AI and Robotics Morphologicalprofiling experiments Cell lines / Primary cells Profiling of individual drugs with known mechanisms …. Drugs Profiling and screening for drug combinations …. …. A discovery engine! Iterative exploration of individualized drug combinations Primary cells and organoids Individualized drug combination Patient
  • 25.
    • Manage lifecycle of AI models, improve FAIR1 • Deploy and use apps and AI models in production • Effort at SciLifeLab Data Center, Sweden Working with ML models 1 Spjuth O, Frid J, and Hellander A. The Machine Learning Life Cycle and the Cloud: Implications for Drug Discovery Expert Opinion On Drug Discovery. 16, 9, 1071-1079. (2021). DOI: 10.1080/17460441.2021.1932812 https://serve.scilifelab.se
  • 26.
    Teaching next-generation scientists Introductionto Lab Automation, 7.5 c Learn to use automated pipetting, plate handling, microscopy. Big Data in Life Science, 5c Compute infrastructure and analysis methods to analyze large data. Other courses (italic=online): Pharmaceutical Bioinformatics, 7.5 c Pharmaceutical Bioinformatics with Sequence Analysis, 7.5c Applied Pharmaceutical Bioinformatics, 5c Applied Pharmaceutical Structural Bioinformatics, 5c Artificial Intelligence in drug discovery, 7.5c Online introductory course.
  • 27.
    Acknowledgements Funding: Pharmb.io research group JordiCarreras-Puigvert Wesley Schaal Jonathan Alvarsson Maris Lapins Polina Georgieva Malin Jarvius Anders Larsson Dan Rosén Andreina Rodrigues Christa Ringers Staffan McShane Amelie Wenz Ebba Bergman Phil Harrison Jonne Rietdijk David Holmberg Akshai Sreenivasan Martin Johansson HASTE project Carolina Wählby, UU Andreas Hellander, UU Salman Toor, UU Håkan Wieslander, UU Ankit Gupta, UU Tianru Zhang, UU Xiaobo Zhao, UU Ben Blamey, UU Alan Sabirsh, AstraZeneca Ida-Maria Sintorn, Vironova Karolinska Institutet Päivi Östling Brinton Seashore-Ludlow Marianna Tampere SciLifeLab Data Center The Serve team NTNU/Trondheim Åsmund Flobak Astrid Laegrid Ulf Norinder Ernst Ahlberg Lars Carlsson Fredrik Svensson CBCS
  • 28.
    Research group website:http://pharmb.io Ola Spjuth ola.spjuth@farmbio.uu.se https://www.linkedin.com/in/olaspjuth/ Thank you

Editor's Notes

  • #9 Note: we write a python program that generates this program Parts in green are free variables and are determined by the constraint solver
  • #18 - Minimize number of concentrations and replicates - Robust to potential systematic errors - Limit effect of failed experiments (e.g. individual wells, areas on plates, or columns/rows in plate)
  • #23 Test all combinations of 3 drugs from library of 500 oncology drugs: >20 M experiments
  • #25 Can we use iterative experimentation where an AI selects the next batch of experiments and have an automated lab perform the experiments? Selectively kill cancer cells while not harming non-malignant cells or Push cancer cells towards less aggressive state in morphological space while affecting non-malignant cells as little as possible