Download to read offline
Elephants Ibises and a more Pythonic way to work with databases
- 1. Presented by: Elephants, ibisesand a more Pythonic way to work with databases July 13, 2022 Marlene Mhangami
- 2. 2 ● Harare, Zimbabwe ●Developer Advocate Voltron Data ● Previous Vice-Chair and Director for the Python Software Foundation @marlene_zw marlenemhangami.com Marlene Mhangami
- 3. 3 Art Around Dublin post conferenceexploration
- 4. 4 MySQL database - Notas familiar - Had to switch to terminal - Working with specific data types (e.g geographic data) - Not as maintainable (tests, readability etc)
- 5. 5 There’s got tobe a more Pythonic way?
- 6.
- 7. 7 Ibis - A Pythonpackage that provides us with a more pythonic way to work to databases - A library for analytical SQL - Inspired by dplyr
- 8. 8 Ibis the idea behindibis is that we want lightweight Python code telling the backend to do the heavy lifting Hey! Can you turn to the right and take us there
- 9. 9 - pandas-like - compilesto sql (executed efficiently) * anything you can write with a SELECT statement you can write in ibis
- 10. t.groupby(‘region’) .aggregate(topk=t.funding_rounds.count()) .sort_by(ibis.desc(‘topk’)) .head() Produces SQLAlchemy Expression <sqlalchemy.sql.select able.Select object> SQL BACKENDS Ibis producesExpressions or Strings for SQL backends SELECT region, count(founding_rounds) AS topk FROM companies GROUP BY region ORDER BY topk SQL STRINGS EXECUTED Ibis expression
- 11. 11 Ibis Elephants are largeand your data can be too
- 12.
- 13.
- 14. 14 Installing Ibis pip installibis-framework import ibis
- 15. 15 Connecting to database -switchto jupyter notebook (using vscode with theme) -import ibis -connect to database
- 16. 16 Table columns Add description... ●access sqlite table ● column names ● pandas-like
- 17. 17 Query the dataset Adddescription... - 420 artists part of the project - 5 Councils took part
- 18. 18 Methods in Ibis Adddescription... -methods available -table. (hit tab) -visit ibis-project.org for docs
- 19. 19 Displaying the artin a notebook Add description...
- 20. 20 Dublin Convention Center Find thecanvas art closest to us
- 21. 21 Switching to PostgreSQL Adddescription... - GeoJSON file - PostGIS for working with geospatial geometries - Data stored in wkb format
- 22. 22 Working with GeoPandas Adddescription... - get wkb data - get corresponding art titles - use geopandas to change wkb to longitude and latitude - Create a dataframe containing titles and lats and long - add latitude and longitude of the convention center
- 23. 23 Working with GeoPandas Adddescription... Use Geopandas explore method on the dataframe to visualise geographic location of the art
- 24.
- 25.
- 26. marlene@voltrondata.com marlene_zw Contact READ THE BLOGPOST! Twitter SOCIAL MEDIA GENERAL CONTACT marlenemhangami.com
- 27.