0

I have a huge database (~70 gigabytes) all stored in a sql table.

I cannot pull the whole database into Spyder due to memory issues, but I only need a select portion of that sql table anyways.

I have a data frame of 576 rows/elements that have an ID that correlates to an ID in the 70 gig table.

Is there a way to pull in only the data that have matching IDs from the SQL table?

This should allow for the table to be small enough for Spyder to handle plus it is the only data I need in the end so it would take care of narrowing the desired data.

I realize this is a bad question for Stack, but I am more so curious if it possible to do this pull in python without pulling the whole table in in the first place.

I have no issues finding the data once the table is in Spyder, but I have not run into this issue yet and I am new to python with SQL.

When I try to search on one ID let alone the 576, the search takes a very long time.

Events = pd.read_sql('SELECT SystemID FROM Events WHERE SystemID = 14237 ', conn)
Improve this question
2
Related questions
0
Extracting values from SQLite database in Python
1
Retrieve row from sqlite with only part of column's data
0
SQLIte3 querying a database
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.