INSERT Pandas Dataframe to SQL-Server using Python - Jupyter Notebook

Question

Till now, I've been requesting data from my SQL-server, using an API, php file basically and using the requests module in Python. So here's my code for that:

# importing the requests library
import requests

# api-endpoint
URL = "https://******.co.in/******/queryRandom.php"

# location given here
token= '***************'

query= 'SELECT userId,createdAt,.... FROM xyz WHERE date >= CURDATE() - INTERVAL 60 DAY'

# defining a params dict for the parameters to be sent to the API
PARAMS = {'token':token, 'query':query}

# sending get request and saving the response as response object
r = requests.post(url = URL, data = PARAMS)
df=pd.DataFrame(r.json())
df.head()

Now after I'm done with my work, I formed a dataframe like this:

Sl.No.  date    nameofthepic    
1   26-08-2018  10-must-see-waterfalls-in-karnataka-8081714182d8    
2   26-08-2018  a-month-in-backpack-one-girl-one-goal-to-explore-the-himalayas-22813ee67e15 
3   26-08-2018  a-month-in-backpack-one-girl-one-goal-to-explore-the-himalayas-week-1-8d113ee673db  
4   26-08-2018  a-month-in-backpack-one-girl-one-goal-to-explore-the-himalayas-week-1-22b13ee67b87  
5   26-08-2018  backpacking-for-a-month-in-the-himalayas-week-1-30813ee674a6

Now using the similar requests module, is there a way I can write something like this: INSERT INTO table_name... and upload the dataframe into the SQL table.

The main problem I'm not able to figure out is:

i) How do I upload the dataframe column values into the table in one go?

ii) If its not possible through requests module, is there any other way I can upload the Pandas dataframe values into SQL-server table directly from jupyter-notebook using Python code?

vishalk · Accepted Answer · 2018-08-30 07:21:42Z

1

To import this dataframe into a MySQL table:

# Import dataframe into MySQL
import sqlalchemy
database_username = 'ENTER USERNAME'
database_password = 'ENTER USERNAME PASSWORD'
database_ip       = 'ENTER DATABASE IP'
database_name     = 'ENTER DATABASE NAME'
database_connection = sqlalchemy.create_engine('mysql+mysqlconnector://{0}:{1}@{2}/{3}'.format(database_username, database_password, database_ip, database_name))
frame.to_sql(con=database_connection, name='table_name_for_df', if_exists='append')

edited Aug 30, 2018 at 7:21

answered Aug 30, 2018 at 5:03

vishalk

1822 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Debadri Dutta Over a year ago

but this will replace the table everytime i run the code right?! I want it to append everyday's data

vishalk Over a year ago

if_exists='append' will append everyday data to mysql database

Collectives™ on Stack Overflow

INSERT Pandas Dataframe to SQL-Server using Python - Jupyter Notebook

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related