How to sequentially create and save csv files using Pandas dataframe in python

Question

I have this simple code which saves a pandas dataframe to a csv file. As of now, it works by overwriting the filename so each time I run it it just replaces the old file with a new one with the same name. Is it possible to save this dataframe but have it create new files sequentially, ie if there is already some file called "filename1" in the directory make the new one called "filename2" so the data from the original file is not just lost?

import pandas
datamatrix= [[1,2,3],[1,2,3],[1,2,3]])
x=pandas.DataFrame(datamatrix)  
pandas.DataFrame.to_csv(x,"filename.csv",',')

Have a look at this question asked here already stackoverflow.com/questions/13852700/… — Raja Sattiraju
– Raja Sattiraju, Commented Jun 15, 2018 at 20:07
This is not a pandas specific question. You want to read the files in folder before save a new one — rpanai
– rpanai, Commented Jun 15, 2018 at 20:08
Possible duplicate of python: Create file but if name exists add number — rpanai
– rpanai, Commented Jun 15, 2018 at 20:09
You can explicitly check if a file exists before writing by using os.path.exists(filename), but enumerating it into infinity may not really what you want, it may be a better idea to add a timestamp in one way or another to it that conflicts only once. Something like filename_201806150311.csv — d parolin
– d parolin, Commented Jun 15, 2018 at 20:12

RK1 · Accepted Answer · 2018-06-15 20:22:08Z

2

I would probably save it with a different timestamp, unless there is a specific reason to have numerical numbering.

Using time stamp

import pandas
import time
datamatrix= [[1,2,3],[1,2,3],[1,2,3]]
x=pandas.DataFrame(datamatrix)  
pandas.DataFrame.to_csv(x,"filename_" + time.strftime('%Y-%m-%d %H-%S') + ".csv",',')

You could also just do a unix time stamp

pandas.DataFrame.to_csv(x,"filename_" + str(int(time.time())) + ".csv",',')

How to get current time in python and break up into year, month, day, hour, minute?

What is the easiest way to get current GMT time in Unix timestamp format?

answered Jun 15, 2018 at 20:22

RK1

2,5702 gold badges22 silver badges40 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

rahlf23 · Accepted Answer · 2018-06-15 20:10:06Z

0

As some of the comments have mentioned, this is not a question that has a pandas-specific answer. Simply use os.listdir(os.getcwd()) to list all of the files in your current working directory that you want to save the file to. If the filename that you are trying to save as already exists in the list returned from the above command, then increment your version number by 1, or save as a different filename entirely, up to you.

answered Jun 15, 2018 at 20:10

rahlf23

9,0494 gold badges30 silver badges57 bronze badges

Comments

vampirekabir · Accepted Answer · 2021-06-09 15:19:50Z

0

by using time.strftime and combinding f-string and raw string literals you can attach timestamp to your desired format

import pandas
import time
timestr = time.strftime("%Y%m%d-%H%M%S")
datamatrix= [[1,2,3],[1,2,3],[1,2,3]]
x=pandas.DataFrame(datamatrix)  
pandas.DataFrame.to_csv(fr'D:\filename_{timestr}.csv',encoding="utf-8", index=False, header=True)

answered Jun 9, 2021 at 15:19

vampirekabir

9912 bronze badges

Collectives™ on Stack Overflow

How to sequentially create and save csv files using Pandas dataframe in python

3 Answers 3

Using time stamp

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Using time stamp

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related