EC2 instance for Python 3 script

Question

I have a script written in Python and it's taking a fairly long time to process (around 30 minutes) on my laptop. I was thinking I could create a EC2 instance in AWS and try to see if it's possible to speed up the process. I have a AWS account so my question:

Which EC2 instance type should I create in order to run the process in a faster way? Process reads a csv file does some calculations and then writes csv with results. Script bottleneck is in the mathematical calculations as csv files are fairly small.

I can go with either a free tier or paid tier instance.

Ajinkya · Accepted Answer · 2019-07-16 17:07:13Z

2

I would say go with p2 extra large If you have to use ec2.

Try to understand what's causing this delay. Which library are you using to read csv. There are various ways in python through which you can manipulate the csv file For example see the image:

Image source here

NumPy, SciPy, joblib and hdf5 are the recommended options for quickly saving and loading csv data.

Try to change your algorithm. In my experience pandas is not speedy when it comes to csv operations. Try to tweak your code if that dosent work switch to p2 extra large

answered Jul 16, 2019 at 17:07

Ajinkya

1,8853 gold badges28 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

EC2 instance for Python 3 script

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related