append is slower with integer than string in python?

Question

I come across this code while looking for a way to measure functions speed.

def test():
    """Stupid test function"""
    L = []
    for i in range(100):
        L.append(i)

if __name__ == '__main__':
     import timeit
     print(timeit.timeit("test()", setup="from __main__ import test"))

when I run it, it takes about 10 seconds, which is very strange for me. I recall that when I am appending 100.000 string into a string, it only takes 0.06 seconds. why does appending 100 integers into a list take 10 seconds? here are the code that I use to append stringt to list.

def wordlist1():
    fin = open("words.txt")
    word_list = []
    for line in fin:
        word = line.strip()
        word_list.append(word)
    return word_list

You are also using range() for your integers, which might be slower than for line in fin — Flob
– Flob, Commented Mar 19, 2019 at 14:22
Based on your title you should be comparing test with another function the same apart rom L.append('a')- you're comparing 2 completely different things — Chris_Rands
– Chris_Rands, Commented Mar 19, 2019 at 15:08
@Flob range in Python 3 doesn't create a list of integers in memory; it's a range object with a __next__ method just like a file iterator. — chepner
– chepner, Commented Mar 19, 2019 at 15:08
@chepner thank you for clarifying that! :-) i don't know that much about python, it was just an idea^^ — Flob
– Flob, Commented Mar 19, 2019 at 16:38

DocDriven · Accepted Answer · 2019-03-19 14:59:40Z

1

I think your problem is the usage of the timeit module. When not specified, the param number defaults to 1e6. So what you are effectively measuring is how long it would take to append a hundred numbers to an array 1e6 times.

To verify this, I used this code snippet:

import timeit

alist = list(range(100))

L = []

def test():
    for i in alist:
        L.append(i)

elapsed = timeit.timeit("test()", setup="from __main__ import test, alist, L", number=1000000)

print('average time elapsed', elapsed/1000000)

On my machine, I got the following result:

7.74517...e-06

I do not know how you measured the append function for your txt file, but you could try to test it the same way.

answered Mar 19, 2019 at 14:59

DocDriven

4,0447 gold badges29 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

chepner · Accepted Answer · 2019-03-19 16:43:54Z

1

If you use iPython, you can see a clearer explanation of the results using its %timeit magic command.

In [1]: def test():
   ...:     """Stupid test function"""
   ...:     L = []
   ...:     for i in range(100):
   ...:         L.append(i)
   ...:

In [2]: %timeit test()
7.12 µs ± 22.9 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

Here the result is an average of 7.12 microseconds per call to test, with the average computed over 7 sets of 100,000 calls to test. This is consistent with the 7.75 microseconds reported by DocDriven in his answer.

answered Mar 19, 2019 at 16:43

chepner

538k77 gold badges594 silver badges746 bronze badges

Comments

elvis · Accepted Answer · 2019-03-19 14:26:27Z

-1

Try this one

def test():

L = []

for i in range(100)

    count + = i

    L.append(count )

    return count

answered Mar 19, 2019 at 14:26

elvis

93 bronze badges

1 Comment

Rory Daulton Over a year ago

Welcome to StackOverflow. But how is this an answer to the question? Also, please format your code correctly clicking the edit link, highlighting the code, then clicking the {} button (or press Ctrl+K).

Collectives™ on Stack Overflow

append is slower with integer than string in python?

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related