Python csv.reader vs csv.dictreader differences?

Question

Have two scripts that give very different results. First using csv.reader and it works fine, then csv.dictreader which has given me problems. Both scripts use the same set of data, only difference between the two files is that the one dictreader uses has headers.

import csv

inv = csv.reader(open('inv.txt', 'rU'), dialect='excel', delimiter="\t")

for PART_CODE,MODEL_NUMBER,PRODUCT_NAME,COLOR,TOTAL_ONHAND,TOTAL_ON_ORDER,TOTAL_SALES,SALES_YEAR_TO_DATE,SALES_LASTYEAR_TO_DATE,\
TOTAL_NUMBER_OF_QTYsSOLD,TOTAL_PURCHASES,PURCHASES_YEAR_TO_DATE,PURCHASES_LASTYEAR_TO_DATE,TOTAL_NUMBER_OF_QTYpurchased,DATE_LAST_SOLD,DATE_FIRST_SOLD in inv:
    if int(TOTAL_ON_ORDER) >= 1:
        print ('%-20s %-100s OnHand: %-4s OnOrder: %-4s') % (MODEL_NUMBER,PRODUCT_NAME,TOTAL_ONHAND,TOTAL_ON_ORDER)

The above works just fine, it'll parse through the 20,000 plus items with no error. Now if I choose to use dictreader as in the below, the script will run into problems after a while...

import csv

inv = csv.DictReader(open('ireport.txt', 'rU'), dialect='excel', delimiter="\t")

for row in inv:
    if int(row['TOTAL_ON_ORDER']) >= 1:
        print ('%-20s %-100s OnHand: %-4s OnOrder: %-4s') % (row['MODEL_NUMBER'],row['PRODUCT_NAME'],row['TOTAL_ONHAND'],row['TOTAL_ON_ORDER'])

Prints out about 100 or so, then fails and reports this error:

if int(row['TOTAL_ON_ORDER']) >= 1:

ValueError: invalid literal for int() with base 10: 'False'

Puzzles me as both scripts use the same data (other than the reader one has no header row and the dictreader one does) one works flawlessly, the other complains. Any clues?

Snippet of inv.txt:

61965901576 383964  Sandisk 128MB 3.3V Smartmedia Card      0   0   0   0   0   0   0   0   0   0   00/00/00    00/00/00
61965901521 348236  Sandisk 128MB Compactflash Card     0   0   54.26   0   0   1   0   0   0   0   01/09/02    01/09/02
61965902011 SDCZ2-1024-A10  Sandisk 1GB Cruzer Mini USB Flash Drive     0   0   0   0   0   0   0   0   0   0   00/00/00    00/00/00
61965901571 266796  Sandisk 256MB CompactFlash Disk     0   0   678.22  0   0   5   0   0   0   0   06/27/02    03/08/02

What happens if you change (in your DictReader code) dialect='excel-tab' to dialect='excel'? — unutbu
– unutbu, Commented Nov 24, 2012 at 13:22
Then it would break as ireport.txt is inv.txt, but has a header row. — user1764417
– user1764417, Commented Nov 24, 2012 at 13:30
Are there False values in the csv? Could TOTAL_ON_ORDER somehow be associated with the wrong column? — unutbu
– unutbu, Commented Nov 24, 2012 at 13:34
No, but it indicates either badly formed input, or your dialect isn't quite right. Can you post an actual sample via pastebin that can reproduce the error with your code? — Mark Tolonen
– Mark Tolonen, Commented Nov 24, 2012 at 14:20

Dan Steingart · Accepted Answer · 2012-11-24 14:33:40Z

3

It looks like it's reading something as a Boolean into an integer (which it's not happy with) in the DictReader function, whereas in the reader function it is not getting that cast as such.

Try this:

import csv

inv = csv.DictReader(open('ireport.txt', 'rU'), dialect='excel', delimiter="\t")

for row in inv:
    try:
      if int(row['TOTAL_ON_ORDER']) >= 1:
          print ('%-20s %-100s OnHand: %-4s OnOrder: %-4s') % (row['MODEL_NUMBER'],row['PRODUCT_NAME'],row['TOTAL_ONHAND'],row['TOTAL_ON_ORDER'])
    except Exception as Err:
      print row['TOTAL_ON_ORDER'],Err
      break #if you want to end the function)

This will show you what line it's choking on, and if you remove the break should chug through it.

Good luck!

answered Nov 24, 2012 at 14:33

Dan Steingart

7752 gold badges7 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user1764417 Over a year ago

Thank you, the error report I was getting from python wasn't accurate enough, apparently there was something in there causing it to set to None, so I added an or row['DATE_LAST_SOLD'] == None and set it to 0 and all was well.

Collectives™ on Stack Overflow

Python csv.reader vs csv.dictreader differences?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related