Unicode Error when importing a csv file in python?

Question

Hi I am using following code to remove duplicate records from my csv file:

inFile = open('I:\SIT\Monthly\LatestMonthly\source\Network1.csv','r')
outFile = open('I:\SIT\Monthly\LatestMonthly\source\Network2.csv','w')

listLines = []

for line in inFile:

    if line in listLines:
        continue

    else:
        outFile.write(line)
        listLines.append(line)

outFile.close()
inFile.close()

When I run the script I am getting an error:

unicodeescape' codec can't decode bytes in position 35-36: malformed \N character escape.

Why do I get this error?

I advice to add some control code to show what line is the problem occurring, in order to inspect the right line of the csv file - at the bare minimum: for (lineNo,line) in enumerate(inFile): print(lineNo) [...] — Pierre Prinetti
– Pierre Prinetti, Commented Jun 17, 2015 at 7:29
Is your csv fle in ASCII format.? The error message suggest unicode, which means more than one byte per character is possible. You could first convert your input files to ASCII, or use codecs to decode them with Python. — Kim Ryan
– Kim Ryan, Commented Jun 17, 2015 at 7:33

nneonneo · Accepted Answer · 2015-06-17 08:07:18Z

2

Your error occurs before you even open the file!

You are not escaping the backslashes in your filename

'I:\SIT\Monthly\LatestMonthly\source\Network1.csv'

and thus \N is interpreted as a Unicode escape character (\N inserts a Unicode character by name, e.g. '\N{MUSICAL SYMBOL G CLEF}')

You can try using a raw literal here:

r'I:\SIT\Monthly\LatestMonthly\source\Network1.csv'

The prefix r tells Python to treat all backslashes as literal backslashes. Alternatively, you can remember to always escape your backslashes:

'I:\\SIT\\Monthly\\LatestMonthly\\source\\Network1.csv'

or finally, you can use forward-slashes:

'I:/SIT/Monthly/LatestMonthly/source/Network1.csv'

answered Jun 17, 2015 at 8:07

nneonneo

181k37 gold badges331 silver badges412 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user3514648 Over a year ago

Thanks nnenonneo...I tried all these but still itz not working

nneonneo Over a year ago

Well, keep in mind you have to apply this to both of your paths. If you still have problems you may consider adding your new code to your question.

Collectives™ on Stack Overflow

Unicode Error when importing a csv file in python?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related