Delete first rows of csv file in python

Question

We want to delete the first 34 rows of our csv file, as it is useless text headers, which is to no use. We are trying to do so by running the following lines of code in Python 3:

with open("test.csv",'r') as f, open("temp.csv",'w') as f1:
    next(f) # skip header line
    for line in f:
        f1.write(line)

Above should only 'delete' first line, but we imagine we could make a for loop with range(0, 35) around next(f). Although that doesn't seem like the "pythonical" way of solving this.

Our data is in test.csv, and we have an empty csv file called temp.csv. The code described above is supposed to skip first line of test.csv, and then copy the rest into temp.csv.

Unfortunately we receive this error:

Traceback (most recent call last):
  File "delete.py", line 2, in <module>
    next(f) # skip header line
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf8 in position 2625: invalid start byte

Why does this occur? And what is the pythonical way of deleting the first 34 rows in a csv file?

try with rb and wb options for both open() functions. r option try to decode the text from your file using the default encoding (utf-8) — Corentin Limier
– Corentin Limier, Commented Jan 29, 2020 at 16:02
Does this answer your question? UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c — AMC
– AMC, Commented Jan 29, 2020 at 18:27

A l w a y s S u n n y · Accepted Answer · 2025-05-07 14:09:05Z

11

I know you want to skip rows using the with context manager when opening and reading a CSV file, but I would suggest you use an excellent library called pandas to read and skip rows from the CSV file like below, also you can save that to another CSV file from the df data frame very easily

without pandas,

import csv

with open('my_csv_file.csv', 'r') as file:
    # Skip rows by advancing file pointer (very memory efficient)
    for _ in range(34):
        next(file)
    
    # Only now start parsing as CSV
    csv_reader = csv.reader(file)
    # Process one row at a time instead of loading all to memory
    for row in csv_reader:
        # Process row here
        pass

with pandas,

import pandas as pd
# skiprows=34 will skip the first 34 lines and try to read from 35 line
df = pd.read_csv('my_csv_file.csv', skiprows=34)
# print the data frame
df

edited May 7 at 14:09

answered Jan 29, 2020 at 16:08

A l w a y s S u n n y

38.8k9 gold badges68 silver badges120 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Buster3650 Over a year ago

This works perfectly! We wanted to clean our csv file, so we could use it in another program we have. We had no idea, we could easily just use 'skiprows' instead. Thanks a lot

AMC Over a year ago

Isn’t Pandas overkill for this?

Dan Over a year ago

I wouldn't install pandas just for this.

A l w a y s S u n n y May 7 at 14:09

ok updated my answer without using pandas.

Collectives™ on Stack Overflow

Delete first rows of csv file in python

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related