How to write bean to CSV using OpenCSV without converting to String[]?

Question

I have a large array of a bean that needs to be written to a CSV file. I am trying to use OpenCSV but all the examples I see on the web ask me to convert the bean (POJO) that I have to a string array. I don't want to do this since this would mean I will have to go through the million beans and convert each of those beans to String[] which would be a very memory-expensive process.

This is the piece of code I have right now for doing this:

private static void writeRowsToCsv(Path filePath, List<MyBean> rows)
        throws IOException {

    StringWriter writer = new StringWriter();
    CSVWriter csvWriter = new CSVWriter(writer, '#', '\'');
    ColumnPositionMappingStrategy mappingStrategy =
            new ColumnPositionMappingStrategy();
    mappingStrategy.setType(MyBean.class);
    List<String[]> rowsInStringArray = convertBeanToStringArray(rows)
    csvWriter.writeAll(rowsInStringArray);
}

Is there a way I can avoid the conversion to String[] and use the list of beans that I have to write to a CSV file?

Why would that be very memory-expensive? You call writeNext() in a loop, converting one bean at a time. — Andreas
– Andreas, Commented Apr 10, 2017 at 14:51
That would mean I am writing one row at a time. I have a millions of rows to be written, which would make it extremely slow. — jobin
– jobin, Commented Apr 10, 2017 at 14:52
What do you think writeAll() does? It iterates the list and writes them, one at a time. How else would it process the list? — Andreas
– Andreas, Commented Apr 10, 2017 at 14:52
I haven't investigated but I believe it writes a batch of rows at a time. — jobin
– jobin, Commented Apr 10, 2017 at 15:04
What batching? There is no batching. Stop trying to guess when you have no clue. writeAll() is nothing but a loop calling writeNext(). See source code. — Andreas
– Andreas, Commented Apr 10, 2017 at 15:09

Garis M Suero · Accepted Answer · 2017-10-31 07:22:01Z

7

Jobin, seriously, if you are using OpenCSV then use the BeanToCsv or the newer StatefulBeanToCsv. It will save your sanity.

private static void writeRowsToCsv(Path filePath, List<MyBean> rows)
        throws IOException {

    StringWriter writer = new StringWriter();
    ColumnPositionMappingStrategy mappingStrategy =
            new ColumnPositionMappingStrategy();
    mappingStrategy.setType(MyBean.class);

    StatefulBeanToCsvBuilder<MyBean> builder = new StatefulBeanToCsvBuilder(writer);
    StatefulBeanToCsv beanWriter = builder
              .withMappingStrategy(mappingStrategy)
              .withSeparator('#')
              .withQuotechar('\'')
              .build();

    beanWriter.write(rows);
}

edited Oct 31, 2017 at 7:22

Garis M Suero

8,1997 gold badges48 silver badges68 bronze badges

answered May 8, 2017 at 20:31

Scott Conway

1,0838 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

ledlogic Over a year ago

The constructor StatefulBeanToCsvBuilder(CSVWriter) does not exist.

Scott Conway Over a year ago

ledlogic - you are absolutely correct. I corrected my code example above as the StatefulBeanToCsv creates a CSVWriter based on values passed in from the builder but it does not take a CSVWriter directly. Not exactly sure what I was thinking back in May but thank you for pointing that out because who knows how long that would have stayed there.

Bharath Over a year ago

this is the easier way to do it.

pyb Over a year ago

In OpenCSV 5.7.0, there is a constructor StatefulBeanToCsvBuilder(ICSVWriter) and CSVWriter implements it via AbstractCSVWriter.

djna Dec 7, 2024 at 11:12

why are the filePath and rows parameters not referenced in this method?

Andreas · Accepted Answer · 2017-04-10 15:18:43Z

1

You call writeNext() in a loop, converting one bean at a time. Not memory-expensive at all.

answered Apr 10, 2017 at 15:18

Andreas

160k13 gold badges164 silver badges262 bronze badges

Collectives™ on Stack Overflow

How to write bean to CSV using OpenCSV without converting to String[]?

2 Answers 2

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related