Skip blank lines while reading .csv file using opencsv (java)

Question

Good day everyone! My target is to make csv reader to skip the blank lines while parsing a file, do nothing basically, only get me the rows with at least one value. At the moment I have two methods -> 1st is just reading all rows as List of Strings array and returns it, 2nd converts the result into List of Lists of Strings, both are bellow:

private List<String[]> readCSVFile(File filename) throws IOException {

    CSVReader reader = new CSVReader(new FileReader(filename));
    List<String[]> allRows = reader.readAll();

    return allRows;

}

public List<List<String>> readFile(File filename) throws IOException {

        List<String[]> allRows = readCSVFile(filename);     
        List<List<String>> allRowsAsLists = new ArrayList<List<String>>();      
        for (String[] rowItemsArray :  allRows) {
            List<String> rowItems = new ArrayList<String>();
            rowItems.addAll(Arrays.asList(rowItemsArray));
            allRowsAsLists.add(rowItems);

        }
    return allRowsAsLists;

}

My first thought was to check (in the 2'nd method) the length of an array if its 0 just to ignore it - which would be something like this:

for (String[] rowItemsArray :  allRows) {
            **if(rowItemArray.length == 0) continue;**
            List<String> rowItems = new ArrayList<String>();
            rowItems.addAll(Arrays.asList(rowItemsArray));
            allRowsAsLists.add(rowItems);

}

Unfortunately that didn't work for the reason that even if the row is blank it still returns an array of elements - empty Strings in fact. Checking an individual String is not an option as there are 100+ columns and this is variable. Please suggest what’s the best way to achieve this. Thanks.

Sorted it out this way:

    public List<List<String>> readFile(File filename) throws IOException {

            List<String[]> allRows = readCSVFile(filename, includeHeaders, trimWhitespacesInFieldValues);       
            List<List<String>> allRowsAsLists = new ArrayList<List<String>>();      
            for (String[] rowItemsArray :  allRows) {
                **if(allValuesInRowAreEmpty(rowItemsArray)) continue;**
                List<String> rowItems = new ArrayList<String>();
                rowItems.addAll(Arrays.asList(rowItemsArray));
                allRowsAsLists.add(rowItems);

            }
            return allRowsAsLists;

        }

    private boolean allValuesInRowAreEmpty(String[] row) {
        boolean returnValue = true;
        for (String s : row) {
            if (s.length() != 0) {
                returnValue = false;
            }
        }
        return returnValue;
    }

You should avoid reading the empty lines at all. That means, the returned List of String arrays should not contain empty entries. Therefore, update your readCSVFile method. — Tom
– Tom, Commented Jan 22, 2015 at 12:13

SubOptimal · Accepted Answer · 2015-01-22 12:12:09Z

9

You could check the length and the first element. If the line contains only a field separator then the lenght > 1. If the line contains a single space character then the first element is not empty.

if (rowItemsArray.length == 1 && rowItemsArray[0].isEmpty()) {
    continue;
}

answered Jan 22, 2015 at 12:12

SubOptimal

23.1k3 gold badges57 silver badges70 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Martin · Accepted Answer · 2020-02-03 08:19:32Z

8

For opencsv 5.0 there is an API-option to read CSV lines directly into a Bean.

For people who prefer using the "CsvToBean" feature, the following solution is using the (sadly deprecated) #withFilter(..) method on CsvToBeanBuilder to skip blank lines in the Inputstream:

InputStream inputStream; // provided
List<MyBean> data = new CsvToBeanBuilder(new BufferedReader(new InputStreamReader(inputStream)))
    .withType(MyBean.class)
    .withFilter(new CsvToBeanFilter() {
        /*
         * This filter ignores empty lines from the input
         */
        @Override
        public boolean allowLine(String[] strings) {
            for (String one : strings) {
                if (one != null && one.length() > 0) {
                    return true;
                }
            }
            return false;
        }
    }).build().parse();

Update: With opencsv Release 5.1 (dated 2/2/2020), CsvToBeanFilter got undeprecated as per feature request #120.

edited Feb 3, 2020 at 8:19

answered Jan 7, 2020 at 16:11

Martin

1451 silver badge6 bronze badges

3 Comments

logbasex Over a year ago

What would you do with opencsv ver 5.1 @Martin

Martin Over a year ago

@logbasex CsvToBeanFilter was only marked as deprecated in 5.0 and that deprecation mark was removed in 5.1 again. You can use the above code with opencsv 5.0 and with opencsv 5.1

trebor Over a year ago

As of opencsv 5.1, there is a withIgnoreEmptyLines method on CsvToBeanBuilder.

Tomerikoo · Accepted Answer · 2021-10-14 11:59:13Z

2

Here is an updated solution with lambdas based on @Martin's solution:

InputStream inputStream; // provided
List<MyBean> data = new CsvToBeanBuilder(new BufferedReader(new InputStreamReader(inputStream)))
    .withType(MyBean.class)
    // This filter ignores empty lines from the input
    .withFilter(stringValues -> Arrays.stream(stringValues)
        .anyMatch(value -> value != null && value.length() > 0))
    .build()
    .parse();

edited Oct 14, 2021 at 11:59

Tomerikoo

19.6k16 gold badges57 silver badges68 bronze badges

answered Oct 14, 2021 at 7:44

Patrik

978 bronze badges

Comments

i.karayel · Accepted Answer · 2022-02-24 17:42:33Z

1

You can use a filter with lambda: like below:

CsvToBean<T> csvToBean = new CsvToBeanBuilder<T>(new StringReader(CSV_HEADER + "\n" + lines))
    .withType(clazz)
    .withFieldAsNull(CSVReaderNullFieldIndicator.EMPTY_SEPARATORS)
    .withSeparator(delimiter)
    .withSkipLines(skipLines)
    .withIgnoreLeadingWhiteSpace(true).withFilter(strings -> {
      for (String r : strings) {
        if (r != null && r.length() > 0) {
          return true;
        }
      }
      return false;
    }).build();

Your lambda filter:

.withFilter(strings -> {
      for (String r : strings) {
        if (r != null && r.length() > 0) {
          return true;
        }
      }
      return false;
    })

answered Feb 24, 2022 at 17:42

i.karayel

4,9452 gold badges25 silver badges29 bronze badges

Comments

chris · Accepted Answer · 2015-01-22 12:23:08Z

0

You could summarize all string values per row after trimming them. If the resulting string is empty, there are no values in any cell. In that case ignore the line.
Something like this:

private boolean onlyEmptyCells(ArrayList<String> check) {
    StringBuilder sb = new StringBuilder();
    for (String s : check) {
        sb.append(s.trim());
    }
    return sb.toString().isEmpty(); //<- ignore 'check' if this returns true
}

answered Jan 22, 2015 at 12:23

chris

1,8153 gold badges19 silver badges30 bronze badges

Comments

JiP · Accepted Answer · 2021-10-29 11:51:43Z

If you do not parse into a Bean, you can use Java Streams API to help you with filtering of invalid CSV rows. My approach is like this (where is is java.io.InputStream instance with CSV data and YourBean map(String[] row) is your mapping method that maps a CSV row to a your Java object:

CSVParser csvp = new CSVParserBuilder()
    .withSeparator(';')
    .withFieldAsNull(CSVReaderNullFieldIndicator.BOTH)
    .build();
CSVReader csvr = new CSVReaderBuilder(new InputStreamReader(is))
    .withCSVParser(csvp)
    .build();
List<YourBean> result = StreamSupport.stream(csvr.spliterator(), false)
    .filter(Objects::nonNull)
    .filter(row -> row.length > 0)
    .map(row -> map(row))
    .collect(Collectors.toList());

theINtoy · Accepted Answer · 2022-01-12 08:59:01Z

The JavaDoc for CsvToBeanFilter states "Here's an example showing how to use CsvToBean that removes empty lines. Since the parser returns an array with a single empty string for a blank line that is what it is checking." and lists an example of how to do this:

private class EmptyLineFilter implements CsvToBeanFilter {

    private final MappingStrategy strategy;

    public EmptyLineFilter(MappingStrategy strategy) {
        this.strategy = strategy;
    }

    public boolean allowLine(String[] line) {
        boolean blankLine = line.length == 1 && line[0].isEmpty();
        return !blankLine;
    }

 }

 public List<Feature> parseCsv(InputStreamReader streamReader) {
    HeaderColumnNameTranslateMappingStrategy<Feature> strategy = new HeaderColumnNameTranslateMappingStrategy();
    Map<String, String> columnMap = new HashMap();
    columnMap.put("FEATURE_NAME", "name");
    columnMap.put("STATE", "state");
    strategy.setColumnMapping(columnMap);
    strategy.setType(Feature.class);
    CSVReader reader = new CSVReader(streamReader);
    CsvToBeanFilter filter = new EmptyLineFilter(strategy);
    return new CsvToBean().parse(strategy, reader, filter);
 }

Muchtar · Accepted Answer · 2023-02-24 11:42:51Z

Another way to do it, is to override the allowLine() method of the Class CsvToBeanFilter and pass it to .withFilter() when building the bean (example in Kotlin, should also work for Java):

import com.opencsv.bean.CsvToBeanFilter
import com.opencsv.bean.CsvToBeanBuilder
import java.io.FileReader

class NonBlankLinesFilter : CsvToBeanFilter {
    override fun allowLine(line: Array<String?>?): Boolean {
        if (line == null) {
            return false
        }
        for (value in line) {
            if (!value.isNullOrBlank()) {
                return true
        }
    }
    return false
  }
}

fun main() {
  val reader = FileReader("path/to/csv/file.csv")

  val csvToBean = CsvToBeanBuilder<MyBean>(reader)
        .withType(MyBean::class.java)
        .withSeparator(',')
        .withFilter(NonBlankLinesFilter())
        .build()

  val uploadList = csvToBean.parse()

  println(uploadList)
}

Collectives™ on Stack Overflow

Skip blank lines while reading .csv file using opencsv (java)

8 Answers 8

Comments

3 Comments

Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

8 Answers 8

Comments

3 Comments

Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related