24

Is there any way InputStream wrapping a list of UTF-8 String? I'd like to do something like:

InputStream in = new XyzInputStream( List<String> lines )
4
  • Can you share the situation, why you want this? Commented Mar 23, 2012 at 10:41
  • 2
    Note that InputStream deals with binary data. Strings are text data. Which encoding are you interested in? Commented Mar 23, 2012 at 10:43
  • You're right I forgot to precise : UTF-8 only. Commented Mar 23, 2012 at 10:47
  • How did you finally accomplish creating an InputStream from List<String> Marc? I have to do the same thing. Any Input will help. Commented Sep 25, 2014 at 19:05

9 Answers 9

18

You can read from a ByteArrayOutputStream and you can create your source byte[] array using a ByteArrayInputStream.

So create the array as follows:

 List<String> source = new ArrayList<String>();
 source.add("one");
 source.add("two");
 source.add("three");
 ByteArrayOutputStream baos = new ByteArrayOutputStream();

 for (String line : source) {
   baos.write(line.getBytes());
 }

 byte[] bytes = baos.toByteArray();

And reading from it is as simple as:

 InputStream in = new ByteArrayInputStream(bytes);

Alternatively, depending on what you're trying to do, a StringReader might be better.

Sign up to request clarification or add additional context in comments.

2 Comments

This is the best way of doing it if it's a one-off. If you have to reuse instances of the InputStream all over the place though, you could always implement the InputStream interface, and do all of the above under the hood.
Good example, but it would be nice to encourage best practice by specifying a charset during getBytes().
5

You can concatenate all the lines together to create a String then convert it to a byte array using String#getBytes and pass it into ByteArrayInputStream. However this is not the most efficient way of doing it.

3 Comments

You can implement a InputStream yourself and convert each String to a byte array as you go along and wrap it in a ByteArrayInputStream and forward the calls to the ByteArrayInputStream. Though, seeing as you will have to have logic dealing with splitting reads across Strings it's probably just as easy to do all the logic yourself and not use ByteArrayInputStream at all.
@Marc avoiding this copy is not necessary a good idea. It allows you to process all the data at once. If the strings are small this can save you a lot of object allocations. It can be faster and in some cases even consume less memory. Whatever you do, benchmark it on sample data and make sure it really is more efficient.
@Marc You can avoid the copy, but not without writing a significant amount of code. You would have to implement InputStream yourself, and somehow adapt all of its methods to working over a List of Strings, which will be an enormous pain to do, and won't buy you that much in terms of performance or reduced memory footprint.
5

In short, no, there is no way of doing this using existing JDK classes. You could, however, implement your own InputStream that read from a List of Strings.

EDIT: Dave Web has an answer above, which I think is the way to go. If you need a reusable class, then something like this might do:


public class StringsInputStream<T extends Iterable<String>> extends InputStream {

   private ByteArrayInputStream bais = null;

   public StringsInputStream(final T strings) throws IOException {
      ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
      for (String line : strings) {
         outputStream.write(line.getBytes());
      }
      bais = new ByteArrayInputStream(outputStream.toByteArray());
   }

   @Override
   public int read() throws IOException {
      return bais.read();
   }

   @Override
   public int read(byte[] b) throws IOException {
      return bais.read(b);
   }

   @Override
   public int read(byte[] b, int off, int len) throws IOException {
      return bais.read(b, off, len);
   }

   @Override
   public long skip(long n) throws IOException {
      return bais.skip(n);
   }

   @Override
   public int available() throws IOException {
      return bais.available();
   }

   @Override
   public void close() throws IOException {
      bais.close();
   }

   @Override
   public synchronized void mark(int readlimit) {
      bais.mark(readlimit);
   }

   @Override
   public synchronized void reset() throws IOException {
      bais.reset();
   }

   @Override
   public boolean markSupported() {
      return bais.markSupported();
   }

   public static void main(String[] args) throws Exception {
      List source = new ArrayList();
      source.add("foo ");
      source.add("bar ");
      source.add("baz");

      StringsInputStream<List<String>> in = new StringsInputStream<List<String>>(source);

      int read = in.read();
      while (read != -1) {
         System.out.print((char) read);
         read = in.read();
      }
   }
}

This basically an adapter for ByteArrayInputStream.

Comments

5

You can create some kind of IterableInputStream

public class IterableInputStream<T> extends InputStream {

    public static final int EOF = -1;

    private static final InputStream EOF_IS = new InputStream() {
        @Override public int read() throws IOException {
            return EOF;
        }
    };

    private final Iterator<T> iterator;
    private final Function<T, byte[]> mapper;

    private InputStream current;

    public IterableInputStream(Iterable<T> iterable, Function<T, byte[]> mapper) {
        this.iterator = iterable.iterator();
        this.mapper = mapper;
        next();
    }

    @Override
    public int read() throws IOException {
        int n = current.read();
        while (n == EOF && current != EOF_IS) {
            next();
            n = current.read();
        }
        return n;
    }

    private void next() {
        current = iterator.hasNext() 
            ? new ByteArrayInputStream(mapper.apply(iterator.next())) 
            : EOF_IS;
    }
}

To use it

public static void main(String[] args) throws IOException {
    Iterable<String> strings = Arrays.asList("1", "22", "333", "4444");
    try (InputStream is = new IterableInputStream<String>(strings, String::getBytes)) {
        for (int b = is.read(); b != -1; b = is.read()) {
            System.out.print((char) b);
        }
    }
}    

3 Comments

That's actually a great idea. In my case I can't operate on whole list or else I'd get OutOfMemory.
Top solution, the only one that avoid memory problems with big data.
Only drawback is that it is slow. See my solution stackoverflow.com/a/79747881/7251133 where I also implement read(byte[], int, int) which performs much better for a few scenarios.
3

In my case I had to convert a list of string in the equivalent file (with a line feed for each line).

This was my solution:

List<String> inputList = Arrays.asList("line1", "line2", "line3");

byte[] bytes = inputList.stream().collect(Collectors.joining("\n", "", "\n")).getBytes();

InputStream inputStream = new ByteArrayInputStream(bytes);

6 Comments

Thanks for sharing. Awesome solution! Worked perfectly for me.
Thanks, nice solution. But because the Collectors.joining does not add a final line feed I switched to a enhanced-loop solution (so last line in file also ends with a line feed).
@StephanvanHoof for the final line feed just try Collectors.joining("\n", "", "\n"). BTW I've just updated my answer :)
I copied this solution, but as I needed a EOF character for different platforms (Windows and Linux), I used System.lineSeparator() instead of "\n".
@FelipeWindmoller thanks for the suggestion,I’ll update My answer
|
1

You can do something similar to this:

https://commons.apache.org/sandbox/flatfile/xref/org/apache/commons/flatfile/util/ConcatenatedInputStream.html

It just implements the read() method of InputStream and has a list of InputStreams it is concatenating. Once it reads an EOF it starts reading from the next InputStream. Just convert the Strings to ByteArrayInputStreams.

Comments

0

you can also do this way create a Serializable List

List<String> quarks = Arrays.asList(
      "up", "down", "strange", "charm", "top", "bottom"
    );

//serialize the List
//note the use of abstract base class references

try{
  //use buffering
  OutputStream file = new FileOutputStream( "quarks.ser" );
  OutputStream buffer = new BufferedOutputStream( file );
  ObjectOutput output = new ObjectOutputStream( buffer );
  try{
    output.writeObject(quarks);
  }
  finally{
    output.close();
  }
}  
catch(IOException ex){
  fLogger.log(Level.SEVERE, "Cannot perform output.", ex);
}

//deserialize the quarks.ser file
//note the use of abstract base class references

try{
  //use buffering
  InputStream file = new FileInputStream( "quarks.ser" );
  InputStream buffer = new BufferedInputStream( file );
  ObjectInput input = new ObjectInputStream ( buffer );
  try{
    //deserialize the List
    List<String> recoveredQuarks = (List<String>)input.readObject();
    //display its data
    for(String quark: recoveredQuarks){
      System.out.println("Recovered Quark: " + quark);
    }
  }
  finally{
    input.close();
  }
}
catch(ClassNotFoundException ex){
  fLogger.log(Level.SEVERE, "Cannot perform input. Class not found.", ex);
}
catch(IOException ex){
  fLogger.log(Level.SEVERE, "Cannot perform input.", ex);
}

Comments

0

I'd like to propose my simple solution:

public class StringListInputStream extends InputStream {
    private final List<String> strings;
    private int pos = 0;
    private byte[] bytes = null;
    private int i = 0;

    public StringListInputStream(List<String> strings) {
        this.strings = strings;
        this.bytes = strings.get(0).getBytes();
    }

    @Override
    public int read() throws IOException {
        if (pos >= bytes.length) {
            if (!next()) return -1;
            else return read();
        }
        return bytes[pos++];
    }

    private boolean next() {
        if (i + 1 >= strings.size()) return false;
        pos = 0;
        bytes = strings.get(++i).getBytes();
        return true;
    }
}

Comments

0

Here a memory-sparing and time-efficient solution. I started with the solution https://stackoverflow.com/a/53330879/7251133 and make it time-efficient by implementing read(byte[], int, int) like that:

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.Iterator;
import java.util.Objects;
import java.util.function.Function;

/**
 * @author Mike Shauneu https://stackoverflow.com/a/53330879/7251133 .
 * @author Florian Hof for the {@link #read(byte[], int, int)}.
 */
public class IterableInputStream<T> extends InputStream {

    public static final int EOF = -1;

    private static final InputStream EOF_IS = new InputStream() {
        @Override public int read() throws IOException {
            return EOF;
        }
        @Override
        public int read(byte[] b, int off, int len) throws IOException {
            return EOF;
        }
    };

    private final Iterator<T> iterator;
    private final Function<T, byte[]> mapper;

    private InputStream current;

    public IterableInputStream(Iterable<T> iterable, Function<T, byte[]> mapper) {
        this.iterator = iterable.iterator();
        this.mapper = mapper;
        next();
    }

    @Override
    public int read() throws IOException {
        int n = current.read();
        while (n == EOF && current != EOF_IS) {
            next();
            n = current.read();
        }
        return n;
    }

    private void next() {
        current = iterator.hasNext() 
            ? new ByteArrayInputStream(mapper.apply(iterator.next())) 
            : EOF_IS;
    }

    @Override
    public int read(byte[] b, int off, int len) throws IOException {
        Objects.checkFromIndexSize(off, len, b.length);
        if (len == 0) {
            return 0;
        } else if (current == EOF_IS) {
            return EOF;
        }

        int nbRead = current.read(b, off, len);
        int nbReadTotal = (nbRead >= 0) ? nbRead : 0;
        while((nbReadTotal < len || nbRead == EOF) && current != EOF_IS) {
            next();
            nbRead = current.read(b, off + nbReadTotal, len - nbReadTotal);
            nbReadTotal += (nbRead >= 0) ? nbRead : 0;
        }
        return nbReadTotal;
    }
}

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.