Java - File To Byte Array - Fast One

Question

I want to read a file into a byte array. So, I am reading it using:

    int len1 = (int)(new File(filename).length());
    FileInputStream fis1 = new FileInputStream(filename);
    byte buf1[] = new byte[len1];
    fis1.read(buf1);

However, it is realy very slow. Can anyone inform me a very fast approach (possibly best one) to read a file into byte array. I can use java library also if needed.

Edit: Is there any benchmark which one is faster (including library approach).

How slow is it? Which part of it is slow? I for one am not aware of it being very slow. — Marko Topolnik
– Marko Topolnik, Commented Jul 31, 2012 at 13:57
Slow as compared to what? Are you sure it's just not a case of percieved slowness in which case the solution is to try not to read the entire file in a single go and think of processing it chunk by chunk? — Sanjay T. Sharma
– Sanjay T. Sharma, Commented Jul 31, 2012 at 13:59
@MarkoTopolnik, Takes 15-20 sec to read 1616007480 bytes. Last line is slow :). — alessandro
– alessandro, Commented Jul 31, 2012 at 14:00
Let's see... 1.5GB in 15 seconds... WOW, that's 100MB/s! That's FAST! — Marko Topolnik
– Marko Topolnik, Commented Jul 31, 2012 at 14:08

AlexR · Accepted Answer · 2012-07-31 14:03:16Z

16

It is not very slow, at least there is not way to make it faster. BUT it is wrong. If file is big enough the method read() will not return all bytes from fist call. This method returns number of bytes it managed to read as return value.

The right way is to call this method in loop:

  public static void copy(InputStream input,
      OutputStream output,
      int bufferSize)
      throws IOException {
    byte[] buf = new byte[bufferSize];
    int bytesRead = input.read(buf);
    while (bytesRead != -1) {
      output.write(buf, 0, bytesRead);
      bytesRead = input.read(buf);
    }
    output.flush();
  }

call this as following:

ByteArrayOutputStream baos = new ByteArrayOutputStream();
copy(new FileInputStream(myfile), baos);
byte[] bytes = baos.toByteArray();

Something like this is implemented in a lot of packages, e.g. FileUtils.readFileToByteArray() mentioned by @Andrey Borisov (+1)

EDIT

I think that reason for slowness in your case is the fact that you create so huge array. Are you sure you really need it? Try to re-think your design. I believe that you do not have to read this file into array and can process data incrementally.

answered Jul 31, 2012 at 14:03

AlexR

116k16 gold badges137 silver badges216 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

divanov Over a year ago

Note how you pass 2 parameters into copy method, which has 3 parameters.

Andrey Borisov · Accepted Answer · 2012-07-31 13:58:27Z

10

apache commons-io FileUtils.readFileToByteArray

answered Jul 31, 2012 at 13:58

Andrey Borisov

3,17020 silver badges18 bronze badges

1 Comment

Andrey Borisov Over a year ago

internally this method invokes copyLarge(InputStream input, OutputStream output) - so should be better =:) Also apache commons-io uses it's own ByteArrayOutputSteam. from docs ---> This is an alternative implementation of the java.io.ByteArrayOutputStream class. The original implementation only allocates 32 bytes at the beginning. As this class is designed for heavy duty it starts at 1024 bytes. In contrast to the original it doesn't reallocate the whole memory block but allocates additional buffers. This way no buffers need to be garbage collected...

Collectives™ on Stack Overflow

Java - File To Byte Array - Fast One

2 Answers 2

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related