In Java - Convert ArrayList<Integer> to Byte Array then convert Byte Array to BLOB

Question

I have climate data such as moisture, pressure, temp etc ... that are daily values, and need to do calculations over a month, and many different months. This leads to data that is either DOUBLE or INTEGER data that are put into Arraylists.

I want to be able to convert this ArrayList to a Byte Array, then compress this to a Binary BLOB, and store in a database in a BLOB field. I knew someone before that did this in python was able to convert Array to ByteArray and then put into database BLOB field.

I have seen some links here, but it is not exactly the same. It is really confusing the different ways to do this. For example here from this link, they encode to base64. How to convert int array to base64 string in Java?

Hope someone can provide guidance into this area.

P.

An int is 4 bytes. So create a byte array 4x the size of the list, then for each int, put the various bytes of the int (use bitshift operators). — Michael
– Michael, Commented Jul 19, 2023 at 18:13

Oleg Cherednik · Accepted Answer · 2023-07-19 19:13:54Z

Standard java serizlizations works fine in this case:

public static void main(String... args) throws IOException, ClassNotFoundException {
    List<Integer> numbers = IntStream.range(0, 100).boxed().collect(Collectors.toList());
    byte[] buf = serialize(numbers);
    List<Integer> res = deserialize(buf);
}

public static byte[] serialize(List<Integer> numbers) throws IOException {
    try (ByteArrayOutputStream out = new ByteArrayOutputStream();
         ObjectOutputStream oos = new ObjectOutputStream(out)) {
        oos.writeObject(numbers);
        return out.toByteArray();
    }
}

public static List<Integer> deserialize(byte[] buf) throws IOException, ClassNotFoundException {
    try (ByteArrayInputStream bais = new ByteArrayInputStream(buf);
         ObjectInputStream in = new ObjectInputStream(bais)) {
        return (List<Integer>) in.readObject();
    }
}

Slaw · Accepted Answer · 2023-07-19 19:42:31Z

You can make use of ByteBuffer to convert a collection of integers to a byte array.

import java.nio.ByteBuffer;
import java.util.Collection;

public class Foo {

    public static byte[] toByteArray(Collection<Integer> ints) {
        // an integer in Java is a 32-bit number (i.e., 4 bytes)
        int capacity = Math.multiplyExact(ints.size(), 4);
        ByteBuffer buffer = ByteBuffer.allocate(capacity);
        ints.forEach(buffer::putInt);
        return buffer.array();
    }
}

^{Note: The above ByteBuffer will be using the BIG_ENDIAN byte order. You can change that to LITTLE_ENDIAN if you need to.}

Then, assuming you're using JDBC directly, you just need to make use of:

Note there's also the PreparedStatement#setBlob(int,InputStream) method. Meaning you could create an InputStream that reads the bytes from your integers. This may have significant memory-use improvements depending on how large your integer collection is (and the JDBC driver implementation), as you don't have to allocate a byte buffer/array.

import java.io.InputStream;
import java.util.Iterator;

public class IntegerInputStream extends InputStream {

    private final Iterator<Integer> ints;

    private int currentInt;
    private int byteIndex = -1;

    public IntegerInputStream(Iterable<Integer> ints) {
        this.ints = ints.iterator();
    }

    @Override
    public int read() {
        if (byteIndex == -1) {
            if (ints.hasNext()) {
                currentInt = ints.next();
                byteIndex = 3;
            } else {
                return -1;
            }
        }
        int next = (currentInt >> 8 * byteIndex) & 0xFF;
        byteIndex--;
        return next;
    }
}

Collectives™ on Stack Overflow

In Java - Convert ArrayList<Integer> to Byte Array then convert Byte Array to BLOB

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related