2

I have an image saved as a bytes array in C#. Below would be an example of how you could create such an array:

byte[] image_bytes= new byte[100] {137, 80, 78, 71, ..., 130};

Note: The above is a simplified example of I achieve the byte array. In reality, I am using Unity to generate the array. Here is a brief snippet on how I actually go about generating the byte array in case this sheds light on latter parts of the question:

camera.Render();
...
Texture2D image = new Texture2D(camera.targetTexture.width, camera.targetTexture.height);
image.ReadPixels(new Rect(0, 0, camera.targetTexture.width, camera.targetTexture.height), 0, 0);
image.Apply();
byte[] image_bytes = image.EncodeToPNG();
...

In C# I can use the function WriteAllBytes to save this image as a file that I can later open on my computer. I have tested this and thus know that the aboveimage_bytes array is not corrupted as I can view the final image image_name.png:

File.WriteAllBytes("image_name.png", image_bytes);

The C# scirpt then sends the byte array as a JSON string over TCP connection to a python script. I have confirmed that the byte array arrives in the python script and has the same length. I have also confirmed that the first and last four bytes are the same, for example:

>>> data = json.loads(message)
>>> img_bytes = np.array(data["image"], dtype=bytes)
>>> print(img_bytes[0])
b'137'
>>> print(img_bytes[1])
b'80'
...
>>> print(img_bytes[-1])
b'130'

My goal is to eventually convert the byte array to an OpenCV frame so I can use OpenCV functions on it. However, I can't even seem to do the basics with it, for example, saving it to a file. When I try to save the data to a file in the python script, the resultant files can not be opened by standard image viewing software. I have tried a basic file save operations:

>>> with open("image_name.png", 'wb') as w:
>>>   w.write(img_bytes)

I have tried using OpenCV:

>>> img_np = cv2.imdecode(img_bytes, cv2.IMREAD_COLOR)
>>> cv2.imwrite("image_name.png", img_np)

I have also tried using PIL (Note: here I used the dimensions of the image):

>>> img_np = Image.frombytes('RGBA', (512, 384), img_bytes, 'raw')
>>> img_np.save("image_name.png")

None of the above Python save functions work (they all create files, but none of the files can be opened and viewed). What am I doing wrong? How can I save the file or even better convert the data to an OpenCV frame in Python?

TLDR: I have a byte array that represents an image in C#. I can save the byte array in C# using WriteAllBytes. I send the byte array to a Python script and have confirmed that it arrives uncorrupted. How can I save the file or even better convert the file to an OpenCV frame in Python?

7
  • 1
    Does this answer your question? "The C# scirpt then sends the byte array as a JSON string over TCP connection to a python script" - this sounds horribly inefficient. If possible, it would be better to just send the raw binary data. Commented Jul 3, 2020 at 1:29
  • In the future, that's maybe how I will go about it. Right now, I use JSON as there are many other variables being sent; the image is only 1 of them. With regard to this question, the reason I brought up the JSON was to convey that to the best of my knowledge, the python script receives the data uncorrupted from the C# script. I just can't seem to use it in any meaningful way. Commented Jul 3, 2020 at 1:33
  • Your numpy.ndarray doesn't make sense to me, dtype=bytes will make each item a byte string. I'm assuming your data is a list of ints? Likely, you want dtype=np.uint8, but to get the raw bytes, just use bytes(data), which should reproduce the actual bytes you want. Commented Jul 3, 2020 at 2:23
  • So likely, you want cv2.imdecode(bytes(data), cv2.IMREAD_COLOR) Commented Jul 3, 2020 at 2:26
  • Also, I agree with @John you should probably just be sending the raw data Commented Jul 3, 2020 at 2:29

1 Answer 1

1

Using dtype=bytes doesn't make sense. This makes each item a byte string, hence img_bytes[0] isn't. byte 137, it's the byte string b'137, because data["image"] is a list of ints.

So likely, you wanted:

img_bytes = np.array(data["image"], dtype=np.uint8)

But, you can also use:

img_bytes = bytes(data["image"])
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.