260

I have a long Hex string that represents a series of values of different types. I need to convert this Hex String into bytes or bytearray so that I can extract each value from the raw data. How can I do this?

For example, the string "ab" should convert to the bytes b"\xab" or equivalent byte array. Longer example:

>>> # what to use in place of `convert` here?
>>> convert("8e71c61de6a2321336184f813379ec6bf4a3fb79e63cd12b")
b'\x8eq\xc6\x1d\xe6\xa22\x136\x18O\x813y\xeck\xf4\xa3\xfby\xe6<\xd1+'
Improve this question
1
  • 1
    How does that hex string look like? Commented Apr 13, 2011 at 12:46

7 Answers 7

Reset to default
423

Suppose your hex string is something like

>>> hex_string = "deadbeef"

Convert it to a bytearray (Python 3 and 2.7):

>>> bytearray.fromhex(hex_string)
bytearray(b'\xde\xad\xbe\xef')

Convert it to a bytes object (Python 3):

>>> bytes.fromhex(hex_string)
b'\xde\xad\xbe\xef'

Note that bytes is an immutable version of bytearray.

Convert it to a string (Python ≤ 2.7):

>>> hex_data = hex_string.decode("hex")
>>> hex_data
"\xde\xad\xbe\xef"
Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Note that bytes.fromhex throws an error when the input string has an odd number of characters: bytes.fromhex("aab") → ValueError: non-hexadecimal number found in fromhex() arg at position 3.
The new version of hex_string.decode("hex") is codecs.decode(hex_string, 'hex').
@КонстантинВан Good! That's what it SHOULD do! "aab" is not a hex number, at all. Hex numbers are ALWAYS in pairs of 2 characters. You cannot do \x1 for example. That is not a valid hex number. \x01 is a valid hex number. Just try it yourself: bytes(b"\x1").hex() = invalid \x escape at position 0. Then try again with \x01, which will be encoded to hex as "01". So of course decoding bytes from hex requires valid hex sequences too! Always pairs of 2 characters!
@MitchMcMabers Sure, just as (1101)₂ is not a binary number because it’s not written in pairs of 8 characters. What I meant by that is that it doesn’t assume the presence of the zeros, which would make sense in numbers. It’s treating the input as an octet string, rather than a number.
167

There is a built-in function in bytearray that does what you intend.

bytearray.fromhex("de ad be ef 00")

It returns a bytearray and it reads hex strings with or without space separator.

Improve this answer

2 Comments

This works in Python 3, whereas hex_string.decode("hex") does not.
>>> bytearray.fromhex("8 ad be ef 00") Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: non-hexadecimal number found in fromhex() arg at position 1
20

provided I understood correctly, you should look for binascii.unhexlify

import binascii
a='45222e'
s=binascii.unhexlify(a)
b=[ord(x) for x in s]
Improve this answer

1 Comment

I agree that unhexlify is the most efficient way to go here, but would suggest that b = bytearray(s) would be a better than using ord. As Python has a built-in type just for arrays of bytes I'm surprised no one is using it
8

Assuming you have a byte string like so

"\x12\x45\x00\xAB"

and you know the amount of bytes and their type you can also use this approach

import struct

bytes = '\x12\x45\x00\xAB'
val = struct.unpack('<BBH', bytes)

#val = (18, 69, 43776)

As I specified little endian (using the '<' char) at the start of the format string the function returned the decimal equivalent.

0x12 = 18

0x45 = 69

0xAB00 = 43776

B is equal to one byte (8 bit) unsigned

H is equal to two bytes (16 bit) unsigned

More available characters and byte sizes can be found here

The advantages are..

You can specify more than one byte and the endian of the values

Disadvantages..

You really need to know the type and length of data your dealing with

Improve this answer

2 Comments

Disadvantages: that is a byte string, not a hex string, so this is not an answer to the question.
It is an answer to the 2nd part of the question "... so that I can shift each value out and convert it into its proper data type".
2

You can use the Codecs module in the Python Standard Library, i.e.

import codecs

codecs.decode(hexstring, 'hex_codec')
Improve this answer

Comments

0

You should be able to build a string holding the binary data using something like:

data = "fef0babe"
bits = ""
for x in xrange(0, len(data), 2)
  bits += chr(int(data[x:x+2], 16))

This is probably not the fastest way (many string appends), but quite simple using only core Python.

Improve this answer

Comments

-7

A good one liner is:

byte_list = map(ord, hex_string)

This will iterate over each char in the string and run it through the ord() function. Only tested on python 2.6, not too sure about 3.0+.

-Josh

Improve this answer

2 Comments

perfect. Working on python 2.7
This doesn't convert hex - it converts each character of a string to an integer. For hex each pair of characters would represent a byte. You might as well just say byte_list = bytearray(hex_string)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.