How can I unpack binary hex formatted data in Python?

Using the PHP pack() function, I have converted a string into a binary hex representation:

$string = md5(time); // 32 character length
$packed = pack('H*', $string);

The H* formatting means "Hex string, high nibble first".

To unpack this in PHP, I would simply use the unpack() function with the H* format flag.

How would I unpack this data in Python?


Asked by: Madaline919 | Posted: 24-09-2021






Answer 1

There's an easy way to do this with the binascii module:

>>> import binascii
>>> print binascii.hexlify("ABCZ")
'4142435a'
>>> print binascii.unhexlify("4142435a")
'ABCZ'

Unless I'm misunderstanding something about the nibble ordering (high-nibble first is the default… anything different is insane), that should be perfectly sufficient!

Furthermore, Python's hashlib.md5 objects have a hexdigest() method to automatically convert the MD5 digest to an ASCII hex string, so that this method isn't even necessary for MD5 digests. Hope that helps.

Answered by: Fenton737 | Posted: 25-10-2021



Answer 2

There's no corresponding "hex nibble" code for struct.pack, so you'll either need to manually pack into bytes first, like:

hex_string = 'abcdef12'

hexdigits = [int(x, 16) for x in hex_string]
data = ''.join(struct.pack('B', (high <<4) + low) 
               for high, low in zip(hexdigits[::2], hexdigits[1::2]))

Or better, you can just use the hex codec. ie.

>>> data = hex_string.decode('hex')
>>> data
'\xab\xcd\xef\x12'

To unpack, you can encode the result back to hex similarly

>>> data.encode('hex')
'abcdef12'

However, note that for your example, there's probably no need to take the round-trip through a hex representation at all when encoding. Just use the md5 binary digest directly. ie.

>>> x = md5.md5('some string')
>>> x.digest()
'Z\xc7I\xfb\xee\xc96\x07\xfc(\xd6f\xbe\x85\xe7:'

This is equivalent to your pack()ed representation. To get the hex representation, use the same unpack method above:

>>> x.digest().decode('hex')
'acbd18db4cc2f85cedef654fccc4a4d8'
>>> x.hexdigest()
'acbd18db4cc2f85cedef654fccc4a4d8'

[Edit]: Updated to use better method (hex codec)

Answered by: David197 | Posted: 25-10-2021



Answer 3

In Python you use the struct module for this.

>>> from struct import *
>>> pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'
>>> unpack('hhl', '\x00\x01\x00\x02\x00\x00\x00\x03')
(1, 2, 3)
>>> calcsize('hhl')
8

HTH

Answered by: Owen156 | Posted: 25-10-2021



Similar questions

Properly formatted example for Python iMAP email access?

tldr: Can someone show me how to properly format this Python iMAP example so it works? from https://docs.python.org/2.4/lib/imap4-example.html import getpass, imaplib M = imaplib.IMAP4() M.login(getpass.getuser(), getpass.getpass()) M.select() typ, data = M.search(None, 'ALL') for num in data[0...


python - Passing Formatted Text Through XSLT

I have formatted text (with newlines, tabs, etc.) coming in from a Telnet connection. I have a python script that manages the Telnet connection and embeds the Telnet response in XML that then gets passed through an XSLT transform. How do I pass that XML through the transform without losing the original formatting? I have access to the transformation script and the python script but not the transform invocation itself.


How are booleans formatted in Strings in Python?

I see I can't do: "%b %b" % (True, False) in Python. I guessed %b for b(oolean). Is there something like this?


Is there a way for me to get detailed formatted information on a Python class?

So, I know I can use dir() to get information about class members etc. What I'm looking for is a way to get a nicely formatted report on everything related to a class (the members, docstrings, inheritance hierarchy, etc.). I want to be able to run this on the command-line so I can explore code and debug better.


python - how to extract formatted text content from PDF

How can I extract the text content (not images) from a PDF while (roughly) maintaining the style and layout like Google Docs can?


datetime - Python: date formatted with %x (locale) is not as expected

I have a datetime object, for which I want to create a date string according to the OS locale settings (as specified e.g. in Windows'7 region and language settings). Following Python's datetime formatting documentation, I used the %x format code which is supposed to output "Locale’s a...


python - Removing broken tags and poorly formatted html from some text

i have a huge database of scraped forum posts that i am inserting into a website. however alot of people try to use html in their forum posts and often times do it wrong. because of this, there are always stray &lt;strike&gt; &lt;b&gt; &lt;/strike&gt; &lt;/div&gt; &lt;/b&gt; tags in the posts which will end up messing up the webpage format when i add say 15 forum posts. for now i have just been append...


Python convert formatted string to list

I have a string "[u'foo']" (Yes, it includes the square brackets and the u''). I have to convert that to a list which looks like [u'foo']. list("[u'foo']") won't work. Any suggestions?


Getting formatted datetime in Python like in PHP

How to get formatted date time in Python the same way as in PHP date('M d Y', $timestamp);?


python - Create an html formatted report

I have a Python 2.6 app running on Linux that creates a CSV file. From the app, I need to create an HTML report, as a single HTML file, that presents the data from the CSV (probably as a table) and also highlights fields where the values meet certain criteria. Charting type functionality would be a nice to have. What's the best way to do this? No GPL stuff please.






Still can't find your answer? Check out these communities...



PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python



top