Byte Packing: Turning a u32 into Four u8s by Hand

Picture a suitcase with a fixed size limit, and you need to get a bunch of stuff across to another side, say into a file, across a network, wherever. Until I decided to work with binary files, I had never thought about this: The thing is a file or a socket doesn’t know what a u32 is. At that layer it’s just raw bytes, 0–255 each, and there’s no “write the number as it is” button.

Back to our example, the channel only takes one suitcase-sized chunk at a time (a byte), and a u32 can be in the billions, way too much for one trip. So you split it into four byte-sized pieces, carry them across one at a time, and put them back together on the other side. That’s byte packing: taking a number apart into its individual u8 bytes, and back again. Here’s the bit manipulation behind it, and why the order you pack those bytes in matters.

Worth being clear about what actually needs this, since it’s not about size in bytes. In the naive DB I am working on, the key and value in a record can easily be more than one byte long, but they’re already just raw bytes which can be read back in order, byte-for-byte, no reconstruction needed. The tag is the same idea at one byte: a single 0 or 1, nothing to combine. But a size is different, in the sense that it’s not a sequence of bytes, it’s one number, and you need to decide how it’s spread across multiple bytes so it can be put back together correctly. So packing is for numbers whose value has to be reconstructed from multiple bytes by position, not for anything that’s already just well… bytes.

A quick refresher on the integer types involved:

We’ll focus on u32. Laid out as four bytes, it looks like this (all zero for now):

Right Shifting

Right shifting moves every bit in a binary number to the right by a specified number of positions. Each shift divides the number by two, with any remainder dropped (floor division).

From here on we’ll only show the bits that matter, since a fully populated u32 would look like this:

One note on notation: when we write out a byte, the most significant one comes first, byte 3 -> byte 2 -> byte 1 -> byte 0 (or byte 1 -> byte 0 if only two bytes have data). This is called big-endian, since the “big end”, the largest place value, comes first. The reverse is little-endian.

Why Big-Endian

Neither ordering is more “correct.” A computer doesn’t care which order you pick, as long as whatever decodes the bytes agrees with whatever encoded them. That’s the only hard rule: encode and decode have to use the same order, or the numbers come out wrong. You could just as easily write byte_packing to emit the least-significant byte first, and it would still work, provided decode_packed_byte used the same order.

I picked big-endian for two reasons:

• It matches how you already read numbers. Write 300 on paper and the 3 (hundreds, the largest place value) comes first, on the left. Big-endian does the same thing at the byte level: [0, 0, 1, 44] reads left to right the same way the number does. Little-endian would print as [44, 1, 0, 0], just as correct, but harder to eyeball in a hex dump.
• It’s the standard for “bytes on the wire.” Big-endian is often called network byte order, precisely because two machines exchanging multi-byte numbers need to agree on an order, and this has long been the default.

Rust’s u32::to_be_bytes() is named for exactly this reason (the “be” is “big-endian”). Its sibling, to_le_bytes(), produces little-endian, which is what most CPUs (x86, ARM) actually use internally for their own registers and memory, for unrelated performance reasons. Different concern, different convention. We’re using big-endian here because it’s the more common, more readable choice for a file format.

Worked Examples

Take 8, represented in bits as 00001000. Since it only occupies byte 0, here’s the right-shift sequence:

Right-shifting by 1 is the same as floor-dividing by 2:

8 / 2 = 4
4 / 2 = 2
2 / 2 = 1
1 / 2 = 0.5 ≈ 0

Take another example, 150, represented as 10010110:

150 / 2 = 75
75 / 2  = 37.5 ≈ 37
37 / 2  = 18.5 ≈ 18
18 / 2  = 9
9 / 2   = 4.5 ≈ 4
4 / 2   = 2
2 / 2   = 1
1 / 2   = 0.5 ≈ 0

In general, shifting right by i places gives $\lfloor \frac{n}{2^{i}} \rfloor$ (floor division: drop the remainder, exactly like the examples above). In both examples we shifted by 1 each time. When byte packing, we usually shift by 8, a full byte at a time.

Putting It Together

I’ve been working on a project that encodes data into bytes and back, and while pairing with Claude Code on it, I decided to hand-roll the encoding step myself rather than reach for the standard library right away. Rust, like most languages, already ships helpers for this in both big-endian and little-endian, but doing it by hand is the fastest way to actually understand what those helpers are doing.

Say we want to convert 300 into a 4-byte array, bytes = [0u8; 4]:

300 in binary is 1 0010 1100, which spills one bit into a second byte. Written out fully:

So [0, 0, 1, 44] is the byte representation of 300.

Working this out by hand is a good way to build intuition, but it only handles one number at a time. To pack any u32, we need a loop, shifting by 8 for each byte:

300 = 0010 1100 (no shift) = 44 as u8
(300 >> 8)  = 0000 0001 = 1 as u8
(300 >> 16) = 0000 0000 = 0 as u8
(300 >> 24) = 0000 0000 = 0 as u8

In code:

pub(crate) fn byte_packing(value: u32) -> [u8; 4] {
    let mut bytes = [0u8; 4];
    for i in 0..4 {
        bytes[i] = (value >> (8 * (3 - i))) as u8;
    }
    bytes
}

That’s the whole thing: shift the value right by 0, 8, 16, and 24 bits, truncate each result down to a u8, and you’ve packed a u32 into four bytes, most significant first.

Decoding It Back

Getting the number back out is the mirror image: left shift instead of right, and rebuild instead of truncate.

Left shifting moves bits left, multiplying by two per shift instead of dividing. Since byte 3 holds the most significant bits, it needs to go back to position 24; byte 2 to 16; byte 1 to 8; byte 0 stays put. Using [0, 0, 1, 44] from earlier:

(0  as u32) << 24 = 0
(0  as u32) << 16 = 0
(1  as u32) << 8  = 256
(44 as u32) << 0  = 44

Adding those up (or OR-ing them, equivalent here since none of the shifted ranges overlap) gives 0 + 0 + 256 + 44 = 300, the original number.

pub(crate) fn decode_packed_byte(bytes: [u8; 4]) -> u32 {
    let mut value: u32 = 0;
    for i in 0..4 {
        value |= (bytes[i] as u32) << (8 * (3 - i));
    }
    value
}

The cast to u32 before shifting matters because a u8 only has 8 bit-positions to begin with so shifting it left just pushes bits off the top and they’re lost. Casting up to u32 first gives the bits 32 positions to land in, so nothing gets discarded when a byte moves up to position 24.

Try It Yourself

The widget below runs both functions above on the same three examples used throughout this post. The highlighted window is the u8 boundary: for byte_packing it’s where the low byte gets read off after each right shift, for decode_packed_byte it’s where each byte starts before it slides left into place. Step through manually, or press play.

References

1. Right Shift Explained - Bit Manipulation Tutorial (YouTube)

This is part of a series from my DB learning with Claude Code as a pair-programming/tutoring partner. You can find the curriculum here.