ViPaq Protocol

This page describes the ViPaq wire format as produced and read by Binacle.Net v3.0.x. For what ViPaq is and why it exists, see ViPaq Protocol.

🚨 The format changed in v3.0.0. Strings produced by v2.1.1 and earlier do not decode here, and strings produced here do not decode there. There is no fallback reader. An old string is rejected with a format error rather than misread, so the failure is visible - but stored strings are worth regenerating after the upgrade.

📌 Structure

A ViPaq blob is a two byte header followed by a body:

[ Header: 2 bytes ][ Body ]

• The header is never compressed. The body is built first, compressed if that makes it smaller, and the header is put in front afterwards.
• Every blob is therefore self-describing: a decoder reads the two header bytes, then reads the body exactly as they say.
• All multi-byte integers are unsigned and little-endian.

🧾 Header

Two bytes, one job each. Byte 0 says how to read the body, byte 1 says how wide its integers are.

Byte 0 - form

Byte 1 - integer widths

Compressed and Layout describe that one blob, not the format. An encoder picks them per blob and a decoder obeys them, so two blobs from the same version can differ in both.

📦 Body

The body carries the item count, the bin dimensions, then the items:

[ Item count: uint16 ][ Bin L, W, H ][ Items ]

• Item count is always a 2 byte uint16, whatever the width fields say.
• Bin dimensions are Length, Width, Height, in that order.
• Items come back in the order they were sent.

Layout decides only how the items are arranged. The count and the bin dimensions are the same either way.

Row-major (Layout = 0)

Each item is written whole - its three dimensions, then its three coordinates - before the next item.

L W H X Y Z | L W H X Y Z | L W H X Y Z | ...

Columnar (Layout = 1)

Each field is written for every item before moving on to the next field. Six runs, each as long as the item count.

L L L ... | W W W ... | H H H ... | X X X ... | Y Y Y ... | Z Z Z ...

Columnar puts values of similar size next to each other, which usually compresses better. Row-major is easier to read in a hex dump. Neither is required - the header bit says which was used.

🔢 Integer Widths

Each of the three sections - bin dimensions, item dimensions, item coordinates - is stored at one width, chosen independently:

Widths are fixed per section, not per value: every item shares one dimensions width and one coordinates width. An encoder picks the smallest width that holds the largest value in the section.

The three sections are sized separately because real data needs them to be. A bin of 5000x2000x2000 holding items no larger than 200 gives 16 bit bin dimensions, 8 bit item dimensions and 16 bit coordinates.

🗜️ Compression

The Compressed header bit records what the encoder did. It is not a hint to guess at.

• Compressed = 0 - the bytes after the header are the body.
• Compressed = 1 - the bytes after the header are a compressed stream holding the body.

The codec is raw DEFLATE (RFC 1951), with no zlib or gzip wrapper. The compressed stream is the DEFLATE bit stream and nothing else.

Compression is not applied by size. The encoder compresses the body, keeps whichever of the two is shorter, and sets the bit to say which it kept - so a blob is never made larger by compressing it. A decoder accepts both.

🔤 Text Form

The stored and shared artifact is base64, and the format is optimized for it.

• Standard base64 (RFC 4648 §4), alphabet A-Z a-z 0-9 + /, padded with =. Not the URL-safe alphabet.
• No line wrapping and no whitespace.

📏 Limits

A value outside these ranges is an error when encoding. Nothing is clamped or widened silently.

🧪 Example

A bin of 10x20x30 holding one item 1x2x3 at (4,5,6).

Every value fits in a byte, so all three sections use the 1 byte width, and the encoder wrote it row-major and uncompressed. Both header bytes are therefore 0x00:

00 00  01 00  0A 14 1E  01 02 03  04 05 06
^byte0 ^count ^bin      ^dims     ^coords
   ^byte1

📖 Full Specification

The normative wire specification, including the decoding order, everything a decoder must reject, and further worked examples, lives with the source:

• 📖 ViPaq Protocol Specification