Protobuf identifies each field by its tag number, not its name. Reuse a tag number for a different field of the same wire type and old bytes are silently reread as the new field — an epoch-ms timestamp 1719950400000 comes back as a plausible user_id, with no error. The safe evolution is a fresh tag number.
Protocol Buffers identifies each field by its tag number, not its name — the name lives only in the schema, and the wire bytes carry (tag, value) pairs. So if a new schema version reuses an old tag number for a different field of the same wire type, old encoded bytes are silently reinterpreted as the new field: no error, wrong meaning. You will encode an event's timestamp_ms under tag 2, then decode those same bytes under a v2 schema that reuses tag 2 for user_id, and watch 1719950400000 — an epoch-millisecond timestamp — come back as a plausible-looking user id. The safe evolution is a fresh tag number.
Designing Data-Intensive Applications, 2nd ed. (Kleppmann & Riccomini, 2025), Chapter 5 — Encoding and Evolution, §"Field tags and schema evolution":
you cannot change a field's tag, since that would make all existing encoded data invalid.
The book states the rule; here you break it on purpose and watch exactly how the data goes bad — silently, with no error, into a plausible wrong value.
The surprise is that a schema change compiles and decodes cleanly while meaning something entirely different. These help; each line notes where it shows up.
(tag number, value) pairs; the field name exists only in the .proto. This is the whole mechanism: tag 2 is tag 2 regardless of what you call it (v1 timestamp_ms vs v2 user_id).int64, 2 = length-delimited for string). The decoder uses this to know how many bytes the value spans — and, in v2c, to notice a mismatch.user_id = 0); a tag whose wire type doesn't match the schema is treated as unknown and skipped (v2c's dropped string).int64 field on tag 2, so only the tag reuse varies between versions and nothing else can explain the result.(field_number << 3) | wire_type key byte). Default and unknown fields (#3): the Protobuf "Proto3 Language Guide" (default values) and the encoding page (unknown fields). If only one is new, make it #1 — that tags, not names, identify fields is the entire landmine.
uv, with Protocol Buffers (grpcio-tools compiling each .proto at runtime + the protobuf runtime) — no Docker. One event value: timestamp_ms = 1719950400000 (epoch-ms, 2024-07-02T20:00:00Z).
One event, encoded once under v1, then decoded under three v2 schemas:
message Event { int64 timestamp_ms = 2; }. Encode the timestamp; the wire bytes are (tag 2, varint value).message Event { int64 user_id = 2; }. Reuses tag 2 for a different int64. Same wire type, so the old value decodes cleanly — as a user_id.message Event { int64 timestamp_ms = 2; int64 user_id = 3; }. user_id gets a fresh tag 3. Old bytes keep the timestamp; user_id defaults to 0.message Event { string user_id = 2; }. Reuses tag 2 but as a string (different wire type), so the old int is a wire-type mismatch.We decode the same old bytes under each and print the result. The whole exercise is one script, code/field_tag_landmine.py.
cd study/ddia/ch05/code
uv run --python 3.12 --with grpcio-tools --with protobuf \
python3 field_tag_landmine.pyDo not read the output yet — make each prediction first.
Predict. timestamp_ms = 1719950400000 is encoded under int64 ... = 2. Roughly how many bytes, and what does the first byte encode?
Seven bytes: one tag byte plus a six-byte varint. 0x10 is (2 << 3) | 0 — tag number 2, wire type 0 (varint). Note what is not here: the string "timestamp_ms" appears nowhere. The name lived only in the schema.
Predict. v2 reuses tag 2 for int64 user_id. Decode the same v1 bytes. Does it raise an error? If not, what is user_id?
No error. The bytes still say "tag 2, this varint," and v2 says "tag 2 is user_id," so a timestamp becomes a user id of 1.7 trillion. This is the landmine: it compiles, it decodes, it is wrong.
Predict. v2b keeps timestamp_ms on tag 2 and adds user_id on a fresh tag 3. v2c reuses tag 2 but as a string. What does each do to the old bytes?
The fresh tag is the fix: the old timestamp keeps its meaning on tag 2, and user_id — never present in the old bytes — reads as its zero default. The wrong-type reuse (v2c) is the interesting middle case: it also raises no error, but because the wire types differ, protobuf can tell the byte isn't a valid string field and silently drops it rather than handing you a plausible wrong value.
10 80 b4 a1 a8 87 32, a tag byte plus a varint — and no field name anywhere.user_id = 1719950400000, with no error.timestamp_ms = 1719950400000 and defaults the new user_id to 0.user_id = '').Protobuf's wire format is deliberately minimal: each field is a key byte — (field_number << 3) | wire_type — followed by the value, and nothing else. The field name is compile-time-only; it never travels. That is exactly what makes Protobuf compact (the size-ladder exercise), and it is exactly what makes tag reuse dangerous. The decoder does not — cannot — check that "the field on tag 2 used to be called timestamp_ms." It reads tag 2, looks up tag 2 in its schema, finds user_id, and assigns the value. The bytes are valid; only the agreement about what tag 2 means has been broken, and Protobuf has no way to detect a broken agreement.
The wire type is the one consistency check the format offers, and it is a check about shape, not meaning. In v2 the shapes match — both int64 are varints — so nothing looks wrong and the value flows straight into the wrong field. In v2c the shapes differ — a varint is not a length-delimited string — so the decoder notices tag 2 doesn't fit and treats it as an unknown field, skipping it. Neither case errors; the difference is whether you get a plausible wrong value or a missing one. Both are silent, and silent is the hazard.
.proto and valid Protobuf. What saves you is process — never reuse a tag number, and mark retired tags reserved so the compiler refuses to reassign them. Note the failure modes rank by danger: same wire type (v2) gives a believable wrong value and is worst; different wire type (v2c) loses the data but at least won't hand you garbage that looks real; a truly fresh tag (v2b) is the only forward- and backward-compatible move. And this is a Protobuf-shaped hazard specifically — Avro has no tags at all and instead depends on the exact writer schema (the Avro exercise), trading tag-reuse for schema-resolution as its evolution mechanism.
reserved as the guardrail. Change v2 to message Event { reserved 2; int64 user_id = 3; } and try to compile a version that puts a field back on tag 2. Predict whether protoc errors — and why reserved is the real fix the book recommends over "just remember not to reuse it."user_id bytes — this is the forward-compatibility direction, the mirror of Step 3.Sources: DDIA 2e, Ch. 5, §"Field tags and schema evolution" · the Protobuf "Encoding" and "Proto3 Language Guide" spec pages.