Protocol spec, written once, generated into four programming languages

Protocols are where good intentions fall apart. On paper there is a clean interface; in practice C firmware runs on the microcontroller, C++ code in the gateway, Rust in the edge service, and Python in the test pipeline, and each of the four has its own idea of what a byte on the wire looks like. Reviews no longer catch it, because four reviewers in four repositories maintain four copies.

The way out is a single source. The specification must be human readable, because it is discussed, reviewed and negotiated. It also has to be strict enough that code can be generated from it without any later enrichment being needed. We solve this in two steps: first Markdown for the human, then JSON for the machine. From the JSON an AI agent generates the four language implementations (see Spec-driven development, Scrum, SAFe and waterfall for the methodological framing).

The previous article on the SSOT pipeline for firmware, tests and docs walked through the basic principle for a single device. This article extends it to a multi-language protocol and describes the iteration cycle as it has worked for us in practice.

Why not write the JSON directly?

JSON is a wonderful intermediate language and a poor basis for a discussion. Nobody enjoys reading a hundred lines of nested braces in a review, commenting on fields between commas, or arguing about architecture over "frame": {"sof": "0xAA"}. Markdown is built for exactly that: a heading per message, tables for fields, prose for the rationale, Mermaid for state diagrams, source references for standards.

So the spec is born as a Markdown document. It carries everything a reviewer needs: stories, reasons, examples, diagrams, cross references. What it lacks: machine-grade strictness.

That strictness is added by a parser.

Migration from DOORS, Polarion, Word or ReqIF

In most embedded organisations the requirements do not live in Markdown but in tools like IBM DOORS, Siemens Polarion, Jama Connect, codeBeamer, in ReqIF exports or simply in Word and Excel. Nobody is going to retype thousands of pages by hand to start the SSOT pipeline.

This is where a local language model earns its keep. We run the exports from the existing tool through an LLM that converts them into our Markdown spec format. Concretely we use a local LLM in the 30 to 35B class such as Qwen3 Coder, on a workstation with enough VRAM. Local is not a taste question here, it is a condition on confidentiality grounds: protocol specifications, register maps and requirement texts are typically IP that must not leave the corporate network. A cloud LLM is disqualified, a local open model like Qwen3 Coder meets the requirement.

The LLM does the tedious translation: DOORS modules become Markdown sections, ReqIF attributes become Markdown tables, Word prose becomes structured text with the right conventions. A human reviews, corrects where needed, and from then on the Markdown file is the source. The legacy tool may continue to live in read mode, new changes happen in the Markdown spec.

The transition is one-off per component. Once the Markdown spec exists, the iteration cycle described in the next sections takes over.

Ideally the transformation from the system engineering tool into the JSON intermediate form runs fully automatic. Every new spec revision that requires a manual step in between costs time and opens a channel for errors: a field name mistyped, an enumeration dropped, a range copied wrong, and the generator silently builds on a wrong foundation. The same holds when an organisation keeps DOORS or Polarion as source and skips the Markdown step: only when the chain from requirements tool to JSON form runs without a human in between does the approach scale across many iterations.

Hints for spec authors

Four rules that have paid off in practice:

Describe the context. What is this spec about, why are we writing it, how do the telegrams flow in a typical exchange. It does not take much, a few paragraphs are enough. But exactly that little bit decides whether the test generator can immediately derive sensible automated tests, or whether you later have to painfully patch things up with a separate use case document.

Keep it short. A spec nobody reads in one sitting will also not be processed by any agent in one sitting. Both human reviewers and an LLM context window have limits. Spec documents stay compact, redundant phrasings go, one statement lives in one place.

Precise over woolly. “Usually”, “should normally”, “as a rule” are not descriptions in a spec, they are bug sources. If something is optional, mark it as optional in the table. If something is conditional, state the condition explicitly. Woolly wording misleads both humans and generators, and the generator will reliably take the wrong assumption.

Textual over visual. The nicest drawing of a bus with all its participants is helpful for human reviewers, but in the pipeline routine it costs an extra step: images have to be extracted, interpreted and turned into a machine-readable form, and that step is rarely free of manual rework. Spec content therefore belongs as far as possible in textual form: tables, ordered lists, clean sections, clearly typed code blocks. When a drawing is genuinely needed, it pays off to keep it in a text-based source format like Mermaid or PlantUML. The repository holds both: the rendered image for the documentation, the source for the parser.

Spec language and English interface

The spec can be written in the language in which the domain is actually discussed, usually German or English, occasionally French or Italian. That is generally not a problem for the AI, modern LLMs handle technical text in all common European languages reliably. What matters is that the generated code identifiers stay consistently in English and that the same technical term always maps to the same translation. For this we keep a short glossary per project at the repo root: the source-language spec terms on one side, the English field names, enum values and function names on the other. The generator reads this glossary in and guarantees that an ambiguous word like “Zustand” always maps to the project-defined translation, whether that is state (state machine), condition (health or fault status) or status (general operational situation). Without such a mapping the AI would pick a different translation depending on how it reads the surrounding sentence.

Markdown to JSON: the parser

The parser is a short Python script that reads the Markdown and lifts every structure relevant to code generation into a JSON form: messages with their fields, data types, ranges, defaults, state transitions, version markers. The rules for what gets extracted are marked in the Markdown via clear conventions (a table in a section titled “Fields” becomes the field list, a state code block annotation becomes the state diagram, and so on).

Crucially, the parser is deterministic and idempotent. The same Markdown produces the same JSON byte for byte. That makes the JSON a reliable build artefact against which every downstream generator runs.

The JSON itself is never edited by hand. Anyone who wants to change the protocol changes the Markdown spec. The parser runs as the first build step and fails hard if the Markdown conventions are violated. That catches the most common error source, namely inconsistencies a human misses in Markdown but the parser sees as ambiguity.

JSON to code: one generator per language

From the JSON form an AI agent generates the implementation per target language. Concretely we run four generators:

• C: header with struct definitions for the wire layouts, plus pack/unpack functions without heap, fit for microcontrollers. Endianness and padding are explicit, no reliance on compiler defaults.
• C++: same layout, plus std::span, std::optional, explicit pack/unpack functions with defined endianness, and a thin typed wrapper layer over the C structs. For the gateway that forwards and transforms messages. The moment the MCU folks need a different C++ library than the embedded Linux developers, because dynamic memory allocation is off the table in a firmware context, pure diversification chaos breaks out, unless the generator knows both profiles and produces the right variant per target.
• Rust: #[repr(C)] structs for wire compatibility, safe parsers with Result types, serde bindings for the test side. For the edge service, where error handling is front and centre.
• Python: classes with dataclasses, struct.pack/unpack for the wire representation, type hints derived from the spec definitions. For the test pipeline and for tools that need fast iteration.

Each generator emits only what can be derived from the spec. Application logic, transport, threading, anything that is not in the wire format is left to the developer. This separation matters: generated code is overwritten ruthlessly on every spec change. Anything that is not generated lives in its own files and stays untouched.

Idiomatic per language, not “C in four languages”

The most common failure mode of multi-language generators is that the C code is mechanically translated into the other languages and looks foreign everywhere. We give the agents a per-language idiomatic skill: in Rust, error types are explicit, in Python, fields are declared as dataclass rather than dict, in C++ enum class is used over bare enums. A reviewer on the Rust side must see Rust code, not a machine translation of C.

This is the point where an AI agent has more leverage than a classical generator: it can pick the right form per language because it understands the language as a language, not just as a template engine.

The iteration cycle

A typical change looks like this:

1. A reviewer spots that a field in a message needs an extended value range.
2. The Markdown spec is updated, in the same document at the same place where the value was described. The spec version is bumped.
3. The build runs. The parser regenerates the JSON. All four generators regenerate their language outputs.
4. The diffs are shown in the pull request. Each language shows a comparable, small change. Reviewers can check whether the translation into every language is plausible.
5. Tests run against the new code. If they are green, the change merges.

When a generator gets something wrong (e.g. it breaks a Rust idiom), the generated code is not fixed by hand, the generator itself is sharpened. Errors must be fixed at the source. The consequence otherwise would be that you have to adjust the generated code by hand after every run, and that is exactly what we do not want. Generated code is generated code, always, no exceptions.

Versioning the spec itself

Every spec carries a version. The version is part of the wire representation so that a receiver immediately knows whether it can decode the message safely at all. Incompatible changes bump the major version, backward-compatible additions the minor.

The version rules also live in the Markdown spec, and the parser enforces them. Renaming a field without bumping the major version is a build error. Adding an optional field without bumping the minor version, the same.

Honest pitfalls

• ABI stability in C and Rust is tricky. Padding and endianness must be explicit, otherwise the same code behaves differently on ARM® and x86. The generator must enforce this actively, not just document it.
• Version mix in the field is the most common error source in deployed systems. A sender on version 1.3, a receiver on 1.1, both believe they understand each other. Tests must cover version combinations explicitly.
• The temptation of exceptions. The moment someone writes a special case into the generated code “just this once”, the discipline is broken. Better to sharpen the generator, even if it takes longer short term.

What stays

A single Markdown file, one parser, four generators, four language outputs. Spec changes are a pull request that is traceable in every language. Reviewers discuss at the spec level, not over hand-written copies of the protocol. The truth lives in the Markdown, not in the code.

In the next article we show how use cases written in the same Markdown / JSON / pytest style produce a test bench that exercises the generated protocol on real hardware.

Keywords: Spec Driven Development, protocol specification, Markdown to JSON, code generation C C++ Rust Python, Qwen3 Coder, DOORS Polarion migration, protocol generator consulting, wire format ABI stability, firmware protocols, Embedded Consulting Switzerland, swissEmbedded firmware engineering