Technology
One pipeline for the live picture and the post-operational answer.
Most vendors run a real-time system and a separate analysis system, and the two disagree. In DataBeacon the live feed and the delayed bulk upload converge on the same event stream, and one set of code produces the tower display, the replay, the nightly KPIs and the safety events.
The spine
An event log, not a request queue.
Surveillance arrives two ways: Mode-S and ADS-B streaming off antennas in near-real time, and ASTERIX radar delivered as files, hours later. Both are decoded and published onto the same Kafka topics. Everything downstream — consolidation, enrichment, the one-second snapshot, the position store, the overnight analytics — consumes from there.
That is what makes a replay agree with what the controller saw at the time: it is not a second reconstruction, it is the same data taking a different path off the same log.
- Strict topic pipeline with independent consumer groups per stage
- The same code path serves streaming and delayed-batch ingest
- Self-managed brokers — no dependency on a specific cloud's managed service
Making conflict detection tractable
Six hundred million pairs, reduced to the ones that matter.
Checking every aircraft against every other aircraft across a day of traffic is a cross-product in the hundreds of millions. Hexagonal spatial indexing with a separation-sized ring cuts that to the candidates that could plausibly conflict, before any geometry runs.
Only then does the full four-dimensional geometry execute, per candidate, returning the diagnostic points that describe the encounter. Work is chunked to bound memory, and because the join is an equality on time, the chunking is lossless rather than approximate.
Rust where it counts
Fourteen services from receiver socket to browser.
The entire data path — ingest, decode, consolidate, enrich, snapshot, stream — is Rust, including an in-house ASTERIX codec. Sub-second latency at nearly two hundred positions per second on modest hardware is a consequence of that choice rather than a claim about it.
The batch analytics is Python, where the work is dataframe-shaped and the libraries are better. Both talk to stock PostgreSQL, Kafka, Redis and S3-compatible object storage — no exotic datastores, nothing that cannot be run somewhere else.
Where it runs
Three deployments, all of them real.
Not "deploys anywhere" as a slogan — three shapes that are running today, each with a different trade.
| Public cloud | Kubernetes on AWS, multi-environment with production and test in separate accounts, provisioned by Terraform. This is where the hosted products run |
|---|---|
| Private / self-hosted | The entire platform — ingest, brokers, databases, all products — on a single Linux box under Docker Compose, carrying live feeds. Its acceptance criterion is identical numbers to production |
| On-premise appliance | For a tower display, one Rust binary with its own web server and map tiles compiled in. No container runtime, no internet connection required |
Portability comes from the shape of the stack — OCI containers, self-managed Kafka, stock datastores — rather than from an abstraction layer. Cloud-specific pieces exist, chiefly around secrets and object storage, and they are the work in any move.
An unusual discipline
Bit-parity, checked every day.
When the self-hosted deployment was built to replace the cloud one, the acceptance test was not "does it work" — it was "does it produce identical numbers", verified by scripted daily comparison against production.
That is how a change that shifted conflict counts by nine times was caught: by the comparison, not by a customer. It is an uncomfortable test to adopt and a good one to have.
How the platform is built
AI in the engineering, not in the claims.
DataBeacon makes extensive use of large language models in building and operating the platform — in development, in infrastructure work, and in the automated checks that keep the two deployments in agreement.
What we do not claim is a machine-learning model in the operational path. The conflict detection, complexity scoring and wind derivation described above are deterministic algorithms with published parameters, which is what makes them explainable to a regulator and reproducible in a safety review. When that changes, this page will say so.
This section needs your input before publication — see the notes accompanying this build.
Questions an engineer would ask?
We would rather have that conversation than write a longer page.