
Hundreds of millions of geospatial data points ingested
Overview
I built Vale to make fragmented European land data usable by software and AI agents, ingesting hundreds of millions of geospatial data points into a normalized land-intelligence system. A coordinate resolves to a normalized site report covering parcel identity, buildings, terrain, planning, designations and hazards where sources support them. Field-level evidence, coverage and limitations carry through to the answer layer.
My role: Founder and engineer, owning data architecture, geospatial pipelines, cloud operations, the MCP interface, grounded answers and evaluation design.
Stack: Python, PostgreSQL / PostGIS, AWS EC2 / S3, OpenAI API, structured outputs and MCP.
The problem
Land information is spread across authorities, formats and update cycles. A model can produce a fluent assessment while confusing missing data with a negative finding or treating nearby evidence as a parcel-wide fact. I needed useful evidence with boundaries the answer layer could preserve.
Data infrastructure at scale
I built ingestion pipelines spanning hundreds of millions of geospatial data points and heterogeneous parcel, planning, environmental, terrain and infrastructure sources. The complexity sits in turning those sources into a maintained spatial data product with stable meaning and inspectable evidence.
The implementation separates source-specific acquisition and normalization from customer queries. PostGIS stores normalized vectors, source metadata and immutable snapshot identity; larger source archives and terrain raster files stay in object storage. Spatial measurements use each source’s declared metric coordinate system, while building coverage requires clipping and combining footprints against the selected parcel.
Expanding coverage also requires operational control. Data is acquired and validated, normalized into staged snapshots, then activated through versioned geographic bundles with exact source membership and rollback metadata. Downloading a large dataset alone does not make it available to a customer.
These decisions connect the scale of ingestion to the reliability of an individual answer: the system must know which sources apply to a site, what they actually establish and which gaps remain.
From local pipelines to cloud operations
I moved Vale’s ingestion workflow from my laptop to a dedicated AWS worker, where a remote coding agent runs persistent, operator-directed jobs. Scoped access, versioned S3 recovery evidence and customer verification support repeatable publication. Selective refreshes reuse verified unchanged source snapshots while preserving provenance and rollback.
Each England run freezes its source inputs, checksums and engine version before staging. Checkpoint receipts record completed steps; after an interruption, recovery checks the active database revision before resuming. A missing receipt cannot safely be treated as a failed publication. Local payloads are eligible for cleanup only after customer verification and verified cloud backups.
Architecture
The CLI and six MCP tools use the same catalogue, coverage and query services. Source-specific ingestion and bounded providers feed PostGIS and object storage. One deterministic parcel lookup gives every analysis in a report the same site identity.
The normalized SiteReport sits between retrieval and generation:
- Resolve the coordinate and jurisdiction, then select a parcel deterministically.
- Retrieve supported fields from normalized data and bounded providers.
- Preserve source, analysis unit, effective date, confidence and coverage state for each field.
- Generate structured model outputs and use application validation to constrain claims to evidence and communicate uncertainty.
The CLI provides an inspection surface. MCP exposes the same contract to agents; ingestion remains separate from the customer query path.
Implementation boundaries
The core keeps interfaces, query logic and publication responsibilities separate:
src/eu_land_api/
├── cli.py / mcp.py # Thin interfaces
├── contracts.py # Reports, fields, evidence rules
├── queries/ # Retrieval and grounded answers
├── repository.py # Normalized PostGIS persistence
├── coverage.py # Activation and rollback
└── ingestion/ # Country-specific source pipelines
The CLI and MCP are thin interfaces over those shared services. Runtime query modules do not import ingestion, and MCP reads the data plane while publication remains an operator action. This keeps a customer query from becoming a data-maintenance job.
Decisions that mattered
Missing data is part of the contract
I kept five states distinct: found, null, not_covered, unsupported and unavailable. A source publishing no value differs from missing coverage, an unsupported field or a temporary provider failure. A covered zero or false remains a real value.
This gives the answer layer a concrete basis for explaining what is unknown instead of treating missing evidence as an absence of risk.
Evidence keeps its geographic scope
Fields retain their analysis unit and spatial relationship. Point evidence, administrative information and proximity observations cannot automatically become parcel-wide conclusions. Mapped-road distance provides context; it does not establish legal access.
Availability must reflect validated coverage
Downloaded or staged data does not count as customer availability. Versioned geographic bundles expose validated capabilities and source membership, with atomic activation and rollback. Country differences remain visible.
Evaluation and iteration
Release checks combine unit tests, PostGIS integration tests against a disposable database, and checks of the installed CLI and MCP tools. Comparing both interfaces against the same active data verifies that field states, source identities and customer results agree.
I built a 100-case evaluation suite across 25 English authorities, separating retrieval quality from answer grounding and uncertainty handling. This distinguished missing evidence from a model or validator mishandling evidence that was present.
Targeted prompt and validator changes improved the rubric pass rate from 62% to 89% and reduced false rejections from 18% to 3% on that evaluation set.
These are results from a defined internal test set, not estimates of accuracy across all locations or questions. The set’s size and geographic scope limit what the numbers establish. Broader coverage and new source types need their own validation.
What this demonstrates
Vale combines hands-on data engineering and cloud operations with model integration and failure analysis. The central design choice was to make evidence, coverage and uncertainty inspectable before asking a model to explain them. The evaluation loop then tested whether those distinctions survived into the answer.
The system supports England and France through concrete source integrations; fields vary by jurisdiction and active coverage. This implementation does not establish full European coverage or customer adoption.