Every response includes where the data came from (lineage), how fresh it is (freshness), and who asked (audit). Schema changes don't break agents — the Gold layer acts as a buffer. Per-agent access control, per-UC scoping, and row-level data isolation are built in.
Ask "what breaks if I change this Silver table?" and get an instant answer. Trace any column back to its source. See the full Silver → Gold → UC dependency graph. REST has none of this.
20 UCs consolidated into 7 Gold views (measured). Scheduled refresh (primary mode) checks TTLs and rebuilds only stale views. Real-time mode (optional, Postgres LISTEN/NOTIFY) available for latency-critical UCs. 49% less storage. REST refreshes everything blindly. Supports incremental adoption — connect to existing Silver (Mode 1) or manage the full pipeline with SQLMesh (Mode 2).
REST: design Gold table + write endpoint + write model + write tests (4 files). CogniMesh: write a 12-line JSON definition. System derives everything else. Unsupported questions get T2 answers, not 404s.
These terms are used throughout this report. If a concept is unfamiliar, refer back here.
| Term | What It Means |
|---|---|
| Bronze Layer | Raw ingested data. No transformations applied. Direct copy from source systems. |
| Silver Layer | Cleaned and enriched data. Joins applied, business logic added, duplicates removed. |
| Gold Layer | Pre-aggregated, query-optimized views built for specific use cases. What agents read from. |
| UC (Use Case) | A specific question an agent can ask. E.g., "What is the health status of customer X?" |
| SLOC | Source Lines of Code -- meaningful lines excluding blanks and comments. |
| LOC | Lines of Code -- total lines including blanks and comments. |
| T0 | Pre-materialized Gold query. Fastest tier. Direct SELECT from a Gold table. |
| T1 | Composed from existing Gold views. Joins across UCs. Slightly slower. |
| T2 | Dynamic query against Silver layer. Query Composer reads metadata, composes SQL. Guardrails enforced. |
| T3 | Reject with explanation. Question cannot be safely answered. Returns ETA for when UC might be available. |
| Lineage | Column-level source mapping. For each Gold column, traces back to the Silver/Bronze source. |
| Freshness | How recently data was refreshed. Measured as age (seconds since last refresh) vs TTL (max allowed staleness). |
| TTL | Time To Live. The maximum allowed age for a Gold table before it is considered stale. |
| Audit Trail | Immutable log of every query: who asked, what UC, which tier, latency, row count, cost. |
| MCP | Model Context Protocol. A standard interface for AI agents to discover and consume data tools. CogniMesh implements an MCP server with 6 tools (query, discover, check_drift, refresh, impact_analysis, provenance) alongside its REST API. |
Both approaches operate on the same synthetic e-commerce dataset. Three medallion layers, nine tables, 220,500 total rows. The data is realistic: 12 months of orders, 8 product categories, 5 customer regions.
ID, name, email, signup date, region (NA / EMEA / APAC / LATAM / MEA)
ID, name, category (8 types), price, supplier
Customer -> product, amount, status, timestamp (12 months of history)
+ total_orders, total_spend, days_since_last_order, ltv_segment
+ units_sold_30d, revenue_30d, return_rate, stock_status
+ customer_region, product_category, amount_usd
health_status (healthy / warning / critical) computed from recency + LTV
Products ranked by revenue within each category
Customers likely to churn (risk_score 0-99)
Individual lookup. Returns one row per customer with computed health status.
| Field | Source | How Computed |
|---|---|---|
| customer_id | silver.customer_profiles | Passthrough |
| name | silver.customer_profiles | Passthrough |
| total_orders | silver.customer_profiles | COUNT from orders |
| total_spend | silver.customer_profiles | SUM from orders |
| days_since_last | silver.customer_profiles | NOW() - last_order_date |
| health_status | Derived | CASE: <30d + high LTV = healthy, >90d = critical, else warning |
Bulk query. Returns all products in a category ranked by revenue.
| Field | Source | How Computed |
|---|---|---|
| product_id | silver.product_metrics | Passthrough |
| name | silver.product_metrics | Passthrough |
| category | silver.product_metrics | Passthrough |
| revenue_30d | silver.product_metrics | SUM from orders last 30d |
| units_sold_30d | silver.product_metrics | COUNT from orders last 30d |
| rank | Derived | ROW_NUMBER() OVER (PARTITION BY category ORDER BY revenue_30d DESC) |
Bulk query. Returns customers sorted by churn risk score (0-99, higher = more risk).
| Field | Source | How Computed |
|---|---|---|
| customer_id | silver.customer_profiles | Passthrough |
| name | silver.customer_profiles | Passthrough |
| days_since_last | silver.customer_profiles | NOW() - last_order_date |
| ltv_segment | silver.customer_profiles | Derived from total_spend |
| risk_score | Derived | (days_since_last / 365) * 50 + LTV factor |
That's it. No lineage. No freshness. No audit. Just raw data.
Each use case requires a hand-written endpoint, a response model, and a Gold table with hand-written SQL. The developer designs everything. The system executes it.
Every query passes through six stages. Audit logging (step 7) runs async after the response is returned.
Every query follows this exact seven-step pipeline. Nothing is optional. The system is always observable.
uc_id + params. Gateway validates schema and extracts agent identity.SELECT * FROM gold_cognimesh.customer_health WHERE customer_id = $1. Pre-materialized, indexed, no joins.cognimesh_internal.lineage. Map each Gold column to its Silver source. Attach column-level provenance to response.cognimesh_internal.freshness. Compute age in seconds. Compare to TTL. Flag stale data.QueryResult with data + lineage + freshness + tier. Agent has full context to explain the answer.audit_log: UC-01, T0, 3.2ms, 1 row, cost=1.001. Immutable record.When the question does not match any registered UC, the system falls back to composing a dynamic query against the Silver layer.
Both approaches respond in under 6ms — the latency difference is the cost of governance and is negligible in real-world agent pipelines.
| Use Case | REST Mean | CogniMesh Mean | Overhead | What the Overhead Buys |
|---|---|---|---|---|
| UC-01: Customer Health | 2.77 ms | 3.38 ms | +0.61 ms | Lineage + freshness on every query |
| UC-02: Top Products | 1.40 ms | 3.31 ms | +1.91 ms | Same + larger lineage for 8 columns |
| UC-03: At-Risk Customers | 2.15 ms | 5.48 ms | +3.33 ms | Same + more rows -> larger lineage payload |
The ~2ms overhead covers lineage lookup and freshness check on every query. Audit logging runs async in a background thread and adds zero latency. Measured with pytest-benchmark (50+ iterations, 5 rounds, FastAPI TestClient in-process).
These are the things that a production data serving layer needs. REST doesn't include any of them — you'd have to build each one yourself (weeks of work). CogniMesh includes all of them from the first query.
Every property below was tested with automated assertions. Not opinions -- binary pass/fail tests in the benchmark suite.
| # | Property | What It Means | REST | CogniMesh | How Tested |
|---|---|---|---|---|---|
| 1 | Discovery | Agent can ask 'what questions can you answer?' and get a list | NO | YES | GET /discover returns UC list with descriptions |
| 2 | Lineage | Every response shows exactly which database table and column each value came from | NO | YES | Check lineage field in response body |
| 3 | Audit Trail | Every query is logged: who asked, what they asked, when, how long it took | NO | YES | Check audit_log table after query |
| 4 | Cost Attribution | You can see how much each use case costs and which agent uses it most | NO | YES | SUM(cost_units) GROUP BY uc_id |
| 5 | Change Governance | When someone changes a use case definition, the before/after is recorded | NO | YES | Check uc_change_log table |
| 6 | Freshness | The response tells you how old the data is and whether it's past its expiry | NO | YES | Check freshness field in response |
| 7 | Tiered Fallback | When an agent asks something new, the system tries to compose an answer from existing data instead of returning an error | 404 | T2/T3 | POST unsupported question, check response tier |
| 8 | Schema Drift | When upstream data tables change, your agents keep working — the Gold layer acts as a buffer | 500 | Isolated | Rename column, query both systems |
| 9 | Impact Analysis | Ask "what breaks if I change this Silver table?" and get an instant answer with affected Gold views and UCs | NO | YES | GET /dependencies/impact |
| 10 | Provenance | Trace any Gold column back to its Silver source table, column, and transformation | NO | YES | GET /dependencies/provenance |
| 11 | Smart Refresh | Only refresh Gold views whose Silver source actually changed — not all of them | NO | YES | POST /refresh/scheduled (primary), POST /refresh/check (legacy) |
| 12 | Access Control | Per-agent scoping — which UCs each agent can access, row-level data isolation, role-based UC management | NO | YES | agent_id in audit, UC scoping, approval queue |
Three failure modes that every production data system will eventually face. Each scenario was tested end-to-end in the benchmark suite.
Silver column ltv_segment is renamed to lifetime_value_tier. What happens?
Gold refresh SQL references old column name. SQL fails. Endpoint returns stale data or crashes entirely.
Recovery: Developer discovers broken SQL (when? maybe hours, maybe a user reports it). Fixes the query, rebuilds Gold table, redeploys. Hours to days.
Gold table was materialized before the rename. Still has valid data. Queries continue serving from Gold (T0). On next refresh, drift is detected and logged.
Recovery: Update UC derivation SQL, re-register, refresh. Agent never sees the error. Automated.
Agent asks: "What is the total revenue by region for the last quarter?" No UC exists for this.
No endpoint exists. Response: 404 Not Found. No alternatives. No explanation. The agent is stuck.
To fix: Design Gold table, write endpoint, write response model, write tests, deploy. Days of work.
Gateway fails UC match. Query Composer reads Silver metadata. Matches 'revenue' -> amount_usd, 'region' -> customer_region. Composes SQL with SUM + GROUP BY + time filter. Checks guardrails. Returns actual data as T2 with composed SQL in metadata.
Gold table was refreshed 6 hours ago. TTL is 4 hours. Is the data still valid?
Response is the same whether data is 1 minute or 1 week old. No freshness field. Agent cannot know. Could be dangerously stale. No way to detect, no way to warn.
Response includes: freshness: {is_stale: true, age_seconds: 10, ttl_seconds: 1}. Agent decides what to do. Audit log records staleness event. Dashboard shows freshness compliance over time.
Total code written to build each system from scratch for the same 3 use cases.
| Metric | REST API | CogniMesh | Notes |
|---|---|---|---|
| Files | 9 | 17 | CogniMesh includes the full platform |
| Python SLOC | 227 | 1,919 | CogniMesh includes registry, gateway, lineage, audit |
| SQL SLOC | 59 | 0 | REST has hand-written Gold SQL |
| JSON SLOC | 0 | 33 | CogniMesh UC definitions are JSON |
| Total SLOC | 286 | 1,952 | One-time platform investment |
What does it cost to add one more use case? UC-04: "Total revenue by region for the last quarter."
Plus: modify app.py to register the new router.
That's it. Register -> system derives Gold table, lineage, freshness tracking, audit, discovery entry. Everything automatic.
| UC Count | REST Total SLOC | CogniMesh Total SLOC |
|---|---|---|
| 3 (initial) | 286 | 1,952 |
| 4 | 364 | 1,964 |
| 10 | 832 | 2,036 |
| 25 | 2,002 | 2,216 |
| 50 | 3,952 | 2,516 |
REST creates 1 Gold table per UC, always. Even when UC-01 (Customer Health) and UC-03 (At-Risk Customers) both pull from the same silver.customer_profiles, REST creates two separate Gold tables with overlapping columns. At 10 UCs, you have 10 independent Gold tables with 45 overlapping columns. Each table stores its own copy of shared data, each requires its own refresh cycle, and each competes for database cache space.
CogniMesh's capability index detects field overlap at UC registration time. It groups UCs by Silver source, takes the field union, and produces consolidated Gold views. UC-01, UC-03, UC-05, UC-09 all pull from silver.customer_profiles — consolidated into one customer_360 Gold view serving all 4 UCs. The result: fewer tables, less storage, fewer refresh cycles, and better cache utilization.
| UC | Question | Silver Source | REST Gold Table | CogniMesh Gold View |
|---|---|---|---|---|
| UC-01 | Customer Health | customer_profiles |
customer_health | customer_360 |
| UC-02 | Top Products | product_metrics |
top_products | product_catalog |
| UC-03 | At-Risk Customers | customer_profiles |
at_risk | customer_360 |
| UC-04 | Revenue by Region | orders_enriched |
revenue_region | order_analytics |
| UC-05 | Customer LTV | customer_profiles |
customer_ltv | customer_360 |
| UC-06 | Purchase History | orders + profiles |
purchases | customer_orders |
| UC-07 | Regional Distribution | customer_profiles |
regional_dist | regional_summary |
| UC-08 | Product Trends | products + orders |
product_trends | product_catalog |
| UC-09 | Customer Segments | customer_profiles |
segments | customer_360 |
| UC-10 | Order Volume | orders_enriched |
order_volume | order_analytics |
| UC Count | REST Gold Tables | CogniMesh Gold Views | Consolidation Ratio | REST Refresh (ms) | CogniMesh Refresh (ms) |
|---|---|---|---|---|---|
| 3 (measured) | 3 | 3 | 1.00 | 360 | 360 |
| 5 | 5 | 4 | 0.80 | 600 | 480 |
| 10 | 10 | 5 | 0.50 | 1,200 | 600 |
| 20 (measured) | 20 | 7 | 0.35 | 2,400 | 840 |
| 25 | 25 | 8 | 0.32 | 3,000 | 960 |
| 50 | 50 | 12 | 0.24 | 6,000 | 1,440 |
| Dimension | CogniMesh Wins At | Why |
|---|---|---|
| Marginal dev hours per UC | UC = 1 (always) | 12 SLOC JSON vs 78 SLOC code — 15% effort from day one |
| Governance & observability | UC = 1 (always) | 12/12 properties built-in: discovery, lineage, audit, cost, governance, freshness, fallback, drift, impact analysis, provenance, smart refresh, access control |
| Unsupported question handling | UC = 1 (always) | T2 fallback vs 404 — CogniMesh never hard-fails on new questions |
| Gold table count | UC = 5 | First Silver source overlap triggers consolidation |
| Total refresh time | UC = 5 | Fewer consolidated views = fewer refresh cycles |
| Storage cost | UC = 5 | Consolidated views eliminate duplicate rows across Gold tables |
| Total maintenance SLOC | UC = 22 | REST: 286 + (n-3) × 78 overtakes CogniMesh: 1,952 + (n-3) × 12 |
| Query latency (T0) | Equivalent (always) | Equivalent — both under 6ms (see Performance) |
| ALL dimensions | UC = 5 | CogniMesh wins storage, table count, refresh, dev hours, governance. Latency is equivalent. |
| UC Count | REST T0 Latency | CogniMesh T0 Latency | Source | Notes |
|---|---|---|---|---|
| 3 (measured) | 2.77 ms | 3.38 ms | Benchmark run 2 | Equivalent (see Performance) |
| 20 (measured) | 2.17 ms avg | 3.74 ms avg | Scale benchmark | Equivalent — latency unchanged at scale |
CogniMesh's audit log records every T2 hit — questions answered from Silver because no Gold view exists. When a pattern reaches a threshold (e.g., 10 hits in 7 days), the system generates a UC candidate. A human approves, the Gold view is updated, and the next query is T0 (instant). REST has no equivalent — the agent gets 404s until someone notices and builds an endpoint.
| Metric | CogniMesh | REST |
|---|---|---|
| Time to first answer | Immediate (T2) | 2-5 business days |
| Time to optimized answer | Hours (after approval) | Same 2-5 days |
| Code changes | 0 | 4 files, 78 SLOC |
| Agent downtime | 0 seconds | 2-5 business days of 404s |
| Pattern detection | Automatic (audit log) | Manual (someone has to notice) |
| Feedback loop | Closed (usage → optimization) | Open (requires human initiative) |
CogniMesh knows the full dependency graph — which Silver tables feed which Gold views, which Gold views serve which UCs. REST has no awareness of dependencies. This enables four capabilities that REST cannot replicate.
Before changing a Silver table, ask CogniMesh what would be affected:
REST equivalent: nobody knows. A developer has to manually trace which Gold tables reference which Silver columns. If they miss one, it breaks silently.
Trace any Gold column back to its source:
REST equivalent: read the SQL source code. No API, no metadata, no runtime query.
A scheduled job runs all 20 Gold table refresh queries, whether the underlying data changed or not. If Silver hasn't changed, you wasted compute. If only customer data changed, you still refreshed product tables.
CogniMesh supports two refresh modes. Scheduled (primary): a periodic job checks all Gold views, refreshes stale ones, and reports changes — called via API (POST /refresh/scheduled), cron, or Airflow. Real-time (optional): Postgres LISTEN/NOTIFY triggers detect Silver changes and refresh affected views immediately. Most UCs use scheduled refresh. Real-time is for latency-critical use cases like fraud detection. Either way, CogniMesh checks the dependency graph: silver.customer_profiles changed → refresh customer_360 (1 view serving 10 UCs). Leave product_catalog and order_analytics alone. Result: 1 refresh instead of 20.
| Layer | Count | Details |
|---|---|---|
| Silver tables | 3 | customer_profiles, product_metrics, orders_enriched |
| Gold views (CogniMesh) | 7 | Consolidated from 20 UCs (ratio: 0.35) |
| Gold tables (REST) | 20 | One per UC, no consolidation |
| Use Cases | 20 | All served through the same gateway |
When silver.customer_profiles changes:
| Endpoint | What It Does | REST Equivalent |
|---|---|---|
GET /dependencies | Full Silver → Gold → UC graph with consolidation ratio | NONE |
GET /dependencies/impact | What breaks if this Silver table changes? | NONE |
GET /dependencies/provenance | Where does this Gold column come from? | NONE |
GET /dependencies/what-if | Change impact estimation with affected UCs | NONE |
GET /refresh/status | Freshness of all Gold views with served UC count | NONE |
POST /refresh/check | Auto-refresh only stale views | NONE |
GET /refresh/plan | Preview what would be refreshed | NONE |
REST is ~2ms faster per query. In practice, both are under 6ms and the difference is negligible.
286 SLOC vs 1,952. If you need 1-3 UCs and will never add more, REST is simpler.
Every developer knows REST. CogniMesh introduces new concepts (UCs, tiers, lineage). Learning curve is real.
REST is thinner: just FastAPI + Postgres. CogniMesh maintains capability index, writes audit logs, tracks freshness. More moving parts.
| Dimension | Winner | Evidence |
|---|---|---|
| Raw T0 latency | Equivalent | Equivalent |
| System properties | CogniMesh | 12/12 vs 0/12 |
| Schema drift | CogniMesh | Gold isolation vs SQL error |
| Unsupported questions | CogniMesh | T2 fallback vs 404 |
| Freshness awareness | CogniMesh | Built-in vs absent |
| Marginal cost per UC | CogniMesh | 12 SLOC vs 78 SLOC |
| Initial simplicity | REST | 286 SLOC vs 1,952 SLOC |
| Gold table consolidation | CogniMesh | 5 views vs 10 tables at UC=10 |
| Refresh cost at scale | CogniMesh | 960ms vs 3,000ms at UC=25 |
| Latency at scale (measured, UC=20) | Equivalent | Equivalent at all measured scales |
| Self-improving Gold layer | CogniMesh | T2 patterns auto-promote; REST needs manual endpoint work |
| Dependency intelligence | CogniMesh | Impact analysis, provenance, smart refresh — REST has none |
| Access control | CogniMesh | Agent scoping, per-UC permissions, row-level isolation |