Which sites received the suspect batch, and which customers have it now? The answer exists, spread across the ERP, the manufacturing system, the quality register and the CRM, under four different part numbers. Someone opens a ticket. Three weeks later a spreadsheet arrives, reconciled by hand, and the batch has already shipped. At the next steering committee, someone proposes a data warehouse.

A warehouse is a larger pilot

A pilot copies part of the business once and queries the copy. A warehouse copies the business continuously, at ten times the cost. Both rest on the same foundation: a copy.[1]

The programme always runs the same way. A year of pipelines. A schema frozen in month three by a team that never handled the part. A reconciliation layer, then a team dedicated to maintaining it. At the end, the original question is still open, because the warehouse holds what was loaded, in a shape chosen before the question was asked.

IBM describes the problem well: data scattered across systems, applications, clouds and documents, with no single source of truth, and out of date by the time it is used. Among the remedies, it lists consolidation into a warehouse or a lake through ETL pipelines.[2] The diagnosis is right. The remedy adds one more system, one more format and one more version of the part number.

Every format is a border

Within a single organisation, every proprietary format adds friction, delay and reconciliation work.[3] Data does not flow. It is extracted, transformed, degraded, reconciled at quarter-end, and in the end trusted by no one.

A warehouse does not remove those borders. It builds a central one and routes everything through it. The four systems still disagree about the part. The disagreement has simply moved into a nightly job, where it fails silently.

The ticket between question and answer

The people who understand the work cannot build the pipeline, and the people who build it do not understand the work. Between them sits a ticket queue.[4] The quality engineer knows what “batch” means on the shop floor. She writes a request. The request becomes a specification, then a pipeline, validated by people who never had the original knowledge. Meaning is lost at each step, and nobody sees the whole chain.

The real cost is time: days to establish which system is right about a single part number, and another week to get management to accept the answer.[5] The organisation that shortens that interval wins.[6] Your competitors hire the same consultants. The advantage goes to the organisation that can answer a question about itself as fast as it can ask it.

Query the data where it is

The alternative has existed for a long time. Data virtualisation lets an application query sources in place: the data stays where it is, and access goes to the source system rather than to a copy.[7] Nothing is extracted, so nothing goes stale. The limits are known: every source must be reachable, and the approach keeps no historical snapshots.[7:1]

But a federated query across four systems that name the same part four different ways just returns four answers faster. It reconciles nothing, because reconciliation is a vocabulary problem, not a data problem. An organisation with no explicit model of itself cannot see what it has no concept for.[8] That model has to be defined first, in the words of the people who do the work, not written up as documentation after the data is loaded.[8:1]

Beyond the query, three things are needed: an explicit model of what the part is across all four systems, a path from every figure back to its source row, and a named person responsible for the decision the answer leads to.

Our position

Galahad does not migrate your data or build a copy of your business to query. Monarch runs inside your infrastructure, under your accounts and your jurisdiction, and reads across the systems you already run.[9] A copy is stale by design, it becomes a fifth system with a fifth version of the truth, and if it leaves your network it leaves your jurisdiction.

Questions to ask a vendor

  1. Does your product need my data moved, copied or indexed before it can answer? Where does the copy live, and how old is it?
  2. When I ask about a part number, which system’s version do I get, and who decided that?
  3. Can the person who knows the business change the vocabulary, or does that take a ticket?
  4. If one source is unreachable, does the answer say so?
  5. Who owns the model of my business, and do I keep it if I leave?

Monarch reads across the systems you already run, inside your infrastructure. Nothing is migrated. See it on your data.

  1. Machines of Consequence, thesis 3. Read it ↩︎

  2. IBM, What is data fragmentation?: data scattered across systems, applications, clouds, databases and documents; stale by the time it reaches its downstream use; consolidation into a warehouse or lake through ETL/ELT pipelines among the remedies. Source ↩︎

  3. Machines of Consequence, thesis 6. Read it ↩︎

  4. Machines of Consequence, thesis 7. Read it ↩︎

  5. Machines of Consequence, thesis 13. Read it ↩︎

  6. Machines of Consequence, thesis 14. Read it ↩︎

  7. Wikipedia, Data virtualization: the data remains in place and real-time access is given to the source system, unlike ETL; among the drawbacks, every source must be reachable and the approach is not suited to historic snapshots. Source ↩︎ ↩︎

  8. Machines of Consequence, thesis 15. Read it ↩︎ ↩︎

  9. Galahad, the company: Monarch runs in your infrastructure, under your accounts, your keys and your jurisdiction; Galahad hosts none of your data. Read it ↩︎