From Chart Prototype to Governed Analysis Pipeline

The first version was not the final product

Many analytics tools are easy to prototype: load a table, choose a measure, group by a dimension, and draw a chart. That path is useful for exploring an idea, but it is not enough for operational data where a plausible answer can still be wrong.

LightBI has been evolving around that distinction. The goal is not only to render a visualization. The goal is to establish a trustworthy path from a raw source to a decision-support result.

Separate the responsibilities

The current architecture separates five concerns that are often mixed together:

• Intake records the physical source and its identity.
• Understanding profiles the source, resolves semantic candidates, reasons about grain and relationships, and computes readiness.
• Authorization exposes only the questions and metrics supported by the available evidence.
• Execution runs a guarded plan against the bound source through DuckDB.
• Presentation turns executed evidence into charts, decision briefs, Deep BA views, dashboards, and exports.

This separation matters because a chart component should not be allowed to decide that a column is revenue, or that two files can be joined, simply because the result looks useful.

Understanding is more than matching headers

A column called Qty may be stock, ordered quantity, sold quantity, or a count. A field called Status may describe delivery, payment, approval, or something else entirely. The semantic registry can produce candidates, but context, values, neighboring fields, type evidence, and row grain are needed before a meaning becomes trustworthy.

LightBI therefore distinguishes signal recognition from dataset understanding. It also distinguishes understanding from business interpretation and executable decision support. A large dictionary is not a license to advertise every possible domain.

Questions provide the user boundary

The Question-first direction gives the operator a usable entry point without requiring SQL or a full data model at the start. Questions are not free-form commands that bypass the engine. They are mapped to approved measures, dimensions, source identity, aggregation rules, and evidence scope.

If a source cannot support the requested analysis, the correct outcome is a visible limitation or a blocked action with a reason. A narrow truthful result is better than a broad result built on an unstated assumption.

Why execution fails closed

Before a query runs, LightBI checks source continuity, physical field availability, metric preflight, runtime policy, and the execution plan. A stale fingerprint, missing field, ambiguous measure, or unsafe relationship must prevent the pipeline from silently switching to another interpretation.

The same principle applies to multi-source work. Files keep their own identities and scopes. A combined analysis needs evidence for role, period, currency, relationship, and duplication risk before it can be authorized.

What this changes for the product

Simple Mode can remain approachable because the complexity is handled by contracts beneath the interface. Advanced Mode can expose schema, SQL, filters, imports, exports, and controlled writeback without creating a second definition of business truth.

LightBI is currently a Public Beta, so this pipeline is still being proven across corpus, holdout, dirty-data, and production scenarios. The important design decision is that every new feature must strengthen the shared understanding and evidence path instead of creating another independent shortcut.

The live distribution is available at lightbi.thaiduy.digital, with the web app at lightbi.thaiduy.digital/app. The implementation is published in the LightBI GitHub repository.

Technical references

The implementation and evidence behind this entry are documented in the LightBI README, the architecture overview, and the MVP decision record. Privacy boundaries are described in PRIVACY.md. The provider extension boundary is documented in the Provider Plugin SDK manual.