← All work
Working technical prototypeFinancial services

A review-first interface for natural-language analytics

Guarded Text-to-SQL

The business question

How can a data team offer natural-language exploration without letting model-generated structured query language (SQL) execute unchecked?

The answer

Separate model proposal from execution, validate every proposal with deterministic policy, and require human approval before a bounded read-only preview can run.

Demonstrated value

Turns generated SQL into a reviewable proposal and prevents direct model-to-database execution.

Focus

Applied AI engineer / Data engineer

I designed and built the proposal, policy, approval, evaluation, and bounded execution workflow.
Primary stakeholder

Data and analytics leader

Open working prototype View source

Context and stakes

The risky shortcut is also the obvious product idea.

Natural-language analytics lowers the barrier to data access. The tempting implementation, generate SQL and run it, also collapses model uncertainty, database access, and user intent into one unsafe step.

A plausible-looking query can expose restricted fields, reference a schema that does not exist, or return a technically valid but misleading result. The product needed to make review part of the workflow rather than a warning added afterward.

Approach

Move authority away from the model.

Treat the model as a proposal engine, not an execution authority. A deterministic policy layer and explicit human approval can preserve the speed of natural-language exploration while keeping database access bounded and inspectable.

Design constraints

  • The public experience uses a small, hand-authored synthetic payments fixture.
  • Identifier-like fields stay outside model context and result previews.
  • Every execution must remain read-only, resource-bounded, and explicitly approved.

Decision journey

A question becomes a proposal before it becomes a query.

This is the stakeholder workflow. The technical architecture comes later.

  1. Analyst

    Ask

    An analyst states a bounded business question in ordinary language.

  2. AI

    Propose

    Azure OpenAI receives a curated catalog and drafts SQL; it cannot execute it.

  3. Policy

    Validate

    SQLGlot and DuckDB EXPLAIN check structure, schema, functions, and resource limits.

  4. Human

    Review

    The analyst inspects SQL, assumptions, lineage, and policy feedback.

  5. Human

    Approve

    A five-minute, single-use approval binds to the exact SQL and data snapshot.

  6. Data

    Preview

    The service revalidates and returns no more than 100 read-only rows.

Product walkthrough

The interface makes the boundary visible.

These captures come from the working public prototype. They show the user journey as it exists, not a reconstructed mockup.

The live Text-to-SQL prototype showing example questions, the disclosed synthetic fixture, and separate ask and approval panels
1

The boundary is visible before the first question

The interface discloses the synthetic data, offers bounded examples, and separates asking from approval. Nothing executes automatically.

The live Text-to-SQL prototype showing generated SQL, policy checks, curated lineage, and an empty result preview awaiting approval
2

Proposal is a review state, not a result

Generated SQL appears with assumptions, policy checks, and curated lineage. The result panel stays empty until a person explicitly approves the exact proposal.

Interactive architecture

Generation stops at a policy and approval boundary.

The system keeps probabilistic generation on one side of a deterministic policy and approval boundary. Select a stage to see what it owns.

Drag to pan / use controls to zoom / select any stage
100%
questionallowed schemaproposal requestdraft SQLreviewable proposalsingle-use approvalbounded rowsQuestionAnalystBrowser interfaceOrchestrationProposal APIFastAPIData boundaryCurated catalogClassified schemaGenerationSQL proposalAzure OpenAI + Entra IDValidationPolicy engineSQLGlot + DuckDB EXPLAINHuman controlBound approvalSingle-use tokenExecutionRead-only previewDuckDB
Read the architecture as text
  1. AnalystQuestion / Browser interface

    States the question, reviews the proposed SQL, and owns the final approval decision.

  2. Proposal APIOrchestration / FastAPI

    Separates proposal creation from approval and execution endpoints.

  3. Curated catalogData boundary / Classified schema

    Exposes approved tables and fields while excluding identifier-like fields from model context.

  4. SQL proposalGeneration / Azure OpenAI + Entra ID

    Drafts SQL from the question and curated catalog. It has no database execution authority.

  5. Policy engineValidation / SQLGlot + DuckDB EXPLAIN

    Checks statement shape, schema, functions, identifiers, preview limits, and the database plan.

  6. Bound approvalHuman control / Single-use token

    Binds a five-minute approval to the exact SQL and active snapshot checksum.

  7. Read-only previewExecution / DuckDB

    Revalidates policy and the snapshot, then returns a bounded preview over synthetic data.

Technology used
PythonFastAPIDuckDBSQLGlotAzure OpenAIMicrosoft Entra IDAzure Container AppsDockerOpenTelemetry

Pivotal decisions

The choices that define the product boundary.

Each decision connects the goal, constraints, rejected alternatives, trade-off, and result.

Decision

Model proposes; policy decides what may proceed

Keep natural-language exploration useful without giving model output execution authority.

Constraints
  • SQL can be syntactically valid and still unsafe or semantically wrong.
  • The public schema and preview must exclude identifiers.
Alternatives
  • Execute model-generated SQL directly
  • Filter generated SQL with regular expressions
Criteria
Fail closed / Understand SQL structure / Explain rejection / Remain testable
Selected
SQLGlot structural policy followed by DuckDB EXPLAIN
Why
Parser-backed validation can reason about statements, tables, columns, functions, and projections before the database sees an executable request.
Trade-off
The policy deliberately rejects ambiguous or unsupported SQL and still cannot prove that a safe query answers the right business question.
What supported it
All 18 deterministic trusted and refused cases matched their expected policy outcome.
Revisit when
Expand the policy when reviewer-approved equivalent SQL and a broader semantic benchmark are available.
Decision

Approval binds to an exact proposal

Make human review a real control rather than a cosmetic confirmation button.

Constraints
  • SQL or data can change between review and execution.
  • The public prototype must stay inexpensive and bounded.
Alternatives
  • Approve a session
  • Approve a natural-language question
  • Automatically run every validated query
Criteria
Single use / Short lived / Tamper evident / Simple enough for a one-replica prototype
Selected
Five-minute, single-use approval bound to SQL and snapshot checksum
Why
Execution revalidates both policy and the active data boundary instead of trusting earlier state.
Trade-off
Ephemeral SQLite is appropriate for the prototype but does not provide cross-replica or restart-safe consumption.
What supported it
An authenticated local lifecycle generated, approved once, revalidated, and executed a bounded aggregate proposal.
Revisit when
Move approval state to a transactional store before protected multi-replica hosting.
Decision

Disclose a small synthetic fixture

Let reviewers inspect a real workflow without placing personal or source account data in the model or public demo.

Constraints
  • No real customer data
  • No identifier-like fields in model context
  • No raw or approved source snapshot in the image
Alternatives
  • Ship a large approved snapshot
  • Use a private demo with hidden data
  • Show a nonfunctional interface mockup
Criteria
Publicly understandable / Reproducible / Low risk / Enough data to exercise the workflow
Selected
Hand-authored synthetic payments fixture with a curated semantic catalog
Why
The compact fixture makes the boundary legible and keeps the working product honest about what it demonstrates.
Trade-off
The demo cannot establish behavior over enterprise-scale, sensitive, or highly varied schemas.
What supported it
The public interface discloses the active fixture and excludes source account identifiers from context and previews.
Revisit when
Add protected tenant data only after caller authorization, durable controls, and an approved data contract exist.

Results and evaluation

A testable boundary, not a broad claim.

18 / 18 expected outcomesdeterministic cases matched their expected trusted or refused outcome
Method
A generated evaluation report runs trusted and refused cases against the committed synthetic fixture and deterministic SQL policy.
Meaning
Within that fixture and case set, the workflow accepted or refused each proposal as expected. It demonstrates that the product boundary is testable, not merely described.
Limitation
The bounded cases do not establish open-ended natural-language correctness, answer accuracy, enterprise-schema coverage, or production readiness.

Limitations and next steps

The prototype is useful because its limits are explicit.

What this does not prove

  • The public endpoint is anonymous and has no caller authorization.
  • The model can still produce semantically wrong SQL, so human review remains mandatory.
  • Approval state and rate limits are process-local and reset after restart or scale-to-zero.
  • The small synthetic fixture does not represent enterprise data complexity or sensitivity.

What I would do next

  1. Add single-tenant Microsoft Entra authentication and prove unauthenticated denial.
  2. Move approval and rate state to durable distributed controls before multi-replica hosting.
  3. Expand evaluation with reviewer-approved equivalent SQL and business-answer checks.

Next: see how the same evidence-first judgment changes retrieval and investigation systems.

Explore the rest of the work