An application-fraud strategy desk over 1,000,000 loan applications: temporal model evaluation against promotion gates fixed before any result was seen, a capacity-bounded review queue wired to a PostgreSQL fraud schema, monthly KPI and vendor-performance reporting aggregated in SQL, and a daily suspect-application report from linking analysis.
Application Fraud Strategy Portfolio
Should we adopt the proposed fraud screening, and what does each option cost at the review capacity we actually staff?
No. At an identical 4,842 cases worked the proposed model catches 472 more fraud attempts and holds up 472 fewer good customers, and it is still refused, because two of eleven checks fixed before the result fail: its calibration intercept and its population stability. The interim position is a controlled temporary ranking baseline, seven role-assigned monitoring controls, and no automatic applicant decision.
A fraud strategy analyst can compare screening approaches at a fixed review capacity, see what each buys and costs, and take a recommendation or a refusal to governance. On the pre-agreed checks the honest output is a refusal: the proposed model catches 472 more fraud attempts while holding up 472 fewer good customers at identical cost, and is still not promoted, because its calibration and population stability fail checks written before the result was known.
Fraud Strategy Analyst
Refused to promote a fraud model that caught 472 more attempts while holding up 472 fewer good customers at identical review cost, because two of eleven checks fixed before the result failed, and documented the conditions that would withdraw the refusal.A fraud strategy analyst preparing a recommendation for a governance forum, and the non-technical fraud strategy owner who receives it.
Context and stakes
The decision context behind the system.
The proposed approach is better than the incumbent score proxy on both axes at identical cost, catching 472 more fraud attempts while holding up 472 fewer good customers. It is not recommended, because two of eleven checks written before the result was known fail. The interim baseline, monitoring controls, evidence gaps, and conditions for reopening the refusal are stated in advance.
An application-fraud strategy desk over a million loan applications that ends in a documented refusal to promote its own model, with the operating changes worth doing anyway.
Approach
A bounded approach shaped by the published constraints.
No. At an identical 4,842 cases worked the proposed model catches 472 more fraud attempts and holds up 472 fewer good customers, and it is still refused, because two of eleven checks fixed before the result fail: its calibration intercept and its population stability. The interim position is a controlled temporary ranking baseline, seven role-assigned monitoring controls, and no automatic applicant decision.
Design constraints
- BAF is privacy-preserving synthetic account-opening data, not observed lending performance
- Economic figures are declared sensitivity inputs rather than observed money, and neither factor could be sourced to this project's citation bar
- Identity linking is validated on a separate deterministic fixture; no BAF row is linked or described as part of a ring
- Labels are treated as complete at period close, which production labels are not, so recent periods understate fraud
Decision journey
The system moves through an inspectable sequence.
Each stage below comes from the published project manifest.
- Stage
Validate and curate 6,000,000
Checksum-validated ingestion to six typed Parquet artifacts, with raw and curated data excluded from the repository
- Stage
Fix promotion gates before
Temporal train, calibrate and evaluate with rolling-origin selection and one untouched test period
- Stage
Evaluate once on an
PostgreSQL fraud schema holding 1,000,000 applications, 2,000,000 scores, the ranked review queue and an analytics grain
- Stage
Wire the review queue
Monthly KPI and vendor performance aggregated in SQL from the schema, never recomputed in pandas
- Stage
Publish a monthly KPI
Precomputed policy grid of 1,152 combinations rendered by a static page with no server and no database
Interactive architecture
The published architecture, made interactive.
Rolling-origin selection before any access to the final period, one untouched test period, and eleven gates fixed at contract time. A five-fold walk-forward whose last fold reproduces the recorded protocol exactly, which validates the harness before its other numbers count. The review queue is wired to a PostgreSQL fraud schema and the monthly KPI grain is aggregated in SQL, so the reporting layer reads the system of record.
Drag to pan / use controls to zoom / select any stageRead the architecture as text
- Checksum-validated ingestion to sixStage 1 / Python
Checksum-validated ingestion to six typed Parquet artifacts, with raw and curated data excluded from the repository
- Temporal train, calibrate andStage 2 / SQL
Temporal train, calibrate and evaluate with rolling-origin selection and one untouched test period
- PostgreSQL fraud schema holdingStage 3 / PostgreSQL
PostgreSQL fraud schema holding 1,000,000 applications, 2,000,000 scores, the ranked review queue and an analytics grain
- Monthly KPI and vendorStage 4 / CatBoost
Monthly KPI and vendor performance aggregated in SQL from the schema, never recomputed in pandas
- Precomputed policy grid ofStage 5 / scikit-learn
Precomputed policy grid of 1,152 combinations rendered by a static page with no server and no database
- Governed analytics extract feedingStage 6 / Power BI
Governed analytics extract feeding a Power BI semantic model and measure library
Pivotal decisions
Technology choices recorded with their trade-offs.
These choices are derived from the project's published decision record.
Use Static precomputed page within the published boundary
All 1,152 policy combinations and governed controls fit in a 78 KB payload, so the stakeholder surface needs no server, no database and no hosting cost
- Constraints
- Precomputing 1,152 policy combinations removes the server, the database and the cost, and makes row-level filtering impossible, which is why the review queue lives in PostgreSQL instead
- Review capacity is held fixed in every comparison so no option is bought with extra headcount
- A score with ties cannot cut on an exact reviewer count, so the incumbent queue overruns staffing by 1.06% to 2.45% every period
- Alternatives
- A web application querying a database on each interaction
- Criteria
- Fit for the stated scope / Inspectable evidence / Explicit operating boundary
- Selected
- Static precomputed page
- Why
- All 1,152 policy combinations and governed controls fit in a 78 KB payload, so the stakeholder surface needs no server, no database and no hosting cost
- Trade-off
- No row-level filtering is possible on that surface, which is why the review queue lives in PostgreSQL instead
- What supported it
- 9 of 11 Pre-agreed promotion checks passed: The model is refused on the other two, which were fixed before any result was seen
- Revisit when
- The precomputed policy grid is combinatorial at models times capacities times 2^rules, 1,152 rows today and 18,432 with four more rules, so beyond that the database path replaces it
Use PostgreSQL fraud schema within the published boundary
Row-level work the static surface cannot do: the ranked queue, per-application scores, and a KPI grain aggregated in SQL rather than recomputed in pandas
- Constraints
- Precomputing 1,152 policy combinations removes the server, the database and the cost, and makes row-level filtering impossible, which is why the review queue lives in PostgreSQL instead
- Review capacity is held fixed in every comparison so no option is bought with extra headcount
- A score with ties cannot cut on an exact reviewer count, so the incumbent queue overruns staffing by 1.06% to 2.45% every period
- Alternatives
- Keeping every result in flat files
- Criteria
- Fit for the stated scope / Inspectable evidence / Explicit operating boundary
- Selected
- PostgreSQL fraud schema
- Why
- Row-level work the static surface cannot do: the ranked queue, per-application scores, and a KPI grain aggregated in SQL rather than recomputed in pandas
- Trade-off
- Adds a local dependency that the deployed demo deliberately does not require
- What supported it
- 53.6% vs 20.6% Catch rate at 5% review capacity: Proposed approach against the incumbent score proxy, at identical cost
- Revisit when
- An HTTP scoring service is designed and unbuilt; batch scoring against the schema already exists
Use CatBoost against a class-weighted logistic comparator within the published boundary
Gradient boosting handles the categorical inputs natively, and the linear comparator is what makes the simple-versus-complex gap measurable at 4.3 points of catch
- Constraints
- Precomputing 1,152 policy combinations removes the server, the database and the cost, and makes row-level filtering impossible, which is why the review queue lives in PostgreSQL instead
- Review capacity is held fixed in every comparison so no option is bought with extra headcount
- A score with ties cannot cut on an exact reviewer count, so the incumbent queue overruns staffing by 1.06% to 2.45% every period
- Alternatives
- Reporting only the challenger, as the project did before the comparator was found sitting at the base rate
- Criteria
- Fit for the stated scope / Inspectable evidence / Explicit operating boundary
- Selected
- CatBoost against a class-weighted logistic comparator
- Why
- Gradient boosting handles the categorical inputs natively, and the linear comparator is what makes the simple-versus-complex gap measurable at 4.3 points of catch
- Trade-off
- A broken baseline flatters everything measured against it, so the comparator now has its own regression test
- What supported it
- +425 to +519 Paired incremental fraud-catch interval: The fixed-capacity advantage remains positive under paired row resampling and in all five time-ordered folds
- Revisit when
- Label ingestion needs an append-only outcomes table, specified and deliberately not created until a producer writes to it
Use GitHub Pages with a stamped health endpoint within the published boundary
Zero cost for a static artifact, and a file stamped at deploy time is the static equivalent of a service health route
- Constraints
- Precomputing 1,152 policy combinations removes the server, the database and the cost, and makes row-level filtering impossible, which is why the review queue lives in PostgreSQL instead
- Review capacity is held fixed in every comparison so no option is bought with extra headcount
- A score with ties cannot cut on an exact reviewer count, so the incumbent queue overruns staffing by 1.06% to 2.45% every period
- Alternatives
- A container on Azure, whose Terraform is authored here but never applied
- Criteria
- Fit for the stated scope / Inspectable evidence / Explicit operating boundary
- Selected
- GitHub Pages with a stamped health endpoint
- Why
- Zero cost for a static artifact, and a file stamped at deploy time is the static equivalent of a service health route
- Trade-off
- Pages ignores the header file, so the policy is carried in the page and frame-ancestors is lost
- What supported it
- 2.39 points Catch cost of removing the highest-disparity feature: It still leaves two fairness triggers firing, so removal is a partial mitigation rather than a fix
- Revisit when
- Monitoring is tiered by observability rather than severity, because outcome signals lag 30 to 90 days
Results and evaluation
Measured results within the disclosed evaluation boundary.
- Method
- Gates evaluated once on an untouched period
- Meaning
- The model is refused on the other two, which were fixed before any result was seen
- Limitation
- BAF is privacy-preserving synthetic account-opening data, not observed lending performance
Limitations and next steps
Useful evidence, bounded claims.
What this does not prove
- BAF is privacy-preserving synthetic account-opening data, not observed lending performance
- Economic figures are declared sensitivity inputs rather than observed money, and neither factor could be sourced to this project's citation bar
- Identity linking is validated on a separate deterministic fixture; no BAF row is linked or described as part of a ring
- Labels are treated as complete at period close, which production labels are not, so recent periods understate fraud
- The Power BI layer has never been opened in Desktop, which is Windows-only, and no .pbix exists
What I would do next
- The precomputed policy grid is combinatorial at models times capacities times 2^rules, 1,152 rows today and 18,432 with four more rules, so beyond that the database path replaces it
- An HTTP scoring service is designed and unbuilt; batch scoring against the schema already exists
- Label ingestion needs an append-only outcomes table, specified and deliberately not created until a producer writes to it
- Monitoring is tiered by observability rather than severity, because outcome signals lag 30 to 90 days
The source is the public Bank Account Fraud suite: privacy-preserving synthetic account-opening records, not observed lending performance and not data about any real person, institution or portfolio. No confidential, client or personal data is involved, and no raw or curated record is deployed or committed. Nothing here approves, denies, prices or underwrites anything, and no automatic decline exists in the code or the database schema: the product ranks, explains and recommends, and a person decides. The recorded outcome is a refusal to promote the model evaluated.
Explore the rest of the work