CONSUMER LENDING / FINANCIAL CRIME
AnalyticsAI/ML
Application Fraud Strategy Portfolio
A fraud strategy analyst can compare screening approaches at a fixed review capacity, see what each buys and costs, and take a recommendation or a refusal to governance. On the pre-agreed checks the honest output is a refusal: the proposed model catches 472 more fraud attempts while holding up 472 fewer good customers at identical cost, and is still not promoted, because its calibration and population stability fail checks written before the result was known.
STAKEHOLDER VIEW
What this project is for.
- Problem
- An application-fraud strategy desk over a million loan applications that ends in a documented refusal to promote its own model, with the operating changes worth doing anyway.
- Intended user
- A fraud strategy analyst preparing a recommendation for a governance forum, and the non-technical fraud strategy owner who receives it.
- Decision supported
- Recommend or refuse one bounded application-fraud screening strategy for governance review.
- Outcome
- The proposed screening is better than what is in place on both axes at identical cost, catching 472 more fraud attempts while holding up 472 fewer good customers. It is not recommended, because two of eleven checks written before the result was known fail. The recommendation is four operating changes that need no promotion, and the conditions under which the refusal would be withdrawn are stated in advance.
- What to try
- Open the decision exhibit and read the verdict first: not approved for rollout, 9 of 11 pre-agreed checks passed
- Important limitation
- BAF is privacy-preserving synthetic account-opening data, not observed lending performance
Deciding whether to promote the proposed screening
- 01Open the decision exhibit and read the verdict first: not approved for rollout, 9 of 11 pre-agreed checks passed
- 02See what the proposal would buy at the capacity actually staffed: 472 more fraud attempts caught and 472 fewer good customers held up, for the same 4,842 cases worked
- 03Read the two checks that fail and what each means in plain language, without needing to know what a calibration intercept is
- 04Set the three economic assumptions and watch every money figure move as a range, because a review does not stop every fraud it finds
- 05Take the refusal, and the four operating changes that need no promotion, into the governance forum
Technical reviewArchitecture, evidence, controls, deployment, and trade-offs
ARCHITECTURE
One source, one rendered system view.
Rolling-origin selection before any access to the final period, one untouched test period, and eleven gates fixed at contract time. A five-fold walk-forward whose last fold reproduces the recorded protocol exactly, which validates the harness before its other numbers count. The review queue is wired to a PostgreSQL fraud schema and the monthly KPI grain is aggregated in SQL, so the reporting layer reads the system of record.
End-to-end flow
- 01Validate and curate 6,000,000 checksummed rows to typed Parquet
- 02Fix promotion gates before any result is seen, then select on rolling-origin folds
- 03Evaluate once on an untouched period and record the refusal rather than tuning toward a pass
- 04Wire the review queue to a PostgreSQL fraud schema and aggregate the KPI grain in SQL
- 05Publish a monthly KPI pack, a daily suspect report, and a strategy document for governance
Technology stack
PythonSQLPostgreSQLCatBoostscikit-learnPower BISAS (translations)
Technology decisions
| Decision | Why | Alternative | Trade-off |
|---|---|---|---|
| Static precomputed page | All 1,152 policy combinations fit in a 66 KB payload, so the stakeholder surface needs no server, no database and no hosting cost | A web application querying a database on each interaction | No row-level filtering is possible on that surface, which is why the review queue lives in PostgreSQL instead |
| PostgreSQL fraud schema | Row-level work the static surface cannot do: the ranked queue, per-application scores, and a KPI grain aggregated in SQL rather than recomputed in pandas | Keeping every result in flat files | Adds a local dependency that the deployed demo deliberately does not require |
| CatBoost against a class-weighted logistic comparator | Gradient boosting handles the categorical inputs natively, and the linear comparator is what makes the simple-versus-complex gap measurable at 4.3 points of catch | Reporting only the challenger, as the project did before the comparator was found sitting at the base rate | A broken baseline flatters everything measured against it, so the comparator now has its own regression test |
| GitHub Pages with a stamped health endpoint | Zero cost for a static artifact, and a file stamped at deploy time is the static equivalent of a service health route | A container on Azure, whose Terraform is authored here but never applied | Pages ignores the header file, so the policy is carried in the page and frame-ancestors is lost |
Evaluation and evidence
| Metric | Plain-language meaning | Score / value | Dataset / scenario | Threshold / baseline | Interpretation | Evidence |
|---|---|---|---|---|---|---|
| Pre-agreed promotion checks passed | What Pre-agreed promotion checks passed measures for the stated evaluation scope. | 9 of 11 | The model is refused on the other two, which were fixed before any result was seen | No threshold or baseline recorded. | BAF is privacy-preserving synthetic account-opening data, not observed lending performance | Evidence link not recorded |
| Catch rate at 5% review capacity | What Catch rate at 5% review capacity measures for the stated evaluation scope. | 53.6% vs 20.6% | Proposed screening against what is in place today, at identical cost | No threshold or baseline recorded. | BAF is privacy-preserving synthetic account-opening data, not observed lending performance | Evidence link not recorded |
| Honest next-period catch interval | What Honest next-period catch interval measures for the stated evaluation scope. | 45.9% to 59.1% | Replaces a single-period figure that understated the uncertainty a deployment decision faces | No threshold or baseline recorded. | BAF is privacy-preserving synthetic account-opening data, not observed lending performance | Evidence link not recorded |
| Catch cost of removing the highest-disparity feature | What Catch cost of removing the highest-disparity feature measures for the stated evaluation scope. | 2.39 points | It still leaves two fairness triggers firing, so removal is a partial mitigation rather than a fix | No threshold or baseline recorded. | BAF is privacy-preserving synthetic account-opening data, not observed lending performance | Evidence link not recorded |
| Unit and contract tests | What Unit and contract tests measures for the stated evaluation scope. | 84 | Including a regression for every defect found, such as a comparator that sat at the base rate for five milestones | No threshold or baseline recorded. | BAF is privacy-preserving synthetic account-opening data, not observed lending performance | Evidence link not recorded |
These are One untouched test period plus a five-period walk-forward, against gates fixed at contract time. results, not a production service-level objective. Unknown values are shown as “Not recorded”; units and special characters retain their source meaning.
Technical terms and value conventions
| Term | Plain-language use | How this project uses it |
|---|---|---|
| Evaluation score | A measured result from a defined test or benchmark, not a general promise. | Read it with the dataset, scenario, and limitation shown in the evaluation table. |
| Unknown / null | A value that was not observed or is intentionally not claimed. | Rendered as “Not recorded” rather than silently converted to zero or omitted. |
| Special characters | Symbols such as %, currency, Unicode, and escaped input retain their source meaning. | Units and encoding are explained where they affect the decision. |
Data boundary
| Classification | Source | Permitted use | Excluded data |
|---|---|---|---|
| public | Bank Account Fraud (BAF) suite, Feedzai, Jesus et al., NeurIPS 2022 Datasets and Benchmarks track, archive SHA-256 fb8d6d8b96f9a3c670a9ad0014da25ee8bee3fcb2be67bdcd115784bdb37cf8a | Non-commercial retrospective analysis, demonstration and public showcasing, with attribution | customer age, which is audit-only and never a model input; a constant device fraud count and a post-application timing field, both excluded by contract |
The source is the public Bank Account Fraud suite: privacy-preserving synthetic account-opening records, not observed lending performance and not data about any real person, institution or portfolio. No confidential, client or personal data is involved, and no raw or curated record is deployed or committed. Nothing here approves, denies, prices or underwrites anything, and no automatic decline exists in the code or the database schema: the product ranks, explains and recommends, and a person decides. The recorded outcome is a refusal to promote the model evaluated.
Security and privacy controls
| Control | Implementation | Evidence | Limitation |
|---|---|---|---|
| Public data boundary | customer age, which is audit-only and never a model input; a constant device fraud count and a post-application timing field, both excluded by contract | Evidence link not recorded | The source is the public Bank Account Fraud suite: privacy-preserving synthetic account-opening records, not observed lending performance and not data about any real person, institution or portfolio. No confidential, client or personal data is involved, and no raw or curated record is deployed or committed. Nothing here approves, denies, prices or underwrites anything, and no automatic decline exists in the code or the database schema: the product ranks, explains and recommends, and a person decides. The recorded outcome is a refusal to promote the model evaluated. |
Deployment and cost boundary
| Provider | Runtime | State | Exposure | Verified | Production claim |
|---|---|---|---|---|---|
| GitHub Pages | Static single page, 62 KB, with styles, script and above-the-fold facts inlined at build time and all 1,152 policy combinations precomputed into a 66 KB payload. No server, no request handler, no database, zero cost. | live | anonymous | 2026-08-10T16:34:00Z | No |
Known limitations
- BAF is privacy-preserving synthetic account-opening data, not observed lending performance
- Economic figures are declared sensitivity inputs rather than observed money, and neither factor could be sourced to this project's citation bar
- Identity linking is validated on a separate deterministic fixture; no BAF row is linked or described as part of a ring
- Labels are treated as complete at period close, which production labels are not, so recent periods understate fraud
- The Power BI layer has never been opened in Desktop, which is Windows-only, and no .pbix exists
Scalability roadmap
- The precomputed policy grid is combinatorial at models times capacities times 2^rules, 1,152 rows today and 18,432 with four more rules, so beyond that the database path replaces it
- An HTTP scoring service is designed and unbuilt; batch scoring against the schema already exists
- Label ingestion needs an append-only outcomes table, specified and deliberately not created until a producer writes to it
- Monitoring is tiered by observability rather than severity, because outcome signals lag 30 to 90 days