Fraud teams need to investigate alerts faster, but every decision must be explainable, auditable and, when it is risky, made by a human. This assistant does the groundwork of an investigation and hands the analyst a recommendation with cited policies, while code, not the model, decides what must go to a person.
Follow one alert through the system
An illustrative case on synthetic data. The stages and decisions are the ones the live API returns; the values are examples.
The flagged payment
Amount
£4,800
Customer's usual payment
£120
Payee
New, never paid before
Account age
21 days
Payments in last 24 hours
6
1
Risk rules
High risk. The amount is 40 times the usual payment, to a new payee, from a 21-day-old account.
2
Network check
The receiving account gets money from many unrelated senders, a common sign of a mule account.
3
Policy retrieval
Finds the policies on APP scam escalation and mule account indicators.
4
LLM recommendation
Request more information, citing both policies.
5
Guardrail overrides the LLM
The policy minimum for this case is escalation, so code changes the recommendation to escalate to a human, checks every citation is real and adds any required one the LLM left out.
6
Human review
Status: pending human review. The analyst gets the evidence, the cited policies and a full audit trail.
Also built in: a kill switch for the LLM, QA sampling of auto-cleared cases, and strict input validation.
How I run it
1Merge to mainEvery change starts the pipeline
2306 testsUnit, integration and safety rules
3Evaluation gateBlocks the release if quality drops
4Build imageContainer built by GitHub Actions
5DeployTo Azure Container Apps, with passwordless login
6Health checkConfirms the new version is up
7MonitorRequests, errors, latency and LLM cost, with email alerts
Fully automatic: nothing reaches Azure without passing the tests and the evaluation gate. Azure Monitor alerts me to error spikes and slow responses.
How I tested it
Development set
50 cases
Held-out sets
20 and 100 cases
Adversarial set
10 cases
Repeats
Each set run 3 times
I report the LLM's own errors separately from the full system's, with 95% confidence intervals.
Results on 100 held-out cases
0 of 40 missed
The LLM alone missed 2–3 of 40 required escalations across runs. The guardrail caught every one, so the system missed none.
About 9%
The 95% upper bound on the system's true miss rate. Zero misses on 40 cases doesn't prove zero risk, so I report the bound.
17 of 100
Cases sent to a human unnecessarily. I traced 12 to one pattern, the LLM escalating on transaction velocity alone, and 5 to a conservative routing rule.
1 policy
Every citation the LLM missed was the same one, on cross-border context. Code now adds it.
~2.3 s, ~$0.0009
Time and model cost per alert.
306 tests
Run with the evaluation gate on every merge, before anything deploys.
What I'm fixing next. The over-escalation and citation patterns, tuned on the development set, then checked on a fresh held-out set I haven't seen.
Policy retrieval, top-1 accuracy
Hashed word features1.7%
BM25 keyword search30%
OpenAI embeddings75%
Built with Python, FastAPI, LangGraph, OpenAI, RAG, scikit-learn, NetworkX, PostgreSQL and Docker. Run with GitHub Actions, Azure Container Apps and Azure Monitor.
Two more builds for fraud and compliance teams: catching scam payments early, and answering policy questions with sources.
APP Scam Risk Model
Personal project, built on synthetic data
In authorised push payment (APP) scams, the customer sends the money themselves, so passwords and security checks don't help. Since October 2024, UK banks share the cost of reimbursing victims, so spotting scam payments and mule accounts early matters.
I built a synthetic UK banking simulator with 500 customers and 10,000 transactions, injected APP scam and mule patterns, and trained an XGBoost model on a strict chronological split, with SHAP explanations for every flag and a Streamlit dashboard for investigators.
What I'm adding next. My first scores looked too good, so I audited the data and found the simulator made fraud too easy to separate. I'm rebuilding it with realistic overlap, comparing XGBoost against a simple baseline using PR-AUC and recall at a fixed alert rate, and adding graph features for mule-account chains and a test for changing scam patterns.
Built with Python, XGBoost, SHAP, pandas and Streamlit.
Teams in regulated industries need answers from long policy documents, but an answer without a reliable source can't be trusted.
I built a RAG pipeline that extracts documents with Docling, including tables and headings, compares two chunking strategies, stores them in ChromaDB, and asks gpt-4.1-mini to cite its sources or say when it doesn't know. It runs as a FastAPI service with tests, CI and Docker.
What I'm adding next. Strengthening citation accuracy with regression tests, building a labelled question set, and measuring retrieval accuracy, answer faithfulness and citation accuracy for each chunking strategy.
Built with Python, Docling, ChromaDB, OpenAI API, FastAPI and Docker.
The rules I work by, and where they show up in my projects.
Code, not the model, makes the risky call
The LLM recommends; deterministic code enforces the policy minimum and decides what goes to a human. On 100 held-out cases, that guardrail caught every escalation the LLM missed.
Measure before I trust it
I keep test cases aside that I never look at while building. When my APP scam model's first scores looked too good, I audited the data instead of publishing them.
Report failures, not just the headline
I show the model's own errors separately from the system's, and trace mistakes to their cause. 17 unnecessary escalations came down to two patterns, which I'm now fixing.
Compare against something simple
A score means little without something to compare it to. Policy retrieval went from 1.7% with hashed features to 30% with BM25 and 75% with embeddings.
Test the safety rules on every change
The rules that matter most run in CI. Every merge runs 306 tests and an evaluation gate, so a change that weakens the safety rules can't deploy.
Run it like a real service
A project counts when it runs, deploys and is watched without me. The fraud assistant deploys itself to Azure on every merge, and Azure Monitor alerts me when errors spike or responses slow down.
Experience and education
Software Engineer
Mar 2021 – Aug 2023
Atos, client: FedEx, Chennai
Worked with FedEx stakeholders to turn requirements into mapping and vehicle-routing features (ArcGIS API for JavaScript, HERE road-network data).
Migrated Node.js services to Python Flask microservices and REST APIs, including processing US street-network data from Esri file geodatabases (FGDB) into AWS S3.
Automated manual user-administration workflows, and supported production releases through testing, code reviews and CI/CD (Jenkins, Git) in an Agile team.