Sandbox profile
Branch procedure answers
Low intensityBranch staff ask the procedure manual in plain language. Customer account numbers are not a training feature, and the model does not approve a transaction.
- Vertical
- Fintech and banking
- Scale
- Small company
- Organization
- River & Pine Bank
- Shape
- Community bank, about $1.2B in assets, 11 branches.
The business problem
The procedure manual is long, the search is literal, and people paste questions that include an account number into whatever chat box is handy. The bank wants answers from the manual only, on a host in its data center, with a redaction step that refuses to send an account number to the model log in the clear.
What Grae SI recommends
Family
Qwen 2.5 1.5B or Phi-4 mini
Size
About 1.5B–3B, CPU or a single small GPU.
Why this size
This is grounded Q&A over a few hundred pages of procedures. A small model with retrieval is enough. A larger model increases the chance it answers from memory instead of the manual. No fine-tune until retrieval misses a target the ops team cares about.
License
Accepted by the bank’s vendor-management person before install. Recorded on the card.
Data controls
Customer data
The manual should not contain customer data. Staff questions might. Account numbers, SSNs, and card numbers are redacted before the prompt is stored.
De-identification
There is no customer-level training set in the first release. If a later fine-tune uses tickets, those tickets are stripped of direct identifiers first.
Access
Branch role and operations. The model does not query core banking.
Residency
The data center the bank already uses for core adjacent systems. No SaaS prompt log.
Retention
Questions kept 30 days for abuse review, with identifiers redacted, then deleted. The manual version is kept with the model version.
Risk classification
Intensity is a planning tier for evidence. It is not a certification and not a regulatory classification.
- The corpus is the procedure manual. The model does not see the core, does not recommend a credit decision, and does not move money.
- The tier stays low only while those limits hold. A question that arrives with a full customer profile is rejected, not answered.
- If operations later asks the model to suggest a SAR or a credit exception, re-profile it. That use is high.
Compliance requirements to prepare for
FFIEC guidance and GLBA
The control is the same one the bank already applies to internal systems that could see customer data: access, logging, and vendor oversight. This host is internal, so the vendor file is about the base-model license, not a data processor.
SR 11-7 style model risk
A procedure chatbot is at the light end of model risk if it cannot affect a customer outcome. The bank’s model-risk policy may still want a short approval. Grae SI’s low-tier report is that short packet.
SOC 2 or ISO/IEC 27001
As the bank’s own program already defines. Not a new certification.
UDAAP
Out of scope while the model never talks to a customer. Put that in the intended use so a later chatbot on the website is a new review.
Grae SI helps assemble the evidence. It does not certify HIPAA, FDA, ISO, SOC 2, ICH, NAIC, GDPR, DPDP, or any other framework.
Validation KPIs
These are targets to lock before go-live. They are not measured outcomes from a customer deployment.
Manual grounding
At least 90% of answers quote a current manual section on a 80-question set
Ungrounded answers are the defect.
Account-number refusal
100% of prompts with a test account number are redacted or refused
Staff will paste them.
No core action
No tool exists that can post a transaction. Confirmed by design review.
A KPI on absence. The architecture is the control.
Stale manual
Answers cite the manual version, and a new manual forces a re-index before the old one is served
A retired fee schedule is a customer-harm risk even in an internal tool.
Deployment architecture
The manual is indexed on a small host. The core banking network is not reachable from that host. Redaction runs in front of the prompt log.
01
Procedure manual
Versioned. Owned by operations.
02
Redaction
Account numbers and SSNs stripped from the question before it is logged.
03
Small runtime
llama.cpp on CPU, or a small GPU if ops wants faster answers.
04
Branch page
Cites the section. Does not open a customer record.
Each box is a component in the customer’s network. Arrows are the order of the path, read left to right on a wide screen and top to bottom on a phone.
Where this sits in the stack
The same ten layers appear on every profile. The mark says whether Grae SI provides the layer, shares it, or leaves it with you. Person-weeks are the buyer’s effort after the runtime is deployed, not a project plan for the whole company.
- Grae SI providesGrae SI provides the layer
- SharedShared. Grae SI supports part of it.
- You ownYou own the layer after install
- You own
Layer 1
Infrastructure and compute
Inside the bank’s existing private cloud account, or on a host in the data center.
Buyer effort after the runtime is up: 1–3 person-weeks
- Shared
Layer 2
Data ingestion and prep
A checklist of what to collect, a sample upload, and a readiness score. The pipeline from the system of record into an approved extract.
Buyer effort after the runtime is up: 2–5 person-weeks
- Shared
Layer 3
Data controls and governance
Account numbers are masked before any fine-tune. GLBA access reviews stay with the bank.
Buyer effort after the runtime is up: 1–3 person-weeks
- Shared
Layer 4
Model selection and training
Grae SI packages the small model and the job. The first release may not fine-tune at all. The index of the manual is the bank’s.
Buyer effort after the runtime is up: 0.5–1.5 person-weeks
- Shared
Layer 5
Evaluation and validation
Low-tier suite plus the grounding set and the account-number probe.
Buyer effort after the runtime is up: 0.5–1.5 person-weeks
- Shared
Layer 6
Agents and orchestration
Compliance agents that classify the use case and name the frameworks to prepare for. Any production agent that calls tools inside your applications.
Buyer effort after the runtime is up: 1–2 person-weeks
- Grae SI provides
Layer 7
Deployment and serving
Safetensors, GGUF, a model card, and an offline bundle for Ollama, vLLM, or llama.cpp. Moving that bundle into the zone and keeping the host patched.
Buyer effort after the runtime is up: 0.5–1.5 person-weeks
- Shared
Layer 8
Monitoring and drift
Review refused prompts monthly. That is a person, not a new platform.
Buyer effort after the runtime is up: 0.5–1 person-weeks
- You own
Layer 9
Application and integration
An internal page. No core integration, on purpose.
Buyer effort after the runtime is up: 1–3 person-weeks
- You own
Layer 10
Organizational validation and sign-off
Operations and compliance sign a one-page intended use.
Buyer effort after the runtime is up: 0.5–1.5 person-weeks
Buyer effort for this profile
2–4 person-weeks
This is the small-bank shape: a manual, a small model, and a written limit on what it may not do.
- Operations
- Compliance or vendor management
- The infrastructure admin
Quantitative benefits
Illustrative estimate for this sandbox. It is not a measured customer result. Assumptions for this profile are listed under the figures.
Time to a reviewable package
2–3 weeks
No fine-tune, no core integration.
Cost versus a cloud API
About $400–$1,200 a month avoided
Illustrative: 11 branches, steady internal use. The policy win is that account numbers never leave the data center.
Compute
Under $100 a month, often a VM they already have
CPU inference for a 1.5B–3B model is realistic at branch-question volume.
Time to an answer
A few minutes saved versus hunting the PDF
Illustrative. If the manual is already well searched, the gain is smaller.
Accuracy target
90% grounded in the current manual
Gold-set target before staff rely on it.
Assumptions
- Illustrative estimate for this sandbox. It is not a measured customer result.
- No customer-facing use and no connection to core banking.
- A later credit or SAR assistant is a new profile with a high tier.
- Low tier depends on the redaction control actually running. If it is off, re-classify.