Introduction

Reinforcement Learning (RL) is a powerful tool for optimizing digital wallet onboarding incentives, but it comes with an inherent risk: exploration. To learn, an RL agent must try new actions. In a video game, a bad action means losing a life. In fintech, a bad action could mean offering a $500 bonus to a fraudulent user or violating regulatory caps on inducements.

The question every enterprise architect asks is: How do we allow the model to learn without blowing up the budget or breaking the law?

The answer lies in Constrained RL and Architectural Guardrails. We cannot trust the LLM or the RL agent to be safe on its own. Safety must be enforced structurally within the orchestration layer. This article demonstrates how to build a Safe Dynamic Pricing Engine for wallet creation using LangGraph. We will implement action masking, compliance-aware retrieval, and hard-bound verification to ensure that exploration never crosses into unsafe territory.

Disclaimer: This POC uses simulated data and simplified Q-Learning for educational purposes. Always consult legal and risk teams before deploying financial incentive algorithms.

The Danger of Unconstrained Exploration

In standard Q-Learning, an agent uses an epsilon-greedy strategy to explore. If ϵ=0.1ϵ=0.1, 10% of the time it picks a random action. In our wallet context, "random" could mean selecting Large_Bonus for a High_Risk_User. Without guardrails, the agent only learns this was bad after the money is lost. We need to prevent the unsafe action from ever being executed.

Safety Architecture: The Guardrail Pattern

Our safety strategy operates on three levels:

  1. Pre-Computation (RAG): Retrieve hard constraints (e.g., "Max bonus for students is $10") before the RL agent acts.

  2. Action Masking: Filter the RL agent’s available action space based on retrieved constraints. The agent literally cannot choose an illegal action.

  3. Post-Verification (Guardrail Node): A deterministic code-based check that validates the final output against business rules before returning it to the user. If validation fails, it falls back to a safe default.

Real-Time Use Case: Wallet Funding Incentive Optimization

Scenario: A new user signs up for a digital wallet. The system must decide on a funding incentive. Safety Constraints:

System Behavior:

  1. RAG Agent retrieves the specific cap for the user's segment.

  2. RL Agent receives a masked action space (e.g., if Student, Large_Bonus is removed from options).

  3. Guardrail Node verifies the selected action. If the RL agent somehow hallucinated or the mask failed, the node overrides it with No_Incentive.

Technology Stack Overview

Project Architecture

safe-wallet-pricing/
├── backend/
│   ├── app/
│   │   ├── main.py
│   │   ├── config.py
│   │   ├── models/
│   │   │   └── state.py
│   │   ├── agents/
│   │   │   ├── compliance_retriever.py
│   │   │   ├── safe_rl_agent.py
│   │   │   └── safety_guardrail.py
│   │   ├── graph/
│   │   │   └── workflow.py
│   │   └── rag/
│   │       └── ingest.py
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── App.jsx
│   │   └── components/
│   │       └── SafePricingDashboard.jsx
│   └── package.json
└── README.md

Step-by-Step POC Implementation

Backend Setup and Configuration

backend/requirements.txt

fastapi
uvicorn
langgraph
langchain
langchain-community
chromadb
sentence-transformers
numpy
python-dotenv
pydantic

backend/app/config.py

import os
from dotenv import load_dotenv

load_dotenv()

class Settings:
    CHROMA_PATH = "./chroma_db"
    LEARNING_RATE = 0.1
    DISCOUNT_FACTOR = 0.9
    EXPLORATION_RATE = 0.15
    # Hard safety bounds that override everything
    ABSOLUTE_MAX_BONUS = 50.0 

settings = Settings()

Defining the LangGraph State with Safety Bounds

backend/app/models/state.py

from typing import TypedDict, List, Optional, Annotated
from langgraph.graph import add_messages

class WalletState(TypedDict):
    messages: Annotated[list, add_messages]
    user_id: str
    user_profile: dict
    # Safety-specific fields
    compliance_constraints: dict 
    allowed_actions: List[str]
    
    # RL fields
    state_key: str
    selected_action: str
    reward: float
    q_table_updated: bool
    
    # Output fields
    final_offer: str
    safety_override: bool

RAG Component: Compliance & Risk Policy Retrieval

backend/app/rag/ingest.py

from langchain_core.documents import Document
from langchain_community.vectorstores import Chroma
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter

def get_compliance_docs():
    return [
        Document(
            page_content="Student Segment: Maximum cash incentive capped at $10. Fee waivers permitted.",
            metadata={"segment": "Student", "max_cash": 10, "allowed_types": ["Small_Bonus", "Fee_Waiver", "No_Incentive"]}
        ),
        Document(
            page_content="Professional Segment: Maximum cash incentive capped at $50. Requires income verification.",
            metadata={"segment": "Professional", "max_cash": 50, "allowed_types": ["Small_Bonus", "Large_Bonus", "Fee_Waiver", "No_Incentive"]}
        ),
        Document(
            page_content="High Risk KYC: NO cash incentives allowed. Only fee waivers or gamification.",
            metadata={"segment": "High_Risk", "max_cash": 0, "allowed_types": ["Fee_Waiver", "No_Incentive"]}
        )
    ]

def ingest_data():
    docs = get_compliance_docs()
    splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
    split_docs = splitter.split_documents(docs)
    
    embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
    vector_store = Chroma.from_documents(
        documents=split_docs,
        embedding=embeddings,
        persist_directory="./chroma_db"
    )
    print("Compliance policies ingested.")
    return vector_store

The RL Agent with Action Masking

This is the core safety mechanism. The agent does not explore the full space; it explores only the safe subspace.

backend/app/agents/safe_rl_agent.py

import numpy as np
import random
from app.config import settings

ALL_ACTIONS = ["No_Incentive", "Small_Bonus", "Large_Bonus", "Fee_Waiver"]

class SafeQLearningAgent:
    def __init__(self):
        self.q_table = {}

    def get_state_key(self, profile):
        return f"{profile.get('type', 'Unknown')}_{profile.get('risk', 'Low')}"

    def choose_action(self, state_key, allowed_actions):
        if state_key not in self.q_table:
            self.q_table[state_key] = np.zeros(len(ALL_ACTIONS))
        
        # SAFETY MECHANISM: Action Masking
        # Create a list of valid indices based on allowed_actions
        valid_indices = [i for i, a in enumerate(ALL_ACTIONS) if a in allowed_actions]
        
        if not valid_indices:
            return "No_Incentive" # Ultimate fallback
            
        if random.uniform(0, 1) < settings.EXPLORATION_RATE:
            # Explore ONLY within safe bounds
            idx = random.choice(valid_indices)
        else:
            # Exploit ONLY within safe bounds
            # Set invalid actions to -inf so argmax ignores them
            masked_q = np.full(len(ALL_ACTIONS), -np.inf)
            for i in valid_indices:
                masked_q[i] = self.q_table[state_key][i]
            idx = np.argmax(masked_q)
            
        return ALL_ACTIONS[idx]

    def learn(self, state_key, action, reward):
        if state_key not in self.q_table:
            self.q_table[state_key] = np.zeros(len(ALL_ACTIONS))
        idx = ALL_ACTIONS.index(action)
        old_value = self.q_table[state_key][idx]
        next_max = np.max(self.q_table[state_key])
        new_value = old_value + settings.LEARNING_RATE * (reward + settings.DISCOUNT_FACTOR * next_max - old_value)
        self.q_table[state_key][idx] = new_value

rl_agent = SafeQLearningAgent()

def safe_rl_node(state):
    state_key = rl_agent.get_state_key(state["user_profile"])
    state["state_key"] = state_key
    
    # Learn from previous interaction
    if state.get("reward") is not None and state.get("prev_action"):
        rl_agent.learn(state_key, state["prev_action"], state["reward"])
        state["q_table_updated"] = True

    # Choose action using MASKED exploration
    allowed = state.get("allowed_actions", ALL_ACTIONS)
    action = rl_agent.choose_action(state_key, allowed)
    
    state["selected_action"] = action
    state["prev_action"] = action
    return state

The Safety Guardrail Node

Even with masking, we add a deterministic post-check. LLMs or serialization errors can corrupt state. Code does not lie.

backend/app/agents/safety_guardrail.py

from app.config import settings

OFFER_VALUES = {
    "No_Incentive": 0,
    "Small_Bonus": 5,
    "Large_Bonus": 20,
    "Fee_Waiver": 0
}

def guardrail_node(state):
    action = state["selected_action"]
    constraints = state.get("compliance_constraints", {})
    max_cash = constraints.get("max_cash", 0)
    allowed_types = constraints.get("allowed_types", [])
    
    offer_value = OFFER_VALUES.get(action, 0)
    
    # CHECK 1: Absolute global cap
    if offer_value > settings.ABSOLUTE_MAX_BONUS:
        state["final_offer"] = "Standard Account Creation"
        state["safety_override"] = True
        return state
        
    # CHECK 2: Segment-specific cap
    if offer_value > max_cash:
        state["final_offer"] = "Standard Account Creation"
        state["safety_override"] = True
        return state
        
    # CHECK 3: Allowed type verification
    if action not in allowed_types:
        state["final_offer"] = "Standard Account Creation"
        state["safety_override"] = True
        return state
        
    # All checks passed
    offer_map = {
        "Small_Bonus": "$5 Bonus on $50 Deposit",
        "Large_Bonus": "$20 Bonus on $100 Deposit",
        "Fee_Waiver": "0% Fees for 3 Months",
        "No_Incentive": "Standard Account Creation"
    }
    state["final_offer"] = offer_map.get(action, "Standard Account Creation")
    state["safety_override"] = False
    return state

Orchestrating the Safe Workflow

backend/app/graph/workflow.py

from langgraph.graph import StateGraph, END
from app.models.state import WalletState
from app.agents.compliance_retriever import compliance_node
from app.agents.safe_rl_agent import safe_rl_node
from app.agents.safety_guardrail import guardrail_node
from langgraph.checkpoint.sqlite import SqliteSaver

def create_workflow():
    workflow = StateGraph(WalletState)
    
    workflow.add_node("compliance", compliance_node)
    workflow.add_node("safe_rl", safe_rl_node)
    workflow.add_node("guardrail", guardrail_node)
    
    workflow.set_entry_point("compliance")
    workflow.add_edge("compliance", "safe_rl")
    workflow.add_edge("safe_rl", "guardrail")
    workflow.add_edge("guardrail", END)
    
    memory = SqliteSaver.from_conn_string(":memory:")
    return workflow.compile(checkpointer=memory)

wallet_graph = create_workflow()

(Note: compliance_retriever.py should query ChromaDB and populate compliance_constraints and allowed_actions in the state based on user profile.)

FastAPI Endpoints

backend/app/main.py

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from app.graph.workflow import wallet_graph
from app.rag.ingest import ingest_data
import uuid

app = FastAPI(title="Safe Wallet Pricing Engine")

class OnboardingRequest(BaseModel):
    user_type: str
    risk_level: str = "Low"
    conversation_id: str = None
    previous_reward: float = None 

@app.post("/ingest")
def ingest():
    ingest_data()
    return {"status": "Compliance Policies Updated"}

@app.post("/offer")
def get_offer(req: OnboardingRequest):
    thread_id = req.conversation_id or str(uuid.uuid4())
    config = {"configurable": {"thread_id": thread_id}}
    
    initial_state = {
        "messages": [{"role": "user", "content": "Start onboarding"}],
        "user_id": "user_123",
        "user_profile": {"type": req.user_type, "risk": req.risk_level},
        "reward": req.previous_reward,
        "q_table_updated": False,
        "safety_override": False
    }
    
    try:
        result = wallet_graph.invoke(initial_state, config=config)
        return {
            "conversation_id": thread_id,
            "offer": result["final_offer"],
            "action_taken": result["selected_action"],
            "learned": result["q_table_updated"],
            "safety_triggered": result["safety_override"]
        }
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

Frontend Implementation with React

frontend/src/App.jsx

import React, { useState } from 'react';

function App() {
  const [userType, setUserType] = useState('Student');
  const [riskLevel, setRiskLevel] = useState('Low');
  const [result, setResult] = useState(null);
  const [convId, setConvId] = useState(null);
  const [lastReward, setLastReward] = useState(null);

  const handleGetOffer = async () => {
    try {
      const res = await fetch('http://localhost:8000/offer', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ 
          user_type: userType, 
          risk_level: riskLevel,
          conversation_id: convId,
          previous_reward: lastReward
        }),
      });
      const data = await res.json();
      setResult(data);
      setConvId(data.conversation_id);
      setLastReward(null);
    } catch (err) {
      console.error(err);
    }
  };

  const simulateOutcome = (reward) => {
    setLastReward(reward);
    alert(`Simulated: ${reward > 0 ? 'Funded (+10)' : 'Churned (-5)'}. Click 'Generate Offer' to continue learning.`);
  };

  return (
    <div className="p-10 max-w-4xl mx-auto font-sans">
      <h1 className="text-2xl font-bold mb-6">🛡️ Safe RL Wallet Onboarding</h1>
      
      <div className="flex gap-4 mb-4">
        <select value={userType} onChange={(e) => setUserType(e.target.value)} className="p-2 border rounded">
          <option value="Student">Student</option>
          <option value="Professional">Professional</option>
        </select>
        <select value={riskLevel} onChange={(e) => setRiskLevel(e.target.value)} className="p-2 border rounded">
          <option value="Low">Low Risk</option>
          <option value="High">High Risk</option>
        </select>
        <button onClick={handleGetOffer} className="bg-blue-600 text-white px-4 py-2 rounded">
          Generate Safe Offer
        </button>
      </div>

      {result && (
        <div className={`mt-6 p-4 border rounded ${result.safety_triggered ? 'bg-red-50 border-red-300' : 'bg-green-50 border-green-300'}`}>
          <div className="flex justify-between items-center">
            <h2 className="font-bold text-lg">{result.offer}</h2>
            {result.safety_triggered && (
              <span className="bg-red-600 text-white text-xs px-2 py-1 rounded">SAFETY OVERRIDE ACTIVE</span>
            )}
          </div>
          <p className="text-sm text-gray-600 mt-1">Action: {result.action_taken} | Learned: {result.learned ? 'Yes' : 'No'}</p>
          
          {!result.safety_triggered && (
            <div className="mt-4 flex gap-2">
              <button onClick={() => simulateOutcome(10)} className="bg-green-500 text-white px-3 py-1 rounded text-sm">✓ Simulate Funding</button>
              <button onClick={() => simulateOutcome(-5)} className="bg-red-500 text-white px-3 py-1 rounded text-sm">✗ Simulate Churn</button>
            </div>
          )}
        </div>
      )}
    </div>
  );
}

export default App;

Running the Application

  1. Backend: cd backend && pip install -r requirements.txt && uvicorn app.main:app --reload

  2. Ingest: curl -X POST http://localhost:8000/ingest

  3. Frontend: cd frontend && npm install && npm run dev

Analyzing Safety Mechanisms in Action

Try this experiment in the UI:

  1. Select Student + Low Risk → You may see $5 Bonus or $20 Bonus during exploration.

  2. Now select Student + High Risk → The system will always return Standard Account Creation or Fee Waiver. Even if the RL agent's Q-table has high values for Large_Bonus from previous student interactions, the action mask removes it from consideration entirely.

  3. If you manually tamper with the state (simulating a bug), the guardrail node catches it and forces the safe default.

This is how we avoid unsafe exploration: the agent never sees the cliff because we built a fence before it takes a step.

Enterprise Production Considerations

Limitations

Conclusion

Unsafe exploration is not an RL problem it is an architecture problem. By combining RAG-driven constraint retrieval, action masking, and deterministic guardrail nodes within a LangGraph workflow, we create a system where the RL agent can learn and optimize within a provably safe boundary. For digital wallet onboarding, this means better conversion rates without regulatory nightmares. The key insight: never let the model be the last checkpoint before money moves. Always put code between the AI and the transaction.