Enterprise AI Assistant
On-premises RAG and MCP-enabled assistant for a commercial ERP suite
01 — Overview
At Arian System, I lead backend and AI engineering for an intelligent assistant embedded across the company's commercial ERP suite[cite: 6]. The system acts as both an instant product specialist—answering complex procedural questions from enterprise documentation—and an operational co-pilot capable of interacting with live ERP modules through controlled APIs[cite: 6].
Unlike consumer AI wrappers built on public cloud APIs, this platform was engineered from day one to run entirely inside enterprise infrastructure, balancing strict data sovereignty with predictable latency[cite: 6].
02 — Business Challenge
Commercial ERP systems are notoriously dense. Users across accounting, inventory, procurement, and payroll routinely struggle to navigate multi-step workflows, resulting in high support ticket volume and slow onboarding.
However, bringing generative AI into an enterprise ERP environment introduced four hard constraints:
- Strict Data Sovereignty: Financial ledgers, employee records, and proprietary business workflows cannot be transmitted to external third-party LLM providers[cite: 6].
- Multi-Tenant & Role-Based Isolation: A warehouse clerk and a CFO working in the same organization must never access or infer unauthorized data through the assistant[cite: 6].
- Actionability Beyond Q&A: Explaining how to issue a purchase order is useful, but enterprise users expect the assistant to query live statuses and trigger approved workflows safely[cite: 6].
- Hardware Realism: Local inference had to perform reliably on practical on-premises GPU infrastructure without prohibitive hardware costs.
03 — Architecture
To decouple knowledge retrieval, reasoning, and live transactional execution, I designed a modular, graph-based orchestration architecture centered around LangGraph and the Model Context Protocol (MCP)[cite: 5, 6].
Core Architectural Layers
- Token-Gated Gateway & Multi-Tenant Context: Every incoming request carries the user's ERP session token[cite: 6]. Tenant IDs and role permissions are resolved before any retrieval or tool execution occurs[cite: 6].
- AI Coordinator (LangGraph): A stateful routing layer that classifies intent—distinguishing between documentation queries, multi-turn operational tasks, and out-of-scope requests.
- Local RAG Pipeline: An ingestion and hybrid retrieval engine indexing over 100 structured ERP user guides (1,000+ chunks) with metadata filtering per ERP module[cite: 6].
- Local LLM Inference Layer: Self-hosted open-weight models served locally, tuned for low-hallucination grounded synthesis and structured tool calling[cite: 6].
- Standalone ERP MCP Server: A dedicated integration layer developed by a team of six engineers under my leadership, exposing commercial ERP capabilities as standardized, schema-validated MCP tools[cite: 6].
04 — Engineering Decisions
Every major technical choice was driven by production reliability rather than framework hype:
- Why MCP for ERP Integration: Instead of hardcoding brittle REST calls inside prompts, building a standalone Model Context Protocol (MCP) server created a clean contract between the AI layer and the core ERP backend[cite: 6]. It standardized tool schemas, session persistence, and telemetry[cite: 6].
- Why Local Models Over Cloud APIs: Privacy regulations and customer trust in enterprise accounting software make cloud data transmission a dealbreaker[cite: 6]. Benchmarking quantized open-weight models locally proved we could achieve strong reasoning accuracy while keeping all data on-premises[cite: 6].
- Why Structured Chunking Over Naive Splitting: ERP manuals rely heavily on hierarchical menus, form fields, and prerequisite steps. Preserving section hierarchy and module metadata during ingestion dramatically improved retrieval precision compared to fixed-size character chunking.
The LLM never connects directly to the ERP database. All read and write operations pass through the MCP server, which enforces the exact same business rules and permission checks as the ERP user interface.
05 — Implementation
Leading a cross-functional engineering team of six[cite: 6], we broke implementation into two distinct phases to de-risk delivery:
- Phase 1 — Grounded Documentation Assistant: We built the document ingestion pipeline, vector indexing, and evaluation harness first. By establishing an automated benchmark suite early, we could measure the exact impact of chunking strategies, embedding models, and prompt constraints before exposing the system to users.
- Phase 2 — Live ERP Operations via MCP: We designed and shipped the standalone MCP server handling API contracts, stateful multi-step session persistence, and end-to-end telemetry[cite: 6]. When a user asks the assistant to inspect an invoice or initiate a workflow, the coordinator negotiates parameters, confirms intent when required, and executes the tool under the user's own security context.
06 — Results & Evaluation
The system moved from architectural prototype to production deployment with measurable, verifiable outcomes:
- 82 / 100 Evaluation Accuracy: Benchmarked against a curated set of real-world ERP operational and procedural questions, demonstrating high factual grounding and minimal hallucination[cite: 6].
- Complete Data Isolation: 100% of document indexing, embedding generation, and LLM inference executes on company-controlled servers[cite: 6].
- Extensible Platform Foundation: Thanks to the decoupled MCP architecture, new ERP modules can expose tools to the assistant without modifying the core AI orchestration engine[cite: 6].
Current Limitations & Next Iterations
Complex multi-step write transactions still require explicit user confirmation guardrails, and latency on long analytical queries depends heavily on local GPU batching throughput. Ongoing work focuses on semantic caching for frequent procedural queries and expanding automated regression benchmarks as new ERP modules are onboarded.