An internship project exploring how generative AI can extend an existing ERP rather than replace it: instead of building a new system, the work sat inside one already in production, constrained by its data, its users, and the need to keep working while the experiment ran.
The problem & constraints
The business wanted generative AI features inside its ERP — mainly to automate the generation and use of certain documents and information — without disrupting a system already in production. That constraint shaped everything: the solution had to integrate with an existing ERP rather than stand alone, keep business data confidential, and run in a local environment where possible rather than call an external API. Coherent, context-appropriate answers mattered more than raw model capability, and the whole exploration had to fit inside a fixed internship timeline.
Approach
- Studied and compared several language models against the ERP's functional needs, rather than defaulting to the first one that worked.
- Experimented with running an LLM locally and integrating it with the ERP, weighing what local deployment costs in setup and resources against calling a hosted model.
- Explored a Retrieval-Augmented Generation (RAG) approach so the model could draw on knowledge the application already holds, instead of answering only from its general training.
- Contributed to early automated document-generation features, and to the functional tests used to judge whether a model's output was relevant and consistent enough to use.
Trade-offs made
Decision
Local deployment over a hosted model, for part of the experimentation: kept business data and the execution environment under control, at the cost of extra hardware and performance constraints a hosted API doesn't have.
Decision
RAG over direct generation: adds a retrieval and context-preparation step the model has to wait on, but grounds answers in the business's own knowledge instead of the model's general training — worth the extra step for an ERP where a wrong-but-fluent answer is worse than a slow one.
Results
The experimentation integrated and tested several generative-AI features inside the ERP's context, including a first pipeline for using a language model to generate content. It also surfaced real behavioral differences between the LLMs compared, and evidence for where a RAG architecture earns its complexity — cases that need the model grounded in domain-specific knowledge rather than general ones. No quantitative benchmark was kept at this stage: results were judged through functional tests and qualitative comparison between models, not a single scored metric.
What I'd do differently
Replace the qualitative comparison with a more systematic evaluation protocol and metrics built for LLM applications — answer relevance, faithfulness to source documents, error rate, response time. I'd also want an automated evaluation harness, so comparing models or tracking a regression as the solution evolves doesn't depend on someone re-reading outputs by hand.