Let’s replace every business process with AI… Wait!

It’s trendy to put LLM into every workflow nowadays. Generate a summary here, add a chatbot there. People even ask a model to make decisions for them and follow it blindly. In my opinion that's a silent path to disaster as we still live in the world where we design systems that are meant to be used by humans. And only a human can still truly tell and feel whenever the system is built to serve his needs.

A basic LLM model is a Swiss Army knife. A great tool to take on a weekend camping trip. But if you’re building a country house and you have to tighten 10000 screws, you need a proper electric screwdriver.

In one of my recent projects, a healthcare organization had a problem where their staff spent a lot of time trying to tell whether a specific health condition is covered by the provided service or not.

To solve this I built an AI assistant, which searched the information in knowledge base and provided a source to the staff, so they could tell if the answer is legit. The system can say ‘no’ with confidence which is very important in healthcare, and provides more questions to the user if the documents don’t provide the answer. A fast, confident but false answer can cause a lot of trouble in this area.

To implement the system I couldn’t load the whole knowledge base into the model’s context for every question asked. This approach is not scalable as the knowledge base grows and it’s not cost effective. I’ve built a RAG to retrieve only the relevant pieces of data, tested it against real staff questions for both retrieval and response quality.

That was the whole AI feature for the project, as extra was unnecessary at that point. What was necessary is to speak to people to find the recurring problem, understanding the cost of a wrong answer, and implementing the optimal solution specifically for that specific job.

Do you see overuse of AI in your business? Or are you eager to add it into the process?