Austin Ai has been creating intelligent chatbots that understand, summarize, and talk about our clients' internal and external documents. LLM orchestration platforms “glue together” parts of the chat and LLM architecture and simulate things like conversation memory and reasoning.
Many of the tools we employ are open source, allowing us to deliver a customized application exceptionally quickly for a fixed price – with no recurring SaaS fees. Components include vector DBs, large language models (LLMs), and orchestration platforms, which “glue together” various parts of the chat and LLM architecture and simulate things like conversation memory and reasoning.
Our researchers have evaluated the LangChain and Haystack orchestration platforms, two of the most popular. We made six applications with each: a chatbot, a search app, a web scraper, an OCR app, and some simple NLP apps, in addition to production apps for our clients. In this article, we share our resultant impressions of those two frameworks.
In brief, both frameworks function well, and LangChain has more capabilities but is more complicated than Haystack. FWIW, we are making several enterprise-wide chat apps in LangChain, but Haystack remains an option for lighter-weight applications or quick proofs of concepts. The table below shows the details of our comparison across various aspects.
NB: Both frameworks, like the entire LLM space, are new and rapidly changing, and our comments are based on our own particular experience which has necessarily been short, but nonetheless oriented in the real business world. Other notable orchestration platforms include LlamaIndex and Griptape. We have briefly looked at these and the former appears to be more similar to LangChain, and the latter to Haystack. Another tool is Rasa, an open source conversational framework which we have experience with. We continue to experiment with all and will report any further conclusions.