Vectara Agent Framework

The Changing Definition of “Agent”

“Agents” for the past few years have been a very loosely defined term. It often came to mean any arbitrary code that calls a generative model in some way. Frameworks like LangChain and Llamaindex let you stitch together complex workflows to orchestrate generative models in different ways. Vectara has supported such “agents” for the past couple of years by being an excellent source of retrieval and generation. So, being able to host an agent was equivalent to saying you are a general hosting provider.

In recent years, the meaning of “agent” has become more clearly defined. Today, an agent is generally understood to be a system built around three essential components: a prompt (which provides the agent with instructions or context), a set of tools (the functions or capabilities it can call upon), and a goal (the outcome it is working toward).

That’s it. Therefore, most production agents today are essentially tool loops.

Why a Framework is Needed

Running this definition of an agent as a service is not only feasible but also highly valuable. That said, the how still matters. Before you even reach the exciting parts of what an agent can do, there’s a long list of operational details to get right. These include provisioning the servers that host the agent, managing and retrieving sessions, versioning instructions and tools as they evolve, and ensuring reliability and scalability.

Easily running your agent is the first aim of the Vectara Agentic framework. We are able to provide an easy-to-use UX and API that allows users to seamlessly create an agent and leave it to us to take care of running the agent and orchestrating the rest of the data flow. The secondary aim is for your agent to be production grade out of the box, with the ability for the agent to improve over time as the framework, models, and data become better. And all in a manner in which users of the framework would not have to lift a finger to enjoy these updates.

What You Get with Vectara Agent Framework

Here’s what we provide out of the box: 

• Easily created sessions with the ability look them up for later
• Keep an instruction library
• Have the agent definition in simple JSON
• We keep track of session flow and agent events, sending data between the agent and tools
• Running the agent in a durable and contention free fashion
• Automatically scaling up the agent if needed.

Especially exciting is that you will have the ability to dynamically alter tools and binding session data to them. Before, you would have to modify the MCP server code for each agent that wanted to use it. We allow you to modify the tools of a MCP server without having to touch the MCP server and present it to the agent how you’d like.

Additionally, Vectara Agent Framework will allow you to limit context rot that most MCP servers bring by allowing an agent to see a subset of tools. Obviously, Vectara agents are seamlessly integrated into the Vectara platform so they can use native tools to interact with our platform’s leading RAG capabilities.

For engineers, this means fewer moving parts to maintain. You can focus on building while Vectara handles the plumbing, reliability, and to bring more capabilities to your agent .

What’s Next

This is an exciting first step, and gives engineers a production-grade foundation to build on. We’re already working on bringing additional features to expand native Vectara agents. We believe agents should be predictable, secure, and easy to scale, and not become another ops burden. That’s why we built the framework the way we did: to give engineering teams control and clarity without reinventing infrastructure.

If you want to see how Vectara can simplify your agent architecture and let you focus on building features, reach out to us .