Your AI agents are lost: give them a graph

The AI industry keeps pushing toward more capable models, but model intelligence alone does not make an agent reliable. An agent still needs to find the right information, understand how it relates, respect the boundaries around its work, and learn from the outcome.

Anthony Alcaraz works at the intersection of agentic engineering and go-to-market strategy as a GTM Agentic Engineering Lead at AWS. He is also the co-author of O'Reilly's Agentic GraphRAG. In this episode, he makes the case that context engineering, not simply model selection, is becoming the central challenge in building useful agentic systems.

The conversation explores how graphs can give agents a navigable structure for retrieval, memory, planning, and reasoning. Anthony also explains why enterprises need to capture tribal knowledge, create controlled gateways to tools, and build feedback loops that connect agent performance to real business outcomes. From there, he looks ahead to smaller teams managing fleets of agents and roles that increasingly blend engineering, product thinking, and customer contact.

Key Topics Discussed

• (01:07) From public-sector machine learning to AWS and AI startups
• (03:31) The first mistake in production AI: failing to map the problem
• (05:45) RAG, context engineering, and the limits of vector retrieval
• (08:59) Why graphs help agents remember, plan, and reason
• (13:14) Tribal knowledge, temporal context, and incident investigation
• (16:07) How agent fleets could reshape roles and organizations
• (19:06) Building centralized tools and self-improving feedback loops

Where to Find Anthony Alcaraz

• LinkedIn: linkedin.com/in/anthony-alcaraz-b80763155
• AWS: aws.amazon.com
• Book: Agentic GraphRAG

Transcript

Can you tell us about your background and what you work on today?

I have an unusual background. I worked in the French public sector, where I pushed AI initiatives before the current wave of generative AI. At the time, the focus was machine learning operations and predictive models. France's social security system is very data-driven, with many decisions made from large volumes of data, so it was an interesting environment for that work.

After that, I moved into the private sector and worked on machine learning for finance at L'Oréal. Before joining AWS, I was CTO at a startup, where I built an architecture for matching CVs and job offers at scale. The company still exists, has customers, and I remain an adviser.

At AWS, I am a GTM Agentic Engineering Lead. My role is to connect high-growth AI and machine learning startups with buyers. AWS works with many enterprises, so I help create connections between those two groups. I also work with venture capital firms, investor managers, and the internal teams supporting startups. I am fortunate to work with founders who have raised significant funding to push the AI frontier forward.

What common mistake do you see when companies deploy AI to production?

The first thing I look for is a clear understanding of the problem the company is trying to solve. AI is only a tool. If you do not map the problem concretely, the technology will not save you.

That work requires meeting people. I recently met founders building a product for CFOs. They spent three or four months speaking with hundreds of CFOs around the world to understand their processes, the differences between geographies and industries, the relevant stakeholders, and the integration points for an AI system. Some organizations may still run important systems on a mainframe with no external API, so you have to plan for that engineering reality.

After problem definition, one of the most important bottlenecks in agentic AI is context. You need to engineer what context will be available to the agent and how the agent will navigate it. As models become more capable, one of their limits will be access to valuable context that remains valuable over time. This brings us back to MLOps. We need engineering practices for curating context, managing its evolution, and maintaining its quality.

What is RAG, and why does context engineering matter for agents?

An agentic system observes data, reasons over it, and acts. To do that, it needs access to enterprise data. It may also need to cross-check information and perform multi-step reasoning, moving from one piece of evidence to the next.

RAG means retrieval-augmented generation. Before the large language model reasons or responds, the system retrieves the relevant data and context. This matters because giving a model too much poor-quality context can reduce its accuracy and understanding. Models can also pay uneven attention across a long context, often performing better with information near the beginning or end.

These limitations create a need for context engineering: selecting, compressing, and organizing the right information for the model. I think graph methods will become prominent here. While writing the book, I met founders in fields including biology and security who were using graphs to curate context for agents. A graph does not replace every other retrieval method. You can still use vector search on top of a graph, but the graph gives the information structure and relationships.

Why can graphs improve agent memory and planning?

There are several dimensions. One is memory management. A graph makes it easier to remove or decay parts of an agent's memory that are outdated or no longer relevant. If you put everything into memory forever, it becomes bloated, and retrieval struggles to find what the agent actually needs.

Graphs are also useful because agents benefit from navigation. A coding agent can move through a terminal, observe what it has done, determine whether it succeeded, and continue until completion. A graph offers a similar navigable space. The same idea applies to planning, where tasks, dependencies, and progress can be represented as connected elements.

Enterprise systems also fail to capture much of the knowledge an agent needs. Important context lives in conversations, email, and the reasoning behind past decisions. Agents may need to process sources such as Slack and Outlook to capture that tribal knowledge. If the ontology and information architecture are designed well, a graph gives agents a clean space in which to navigate it.

How can graphs make agentic systems safer and more efficient?

A graph can include rules that enforce relationships and definitions. That creates a reasoning layer that complements the reasoning of the language model. It can help ensure that the system respects a schema and stays within the boundaries designed around it.

There is also an efficiency argument. Once your context is well curated, you may be able to use smaller, more efficient models instead of relying on the most capable and expensive model for every task. You could use open models, fine-tuned models, or specialist models trained for work involving the graph data.

Graphs can also connect to reinforcement learning. Training an agent requires an environment in which it can perform tasks and receive feedback. A well-built graph can provide a useful environment for that training and support long-term efficiency gains.

What does this mean for incident investigation and other context-heavy work?

Graphs are a natural way to manage time and hierarchy in a knowledge space. In incident response, an agent needs both durable organizational knowledge and the latest operational evidence. That may include architectural quirks, known weaknesses, recent commits, traces, logs, and dashboards.

Graph architectures are flexible enough to represent different relationships depending on the use case. The goal is to curate a context that lets the agent understand how systems relate and how those relationships change over time. With that foundation, the agent can navigate toward the relevant evidence instead of treating every piece of information as an isolated document.

How do you expect agentic AI to change roles and organizations?

I expect a significant wave of transformation. If an enterprise manages context well and deploys a capable fleet of agents, a large share of knowledge work could be automated or augmented. What remains for people is managing and improving that fleet.

Improvement still requires speaking with customers and working backward from the value they need. I expect roles to converge toward something like product management. People will improve the agentic systems behind the company's value while also speaking with customers, partners, and other external stakeholders. Pure coding roles may decline outside the most complex systems, while more work combines engineering, sales, and product activity.

I do not think companies should treat this primarily as an opportunity for large layoffs. They should aim to create more value. Better technology could support more sales conversations, better targeting, stronger customer interactions, and higher conversion. My optimistic view is that this improves the match between demand and supply and allows companies to grow their top line.

What should enterprises build first to manage agents at scale?

Every role in a company uses a set of tools and skills. One practical step is to create a centralized gateway for those capabilities. You want to rationalize how agents call tools, secure those calls, and map the needs across different roles. You do not want uncontrolled sprawl, with MCP servers and tools growing in every direction without a consistent way to access them.

More fundamentally, agentic systems need to be self-improving. Every success and every failure should feed back into the system. MLOps created this kind of flywheel for machine learning, and enterprises need an equivalent at the organizational level for agents.

That flywheel cannot be technical only. It needs business feedback from sales, go-to-market teams, and other stakeholders. Agent performance matters when it leads to better business action and customer value.

What might teams and companies look like in an agentic future?

A team of five or six people managing hundreds of agents could accomplish a great deal. I can imagine something like Amazon's two-pizza-team model, where a small team pursues a clear set of goals with an agentic system beneath it and collaborates with other small teams managing their own agents.

If a business thinks only in the short term, it may conclude that it needs fewer people. If it wants to capture more markets and create more value, it may need more highly augmented people. Two people with dramatically improved capabilities can achieve more than one.

Innovation always creates some destruction and some creation. Certain sectors and roles will be affected, but human needs expand. If society produces better products and services, people become accustomed to that quality and push the frontier further. I believe that process can create new work as well as transform existing work.

Where can people follow your work?

LinkedIn is my main platform. I write there frequently about AI and machine learning. I also have a Medium publication, although I publish there less often now.