I still remember the day our research agent, designed to summarize complex scientific papers, started producing impressive-looking citations that seemed too good to be true. And they were. Upon further investigation, we found that the agent was hallucinating sources, fabricating references to non-existent studies and researchers. This was a major problem, as the credibility of our entire project hinged on the accuracy and reliability of the information our agent provided.
The issue stemmed from the way we were using LangGraph to generate text based on a given prompt. Our initial approach was to simply provide the agent with a vast amount of text data, including scientific papers, and hope that it would learn to recognize and replicate the patterns of proper citation. However, this method clearly wasn't working, as the agent was not actually understanding the context or the importance of accurate sourcing.
To tackle this problem, we decided to leverage the Model Context Protocol (MCP) to provide our agent with a more structured and explicit understanding of what constitutes a proper citation. We began by defining a custom CitationNode class within our LangGraph model, which would serve as a template for our agent to follow when generating citations.
import langgraph
from mcp import tools
class CitationNode(langgraph.Node):
def __init__(self, author, year, title, publication):
self.author = author
self.year = year
self.title = title
self.publication = publication
def to_string(self):
return f"{self.author} ({self.year}). {self.title}. {self.publication}"
# Create a new LangGraph model
model = langgraph.StateGraph()
# Add a node for our citation template
citation_node = CitationNode("John Doe", 2020, "Example Study", "Journal of Example Research")
model.add_node(citation_node)
# Define a conditional edge to trigger the citation node when the agent encounters a prompt requiring a source
model.add_conditional_edges(
tools.prompt_to_node("What is the source of this information?"),
citation_node,
probability=0.8
)
By using this custom CitationNode and defining a conditional edge to trigger it when the agent encounters a prompt requiring a source, we were able to significantly improve the accuracy of our agent's citations. The agent was now able to recognize when a source was needed and generate a proper citation based on the template we provided.
One practical gotcha we learned from this experience is the importance of thoroughly testing and evaluating the performance of our agent in different scenarios. Simply assuming that our agent will work as intended can lead to embarrassing mistakes, like hallucinated sources. By thoroughly testing our agent and refining its performance, we can ensure that it provides accurate and reliable information.
As we continue to develop and refine our research agent, we're excited to explore new applications and capabilities, such as integrating with external knowledge graphs to further enhance the accuracy and depth of our agent's responses. Tomorrow, we'll dive deeper into the possibilities of combining LangGraph and MCP to create even more powerful and sophisticated agentic AI systems.
