Skip to main content

Handle parsing errors

Occasionally the LLM cannot determine what step to take because its outputs are not correctly formatted to be handled by the output parser. In this case, by default the agent errors. But you can easily control this functionality with handle_parsing_errors! Let’s explore how.

Setup​

from langchain.agents import AgentType, Tool, initialize_agent
from langchain.chat_models import ChatOpenAI
from langchain.utilities import SerpAPIWrapper
search = SerpAPIWrapper()
tools = [
    Tool(
        name="Search",
        func=search.run,
        description="useful for when you need to answer questions about current events. You should ask targeted questions",
    ),
]

Error​

In this scenario, the agent will error (because it fails to output an Action string)

mrkl = initialize_agent(
    tools,
    ChatOpenAI(temperature=0),
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
)
mrkl.run("Who is Leo DiCaprio's girlfriend? No need to add Action")


> Entering new AgentExecutor chain...
OutputParserException: Could not parse LLM output: I'm sorry, but I cannot provide an answer without an Action. Please provide a valid Action in the format specified above.

Default error handling​

Handle errors with Invalid or incomplete response:

mrkl = initialize_agent(
    tools,
    ChatOpenAI(temperature=0),
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
    handle_parsing_errors=True,
)
mrkl.run("Who is Leo DiCaprio's girlfriend? No need to add Action")


> Entering new AgentExecutor chain...

Observation: Invalid or incomplete response
Thought:
Observation: Invalid or incomplete response
Thought:Search for Leo DiCaprio's current girlfriend
Action:
```
{
  "action": "Search",
  "action_input": "Leo DiCaprio current girlfriend"
}
```

Observation: Just Jared on Instagram: “Leonardo DiCaprio & girlfriend Camila Morrone couple up for a lunch date!
Thought:Camila Morrone is currently Leo DiCaprio's girlfriend
Final Answer: Camila Morrone

> Finished chain.
'Camila Morrone'

Custom error message​

You can easily customize the message to use when there are parsing errors.

mrkl = initialize_agent(
    tools,
    ChatOpenAI(temperature=0),
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
    handle_parsing_errors="Check your output and make sure it conforms!",
)
mrkl.run("Who is Leo DiCaprio's girlfriend? No need to add Action")


> Entering new AgentExecutor chain...

Observation: Could not parse LLM output: I'm sorry, but I canno
Thought:I need to use the Search tool to find the answer to the question.
Action:
```
{
  "action": "Search",
  "action_input": "Who is Leo DiCaprio's girlfriend?"
}
```

Observation: DiCaprio broke up with girlfriend Camila Morrone, 25, in the summer of 2022, after dating for four years. He's since been linked to another famous supermodel – Gigi Hadid. The power couple were first supposedly an item in September after being spotted getting cozy during a party at New York Fashion Week.
Thought:The answer to the question is that Leo DiCaprio's current girlfriend is Gigi Hadid. 
Final Answer: Gigi Hadid.

> Finished chain.
'Gigi Hadid.'

Custom Error Function​

You can also customize the error to be a function that takes the error in and outputs a string.

def _handle_error(error) -> str:
    return str(error)[:50]


mrkl = initialize_agent(
    tools,
    ChatOpenAI(temperature=0),
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
    handle_parsing_errors=_handle_error,
)
mrkl.run("Who is Leo DiCaprio's girlfriend? No need to add Action")


> Entering new AgentExecutor chain...

Observation: Could not parse LLM output: I'm sorry, but I canno
Thought:I need to use the Search tool to find the answer to the question.
Action:
```
{
  "action": "Search",
  "action_input": "Who is Leo DiCaprio's girlfriend?"
}
```

Observation: DiCaprio broke up with girlfriend Camila Morrone, 25, in the summer of 2022, after dating for four years. He's since been linked to another famous supermodel – Gigi Hadid. The power couple were first supposedly an item in September after being spotted getting cozy during a party at New York Fashion Week.
Thought:The current girlfriend of Leonardo DiCaprio is Gigi Hadid. 
Final Answer: Gigi Hadid.

> Finished chain.
'Gigi Hadid.'