MH Tsai
LLM workflows vs. AI agents
As LLM context windows grow larger, they are becoming more capable of achieving complex tasks. Anthropic published this article: Building effective agents, which provides a clear definition of workflow and agent.
To clarify some ambiguities in the article, I'll articulate the difference between workflow and agent with examples given my experience of several in-production LLM services.
LLM Workflows
Definition: Sequential LLM operations with predefined steps
The key aspect of a workflow is predetermined routing, even though the routing could be complex or breakdown to several tool during the processing, if the routing can be predicted, it is considered a workflow.
AI Agents
Definition: Self-directing AI systems with agency
The key part for the agent is the Planning and replanning based on context and memory. Ideally, there should be no heuristic if/else logic to control the flow, it will rely on the agent to determine how should it achieve the given goal.
Here I'll use the same vertical service using these two different approaches, to help you compare the difference side by side: Customer Service
Customer Service workflow
A customer service workflow will look like this
Customer: "My subscription didn't renew"
↓
Intent Classification
↓
Subscription Workflow Branch
↓
Payment Status Check
↓
Generate Response from Template
↓
Route to Human if Needed
The workflow follows a rigid, pre-defined process where each step must be completed sequentially.
Customer Service Agent
A customer service agent will look like this:
_Customer: "My subscription didn't renew"
Agent:
- Checks subscription status
- Identifies past payment history
- Proactively looks for common issues
- Suggests multiple solutions
- Can pivot based on customer responses
- Remembers context from previous interactions_
We only provide tools to let the LLM decide how to achieve its goals.
Pros and Cons
The pros and cons of LLM applications can be easily seen from these two examples.
Workflows:
Pros:
- Relatively simple to develop, since the calling steps are fixed, it can guarantee the function is called with given strcuture input.
- Easily to debug and test against each step. For example, the calling steps is fixed, so in testing phase, we can always confirm the unexpected input value is due to it's previous step. Also when prepare testing dataset, we can just consider the possible input for certain functions
Cons:
- There is no flexibility in the flow, so any business requirement change or a subset of the current process will need to create a entirely new workflow, take the customer service as example, if I want to add a new functionality to list all the history payments, there is no break point in the existing flow to achieve it, whereas agent can easily achieve it since the agent has the ability to pick certain tool, and determine if the result is matched the goal, and the agent can respond to user.
- Process time: Given each workflow need to go through each steps every time, it doesn't know if it can early return the result, since the workflow doesn't aware of the goals
Agents:
Pros:
- Flexibility of achieving multiples business requirement at a time, agent can compose different tools and achieve multiple business requirement.
Cons:
- It's relatively hard to debug and evaluate, given the natural randomness of the tool chose by LLM, same input could lead to different approach toward the goal.
Conclusion
Hope this two side-by-side comparsion gain you to have a more clear view on this two different LLM application approachs.
When to use workflows (with example):
- Regulated Industries & Compliance Requirements: Legal document processing with mandatory review stages.
- High-Stakes Operations: Medical diagnosis support systems.
- Predictable, Linear Tasks: Document format conversion.
- Resource-Constrained Environments: When API costs need to be tightly controlled.
When to use agents: (with example):
- Complex Problem-Solving Scenarios: Research and analysis tasks.
- Dynamic Customer Interactions: High-touch customer support.
- Multi-Step, Variable Tasks: Project management.
- Exploratory Processes: User research synthesis.
As LLM continue to evolve, the choice between workflows and agents isn't about finding the perfect solution, but about matching the right approach to your current challenges. Start with workflows where you need reliability and control, then evolve towards agents as your needs grow more complex.