Automating Workflows with CrewAI: From Setup to Execution in Jupyter
Brief Description of the CrewAI Framework
CrewAI is an innovative framework developed to streamline the integration of AI agents into various tasks and workflows. It provides a robust structure for defining and managing AI-driven agents, enabling them to work collaboratively on complex projects. The framework is particularly useful for automating research, content creation, and data analysis tasks.
Key Components of CrewAI:
Agent : Represents an AI entity with a specific role, goal, and backstory. Each agent can be equipped with various tools and capabilities, such as language models and search tools.
Task : Defines a specific piece of work to be completed by an agent. Tasks include a description, expected output, and the agent assigned to complete the task.
Crew : A collection of agents and tasks. The crew coordinates the execution of tasks, ensuring that agents work together efficiently to achieve the overall goal.
Tools : Extensions that provide additional functionalities to agents. These can include search tools, data processing utilities, and more.
LLM (Language Model) : Underlying models, such as OpenAI’s GPT, that power the agents’ natural language understanding and generation capabilities.
Development and Usage :
CrewAI was developed by a collaborative team of AI researchers and engineers aiming to simplify the deployment of AI agents in various applications.
It is increasingly adopted in tech industries for automating repetitive tasks, enhancing productivity, and enabling advanced data-driven insights.
Documentation and Installation:
Documentation for CrewAI can be found at the official CrewAI documentation site.
To install CrewAI, use the following pip command:
pip install crewai crewai[tools]The framework facilitates a modular approach, allowing developers to define and deploy agents for various purposes, such as conducting research, generating content, or analysing data. By leveraging powerful language models and integrating specialized tools, CrewAI enhances productivity and automates complex tasks.
What is SERPER?
Serper is an API designed for extracting structured information from web search results. It is particularly useful for integrating search capabilities into AI workflows, enabling agents to gather and analyze web data efficiently.
How to Get API Keys
To use CrewAI with OpenAI and Serper, you need to obtain API keys for both services.
Getting an OpenAI API Key
Sign Up: Go to the OpenAI website and create an account. Generate API Key: Once logged in, navigate to the API section of your account settings. Copy API Key: Generate a new API key and copy it. Store it securely as it will be used to authenticate your API requests.
Getting a Serper API Key
Sign Up: Visit the Serper website and create an account.
Generate API Key: After logging in, go to the API keys section in your account dashboard.
Copy API Key: Generate a new API key and copy it.
This key will be required to integrate Serper with CrewAI.
Cell 1: Import necessary libraries to set the environment
This cell imports the necessary libraries for setting up the environment. os is used for interacting with the operating system, and load_dotenv from the dotenv package is used to load environment variables from a .env file. Next we load environment variables from the .env file and retrieves the values of OPENAI_API_KEY and SERPER_API_KEY. The keys are printed to ensure they are loaded correctly.
import os
import warnings
from dotenv import load_dotenv
warnings.filterwarnings("ignore", message="Overriding of current TracerProvider is not allowed")
# Load the .env file
load_dotenv()
# Access the environment variables
openai_api_key = os.getenv("OPENAI_API_KEY")
serper_api_key = os.getenv("SERPER_API_KEY")
#print("OPENAI_API_KEY:", openai_api_key)
#print("SERPER_API_KEY:", serper_api_key)
Cell 2: Import libraries for CrewAI and SerperDevTool
This cell imports the required classes and tools from crewai, crewai_tools, and langchain_openai packages. These will be used to create agents, tasks, and manage interactions with the OpenAI API.
import sys
# Verify that the necessary packages are installed and can be imported
try:
from crewai import Agent, Task, Crew
from crewai_tools import SerperDevTool
from langchain_openai import ChatOpenAI
print('crewai and crewai_tools are installed and importable')
except ImportError as e:
print('Error:', e)
except Exception as e:
print('Unexpected error:', e)
crewai and crewai_tools are installed and importable
Cell 3: Initialize SerperDevTool
This cell initializes the SerperDevTool with the Serper API key. This tool will be used by the agent to perform search tasks.
# Initialize SerperDevTool with API key
search_tool = SerperDevTool(api_key=serper_api_key)
Cell 4: Define and initialize SearchAgent
This cell initializes the research agent with a specific role, goal, and backstory. The agent is configured to use the SerperDevTool for searches and the ChatOpenAI model for generating responses. The verbose parameter is set to False, meaning the agent will not print detailed logs.
researcher = Agent(
role = "Travel Research Specialist",
goal = "Identify emerging travel destinations and trends",
backstory = """You work at a leading travel agency, specializing in discovering new travel opportunities.
Your expertise lies in identifying popular destinations, analyzing travel trends, and providing actionable insights to improve travel offerings.""",
verbose = False,
allow_delegation = False,
tools=[search_tool],
llm = ChatOpenAI(model_name="gpt-4-turbo-preview", temperature=0.3, openai_api_key=openai_api_key)
)
In the context of a travel agency, setting allow_delegation to False means that the Travel Research Specialist will handle all research tasks independently without passing them to other agents. If set to True, the Travel Research Specialist could potentially delegate specific research tasks to other specialized agents, allowing for more complex and distributed workflows.
Cell 5: Initialize Itinerary Creator Agent
This cell initializes the itinerary creator agent with a different role, goal, and backstory. The itinerary_creator is responsible for creating engaging content based on the insights provided by the researcher. The verbose parameter is set to True, allowing detailed logs.
itinerary_creator = Agent(
role = 'Travel Itinerary Creator',
goal = 'Craft compelling travel itineraries for various destinations',
backstory = """You are a renowned travel itinerary creator, known for your engaging and informative travel plans.
You have a talent for making travel destinations come alive through your detailed and exciting itineraries, inspiring clients to explore the world.""",
verbose = True,
allow_delegation = True,
llm = ChatOpenAI(model_name="gpt-4-turbo-preview", temperature=0.7, openai_api_key=openai_api_key)
)
The temperature parameter in language models controls the randomness of the output. Lower values make the output more focused and deterministic, while higher values make it more creative and diverse.
For your agents, the temperature settings should reflect their goals:
Research Agent:
The research agent should provide accurate and reliable information. A lower temperature will help achieve this by making the output more deterministic and focused. Suggested temperature: 0.3 to 0.5
Itinerary Creator:
The itinerary creator benefits from creativity to craft engaging and diverse travel plans. A higher temperature can help achieve this by allowing for more creative output. Suggested temperature: 0.7 to 0.9
Cell 6: Define Task 1 for Researcher Agent
This cell defines the first task for the travel research specialist agent. The task involves conducting and identifying emerging travel destinations and trends.
task1 = Task(
description = """Conduct a comprehensive analysis to prepare a one-month travel itinerary for a couple traveling from New Zealand to Portugal.
The itinerary should include:
a) An optimal return flying route from Auckland, New Zealand,
b) Options to visit key historical places in Portugal,
c) Information on the most important museums,
d) Details on the most beautiful spots along the coastal line,
e) Advice on the best options for mobile telephone service providers,
f) Information on medical insurance for Portugal and how to use the Portuguese health system,
g) Guidance on using their NZ driving licenses and hiring a car,
h) Recommendations for the most valuable mobile applications that should help them in Portugal.
The itinerary report should be properly formatted for easy reading.""",
expected_output = "Detailed one-month travel itinerary report.",
agent = researcher
)
Cell 7: Define Task 2 for Itinerary Creator Agent
This cell defines the second task for the itinerary creator agent. The task involves compiling a detailed travel itinerary based on the insights provided by the researcher. The itinerary should be easy to read and cover all the sub-tasks of the researcher.
# Task for Itinerary Creator Agent
task2 = Task(
description = """Using the insights provided by the research agent, create a detailed travel itinerary for the couple.
The itinerary should include:
a) A day-by-day plan outlining the optimal return flying route from Auckland, New Zealand,
b) Visits to key historical places in Portugal,
c) Tours of the most important museums,
d) Exploration of the most beautiful spots along the coastal line,
e) Recommendations for mobile telephone service providers,
f) Information on medical insurance for Portugal and guidance on using the Portuguese health system,
g) Advice on using their NZ driving licenses and car hire options,
h) Suggestions for the most valuable mobile applications that are popular in Porugal that should help them in this country.
The itinerary should be engaging, easy to follow, and visually appealing.""",
expected_output = "Complete and detailed travel itinerary document.",
agent = itinerary_creator
)
Cell 8: Create Crew and Assign Tasks
crew = Crew(
agents = [researcher, itinerary_creator],
tasks = [task1, task2],
verbose = 1
)
This cell creates a Crew comprising the researcher and itinerary creator agents and assigns them their respective tasks. The verbose parameter is set to 1, enabling detailed logging during the execution of tasks.
Cell 9: Kickoff the Crew
import io
import contextlib
# Suppress output
with contextlib.redirect_stdout(io.StringIO()):
result = crew.kickoff()
This cell initiates the execution of the tasks assigned to the crew. The kickoff method runs the tasks sequentially or concurrently as defined in the crew configuration.
Cell 10: Print the Results
# Print the result in a formatted way (for better output use markdown package)
print(result)
**********************************************************************************************
One-Month Travel Itinerary for a New Zealand Couple Exploring Portugal
---
**Welcome to Your Portuguese Adventure!**
Embark on a breathtaking journey from the serene landscapes of New Zealand to the historic and vibrant heart of Portugal. This one-month itinerary is designed to immerse you in Portugal's rich culture, stunning scenery, and remarkable history, ensuring an unforgettable experience.
---
### **Pre-Departure Checklist:**
- **Flights Booked:** Auckland International Airport (AKL) to Lisbon Airport (LIS) via Dubai (DXB) or Singapore (SIN). Remember, booking 2-3 months in advance ensures the best fares.
- **Travel Insurance:** Ensure it covers medical expenses in Portugal.
- **Packing:** Prepare for a variety of climates and occasions. Don’t forget your chargers and an adapter!
---
### **Week 1: Lisbon and Surroundings**
**Day 1-3: Discovering Lisbon**
- **Morning:** Explore the historic Belém Tower and the Monument to the Discoveries.
- **Afternoon:** Visit the Jerónimos Monastery and taste the iconic Pastéis de Belém.
- **Evening:** Enjoy a Fado show in Alfama.
**Day 4: Sintra**
- **Day Trip:** Pena Palace, Moors Castle, and Quinta da Regaleira. Return to Lisbon by evening.
**Day 5: Cascais and Estoril**
- **Day Trip:** Relax on the beaches and explore the quaint streets.
**Day 6-7: Free Days in Lisbon**
- **Suggestions:** Explore local markets, try a cooking class, or take a day to relax.
---
### **Week 2: Porto and the North**
**Day 8: Travel to Porto**
- **Train from Lisbon:** Enjoy the scenic route.
**Day 9-11: Exploring Porto**
- **Highlights:** Lello Bookstore, Clerigos Tower, and a Port wine tour in Vila Nova de Gaia.
- **Day Trip:** Braga and Guimarães for historical insights.
---
### **Week 3: The Algarve Coast**
**Day 15: Travel to Algarve**
- **Option:** Rent a car for flexibility.
**Day 16-20: Coastal Wonders**
- **Highlights:** Lagos, Benagil Cave, and Faro. Enjoy beaches, cliffs, and seafood.
- **Activities:** Kayaking, snorkeling, and hiking.
---
### **Week 4: Central Portugal and Farewell**
**Day 21: Travel to Coimbra**
- **Visit:** University of Coimbra and the historic center.
**Day 22-23: Nazaré and Surrounds**
- **Explore:** The giant waves and traditional fishing culture.
**Day 24-25: Óbidos and Return to Lisbon**
- **Final Days:** Enjoy the medieval town of Óbidos before heading back to Lisbon for last-minute souvenirs and a farewell dinner.
---
### **Practical Information:**
- **Mobile Service:** Vodafone Portugal or MEO for tourist-friendly plans.
- **Medical Insurance:** Check for direct billing options with hospitals. Pharmacies are great for minor issues.
- **Driving:** Your NZ license is valid. Consider renting a GPS with your car.
- **Apps to Download:** Google Maps, XE Currency, Google Translate, Uber, and Zomato.
---
**(Itinerary Creator) Conclusion:**
From the moment you land in Lisbon to your final days soaking up the rich history and natural beauty of Portugal, this itinerary promises a blend of adventure, relaxation, and cultural immersion. Portugal awaits to offer you both the trip of a lifetime and memories that will last forever. Safe travels!
This cell prints the results of the tasks executed by the crew. The output will include the analysis report from the researcher and the itinerary from the itinerary creator.
Conclusion
In this post, we’ve demonstrated the capabilities of CrewAI for automating complex workflows involving multiple AI agents. By leveraging the power of OpenAI’s GPT-4 and Serper’s web search tool, we created a seamless process where one agent conducts comprehensive research and another crafts compelling itinerary based on the research findings. This approach can be extended and customized for various industries, showcasing the versatility and potential of CrewAI in enhancing productivity and driving innovation.
Whether you’re looking to streamline research processes, automate content creation, or explore other advanced AI applications, CrewAI provides a robust framework to achieve these goals. With detailed examples and clear instructions, this notebook serves as a practical guide to getting started with CrewAI, empowering you to harness the full potential of AI agents in your projects.