解锁高通AI推理套件SDK(4):Litellm、CrewAI与自动生成
作家的任务是使用研究员的成果,用 Markdown 语言撰写一篇博客文章。如需在使用 Qualcomm AI On-Prem Appliance 解决方案时使用工具调用,请确保设备盒具备与相关外部 API 通信的功能,以实现工具调用。是一个使用 OpenAI 格式调用 LLM API 的流行框架,许多框架正在使用它来抽象 API 调用。是一个流行的人工智能框架,用于创建自定义代理团队,旨在使用大型
Litellm
Litellm是一个使用 OpenAI 格式调用 LLM API 的流行框架,许多框架正在使用它来抽象 API 调用。
我们可以使用 litellm 来调用 Imagine Playground API。
先决条件
pip install litellm
对话
非流媒体
from rich.pretty import pprint
import litellm
# litellm.api_base=""
# litellm.api_key=""
response = litellm.completion(
model="openai/Llama-3.1-8B",
messages=[
{
"role": "user",
"content": "Tell me about the new wonders of the world",
}
],
max_tokens=1024,
)
pprint(response.choices[0].message.content)
这将打印类似以下内容:
You're referring to the New 7 Wonders of the World, a list compiled in 2007 through a worldwide poll. Here are the seven wonders:
1. **The Great Wall of China**: A series of fortifications built across several Chinese dynasties to protect the country from invasions. It stretches over 13,000 miles (21,000 km) and is one of the longest structures ever built.
2. **The Taj Mahal** (India): A mausoleum built by Mughal Emperor Shah Jahan in memory of his wife, Mumtaz Mahal. It's a stunning example of Mughal architecture, with intricate marble inlays and a perfect blend of Indian, Persian, and Islamic styles.
3. **Machu Picchu** (Peru): An Inca citadel built in the 15th century, abandoned before the arrival of the Spanish conquistadors, and rediscovered in the 20th century. It's a testament to the engineering and architectural skills of the Incas.
4. **Chichén Itzá** (Mexico): A pre-Columbian Mayan city on the Yucatán Peninsula, built by the Mayans around 1100 AD. It features the Pyramid of Kukulkan, a 98-foot-tall (30 meters) pyramid that aligns with the movements of the sun and the stars.
5. **The Roman Colosseum** (Italy): An amphitheater built in the 1st century AD, hosting gladiatorial contests, animal hunts, and public spectacles. It's a symbol of the power and engineering prowess of the ancient Romans.
6. **The Christ the Redeemer statue** (Brazil): A massive Art Deco statue of Jesus Christ, built in Rio de Janeiro between 1922 and 1931. It's 98 feet (30 meters) tall, with outstretched arms and a serene expression.
7. **The Pyramids of Giza** (Egypt): The only remaining ancient wonder from the original list, these pyramids are the oldest and only remaining structure from the original list of Seven Wonders of the Ancient World. The largest pyramid, the Great Pyramid of Giza, is an astonishing 481 feet (147 meters) tall.
...
流媒体
import litellm
# litellm.api_base=""
# litellm.api_key=""
stream_response = litellm.completion(
model="openai/Llama-3.1-8B",
messages=[
{
"role": "user",
"content": "Tell me about the new wonders of the world",
}
],
max_tokens=500,
stream=True,
)
for chunk in stream_response:
print(chunk.choices[0].delta.content or "", end="", flush=True)
这将打印类似以下内容:
You're referring to the New 7 Wonders of the World, a list compiled in 2007 through a worldwide poll. Here are the seven wonders:
1. **The Great Wall of China**: A series of fortifications built across several Chinese dynasties to protect the country from invasions. It stretches over 13,000 miles (21,000 km) and is one of the longest structures ever built.
2. **The Taj Mahal** (India): A mausoleum built by Mughal Emperor Shah Jahan in memory of his wife, Mumtaz Mahal. It's a stunning example of Mughal architecture, with intricate marble inlays and a perfect blend of Indian, Persian, and Islamic styles.
3. **Machu Picchu** (Peru): An Inca citadel built in the 15th century, abandoned before the arrival of the Spanish conquistadors, and rediscovered in the 20th century. It's a testament to the engineering and architectural skills of the Incas.
4. **Chichén Itzá** (Mexico): A pre-Columbian Mayan city on the Yucatán Peninsula, built by the Mayans around 1100 AD. It features the Pyramid of Kukulkan, a 98-foot-tall (30 meters) pyramid that aligns with the movements of the sun and the stars.
5. **The Roman Colosseum** (Italy): An amphitheater built in the 1st century AD, hosting gladiatorial contests, animal hunts, and public spectacles. It's a symbol of the power and engineering prowess of the ancient Romans.
6. **The Christ the Redeemer statue** (Brazil): A massive Art Deco statue of Jesus Christ, built in Rio de Janeiro between 1922 and 1931. It's 98 feet (30 meters) tall, with outstretched arms and a serene expression.
7. **The Pyramids of Giza** (Egypt): The only remaining ancient wonder from the original list, these pyramids are the oldest and only remaining structure from the original list of Seven Wonders of the Ancient World. The largest pyramid, the Great Pyramid of Giza, is an astonishing 481 feet (147 meters) tall.
...
完成
from rich.pretty import pprint
import litellm
# litellm.api_base=""
# litellm.api_key=""
completion_response = litellm.text_completion(
prompt="def fibonacci(n):",
model="text-completion-openai/Llama-3.1-8B",
max_tokens=100,
)
pprint(completion_response.choices[0].text)
这将打印类似以下内容:
\n if n <= 0:\n return "Input should be a positive integer"\n elif n == 1:\n return 0\n elif n == 2:\n return 1\n else:\n a, b = 0, 1\n for _ in range(2, n):\n a, b = b, a + b\n return b\n ...
嵌入
from rich.pretty import pprint
import litellm
litellm.suppress_debug_info = True
# litellm.api_base=""
# litellm.api_key=""
response = litellm.embedding(
model="openai/BAAI/bge-large-en-v1.5", input=["This is amazing", "Quite good stuff"]
)
pprint(response)
CrewAI
CrewAI是一个流行的人工智能框架,用于创建自定义代理团队,旨在使用大型语言模型 (LLM) 执行各种任务。
使用ImagineLLM,我们可以将 高通AI Inference Suite SDK 与 CrewAI 代理一起使用。
先决条件
在运行下面的示例之前,请确保安装了支持 CrewAI 的 Qualcomm AI Inference Suite SDK高通安装了支持 CrewAI 的 Qualcomm AI Inference Suite SDK。
基本示例
以下示例结合了两个代理:一个研究员和一个作家。研究员的任务是解释五大 AI 模型及其具体应用。作家的任务是使用研究员的成果,用 Markdown 语言撰写一篇博客文章。
import os
from crewai import LLM, Agent, Crew, Process, Task
os.environ["OTEL_SDK_DISABLED"] = "true"
llm = LLM(model="openai/Llama-3.1-8B", base_url="", api_key="", max_tokens=1024)
research = Agent(
role="researcher",
goal="the goal of this agent is to research about new AI models and their applications",
backstory="this agent is a data scientist researcher and is interested in AI models",
verbose=True,
allow_delegation=False,
llm=llm,
max_retry_limit=2,
)
writer = Agent(
role="writer",
goal="the goal of this agent is to write about AI models and their applications",
backstory="this agent is a writer for a blog and is interested in AI models",
verbose=True,
allow_delegation=False,
llm=llm,
max_retry_limit=2,
)
task1 = Task(
description="research about new AI models and their applications",
agent=research,
expected_output="give a list of the top 5 AI models and their specific applications, do a short answer",
)
task2 = Task(
description="write about AI models and their applications",
agent=writer,
expected_output="a blog post with at least 3 main parts, in markdown format, use the list given by the researcher",
)
crew = Crew(
name="AI Models Crew",
agents=[research, writer],
tasks=[task1, task2],
verbose=True,
share_crew=False,
process=Process.sequential,
)
result = crew.kickoff()
print("-----------------------------")
print("Crew execution result:")
print("-----------------------------")
print(result)
这是该代码的示例输出:
==================================================
Crew execution result:
==================================================
# Top 5 AI Models and Their Applications
In recent years, Artificial Intelligence (AI) has made tremendous progress, and various
AI models have been developed to solve complex problems. In this blog post, we will
explore the top 5 AI models and their specific applications.
### 1. Generative Adversarial Networks (GANs)
GANs are a type of AI model that uses a neural network to generate new data that is
similar to a given dataset. The model consists of two neural networks: a generator and
a discriminator. The generator creates new data, while the discriminator evaluates the
generated data and tells the generator whether it is realistic or not.
Applications:
* Image generation: GANs can be used to generate realistic images of objects, such as
faces, animals, and buildings.
* Data augmentation: GANs can be used to generate new data that is similar to a given
dataset, which can be used to train machine learning models.
* Style transfer: GANs can be used to transfer the style of one image to another
image.
### 2. Recurrent Neural Networks (RNNs)
RNNs are a type of AI model that is designed to process sequential data, such as speech,
text, and time series data. The model consists of a series of neural networks that are
connected in a loop, allowing the model to keep track of the context of the input data.
Applications:
* Language translation: RNNs can be used to translate text from one language to another.
* Speech recognition: RNNs can be used to recognize spoken language and transcribe it
into text.
* Time series forecasting: RNNs can be used to forecast future values in a time series
based on past values.
### 3. Convolutional Neural Networks (CNNs)
CNNs are a type of AI model that is designed to process data that is composed of grids
of values, such as images and videos. The model consists of a series of neural networks that are connected in a convolutional manner, allowing the model to extract features from the input data.
Applications:
* Image classification: CNNs can be used to classify images into different categories,
such as objects, scenes, and actions.
* Object detection: CNNs can be used to detect objects in images and videos.
* Image segmentation: CNNs can be used to segment images into different regions, such as
objects, textures, and backgrounds.
### 4. Transformers
Transformers are a type of AI model that is designed to process sequential data, such as
text and speech. The model consists of a series of neural networks that are connected in
a loop, allowing the model to keep track of the context of the input data.
Applications:
* Language translation: Transformers can be used to translate text from one language to
another.
* Text summarization: Transformers can be used to summarize long pieces of text into shorter
summaries.
* Question answering: Transformers can be used to answer questions based on a given
piece of text.
### 5. Autoencoders
Autoencoders are a type of AI model that is designed to learn a compact representation
of the input data. The model consists of an encoder and a decoder, which are connected
in a loop. The encoder maps the input data to a lower-dimensional representation, while
the decoder maps the lower-dimensional representation back to the original input data.
Applications:
* Dimensionality reduction: Autoencoders can be used to reduce the dimensionality of
high-dimensional data, such as images and videos.
* Generative modeling: Autoencoders can be used to generate new data that is similar to
a given dataset.
* Anomaly detection: Autoencoders can be used to detect anomalies in a dataset by
identifying data points that are farthest from the mean of the dataset.
In conclusion, the top 5 AI models and their specific applications have the potential
to revolutionize many industries and fields. These models have been used in a variety
of applications, including image generation, data augmentation, style transfer, language
translation, speech recognition, time series forecasting, image classification, object
detection, image segmentation, text summarization, question answering, anomaly
detection, dimensionality reduction, and generative modeling. As AI continues to evolve,
we can expect to see even more innovative applications of these models in the future.
拥有工具的代理
我们还可以实现各种工具与您的代理的集成。
人工智能设备
如需在使用 高通 AI On-Prem Appliance 解决方案时使用工具调用,请确保设备盒具备与相关外部 API 通信的功能,以实现工具调用。
例如,您可以使用Exa 搜索工具或一些简单的工具来执行一些任务。以下示例安装以下依赖项:
然后执行:
import os
from crewai_tools import EXASearchTool, tool
from crewai import LLM, Agent, Crew, Process, Task
os.environ["OTEL_SDK_DISABLED"] = "true"
# https://docs.crewai.com/tools/exasearchtool
# You would have to set the API Key for ExaSearchTool
llm = LLM(model="openai/Llama-3.1-8B", base_url="", api_key="", max_tokens=1024)
@tool
def AdditionTool(a: int, b: int) -> str:
"""
can perform additions
"""
return a + b
@tool
def MultiplicationTool(a: int, b: int) -> str:
"""
can perform multiplications
"""
return a * b
research = Agent(
role="professor",
goal="the goal of this agent is to give the answer of basic calculus questions and history questions",
backstory="this agent is a professor for middle school student",
verbose=True,
allow_delegation=False,
llm=llm,
cache=True,
max_retry_limit=0,
tools=[EXASearchTool(), AdditionTool, MultiplicationTool],
)
task1 = Task(
description="what is the result of 9 + 12? what is the result of 3 * 4? what's the name of the first US president? what is the result of 12 * 7?",
agent=research,
expected_output="the answer of each question",
max_retry_limit=0,
)
crew = Crew(
name="AI Models Crew",
agents=[research],
tasks=[task1],
verbose=True,
share_crew=False,
process=Process.sequential,
)
result = crew.kickoff()
print("=" * 50)
print("Crew execution result:")
print("=" * 50)
print(result)
这将生成类似以下内容的输出:
==================================================
Crew execution result:
==================================================
The result of 9 + 12 is 21.
The result of 3 * 4 is 12.
The first US president was George Washington.
The result of 12 * 7 is 84.
AutoGen
AutoGen是另一个 AI 框架,它允许您使用 LLM 创建自定义代理团队,用于执行多项任务。Autogen 代理还能够运行代码并测试输出,以便根据需要进行修改。
安装
pip install autogen-agentchat~=0.2
基本示例
在这个例子中,我们要求两个代理编写一个快速排序函数。这里有两个代理,一个负责编写函数,另一个负责执行代码。
import httpx
from autogen import ConversableAgent, UserProxyAgent
class HttpClient(httpx.Client):
def __deepcopy__(self, memo):
return self
# Enter base_url and api_key
config_list = [
{
"model": "Llama-3.1-8B",
"base_url": "",
"api_key": "",
"max_tokens": 256,
"temperature": 0.5,
"http_client": HttpClient(verify=False),
}
]
assistant = ConversableAgent("assistant", llm_config={"config_list": config_list})
user_proxy = UserProxyAgent(
"user_proxy",
code_execution_config={
"work_dir": "coding",
"use_docker": False,
},
)
code_writer_agent_system_message = assistant.system_message
print(" system message ".center(100, "="))
print(code_writer_agent_system_message)
print(
" code for prompt: Write python code to perform a quicksort over a python list ".center(
100, "="
)
)
user_proxy.initiate_chat(
assistant,
message="Write python code to perform a quicksort over a python list",
verify=False,
)
print("=" * 100)
这将生成类似以下内容的输出:
Here is an example of a Python implementation of the quicksort algorithm:
```
def quicksort(arr):
if len(arr) <= 1:
return arr
pivot = arr[0]
less = [x for x in arr[1:] if x <= pivot]
greater = [x for x in arr[1:] if x > pivot]
return quicksort(less) + [pivot] + quicksort(greater)
```
This implementation uses the "Lomuto" partition scheme, which is a variation of the standard "Hoare" partition scheme that is slightly faster and more efficient.
Here's an explanation of how the code works:
1. If the length of the input array is 0 or 1, return the original array (since it is already sorted).
2. Choose the first element of the array as the pivot.
3. Create two lists: `less` and `greater`. `less` contains all elements in the array that are less than or equal to the pivot, and `greater` contains all elements that are greater than the pivot.
4. Recursively call the `quicksort` function on the `less` and `greater` lists.
5. Concatenate the results of the recursive calls
火山引擎开发者社区是火山引擎打造的AI技术生态平台,聚焦Agent与大模型开发,提供豆包系列模型(图像/视频/视觉)、智能分析与会话工具,并配套评测集、动手实验室及行业案例库。社区通过技术沙龙、挑战赛等活动促进开发者成长,新用户可领50万Tokens权益,助力构建智能应用。
更多推荐
所有评论(0)