notebooks/zh-CN/phoenix_observability_on_hf_spaces.ipynb (603 lines of code) (raw):
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 在 Hugging Face Spaces 上设置 Phoenix 观察性仪表板以进行 LLM 应用程序追踪\n",
"\n",
"_作者：[Andrew Reed](https://huggingface.co/andrewrreed)_\n",
"\n",
"[Phoenix](https://docs.arize.com/phoenix) 是由 [Arize AI](https://arize.com/) 开发的开源观察性库，旨在进行实验、评估和故障排除。它允许 AI 工程师和数据科学家快速可视化他们的数据、评估性能、追踪问题，并导出数据以进行改进。\n",
"\n",
"<div style=\"display: flex; justify-content: space-between; gap: 20px; margin: 20px 0;\">\n",
" <div style=\"width: 32%; text-align: center; display: flex; flex-direction: column; align-items: center; justify-content: flex-start;\">\n",
" <img src=\"https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/phoenix-tracing.png\" style=\"width: 100%; height: 200px; object-fit: contain;\"/>\n",
" <p style=\"margin-top: 10px; margin-bottom: 0;\">追踪</p>\n",
" </div>\n",
" <div style=\"width: 32%; text-align: center; display: flex; flex-direction: column; align-items: center; justify-content: flex-start;\">\n",
" <img src=\"https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/phoenix-datasets.png\" style=\"width: 100%; height: 200px; object-fit: contain;\"/>\n",
" <p style=\"margin-top: 10px; margin-bottom: 0;\">数据集</p>\n",
" </div>\n",
" <div style=\"width: 32%; text-align: center; display: flex; flex-direction: column; align-items: center; justify-content: flex-start;\">\n",
" <img src=\"https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/phoenix-experiments.png\" style=\"width: 100%; height: 200px; object-fit: contain;\"/>\n",
" <p style=\"margin-top: 10px; margin-bottom: 0;\">实验</p>\n",
" </div>\n",
"</div>\n",
"\n",
"在本 Notebook 中，我们将展示如何在 [Hugging Face Spaces](https://huggingface.co/spaces) 上部署 Phoenix 观察性仪表板，并配置它以自动追踪 LLM 调用，从而全面了解 LLM 应用程序的内部工作原理。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 第一步：在 Hugging Face Spaces 上部署 Phoenix\n",
"\n",
"虽然 Phoenix 提供了一个用于本地开发的 [Notebook 高级选项](https://docs.arize.com/phoenix/deployment/environments#notebooks)，它也可以通过 [Docker 作为独立仪表板部署](https://docs.arize.com/phoenix/deployment/environments#container)。长时间运行的托管仪表板是提供 LLM 应用行为集中视图的好方法，并且有助于团队协作。Hugging Face Spaces 提供了一种简单的方式来托管 ML 应用，并支持可选的持久存储，且它对自定义 Docker 镜像的支持使其成为托管 Phoenix 的理想平台——我们来看看它是如何工作的！\n",
"\n",
"首先，我们将 [复制示例空间](https://huggingface.co/spaces/andrewrreed/phoenix-arize-observability-demo?duplicate=true)\n",
"\n",
"\n",
"\n",
"我们可以将空间配置为私有或公开，并且它可以位于我们的用户命名空间或组织命名空间中。我们可以使用默认的免费 CPU 配置，并且重要的是，我们可以指定要为该空间附加一个持久磁盘。\n",
"\n",
"> [!TIP] 为了使追踪数据在 Space 重启后保持持久，我们 _必须_ 配置持久磁盘，否则所有数据将在 Space 重启时丢失。配置持久磁盘是付费功能，并会产生 Space 存续期间的费用。在本例中，我们将使用小型 - 20GB 磁盘选项，每小时 $0.01。\n",
"\n",
"点击“复制空间”后，Docker 镜像将开始构建。这将需要几分钟时间完成，之后我们将看到一个空的 Phoenix 仪表板。\n",
"\n",
"\n",
"\n",
"## 第二步：配置应用追踪\n",
"\n",
"现在我们已经有了一个运行中的 Phoenix 仪表板，可以使用 [OpenTelemetry TracerProvider](https://docs.arize.com/phoenix/quickstart#connect-your-app-to-phoenix) 配置应用程序，以自动追踪 LLM 调用。在这个例子中，我们将使用 OpenAI 客户端库来给应用程序加上监控，并追踪从 `openai` Python 包发出的 LLM 调用，这些调用会被发送到 [Hugging Face 的 Serverless 推理 API](https://huggingface.co/docs/api-inference/en/index) 上运行的 LLM。\n",
"\n",
"> [!TIP] Phoenix 支持对 [多种 LLM 框架](https://docs.arize.com/phoenix/tracing/integrations-tracing) 进行追踪，包括 LangChain、LlamaIndex、AWS Bedrock 等。\n",
"\n",
"首先，我们需要安装必要的库："
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install -q arize-phoenix arize-phoenix-otel openinference-instrumentation-openai openai huggingface-hub"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"接下来，我们将使用 `huggingface_hub` 库登录到 Hugging Face。这将允许我们生成所需的身份验证，以便为我们的 Space 和 Serverless 推理 API 提供认证。请确保用于身份验证的 Hugging Face 令牌具有正确的权限，以便访问存放你 Space 的组织。"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from huggingface_hub import interpreter_login\n",
"\n",
"interpreter_login()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"现在，我们可以将 [Phoenix 客户端配置](https://docs.arize.com/phoenix/deployment/configuration#client-configuration) 为连接到我们运行中的 Phoenix 仪表板：\n",
"\n",
"1. 注册 Phoenix 追踪提供者：\n",
" - 指定我们选择的 `project_name`\n",
" - 将 `endpoint` 设置为我们 Space 的主机名（可以通过仪表板 UI 中的“设置”标签找到，如下图所示）\n",
" - 将 `headers` 设置为访问 Space 所需的 Hugging Face 头信息\n",
"2. 在我们的应用代码中，使用 OpenAI 追踪提供者进行监控\n",
"\n",
""
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"🔭 OpenTelemetry Tracing Details 🔭\n",
"| Phoenix Project: test\n",
"| Span Processor: SimpleSpanProcessor\n",
"| Collector Endpoint: https://andrewrreed-phoenix-arize-observability-demo.hf.space/v1/traces\n",
"| Transport: HTTP\n",
"| Transport Headers: {'user-agent': '****', 'authorization': '****'}\n",
"| \n",
"| Using a default SpanProcessor. `add_span_processor` will overwrite this default.\n",
"| \n",
"| `register` has set this TracerProvider as the global OpenTelemetry default.\n",
"| To disable this behavior, call `register` with `set_global_tracer_provider=False`.\n",
"\n"
]
}
],
"source": [
"from phoenix.otel import register\n",
"from huggingface_hub.utils import build_hf_headers\n",
"from openinference.instrumentation.openai import OpenAIInstrumentor\n",
"\n",
"# 1. Register the Phoenix tracer provider\n",
"tracer_provider = register(\n",
" project_name=\"test\",\n",
" endpoint=\"https://andrewrreed-phoenix-arize-observability-demo.hf.space\"\n",
" + \"/v1/traces\",\n",
" headers=build_hf_headers(),\n",
")\n",
"\n",
"# 2. Instrument our application code to use the OpenAI tracer provider\n",
"OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 第三步：发起调用并在 Phoenix 仪表板中查看追踪\n",
"\n",
"现在，我们可以调用 LLM 并在 Phoenix 仪表板中查看追踪。我们使用 OpenAI 客户端向 Hugging Face 的 Serverless 推理 API 发起调用，该 API 已经配置为与 Phoenix 配合工作。在这个例子中，我们使用的是 `meta-llama/Llama-3.1-8B-Instruct` 模型。"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Llamas are intelligent and social animals, and they do have ways to entertain themselves and have fun. While we can't directly ask a llama about its personal preferences, we can observe their natural behaviors and make some educated guesses. Here are some things that might bring a llama joy and excitement:\n",
"\n",
"1. **Socializing**: Llamas are herd animals and they love to interact with each other. They'll often engage in play-fighting, neck-wrestling, and other activities to establish dominance and strengthen social bonds. When llamas have a strong social network, it can make them feel happy and content.\n",
"2. **Exploring new environments**: Llamas are naturally curious creatures, and they love to explore new surroundings. They'll often investigate their surroundings, sniffing and investigating new sights, sounds, and smells.\n",
"3. **Playing with toys**: While llamas don't need expensive toys, they do enjoy playing with objects that stimulate their natural behaviors. For example, a ball or a toy that mimics a target can be an entertaining way to engage them.\n",
"4. **Climbing and jumping**: Llamas are agile and athletic animals, and they enjoy using their limbs to climb and jump over obstacles. Providing a safe and stable area for them to exercise their physical abilities can be a fun and engaging experience.\n",
"5. **Browsing and foraging**: Llamas have a natural instinct to graze and browse, and they enjoy searching for tasty plants and shrubs. Providing a variety of plants to munch on can keep them engaged and entertained.\n",
"6. **Mentally stimulating activities**: Llamas are intelligent animals, and they can benefit from mentally stimulating activities like problem-solving puzzles or learning new behaviors (like agility training or obedience training).\n",
"\n",
"Some fun activities you can try with a llama include:\n",
"\n",
"* Setting up an obstacle course or agility challenge\n",
"* Creating a \"scavenger hunt\" with treats and toys\n",
"* Introducing new toys or objects to stimulate their curiosity\n",
"* Providing a variety of plants and shrubs to browse and graze on\n",
"* Engaging in interactive games like \"follow the leader\" or \"find the treat\"\n",
"\n",
"Remember to always prioritize the llama's safety and well-being, and to consult with a veterinarian or a trained llama handler before attempting any new activities or introducing new toys.\n"
]
}
],
"source": [
"from openai import OpenAI\n",
"from huggingface_hub import get_token\n",
"\n",
"client = OpenAI(\n",
" base_url=\"https://api-inference.huggingface.co/v1/\",\n",
" api_key=get_token(),\n",
")\n",
"\n",
"messages = [{\"role\": \"user\", \"content\": \"What does a llama do for fun?\"}]\n",
"\n",
"response = client.chat.completions.create(\n",
" model=\"meta-llama/Llama-3.1-8B-Instruct\",\n",
" messages=messages,\n",
" max_tokens=500,\n",
")\n",
"\n",
"print(response.choices[0].message.content)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"如果我们返回到 Phoenix 仪表板，我们可以看到从 LLM 调用捕获并显示的追踪信息！如果你在配置 Space 时使用了持久磁盘，那么每次重启 Space 时，所有的追踪信息都会被保存。\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 奖励部分：使用 CrewAI 追踪多代理应用程序\n",
"\n",
"观察性的真正优势在于能够追踪和检查复杂的 LLM 工作流。在这个例子中，我们将安装并使用 [CrewAI](https://www.crewai.com/) 来追踪一个多智能体应用程序。\n",
"\n",
"> [!TIP] `openinference-instrumentation-crewai` 包目前要求 Python 3.10 或更高版本。安装 `crewai` 库后，你可能需要重启 Notebook 内核以避免出现错误。"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install -q openinference-instrumentation-crewai crewai crewai-tools"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"和之前一样，我们将注册 Phoenix 追踪提供者并给应用代码加上监控，但这次我们还需要取消现有 OpenAI 追踪提供者的监控，以避免冲突。"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Overriding of current TracerProvider is not allowed\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"🔭 OpenTelemetry Tracing Details 🔭\n",
"| Phoenix Project: crewai\n",
"| Span Processor: SimpleSpanProcessor\n",
"| Collector Endpoint: https://andrewrreed-phoenix-arize-observability-demo.hf.space/v1/traces\n",
"| Transport: HTTP\n",
"| Transport Headers: {'user-agent': '****', 'authorization': '****'}\n",
"| \n",
"| Using a default SpanProcessor. `add_span_processor` will overwrite this default.\n",
"| \n",
"| `register` has set this TracerProvider as the global OpenTelemetry default.\n",
"| To disable this behavior, call `register` with `set_global_tracer_provider=False`.\n",
"\n"
]
}
],
"source": [
"from opentelemetry import trace\n",
"from openinference.instrumentation.crewai import CrewAIInstrumentor\n",
"\n",
"# 0. Uninstrument existing tracer provider and clear the global tracer provider\n",
"OpenAIInstrumentor().uninstrument()\n",
"if trace.get_tracer_provider():\n",
" trace.get_tracer_provider().shutdown()\n",
" trace._TRACER_PROVIDER = None # Reset the global tracer provider\n",
"\n",
"# 1. Register the Phoenix tracer provider\n",
"tracer_provider = register(\n",
" project_name=\"crewai\",\n",
" endpoint=\"https://andrewrreed-phoenix-arize-observability-demo.hf.space\"\n",
" + \"/v1/traces\",\n",
" headers=build_hf_headers(),\n",
")\n",
"\n",
"# 2. Instrument our application code to use the OpenAI tracer provider\n",
"CrewAIInstrumentor().instrument(tracer_provider=tracer_provider)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"现在，我们将使用 CrewAI 定义一个多智能体应用程序，研究并撰写一篇关于观察性和追踪在 LLM 应用中的重要性的博客。\n",
"\n",
"> [!TIP] 这个例子借鉴并修改自 [此处](https://docs.arize.com/phoenix/tracing/integrations-tracing/crewai)。"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-10-31 09:10:49,986 - 8143739712 - __init__.py-__init__:538 - WARNING: Overriding of current TracerProvider is not allowed\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[1m\u001b[95m# Agent:\u001b[00m \u001b[1m\u001b[92mResearcher\u001b[00m\n",
"\u001b[95m## Task:\u001b[00m \u001b[92mConduct comprehensive research and analysis of the importance of observability and tracing in LLM applications.\n",
" Identify key trends, breakthrough technologies, and potential industry impacts.\u001b[00m\n",
"\n",
"\n",
"\u001b[1m\u001b[95m# Agent:\u001b[00m \u001b[1m\u001b[92mResearcher\u001b[00m\n",
"\u001b[95m## Using tool:\u001b[00m \u001b[92mSearch the internet\u001b[00m\n",
"\u001b[95m## Tool Input:\u001b[00m \u001b[92m\n",
"\"{\\\"search_query\\\": \\\"importance of observability and tracing in LLM applications\\\"}\"\u001b[00m\n",
"\u001b[95m## Tool Output:\u001b[00m \u001b[92m\n",
"\n",
"Search results: Title: LLM Observability: The 5 Key Pillars for Monitoring Large Language ...\n",
"Link: https://arize.com/blog-course/large-language-model-monitoring-observability/\n",
"Snippet: Why leveraging the five pillars of LLM observability is essential for ensuring performance, reliability, and seamless LLM applications.\n",
"---\n",
"Title: Observability of LLM Applications: Exploration and Practice from the ...\n",
"Link: https://www.alibabacloud.com/blog/observability-of-llm-applications-exploration-and-practice-from-the-perspective-of-trace_601604\n",
"Snippet: This article clarifies the technical challenges of observability by analyzing LLM application patterns and different concerns.\n",
"---\n",
"Title: What is LLM Observability? - The Ultimate LLM Monitoring Guide\n",
"Link: https://www.confident-ai.com/blog/what-is-llm-observability-the-ultimate-llm-monitoring-guide\n",
"Snippet: Observability tools collect and correlate logs, real-time evaluation metrics, and traces to understand the context of unexpected outputs or ...\n",
"---\n",
"Title: An Introduction to Observability for LLM-based applications using ...\n",
"Link: https://opentelemetry.io/blog/2024/llm-observability/\n",
"Snippet: Why Observability Matters for LLM Applications · It's vital to keep track of how often LLMs are being used for usage and cost tracking. · Latency ...\n",
"---\n",
"Title: Understanding LLM Observability - Key Insights, Best Practices ...\n",
"Link: https://signoz.io/blog/llm-observability/\n",
"Snippet: LLM Observability is essential for maintaining reliable, accurate, and efficient AI applications. Focus on the five pillars: evaluation, tracing ...\n",
"---\n",
"Title: LLM Observability Tools: 2024 Comparison - lakeFS\n",
"Link: https://lakefs.io/blog/llm-observability-tools/\n",
"Snippet: LLM observability is the process that enables monitoring by providing full visibility and tracing in an LLM application system, as well as newer ...\n",
"---\n",
"Title: From Concept to Production with Observability in LLM Applications\n",
"Link: https://hadijaveed.me/2024/03/05/tracing-and-observability-in-llm-applications/\n",
"Snippet: Traces are essential to understanding the full “path” a request takes in your application, e.g, prompt, query-expansion, RAG retrieved top-k ...\n",
"---\n",
"Title: The Importance of LLM Observability: A Technical Deep Dive\n",
"Link: https://www.linkedin.com/pulse/importance-llm-observability-technical-deep-dive-patrick-carroll-trlqe\n",
"Snippet: LLM observability is crucial for any technical team that wants to maintain and improve the reliability, security, and performance of their AI- ...\n",
"---\n",
"Title: Observability and Monitoring of LLMs - TheBlue.ai\n",
"Link: https://theblue.ai/blog/llm-observability-en/\n",
"Snippet: LLM-Observability is crucial to maximize the performance and reliability of Large Language Models (LLMs). By systematically capturing and ...\n",
"---\n",
"\u001b[00m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-10-31 09:11:21,244 - 8143739712 - __init__.py-__init__:100 - WARNING: Invalid type TaskOutput for attribute 'output.value' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m\u001b[95m# Agent:\u001b[00m \u001b[1m\u001b[92mResearcher\u001b[00m\n",
"\u001b[95m## Final Answer:\u001b[00m \u001b[92m\n",
"**Comprehensive Research and Analysis Report: Importance of Observability and Tracing in LLM Applications**\n",
"\n",
"**Introduction**\n",
"\n",
"Large Language Models (LLMs) have revolutionized the field of natural language processing, enabling applications such as language translation, text summarization, and conversational AI. However, as LLMs become increasingly complex and widespread, ensuring their performance, reliability, and security has become a significant challenge. Observability and tracing are crucial components in addressing this challenge, enabling developers to monitor, debug, and optimize LLM applications. This report provides an in-depth analysis of the importance of observability and tracing in LLM applications, highlighting key trends, breakthrough technologies, and potential industry impacts.\n",
"\n",
"**Key Trends:**\n",
"\n",
"* **Increased Adoption of Cloud-Native Observability Tools**: Cloud-native observability tools, such as OpenTelemetry and Signoz, are gaining popularity due to their ability to provide real-time insights into LLM application performance and behavior.\n",
"* **Growing Importance of Tracing**: Tracing has become a critical aspect of LLM observability, enabling developers to understand the flow of requests through the application and identify bottlenecks.\n",
"* **Rise of AI-Specific Observability Platforms**: Specialized observability platforms, such as Arize and LakeFS, are emerging to cater to the unique needs of LLM applications.\n",
"\n",
"**Breakthrough Technologies:**\n",
"\n",
"* **OpenTelemetry**: OpenTelemetry is an open-source observability framework that enables developers to collect and correlate logs, metrics, and traces from LLM applications.\n",
"* **Signoz**: Signoz is a cloud-native observability platform that provides real-time insights into LLM application performance and behavior.\n",
"* **Arize**: Arize is an AI-specific observability platform that enables developers to monitor and optimize LLM applications in real-time.\n",
"\n",
"**Potential Industry Impacts:**\n",
"\n",
"* **Improved Performance**: Observability and tracing enable developers to identify and address performance bottlenecks, leading to improved response times and user experience.\n",
"* **Enhanced Reliability**: By monitoring LLM application behavior, developers can identify and fix issues before they impact users, ensuring high uptime and reliability.\n",
"* **Increased Security**: Observability and tracing enable developers to detect and respond to security threats in real-time, protecting sensitive user data and preventing data breaches.\n",
"\n",
"**Conclusion**\n",
"\n",
"Observability and tracing are critical components of LLM application development, enabling developers to monitor, debug, and optimize their applications. As LLMs continue to evolve and become increasingly complex, the importance of observability and tracing will only continue to grow. By adopting cloud-native observability tools, leveraging breakthrough technologies like OpenTelemetry and Signoz, and staying up-to-date with industry trends, developers can ensure the performance, reliability, and security of their LLM applications.\u001b[00m\n",
"\n",
"\n",
"\u001b[1m\u001b[95m# Agent:\u001b[00m \u001b[1m\u001b[92mTechnical Writer\u001b[00m\n",
"\u001b[95m## Task:\u001b[00m \u001b[92mUsing the insights provided, develop an engaging blog\n",
" post that highlights the importance of observability and tracing in LLM applications.\n",
" Your post should be informative yet accessible, catering to a tech-savvy audience.\n",
" Make it sound cool, avoid complex words so it doesn't sound like AI.\u001b[00m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-10-31 09:11:39,715 - 8143739712 - __init__.py-__init__:100 - WARNING: Invalid type TaskOutput for attribute 'output.value' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m\u001b[95m# Agent:\u001b[00m \u001b[1m\u001b[92mTechnical Writer\u001b[00m\n",
"\u001b[95m## Final Answer:\u001b[00m \u001b[92m\n",
"**Unlocking the Power of Observability and Tracing in LLM Applications**\n",
"\n",
"As Large Language Models (LLMs) continue to revolutionize the field of natural language processing, ensuring their performance, reliability, and security has become a top priority for developers. With the increasing complexity of LLMs, it's essential to have a robust monitoring and debugging system in place to identify and address issues before they impact users. This is where observability and tracing come into play – two critical components that enable developers to monitor, debug, and optimize LLM applications.\n",
"\n",
"**The Importance of Observability and Tracing in LLM Applications**\n",
"\n",
"Observability and tracing are not just buzzwords; they're essential tools for any LLM developer. By implementing observability and tracing, developers can gain real-time insights into their application's performance and behavior, allowing them to identify bottlenecks, optimize resource utilization, and ensure high uptime and reliability. With observability and tracing, developers can:\n",
"\n",
"* **Monitor application performance**: Get real-time insights into application performance, including response times, latency, and error rates.\n",
"* **Identify bottlenecks**: Use tracing to understand the flow of requests through the application and identify areas where performance can be improved.\n",
"* **Optimize resource utilization**: Make data-driven decisions to optimize resource allocation and reduce waste.\n",
"* **Ensure high uptime and reliability**: Identify and fix issues before they impact users, ensuring high uptime and reliability.\n",
"* **Protect sensitive user data**: Detect and respond to security threats in real-time, protecting sensitive user data and preventing data breaches.\n",
"\n",
"**The Rise of Cloud-Native Observability Tools and AI-Specific Observability Platforms**\n",
"\n",
"The observability landscape is evolving rapidly, with cloud-native observability tools and AI-specific observability platforms emerging to cater to the unique needs of LLM applications. Some of the key players in this space include:\n",
"\n",
"* **OpenTelemetry**: An open-source observability framework that enables developers to collect and correlate logs, metrics, and traces from LLM applications.\n",
"* **Signoz**: A cloud-native observability platform that provides real-time insights into LLM application performance and behavior.\n",
"* **Arize**: An AI-specific observability platform that enables developers to monitor and optimize LLM applications in real-time.\n",
"\n",
"**The Future of Observability and Tracing in LLM Applications**\n",
"\n",
"As LLMs continue to evolve and become increasingly complex, the importance of observability and tracing will only continue to grow. By adopting cloud-native observability tools, leveraging breakthrough technologies like OpenTelemetry and Signoz, and staying up-to-date with industry trends, developers can ensure the performance, reliability, and security of their LLM applications. In the future, we can expect to see:\n",
"\n",
"* **Increased adoption of cloud-native observability tools**: As more developers adopt cloud-native observability tools, we can expect to see improved performance, reliability, and security in LLM applications.\n",
"* **Advancements in AI-specific observability platforms**: AI-specific observability platforms will continue to evolve, providing more advanced features and capabilities to support the unique needs of LLM applications.\n",
"* **Greater emphasis on security**: As LLMs become more widespread, the importance of security will only continue to grow, with observability and tracing playing a critical role in detecting and responding to security threats.\n",
"\n",
"In conclusion, observability and tracing are essential components of LLM application development, enabling developers to monitor, debug, and optimize their applications. By embracing cloud-native observability tools, leveraging breakthrough technologies, and staying up-to-date with industry trends, developers can ensure the performance, reliability, and security of their LLM applications.\u001b[00m\n",
"\n",
"\n",
"------------ FINAL RESULT ------------\n",
"**Unlocking the Power of Observability and Tracing in LLM Applications**\n",
"\n",
"As Large Language Models (LLMs) continue to revolutionize the field of natural language processing, ensuring their performance, reliability, and security has become a top priority for developers. With the increasing complexity of LLMs, it's essential to have a robust monitoring and debugging system in place to identify and address issues before they impact users. This is where observability and tracing come into play – two critical components that enable developers to monitor, debug, and optimize LLM applications.\n",
"\n",
"**The Importance of Observability and Tracing in LLM Applications**\n",
"\n",
"Observability and tracing are not just buzzwords; they're essential tools for any LLM developer. By implementing observability and tracing, developers can gain real-time insights into their application's performance and behavior, allowing them to identify bottlenecks, optimize resource utilization, and ensure high uptime and reliability. With observability and tracing, developers can:\n",
"\n",
"* **Monitor application performance**: Get real-time insights into application performance, including response times, latency, and error rates.\n",
"* **Identify bottlenecks**: Use tracing to understand the flow of requests through the application and identify areas where performance can be improved.\n",
"* **Optimize resource utilization**: Make data-driven decisions to optimize resource allocation and reduce waste.\n",
"* **Ensure high uptime and reliability**: Identify and fix issues before they impact users, ensuring high uptime and reliability.\n",
"* **Protect sensitive user data**: Detect and respond to security threats in real-time, protecting sensitive user data and preventing data breaches.\n",
"\n",
"**The Rise of Cloud-Native Observability Tools and AI-Specific Observability Platforms**\n",
"\n",
"The observability landscape is evolving rapidly, with cloud-native observability tools and AI-specific observability platforms emerging to cater to the unique needs of LLM applications. Some of the key players in this space include:\n",
"\n",
"* **OpenTelemetry**: An open-source observability framework that enables developers to collect and correlate logs, metrics, and traces from LLM applications.\n",
"* **Signoz**: A cloud-native observability platform that provides real-time insights into LLM application performance and behavior.\n",
"* **Arize**: An AI-specific observability platform that enables developers to monitor and optimize LLM applications in real-time.\n",
"\n",
"**The Future of Observability and Tracing in LLM Applications**\n",
"\n",
"As LLMs continue to evolve and become increasingly complex, the importance of observability and tracing will only continue to grow. By adopting cloud-native observability tools, leveraging breakthrough technologies like OpenTelemetry and Signoz, and staying up-to-date with industry trends, developers can ensure the performance, reliability, and security of their LLM applications. In the future, we can expect to see:\n",
"\n",
"* **Increased adoption of cloud-native observability tools**: As more developers adopt cloud-native observability tools, we can expect to see improved performance, reliability, and security in LLM applications.\n",
"* **Advancements in AI-specific observability platforms**: AI-specific observability platforms will continue to evolve, providing more advanced features and capabilities to support the unique needs of LLM applications.\n",
"* **Greater emphasis on security**: As LLMs become more widespread, the importance of security will only continue to grow, with observability and tracing playing a critical role in detecting and responding to security threats.\n",
"\n",
"In conclusion, observability and tracing are essential components of LLM application development, enabling developers to monitor, debug, and optimize their applications. By embracing cloud-native observability tools, leveraging breakthrough technologies, and staying up-to-date with industry trends, developers can ensure the performance, reliability, and security of their LLM applications.\n"
]
}
],
"source": [
"import os\n",
"from huggingface_hub import get_token\n",
"from crewai_tools import SerperDevTool\n",
"from crewai import LLM, Agent, Task, Crew, Process\n",
"\n",
"# Define our LLM using HF's Serverless Inference API\n",
"llm = LLM(\n",
" model=\"huggingface/meta-llama/Llama-3.1-8B-Instruct\",\n",
" api_key=get_token(),\n",
" max_tokens=1024,\n",
")\n",
"\n",
"# Define a tool for searching the web\n",
"os.environ[\"SERPER_API_KEY\"] = (\n",
" \"YOUR_SERPER_API_KEY\" # must set this value in your environment\n",
")\n",
"search_tool = SerperDevTool()\n",
"\n",
"# Define your agents with roles and goals\n",
"researcher = Agent(\n",
" role=\"Researcher\",\n",
" goal=\"Conduct thorough research on up to date trends around a given topic.\",\n",
" backstory=\"\"\"You work at a leading tech think tank. You have a knack for dissecting complex data and presenting actionable insights.\"\"\",\n",
" verbose=True,\n",
" allow_delegation=False,\n",
" tools=[search_tool],\n",
" llm=llm,\n",
" max_iter=1,\n",
")\n",
"writer = Agent(\n",
" role=\"Technical Writer\",\n",
" goal=\"Craft compelling content on a given topic.\",\n",
" backstory=\"\"\"You are a technical writer with a knack for crafting engaging and informative content.\"\"\",\n",
" llm=llm,\n",
" verbose=True,\n",
" allow_delegation=False,\n",
" max_iter=1,\n",
")\n",
"\n",
"# Create tasks for your agents\n",
"task1 = Task(\n",
" description=\"\"\"Conduct comprehensive research and analysis of the importance of observability and tracing in LLM applications.\n",
" Identify key trends, breakthrough technologies, and potential industry impacts.\"\"\",\n",
" expected_output=\"Full analysis report in bullet points\",\n",
" agent=researcher,\n",
")\n",
"\n",
"task2 = Task(\n",
" description=\"\"\"Using the insights provided, develop an engaging blog\n",
" post that highlights the importance of observability and tracing in LLM applications.\n",
" Your post should be informative yet accessible, catering to a tech-savvy audience.\n",
" Make it sound cool, avoid complex words so it doesn't sound like AI.\"\"\",\n",
" expected_output=\"Blog post of at least 3 paragraphs\",\n",
" agent=writer,\n",
")\n",
"\n",
"# Instantiate your crew with a sequential process\n",
"crew = Crew(\n",
" agents=[researcher, writer],\n",
" tasks=[task1, task2],\n",
" verbose=True,\n",
" process=Process.sequential,\n",
")\n",
"\n",
"# Get your crew to work!\n",
"result = crew.kickoff()\n",
"\n",
"print(\"------------ FINAL RESULT ------------\")\n",
"print(result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"返回 Phoenix 仪表板后，我们可以在新项目 \"crewai\" 中看到来自我们多智能体应用程序的追踪信息！\n",
"\n",
""
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.11"
}
},
"nbformat": 4,
"nbformat_minor": 2
}