Vibe Coding Discoveropenllmetry
View on GitHubOpen-source observability for your GenAI or LLM application, based on OpenTelemetry
OpenLLMetry provides OpenTelemetry-based instrumentations for LLM providers, vector DBs and agent frameworks, plus a Python SDK that emits standard OTel traces and metrics. Use it to trace, monitor and export LLM app telemetry into Datadog, Grafana, Honeycomb and other backends.
Use Cases
Trace LLM and agent calls with OpenTelemetry spansMonitor token usage, latency and cost of model callsDebug prompts and completions in production LLM appsObserve vector DB queries (Chroma, Pinecone, Qdrant, Weaviate)Export LLM traces to Datadog, Grafana, Honeycomb, New RelicDetect API breaking changes and errors in LLM providersInstrument multi-agent frameworks like CrewAI and LangGraphAdd LLM observability to existing OpenTelemetry stacks
Built With
- Language
- Python
- Frameworks
- OpenTelemetry · LangChain · LangGraph · LlamaIndex · CrewAI · Haystack · LiteLLM · Langflow · Agno · AWS Strands · OpenAI Agents SDK · MCP · Chroma · Pinecone · Qdrant · Weaviate
Tags
observability · opentelemetry · llmops · tracing · monitoring · instrumentation · llm · metrics · telemetry · vector-db · python · eval · genai · apm