Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view
Deliver safe & effective language models
Framework for testing vulnerabilities of large language models (LLM).
MIT-licensed Framework for LLMs, RAGs, Chatbots testing. Configurable via YAML and integrable into CI pipelines for automated testing.
g4f-working is a daily-updated list of working no-auth AI providers and models from @xtekky/gpt4free. It helps developers, testers, and AI enthusiasts instantly find which models are currently online and accessible without any API keys, tokens, or cookies.
Open-source framework for stress-testing LLMs and conversational AI. Identify hallucinations, policy violations, and edge cases with scalable, realistic simulations. Join the discord: https://discord.gg/ssd4S37WNW
Tiny QA Benchmark++ a micro-benchmark suite (52-item gold + on-demand multilingual synthetic packs), generator CLI, and CI-ready eval harness for ultra-fast LLM smoke-testing & regression-catching.
Test your production LLMs and simulate users
🐙 Multi-armed mocks for LLM apps - Drop-in replacement for OpenAI/Anthropic APIs for deterministic testing
API Ninja simplifies API testing by allowing users to define test flows in plain English.
Multi-agent simulation using LLMs. Agents autonomously decide actions for survival, reproduction, and social behavior in a grid world.This project aims to replicate a paper published in 2025 (arXiv:2508.12920).
AI RAG evaluation project using Ragas. Includes RAG metrics (precision, recall, faithfulness), retrieval diagnostics, and prompt testing examples for fintech/banking LLM systems. Designed as an AI QA Specialist portfolio project.
VerifyAI is a simple UI application to test GenAI outputs
A lightweight, console-based Python tool for AI-driven code generation and project management. Optimized for resource-constrained systems, it supports multiple AI providers and self-regeneration.
📊 Monitor AI agents in real-time with Agent Monitor. Gain visibility, track costs, and debug effectively for enhanced performance and reliability.
🚀 Comprehensive testing framework for LLM applications with semantic assertions, multi-provider support, RAG testing, and prompt optimization. Test AI the right way!
🤖 Simulate realistic conversations to test and improve your AI agents, generating evaluation datasets and automating QA for reliable performance.
A community-driven archive of AI prompts and responses. Log, compare, and contribute structured examples to build a searchable public prompt-response database.
Add a description, image, and links to the llm-testing topic page so that developers can more easily learn about it.
To associate your repository with the llm-testing topic, visit your repo's landing page and select "manage topics."