Confident AI

Confident AI is a creator of the DeepEval.

DeepEval is a package for unit testing LLMs. Using DeepEval, everyone can build robust language models through faster iterations using both unit testing and integration testing. `DeepEval provides support for each step in the iteration from synthetic data creation to testing.

Installation and Setup

You need to get the DeepEval API credentials.

You need to install the DeepEval Python package:

pip install deepeval

Callbacks

See an example.

Modern Integration Example

Install the required packages:

pip install deepeval langchain langchain-openai

Authenticate with your API key:

import os
import deepeval

# Load API key from environment variable for security
api_key = os.environ.get("DEEPEVAL_API_KEY")
deepeval.login(api_key)

Use the new callback handler:

from deepeval.integrations.langchain import CallbackHandler

handler = CallbackHandler(
 name="My Trace",
 tags=["production", "v1"],
 metadata={"experiment": "A/B"},
 thread_id="thread-123",
 user_id="user-456"
)

See the full example.