AlphaTrion is an open-source framework to help build GenAI applications, including experiment tracking, adaptive model routing, prompt optimization, performance evaluation and so on. The name comes after the oldest and wisest Transformer - AlphaTrion.

Still under active development.

• Project: A Project is a namespace-level abstraction that isolates experiments from different users or teams.
• Experiment: An Experiment is a logic-level abstraction for organizing and managing a series of related trials. It serves as a way to group together multiple trials that share a common goal or objective.
• Trial: A Trial represents a config-level abstraction for a specific set of hyperparameters or configurations within an experiment. It is a single execution of a particular configuration.
• Run: A Run is a real execution instance of a trial. It represents the actual execution of the code with the specified configuration and hyperparameters defined in the trial.

pip install alphatrion

• Git clone the repository
• Run source start.sh to activate the virtual environment.

Run the following command for setup:

cp .env.example .env & make up

You can login to pgAdmin at http://localhost:8080 to see the Postgres database with following credentials:

Email: alphatrion@inftyai.com Username: alphatrion Password: alphatr1on Server: postgres

Below is a simple example with two approaches demonstrating how to create an experiment and log performance metrics.

import alphatrion as alpha import uuid # Better to use a fixed UUID to identify your project. alpha.init(project_id=uuid.uuid4(), artifact_insecure=True) async def log(): # Run your code here then log metrics. await alpha.log_metrics({"accuracy": 0.95}) async with alpha.CraftExperiment.setup(name="my_first_experiment") as exp: async with exp.start_trial(name="my_trial") as trial: run = trial.start_run(lambda: log()) await run.wait()

Dashboard is coming soon! Meanwhile, you can query the results directly from the database.

make down

We welcome contributions! Please refer to developer.md for more information on how to set up your development environment and contribute to the project.