How to use GitLab Development Kit
Duo Agent Platform

Duo Agent Platform

This guide will help you set up a local copy of Duo Agent Platform, within the GDK environment. For more information, see development of GitLab Duo Agent Platform.

Set up Duo Agent Platform

  1. Set up AI Gateway locally.

    1. When you run the setup script, ensure you select SaaS mode; Duo Agent Platform is only available for GitLab.com

  2. Create an Ultimate group and project with experimental features and GitLab Duo turned on by running the following Rake task in your GitLab repository directory:

    cd gitlab GITLAB_SIMULATE_SAAS=1 bundle exec 'rake gitlab:duo:setup'

  3. In the root of your <gdk-dir> enable duo_workflow and configure your GitLab instance to use this locally running instance:

    gdk config set duo_workflow.enabled true gdk reconfigure gdk restart duo-workflow-service rails

    The source code of Duo Agent Platform Service is located at <gdk-dir>/gitlab-ai-gateway/duo_workflow_service.

Environment Setup 1: IDE Flows

Follow these instructions if you are running Duo Agent Platform via your local IDE.

  1. Clone the Ultimate, Duo-enabled project in VS Code.

    1. The project added by the gitlab:duo:setup Rake task is located at $GDK_BASE_URL/gitlab-duo/test or can be cloned manually.

      Important: Run git remote -v to verify the remote origin uses http (not https or ssh). The GitLab Workflow extension with GDK auth only works with http.

    2. Tip 1: to avoid issues related to switching between GitLab.com and GDK auth in VS Code, create a new VS Code profile and log into the GitLab Workflow extension at the GDK URL in the new profile.

    3. Tip 2: When you authenticate the GitLab extension in VS Code by running >Gitlab: Authenticate in the command palette, the GDK URL automatically populates with https. If you aren’t running GDK with SSL, select “Manually enter instance URL” and enter your GDK URL.

Environment Setup 2: Remote Flows

Follow these instructions if you are running Duo Agent Platform via CI pipelines in Rails.

Remote Flows are executed in pipeline jobs in Rails. In the back end, these are triggered by the parameter start_workflow = true.

To set up Remote Flows:

  1. Follow the steps to create and register a local runner to your GDK. Follow the instructions for executing a runner from within docker. For the image name, enter the name of an Ubuntu image, for example ubuntu:22.04.

  2. Ensure the runner you created in step 1 is the one that will used for Remote Flows. You can manage runners by going to Admin > CI/CD > Runners.

  3. Optional. To view flow execution logs in CI job logs, enable debug logging:

    gdk config set duo_workflow.debug true gdk reconfigure

Testing Remote Flows

Option 1 - Start a Remote Flow via API Call

  1. Get an api scoped personal access token in your local GDK.

  2. Turn on flow execution. Go to the project (such as gitlab-duo/test) and select Settings > General > GitLab Duo > Allow flow execution.

  3. Start a Remote Flow with the API. Replace <PROJECT_ID> in the example below with the ID of a project in your GDK that passes all the required Duo Agent Platform access checks otherwise you will get a 403 response. If you ran the rake gitlab:duo:setup task, you will have a project with full path gitlab-duo/test that you can use, and can replace <PROJECT_ID> with gitlab-duo%2Ftest:

    curl -H "Private-Token: $GDK_PAT" -XPOST "http://$GDK_BASE_URL/api/v4/ai/duo_workflows/workflows?project_id=<PROJECT_ID>&start_workflow=true&goal=Open%20a%20new%20MR%20in%20the%20project%20with%20a%20file%20called%20hello.rb%20which%20is%20a%20simple%20Ruby%20hello%20world&workflow_definition=software_development&pre_approved_agent_privileges[]=1&pre_approved_agent_privileges[]=2&pre_approved_agent_privileges[]=3&pre_approved_agent_privileges[]=4&pre_approved_agent_privileges[]=5&agent_privileges[]=1&agent_privileges[]=2&agent_privileges[]=3&agent_privileges[]=4&agent_privileges[]=5"

  4. If you receive the error message Can not execute workflow in CI, ensure flows are turned on in step 2.

  5. Verify the job started. Go to the project referenced in step 3 and select Build > Jobs. The job called workload should be running.

Option 2 - Migrate a JenkinsFile to GitLab CI/CD

Note: Currently, JenkinsFile to GitLab CI only works on the main branch. This is a known issue.

  1. Have a Jenkinsfile in your project. For examples, see migrating from Jenkins.
  2. Go to the Jenkinsfile in your browser, and select Convert to GitLab CI/CD. You should see a Workflow started successfully message.
  3. Go to Build > Jobs to view the jobs, or go to your merge requests to view the merge request created by GitLab Duo Agent Platform.

Debugging Issues with Remote Flows

  • Make sure that you have Runners available for Remote Flows:

    1. Go to your local GitLab instance.
    2. Go to Admin > CI/CD > Runners.
    3. Make sure you have a respective project, group or instance level runner registered to your project.

  • If the runner is not automatically picking up jobs from your pipeline:

    1. Restart GDK rails background job using gdk restart rails-background-jobs.
    2. Restart runner application using gitlab-runner restart.

  • If you used the Docker executor, ensure that Docker is running.

  • If your workflow CI job (under Build > Jobs) for your project failed with Failed to connect to gdk.test port 3000, follow the steps to create a loopback interface.

  • If you need to test out the OAuth token that Duo Agent Platform Executor is given, set the environment variable DUO_WORKFLOW_SKIP_TOKEN_REVOCATION=true. Executor will not revoke the token, allowing you to test it out.

    • Note: This needs to be set in the environment that the Executor is run in. For Remote Flows, set environment variables for the Duo Workflow Executor in start_workflow_service.rb.

  • If you see a server is unavailable error associated with the duo_workflow_executor make sure that the service_url field is set to <gdk-url>:50052 under duo_workflow in <gdk-dir>/gitlab/config/gitlab.yml.

Optional: Run a different branch of Duo Agent Platform Service

See Run a different branch of AI Gateway and Duo Agent Platform Service.

Optional: Run a different branch of Duo Agent Platform Executor

The duo-workflow-executor repository is cloned into <gdk-dir>/duo-workflow-executor and compiled every time you run gdk reconfigure. The binary is placed in <gdk-dir>/gitlab/public/assets and served up by your local GitLab instance.

To change the version used:

  1. Edit the <gdk-dir>/gitlab/DUO_WORKFLOW_EXECUTOR_VERSION with a valid SHA in the repository.

  2. Recompile the binary:

    gdk reconfigure

Optional: Configure LLM Cache

LLMs are slow and expensive. If you are doing lots of repetitive development with Duo Agent Platform you may wish to enable LLM caching to speed up iteration and save money. To enable the cache:

gdk config set duo_workflow.llm_cache true gdk reconfigure gdk restart duo-workflow-service rails
Last updated on
Docker Registry replicationElasticsearch