How Dependency Proxy via Workhorse works

Injectors

sequenceDiagram
 Client->>+Workhorse: Request
 Workhorse->>+Rails: Propagate the request as-is
 Rails-->>-Workhorse: Respond with a special header that contains instructions for proceeding with the request
 Workhorse-->>Client: Response

An example: Send Git Blob

sequenceDiagram
 Client->>Workhorse: Request for a blob
 Workhorse->>Rails: Propagate the request
 
 Rails->>Workhorse: Respond with a `git-blob:{encoded_data}` header
 Workhorse->>Gitaly: Establish a connection to perform GetBlob RPC
 Gitaly->>Workhorse: Stream the data
 Workhorse->>Client: Stream the data

How GitLab Rails processes the request?

• send_git_blob
• Send a header with a particular information

How Workhorse processes the header?

• Specify a list of injectors
• Iterate over injectors to find a match
• Process a particular request

An example: Send File

sequenceDiagram
 Client->>Workhorse: Request for a blob
 Workhorse->>Rails: Propagate the request
 
 Rails->>Workhorse: Respond with a `send-url:{encoded_data}` header
 Workhorse->>Object Storage: Establish a connection
 Object Storage->>Workhorse: Stream the data
 Workhorse->>Client: Stream the data

---

Pre-authorized requests

sequenceDiagram
 participant Client
 participant Workhorse
 participant Rails
 participant Object Storage

 Client->>+Workhorse: PUT /artifacts/uploads
 Note right of Rails: Append `/authorize` to the original URL and call Rails for an Auth check
 Workhorse->>+Rails: GET /artifacts/uploads/authorize
 Rails-->>-Workhorse: Authorized successfully

 Client->>+Workhorse: Stream the file content
 Workhorse->>+Object Storage: Upload the file
 Object Storage-->>-Workhorse: Success

 Workhorse->>+Rails: Finalize the request
 Note right of Rails: Call the original URL to create a database record
 Rails-->>-Workhorse: Finalized successfully
 Workhorse-->>Client: Uploaded successfully

Docs to read:

• https://docs.gitlab.com/ee/development/uploads/working_with_uploads.html
• https://docs.gitlab.com/ee/development/uploads/index.html#workhorse-assisted-uploads

---

Dependency Proxy

Issue

#11548 (closed)

sequenceDiagram
 Client->>Rails: Request for a file
 alt In Cache
 Rails->>Client: Respond with the file (using send-url injector)
 else Not In Cache
 Rails->>Container Registry: Request to download the file
 Container Registry->>Rails: Respond with the file
 Rails->>Object Storage: Upload file
 Object Storage->>Rails: Uploaded
 Rails->>Client: Respond with the file
 end

sequenceDiagram
 Client->>Workhorse: GET /v2/*group_id/dependency_proxy/containers/*image/blobs/:sha
 Workhorse->>Rails: GET /v2/*group_id/dependency_proxy/containers/*image/blobs/:sha 
 Rails->>Rails: Check DB. Is blob persisted in cache?
 
 alt In Cache
 Rails->>Workhorse: Respond with send-url injector
 Workhorse->>Client: Send the file to the client
 else Not In Cache
 Rails->>Rails: Generate auth token and download URL for the blob in upstream registry
 Rails->>Workhorse: Respond with send-dependency injector
 Workhorse->>Container Registry: Open stream for a blob
 Container Registry->>Workhorse: Stream
 Workhorse->>Rails: GET /v2/*group_id/dependency_proxy/containers/*image/blobs/:sha/authorize
 Rails->>Workhorse: Respond with upload instructions
 Workhorse->>Client: Send the file to the client
 Workhorse->>Object Storage: Save the file
 Workhorse->>Rails: Finalize the upload
 end

Merge request:

• !71890 (merged)