DEV Community

Cover image for Go for Node developers: creating an IDP from scratch - Set-up
Afonso Barracha
Afonso Barracha

Posted on

Go for Node developers: creating an IDP from scratch - Set-up

#backend #go #restapi #oauth

Series Intro

Nowadays, due to performance constraints a lot of companies are moving away from NodeJS to Go for their network and API stacks. This series is for developers interest in making the jump from Node.js to Go.

On this series, I will teach the process of building an IDP (identity provider), a project I started to create a comunity driven self-hosted opensource version of clerk.

Series Requirements

This series requires some basic knowledge of Go, as it will teach you how to structure a Go API, and how to transfer Node.js to Go skills. Before starting this series, I recommend reading the following two books to gain familiarity with the Go programming language:

  1. Go Fundamentals: an introduction to the go programming language.
  2. Network Programming with Go: an introduction to Network Programming, from network nodes and TCP to HTTP fundamentals.

Or if you're in a hurry just take a tour of Go in the official website.

Topics to cover

This series will be divided in 6 parts:

  1. Project structure and Database design: I'll lay the foundation by setting up the project structure and designing the database schema for the IDP.
  2. Local and External Authentication with JWTs: I will touch on how I implemented local authentication using JSON Web Tokens (JWTs) Bearer tokens (RFC-6750) for our Accounts (tenants) including external auth (RFC-6749).
  3. Production mailer: to send emails we will create our own queue using redis and the "net/smtp" package from the standard library.
  4. Component testing with docker-compose: I'll touch on how to ensure the reliability of our API by writing endpoint tests using Docker Compose and Go's standard net/http and testing packages.
  5. Apps and account mapping: add multiple apps support for each account.
  6. Deploying our API: We will deploy our App to a VPS (virtual private server) using coolify.

Intro

A well-organized project structure is a requirement for building maintainable and scalable APIs. In this article, we'll explore how to structure a Go API using a very common pattern in the Node.JS world, using the Model, Service, Controller (MSC) architecture.

We will leverage the following stack:

  • Fiber: Go framework inspired in express.
  • PostgreSQL with SQLC: SQLC is a Go code generator with type safety and compile-time checks for our SQL queries.
  • Redis with fiber's Redis abstraction: a caching storage abstraction for fiber using Redis.

We will also design the database schema and create the necessary migrations to set up our data layer.

Packages structure

For the structure of our code we'll levarage the go-blueprint CLI tool made by the youtuber Melky.

To start the project install and initialize the project:

$ go install github.com/melkeydev/go-blueprint@latest $ makedir devlogs && cd devlogs $ go-blueprint create --name idp --framework fiber --driver postgres --git commit
Enter fullscreen mode Exit fullscreen mode

This will lead to an initial file structure like:

C:/Users/user/IdeProjects/devlogs/idp: ├─── cmd │ ├─── api │ │ main.go ├─── internal │ ├─── database │ │ database.go │ ├─── server │ │ server.go │ │ routes.go │ │ routes_test.go | ...
Enter fullscreen mode Exit fullscreen mode

While the name of the module will be devlogs/idp, this is a bit far of what we want, which would be github.com/your-username/devlogs/idp. So update the module name to github.com/your-username/devlogs/idp and run:

$ go mod tidy
Enter fullscreen mode Exit fullscreen mode

And move the files around for a MSC (Model, Service, Controller) architecture:

C:/Users/user/IdeProjects/devlogs/idp: ├─── cmd │ ├─── api │ │ main.go ├─── internal │ ├─── config │ │ config.go │ │ encryption.go │ │ logger.go │ │ oauth.go │ │ rate_limiter.go │ │ tokens.go │ │ ... │ ├─── controllers │ │ ├─── bodies │ │ │ common.go │ │ │ ... │ │ ├─── params │ │ │ common.go │ │ │ ... │ │ ├─── paths │ │ │ common.go │ │ │ ... │ │ auth.go │ │ controllers.go │ │ ... │ ├─── exceptions │ │ controllers.go │ │ services.go │ ├─── providers │ │ ├─── cache │ │ │ cache.go │ │ │ ... │ │ ├─── database │ │ │ database.go │ │ │ ... │ │ ├─── mailer │ │ │ mailer.go │ │ │ ... │ │ ├─── oauth │ │ │ oauth.go │ │ │ ... │ │ ├─── tokens │ │ │ tokens.go │ │ │ ... │ │ ├─── encryption │ │ │ encryption.go │ │ │ ... │ ├─── database │ │ database.go │ ├─── server │ │ ├─── routes │ │ │ routes.go │ │ │ ... │ │ logger.go │ │ server.go │ │ routes.go │ ├─── services │ │ ├─── dtos │ │ │ common.go │ │ │ ... │ ├─── utils │ │ ... | ...
Enter fullscreen mode Exit fullscreen mode

Most logic will live on the internal folder, where the main structure is as follows:

  • Config: centralizes and loads all environment configurations.
  • Server: contains the Fiber instance initialization and endpoints routing
    • /routes: specifies the routes for a given controller method.
    • logger.go: builds the default configuration for the structure logger.
    • routes.go: has the main method to register all routes RegisterFiberRoutes.
    • server.go: builds a FiberServer instance with the fiber.App instance and routes router.
  • Controllers: handle incoming HTTP requests, process them using services, and return the appropriate HTTP responses.
    • /bodies: specifies the controllers bodies.
    • /params: specifies the controllers URL and Query parameters.
    • /paths: where the routes path constants are defined so they can be easily shared.
  • Services: where most of our business logic and interactions with external providers lives.
    • /dtos: where our data transfer objects are defined,
  • Providers: where we have our external providers such as Data stores, JWTs, etc.
    • /cache: Redis storage interactions.
    • /database: PostgreSQL interactions and model structs implementations.
    • /mailer: connection to our mailing queue.
    • /oauth: oauth interactions with external authentication providers.
    • /tokens: jwt signing and verifying.
    • /encryption: envelope encryption logic provider.

Configuration

As most APIs, the configuration will come from environment variables, we can load them with the os package from the standard library.

For local development we can load the environment variables from a .env file, by installing the following package:

$ go get github.com/joho/godotenv
Enter fullscreen mode Exit fullscreen mode

This variables most of the time act as secrets, hence we will use a OOP style encapsulation with them where all members of configurations structs are private and immutable, while you can get their values with getters.

Our IDP will have main configurations groups apart from the base config:

  • Logger: whether debug is active and the env to chose whether we want a text or JSON handler.
  • Tokens: the private/public keys pairs and TTL for signing and verifying JWTs.
  • OAuth: collection of client ids and secrets for each of the external authentication providers.
  • Rate Limiter: specifies the max number of request withing a window that an IP can make.
  • Encryption: list of KEK (Key encryption keys) provided to the environment.

Logger

On the internal/config directory create a logger.go file with the following struct and New function builder:

package config type LoggerConfig struct { isDebug bool env string serviceName string } func NewLoggerConfig(isDebug bool, env, serviceName string) LoggerConfig { return LoggerConfig{ isDebug: isDebug, env: env, serviceName: serviceName, } } func (l *LoggerConfig) IsDebug() bool { return l.isDebug } func (l *LoggerConfig) Env() string { return l.env } func (l *LoggerConfig) ServiceName() string { return l.serviceName }
Enter fullscreen mode Exit fullscreen mode

NOTE: for most methods in Go it is recommended using pointer receivers. Pointers are just address pointing to the underlying memory of the struct, but for simplicity you can think of them as pass by reference instead of pass by value.

Tokens

JWT have 3 main parts that need to be provided by the environment:

  • Public Key: the key that will be used to verify the token.
  • Private Key: the key that will be used to sign the token.
  • Time to live: the TTL in seconds of the token.

And the service will have 7 different JWTs with different key pairs:

  • Access: for the access token.
  • Account Credentials: for machine to machine access.
  • Refresh: for the refresh token to refresh the access token on client to machine access.
  • Confirmation: for the email confirmation token (JWT for confirmation can be overkill, however it saves on ram memory by not saving the hashed email in cache).
  • Reset: for email resetting.
  • OAuth: for a temporary authorization header for the token exchange.
  • Two Factor: for a temporary authorization header for passing the two factor code.

Single Configuration

Create a tokens.go file on the /internal/config directory with the struct, new method and getters:

package config type SingleJwtConfig struct { publicKey string privateKey string ttlSec int64 } func NewSingleJwtConfig(publicKey, privateKey string, ttlSec int64) SingleJwtConfig { return SingleJwtConfig{ publicKey: publicKey, privateKey: privateKey, ttlSec: ttlSec, } } func (s *SingleJwtConfig) PublicKey() string { return s.publicKey } func (s *SingleJwtConfig) PrivateKey() string { return s.privateKey } func (s *SingleJwtConfig) TtlSec() int64 { return s.ttlSec }
Enter fullscreen mode Exit fullscreen mode

Tokens Configuration

Now do the same as previously but for each token type:

package config // ... type TokensConfig struct { access SingleJwtConfig accountCredentials SingleJwtConfig refresh SingleJwtConfig confirm SingleJwtConfig reset SingleJwtConfig oAuth SingleJwtConfig twoFA SingleJwtConfig } func NewTokensConfig( access SingleJwtConfig, accountCredentials SingleJwtConfig, refresh SingleJwtConfig, confirm SingleJwtConfig, oAuth SingleJwtConfig, twoFA SingleJwtConfig, ) TokensConfig { return TokensConfig{ access: access, accountCredentials: accountCredentials, refresh: refresh, confirm: confirm, oAuth: oAuth, twoFA: twoFA, } } func (t *TokensConfig) Access() SingleJwtConfig { return t.access } func (t *TokensConfig) AccountCredentials() SingleJwtConfig { return t.accountKeys } func (t *TokensConfig) Refresh() SingleJwtConfig { return t.refresh } func (t *TokensConfig) Confirm() SingleJwtConfig { return t.confirm } func (t *TokensConfig) Reset() SingleJwtConfig { return t.reset } func (t *TokensConfig) OAuth() SingleJwtConfig { return t.oAuth } func (t *TokensConfig) TwoFA() SingleJwtConfig { return t.twoFA }
Enter fullscreen mode Exit fullscreen mode

OAuth

External authentication providers have two environment variables each:

  • Client ID: the identifier of the app on the external IDP
  • Client Secret: a secure secret to fetch the user info from the code exchange.

And we will add support for the 5 main western ones:

  • GitHub
  • Google
  • Facebook
  • Apple
  • Microsoft

Single OAuth provider configuration

Create a oauth.go file on the /internal/config directory with the struct, new method and getters:

package config type OAuthProviderConfig struct { clientID string clientSecret string } func NewOAuthProvider(clientID, clientSecret string) OAuthProviderConfig { return OAuthProviderConfig{ clientID: clientID, clientSecret: clientSecret, } } func (o *OAuthProviderConfig) ClientID() string { return o.clientID } func (o *OAuthProviderConfig) ClientSecret() string { return o.clientSecret }
Enter fullscreen mode Exit fullscreen mode

OAuth providers configuration

Create a struct, new method and getter for each provider:

package config // ... type OAuthProvidersConfig struct { gitHub OAuthProviderConfig google OAuthProviderConfig facebook OAuthProviderConfig apple OAuthProviderConfig microsoft OAuthProviderConfig } func NewOAuthProviders(gitHub, google, facebook, apple, microsoft OAuthProviderConfig) OAuthProvidersConfig { return OAuthProvidersConfig{ gitHub: gitHub, google: google, facebook: facebook, apple: apple, microsoft: microsoft, } } func (o *OAuthProvidersConfig) GitHub() OAuthProviderConfig { return o.gitHub } func (o *OAuthProvidersConfig) Google() OAuthProviderConfig { return o.google } func (o *OAuthProvidersConfig) Facebook() OAuthProviderConfig { return o.facebook } func (o *OAuthProvidersConfig) Apple() OAuthProviderConfig { return o.apple } func (o *OAuthProvidersConfig) Microsoft() OAuthProviderConfig { return o.microsoft }
Enter fullscreen mode Exit fullscreen mode

Rate Limiter

The rate limiter has only two options:

  • Max: the number of maximum number of requests per window size.
  • Expiration Seconds: the window size in seconds.

Just create the rate_limiter.go file on the /internal/config directory as follows:

package config type RateLimiterConfig struct { max int64 expSec int64 } func NewRateLimiterConfig(max, expSec int64) RateLimiterConfig { return RateLimiterConfig{ max: max, expSec: expSec, } } func (r *RateLimiterConfig) Max() int64 { return r.max } func (r *RateLimiterConfig) ExpSec() int64 { return r.expSec }
Enter fullscreen mode Exit fullscreen mode

Encryption

Finaly to configure the KEKs for each encryption space:

  • Accounts: encryption of secrets for TOTP two factor auth.
  • Apps: encryption for the APPs private keys.
  • Users: encryption of secrets for TOTP two factor auth.

As well as old keys for easy keys rotation.

Add them to encryption.go file:

package config import "encoding/json" type EncryptionConfig struct { accountSecret string appSecret string userSecret string oldSecrets []string } func NewEncryptionConfig(accountSecret, appSecret, userSecret, oldSecrets string) EncryptionConfig { var secretSlice []string if err := json.Unmarshal([]byte(oldSecrets), &secretSlice); err != nil { panic(err) } return EncryptionConfig{ accountSecret: accountSecret, appSecret: appSecret, userSecret: userSecret, oldSecrets: secretSlice, } } func (e *EncryptionConfig) AccountSecret() string { return e.accountSecret } func (e *EncryptionConfig) AppSecret() string { return e.appSecret } func (e *EncryptionConfig) UserSecret() string { return e.userSecret } func (e *EncryptionConfig) OldSecrets() []string { return e.oldSecrets }
Enter fullscreen mode Exit fullscreen mode

Base config

On the base config you just need to add the environment variables in an array, as well as getters for each config parameter of the struct:

package config import ( "log/slog" "os" "strconv" "strings" "github.com/google/uuid" "github.com/joho/godotenv" ) type Config struct { port int64 maxProcs int64 databaseURL string redisURL string frontendDomain string backendDomain string cookieSecret string cookieName string emailPubChannel string encryptionSecret string serviceID uuid.UUID loggerConfig LoggerConfig tokensConfig TokensConfig oAuthProvidersConfig OAuthProvidersConfig rateLimiterConfig RateLimiterConfig encryptionConfig EncryptionConfig } func (c *Config) Port() int64 { return c.port } func (c *Config) MaxProcs() int64 { return c.maxProcs } func (c *Config) DatabaseURL() string { return c.databaseURL } func (c *Config) RedisURL() string { return c.redisURL } func (c *Config) FrontendDomain() string { return c.frontendDomain } func (c *Config) BackendDomain() string { return c.backendDomain } func (c *Config) CookieSecret() string { return c.cookieSecret } func (c *Config) CookieName() string { return c.cookieName } func (c *Config) EmailPubChannel() string { return c.emailPubChannel } func (c *Config) EncryptionSecret() string { return c.encryptionSecret } func (c *Config) ServiceID() uuid.UUID { return c.serviceID } func (c *Config) LoggerConfig() LoggerConfig { return c.loggerConfig } func (c *Config) TokensConfig() TokensConfig { return c.tokensConfig } func (c *Config) OAuthProvidersConfig() OAuthProvidersConfig { return c.oAuthProvidersConfig } func (c *Config) RateLimiterConfig() RateLimiterConfig { return c.rateLimiterConfig } func (c *Config) EncryptionConfig() EncryptionConfig { return c.encryptionConfig } var variables = [40]string{ "PORT", "ENV", "DEBUG", "SERVICE_NAME", "SERVICE_ID", "MAX_PROCS", "DATABASE_URL", "REDIS_URL", "FRONTEND_DOMAIN", "BACKEND_DOMAIN", "COOKIE_SECRET", "COOKIE_NAME", "RATE_LIMITER_MAX", "RATE_LIMITER_EXP_SEC", "EMAIL_PUB_CHANNEL", "JWT_ACCESS_PUBLIC_KEY", "JWT_ACCESS_PRIVATE_KEY", "JWT_ACCESS_TTL_SEC", "JWT_ACCOUNT_CREDENTIALS_PUBLIC_KEY", "JWT_ACCOUNT_CREDENTIALS_PRIVATE_KEY", "JWT_ACCOUNT_CREDENTIALS_TTL_SEC", "JWT_REFRESH_PUBLIC_KEY", "JWT_REFRESH_PRIVATE_KEY", "JWT_REFRESH_TTL_SEC", "JWT_CONFIRM_PUBLIC_KEY", "JWT_CONFIRM_PRIVATE_KEY", "JWT_CONFIRM_TTL_SEC", "JWT_RESET_PUBLIC_KEY", "JWT_RESET_PRIVATE_KEY", "JWT_RESET_TTL_SEC", "JWT_OAUTH_PUBLIC_KEY", "JWT_OAUTH_PRIVATE_KEY", "JWT_OAUTH_TTL_SEC", "JWT_2FA_PUBLIC_KEY", "JWT_2FA_PRIVATE_KEY", "JWT_2FA_TTL_SEC", "ACCOUNT_SECRET", "APP_SECRET", "USER_SECRET", "OLD_SECRETS", } var optionalVariables = [17]string{ "GITHUB_CLIENT_ID", "GITHUB_CLIENT_SECRET", "GOOGLE_CLIENT_ID", "GOOGLE_CLIENT_SECRET", "FACEBOOK_CLIENT_ID", "FACEBOOK_CLIENT_SECRET", "APPLE_CLIENT_ID", "APPLE_CLIENT_SECRET", "MICROSOFT_CLIENT_ID", "MICROSOFT_CLIENT_SECRET", "OLD_JWT_ACCESS_PUBLIC_KEY", "OLD_JWT_ACCOUNT_CREDENTIALS_PUBLIC_KEY", "OLD_JWT_REFRESH_PUBLIC_KEY", "OLD_JWT_CONFIRM_PUBLIC_KEY", "OLD_JWT_RESET_PUBLIC_KEY", "OLD_JWT_OAUTH_PUBLIC_KEY", "OLD_JWT_2FA_PUBLIC_KEY", } var numerics = [11]string{ "PORT", "MAX_PROCS", "JWT_ACCESS_TTL_SEC", "JWT_ACCOUNT_CREDENTIALS_TTL_SEC", "JWT_REFRESH_TTL_SEC", "JWT_CONFIRM_TTL_SEC", "JWT_RESET_TTL_SEC", "JWT_OAUTH_TTL_SEC", "JWT_2FA_TTL_SEC", "RATE_LIMITER_MAX", "RATE_LIMITER_EXP_SEC", } func NewConfig(logger *slog.Logger, envPath string) Config { err := godotenv.Load(envPath) if err != nil { logger.Error("Error loading .env file") } variablesMap := make(map[string]string) for _, variable := range variables { value := os.Getenv(variable) if value == "" { logger.Error(variable + " is not set") panic(variable + " is not set") } variablesMap[variable] = value } for _, variable := range optionalVariables { value := os.Getenv(variable) variablesMap[variable] = value } intMap := make(map[string]int64) for _, numeric := range numerics { value, err := strconv.ParseInt(variablesMap[numeric], 10, 0) if err != nil { logger.Error(numeric + " is not an integer") panic(numeric + " is not an integer") } intMap[numeric] = value } env := variablesMap["ENV"] return Config{ port: intMap["PORT"], maxProcs: intMap["MAX_PROCS"], databaseURL: variablesMap["DATABASE_URL"], redisURL: variablesMap["REDIS_URL"], frontendDomain: variablesMap["FRONTEND_DOMAIN"], backendDomain: variablesMap["BACKEND_DOMAIN"], cookieSecret: variablesMap["COOKIE_SECRET"], cookieName: variablesMap["COOKIE_NAME"], emailPubChannel: variablesMap["EMAIL_PUB_CHANNEL"], serviceID: uuid.MustParse(variablesMap["SERVICE_ID"]), loggerConfig: NewLoggerConfig( strings.ToLower(variablesMap["DEBUG"]) == "true", env, variablesMap["SERVICE_NAME"], ), tokensConfig: NewTokensConfig( NewSingleJwtConfig( variablesMap["JWT_ACCESS_PUBLIC_KEY"], variablesMap["JWT_ACCESS_PRIVATE_KEY"], variablesMap["OLD_JWT_ACCESS_PUBLIC_KEY"], intMap["JWT_ACCESS_TTL_SEC"], ), NewSingleJwtConfig( variablesMap["JWT_ACCOUNT_CREDENTIALS_PUBLIC_KEY"], variablesMap["JWT_ACCOUNT_CREDENTIALS_PRIVATE_KEY"], variablesMap["OLD_JWT_ACCOUNT_CREDENTIALS_PUBLIC_KEY"], intMap["JWT_ACCOUNT_CREDENTIALS_TTL_SEC"], ), NewSingleJwtConfig( variablesMap["JWT_REFRESH_PUBLIC_KEY"], variablesMap["JWT_REFRESH_PRIVATE_KEY"], variablesMap["OLD_JWT_REFRESH_PUBLIC_KEY"], intMap["JWT_REFRESH_TTL_SEC"], ), NewSingleJwtConfig( variablesMap["JWT_CONFIRM_PUBLIC_KEY"], variablesMap["JWT_CONFIRM_PRIVATE_KEY"], variablesMap["OLD_JWT_CONFIRM_PUBLIC_KEY"], intMap["JWT_CONFIRM_TTL_SEC"], ), NewSingleJwtConfig( variablesMap["JWT_RESET_PUBLIC_KEY"], variablesMap["JWT_RESET_PRIVATE_KEY"], variablesMap["OLD_JWT_RESET_PUBLIC_KEY"], intMap["JWT_RESET_TTL_SEC"], ), NewSingleJwtConfig( variablesMap["JWT_OAUTH_PUBLIC_KEY"], variablesMap["JWT_OAUTH_PRIVATE_KEY"], variablesMap["OLD_JWT_OAUTH_PUBLIC_KEY"], intMap["JWT_OAUTH_TTL_SEC"], ), NewSingleJwtConfig( variablesMap["JWT_2FA_PUBLIC_KEY"], variablesMap["JWT_2FA_PRIVATE_KEY"], variablesMap["OLD_JWT_2FA_PUBLIC_KEY"], intMap["JWT_2FA_TTL_SEC"], ), ), oAuthProvidersConfig: NewOAuthProviders( NewOAuthProvider(variablesMap["GITHUB_CLIENT_ID"], variablesMap["GITHUB_CLIENT_SECRET"]), NewOAuthProvider(variablesMap["GOOGLE_CLIENT_ID"], variablesMap["GOOGLE_CLIENT_SECRET"]), NewOAuthProvider(variablesMap["FACEBOOK_CLIENT_ID"], variablesMap["FACEBOOK_CLIENT_SECRET"]), NewOAuthProvider(variablesMap["APPLE_CLIENT_ID"], variablesMap["APPLE_CLIENT_SECRET"]), NewOAuthProvider(variablesMap["MICROSOFT_CLIENT_ID"], variablesMap["MICROSOFT_CLIENT_SECRET"]), ), rateLimiterConfig: NewRateLimiterConfig( intMap["RATE_LIMITER_MAX"], intMap["RATE_LIMITER_EXP_SEC"], ), encryptionConfig: NewEncryptionConfig( variablesMap["ACCOUNT_SECRET"], variablesMap["APP_SECRET"], variablesMap["USER_SECRET"], variablesMap["OLD_SECRETS"], ), } }
Enter fullscreen mode Exit fullscreen mode

Providers

With the config done, we can start creating the providers:

  • cache: Redis cache;
  • database: PostgreSQL database;
  • mailer: Redis PubSub publisher for emails (the queue will be built on a subsquent article);
  • oauth: external authentication providers;
  • tokens: jwt signing and verifying;
  • encrytion: the provider for envolope encryption (more of an isolation of logic than a proper provider. Encryption logic should always be isolated in production)

Utilities

Observasibility of APIs is important for production debugging, so lets create an utility to build a structure logger.

To make it easy to locate the function and logic, we need to pass the package location, method, and layer. Create the utility on the root utils package. Create a logger.go file and add the following code:

package utils import ( "log/slog" ) type LogLayer = string const ( ControllersLogLayer LogLayer = "controllers" ServicesLogLayer LogLayer = "services" ProvidersLogLayer LogLayer = "providers" ) type LoggerOptions struct { Layer string Location string Method string RequestID string } func BuildLogger(logger *slog.Logger, opts LoggerOptions) *slog.Logger { return logger.With( "layer", opts.Layer, "location", opts.Location, "method", opts.Method, "requestId", opts.RequestID, ) }
Enter fullscreen mode Exit fullscreen mode

Cache

Our cache implementation will just extend the fiber Redis Storage abstraction. On the internal/providers create a /cache directory and add cache.go file with the following struct and new function:

package cache import ( "context" "log/slog" fiberRedis "github.com/gofiber/storage/redis/v3" "github.com/redis/go-redis/v9" ) const logLayer string = "cache" type Cache struct { logger *slog.Logger storage *fiberRedis.Storage } func NewCache(logger *slog.Logger, storage *fiberRedis.Storage) *Cache { return &Cache{ logger: logger, storage: storage, } }
Enter fullscreen mode Exit fullscreen mode

With three common methods to reset the cache, get the redist client and ping the cache:

package cache // ... func (c *Cache) ResetCache() error { return c.storage.Reset() } func (c *Cache) Client() redis.UniversalClient { return c.storage.Conn() } func (c *Cache) Ping(ctx context.Context) error { return c.Client().Ping(ctx).Err() }
Enter fullscreen mode Exit fullscreen mode

Database

The database set-up is a bit more complex. We will write both our migrations and queries in SQL and use migrate to migrate our DB changes, and SQLC to generate safe Go SQL query codes.

Start by installing the necessary dependencies:

$ go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest $ go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest
Enter fullscreen mode Exit fullscreen mode

And create a config file for SQLC as sqlc.yaml on the idp root folder:

version: "2" sql: - schema: "internal/providers/database/migrations" queries: "internal/providers/database/queries" engine: "postgresql" gen: go: package: "database" out: "internal/providers/database" sql_package: "pgx/v5" emit_empty_slices: true overrides: - db_type: "timestamptz" go_type: "time.Time" - db_type: "uuid" go_type: "github.com/google/uuid.UUID"
Enter fullscreen mode Exit fullscreen mode

Database Schema

Since we are creating an IDP we will have three main entitites:

  • Accounts (also known as tenants): the main users/admins that can create APPs that provide authentication to their own services.
  • Apps: the authentication provider configuration for a given service or services.
  • Users: users that have registered for the account's apps.

This leads to the following somewhat complex datamodel:

Database Schema

Initial Migration

With migrate installed run the following command:

$ migrate create --ext sql --dir ./internal/providers/database/migration create_initial_schema
Enter fullscreen mode Exit fullscreen mode

This will generate two files:

  • YYYYMMDDHHMMSS_create_initial_schema.up.sql
  • YYYYMMDDHHMMSS_create_initial_schema.down.sql

To the up migration copy the following SQL:

CREATE TABLE "accounts" ( "id" serial PRIMARY KEY, "first_name" varchar(50) NOT NULL, "last_name" varchar(50) NOT NULL, "username" varchar(109) NOT NULL, "email" varchar(250) NOT NULL, "password" text, "version" integer NOT NULL DEFAULT 1, "is_confirmed" boolean NOT NULL DEFAULT false, "two_factor_type" varchar(5) NOT NULL DEFAULT 'none', "created_at" timestamp NOT NULL DEFAULT (now()), "updated_at" timestamp NOT NULL DEFAULT (now()) ); CREATE TABLE "account_totps" ( "id" serial PRIMARY KEY, "account_id" integer NOT NULL, "url" varchar(250) NOT NULL, "secret" text NOT NULL, "dek" text NOT NULL, "recovery_codes" jsonb NOT NULL, "created_at" timestamp NOT NULL DEFAULT (now()), "updated_at" timestamp NOT NULL DEFAULT (now()) ); CREATE TABLE "account_credentials" ( "id" serial PRIMARY KEY, "account_id" integer NOT NULL, "scopes" jsonb NOT NULL, "client_id" varchar(22) NOT NULL, "client_secret" text NOT NULL, "created_at" timestamp NOT NULL DEFAULT (now()), "updated_at" timestamp NOT NULL DEFAULT (now()) ); CREATE TABLE "auth_providers" ( "id" serial PRIMARY KEY, "email" varchar(250) NOT NULL, "provider" varchar(10) NOT NULL, "created_at" timestamp NOT NULL DEFAULT (now()), "updated_at" timestamp NOT NULL DEFAULT (now()) ); CREATE TABLE "apps" ( "id" serial PRIMARY KEY, "account_id" integer NOT NULL, "name" varchar(50) NOT NULL, "client_id" varchar(22) NOT NULL, "client_secret" text NOT NULL, "dek" text NOT NULL, "callback_uris" varchar(250)[] NOT NULL DEFAULT '{}', "logout_uris" varchar(250)[] NOT NULL DEFAULT '{}', "user_scopes" jsonb NOT NULL DEFAULT '{ "email": true, "name": true }', "app_providers" jsonb NOT NULL DEFAULT '{ "email_password": true }', "id_token_ttl" integer NOT NULL DEFAULT 3600, "jwt_crypto_suite" varchar(7) NOT NULL DEFAULT 'ES256', "created_at" timestamp NOT NULL DEFAULT (now()), "updated_at" timestamp NOT NULL DEFAULT (now()) ); CREATE TABLE "app_keys" ( "id" serial PRIMARY KEY, "app_id" integer NOT NULL, "account_id" integer NOT NULL, "name" varchar(10) NOT NULL, "jwt_crypto_suite" varchar(7) NOT NULL, "public_key" jsonb NOT NULL, "private_key" text NOT NULL, "created_at" timestamp NOT NULL DEFAULT (now()), "updated_at" timestamp NOT NULL DEFAULT (now()) ); CREATE TABLE "users" ( "id" serial PRIMARY KEY, "account_id" integer NOT NULL, "email" varchar(250) NOT NULL, "password" text, "version" integer NOT NULL DEFAULT 1, "two_factor_type" varchar(5) NOT NULL DEFAULT 'none', "user_data" jsonb NOT NULL DEFAULT '{}', "created_at" timestamp NOT NULL DEFAULT (now()), "updated_at" timestamp NOT NULL DEFAULT (now()) ); CREATE TABLE "user_totps" ( "id" serial PRIMARY KEY, "user_id" integer NOT NULL, "url" varchar(250) NOT NULL, "secret" text NOT NULL, "dek" text NOT NULL, "recovery_codes" jsonb NOT NULL, "created_at" timestamp NOT NULL DEFAULT (now()), "updated_at" timestamp NOT NULL DEFAULT (now()) ); CREATE TABLE "user_auth_providers" ( "id" serial PRIMARY KEY, "user_id" integer NOT NULL, "email" varchar(250) NOT NULL, "provider" varchar(10) NOT NULL, "account_id" integer NOT NULL, "created_at" timestamp NOT NULL DEFAULT (now()), "updated_at" timestamp NOT NULL DEFAULT (now()) ); CREATE TABLE "blacklisted_tokens" ( "id" uuid PRIMARY KEY, "expires_at" timestamp NOT NULL, "created_at" timestamp NOT NULL DEFAULT (now()) ); CREATE UNIQUE INDEX "accounts_email_uidx" ON "accounts" ("email"); CREATE UNIQUE INDEX "accounts_username_uidx" ON "accounts" ("username"); CREATE UNIQUE INDEX "accounts_totps_account_id_uidx" ON "account_totps" ("account_id"); CREATE UNIQUE INDEX "account_credentials_client_id_uidx" ON "account_credentials" ("client_id"); CREATE INDEX "account_credentials_account_id_idx" ON "account_credentials" ("account_id"); CREATE INDEX "auth_providers_email_idx" ON "auth_providers" ("email"); CREATE UNIQUE INDEX "auth_providers_email_provider_uidx" ON "auth_providers" ("email", "provider"); CREATE INDEX "apps_account_id_idx" ON "apps" ("account_id"); CREATE UNIQUE INDEX "client_id_uidx" ON "apps" ("client_id"); CREATE INDEX "app_keys_app_id_idx" ON "app_keys" ("app_id"); CREATE INDEX "app_keys_account_id_idx" ON "app_keys" ("account_id"); CREATE UNIQUE INDEX "app_keys_name_app_id_uidx" ON "app_keys" ("name", "app_id"); CREATE UNIQUE INDEX "users_account_id_email_uidx" ON "users" ("account_id", "email"); CREATE INDEX "users_account_id_idx" ON "users" ("account_id"); CREATE UNIQUE INDEX "user_totps_user_id_uidx" ON "user_totps" ("user_id"); CREATE INDEX "user_auth_provider_email_idx" ON "user_auth_providers" ("email"); CREATE INDEX "user_auth_provider_user_id_idx" ON "user_auth_providers" ("user_id"); CREATE UNIQUE INDEX "user_auth_provider_account_id_provider_uidx" ON "user_auth_providers" ("email", "account_id", "provider"); CREATE INDEX "user_auth_provider_account_id_idx" ON "user_auth_providers" ("account_id"); ALTER TABLE "account_totps" ADD FOREIGN KEY ("account_id") REFERENCES "accounts" ("id") ON DELETE CASCADE; ALTER TABLE "account_credentials" ADD FOREIGN KEY ("account_id") REFERENCES "accounts" ("id") ON DELETE CASCADE; ALTER TABLE "auth_providers" ADD FOREIGN KEY ("email") REFERENCES "accounts" ("email") ON DELETE CASCADE ON UPDATE CASCADE; ALTER TABLE "apps" ADD FOREIGN KEY ("account_id") REFERENCES "accounts" ("id") ON DELETE CASCADE; ALTER TABLE "app_keys" ADD FOREIGN KEY ("app_id") REFERENCES "apps" ("id") ON DELETE CASCADE; ALTER TABLE "app_keys" ADD FOREIGN KEY ("account_id") REFERENCES "accounts" ("id") ON DELETE CASCADE; ALTER TABLE "users" ADD FOREIGN KEY ("account_id") REFERENCES "accounts" ("id") ON DELETE CASCADE; ALTER TABLE "user_totps" ADD FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE CASCADE; ALTER TABLE "user_auth_providers" ADD FOREIGN KEY ("account_id") REFERENCES "accounts" ("id") ON DELETE CASCADE; ALTER TABLE "user_auth_providers" ADD FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE CASCADE;
Enter fullscreen mode Exit fullscreen mode

While on the down drop all tables if they exist:

DROP TABLE IF EXISTS "user_auth_providers"; DROP TABLE IF EXISTS "user_totps"; DROP TABLE IF EXISTS "users"; DROP TABLE IF EXISTS "app_keys"; DROP TABLE IF EXISTS "apps"; DROP TABLE IF EXISTS "auth_providers"; DROP TABLE IF EXISTS "account_credentials"; DROP TABLE IF EXISTS "account_totps"; DROP TABLE IF EXISTS "accounts"; DROP TABLE IF EXISTS "blacklisted_tokens";
Enter fullscreen mode Exit fullscreen mode

SQLC Code generation

To set up the SQLC generated code we need to start by writting a query on the queries folder. Create a accounts.sql file in the queries directory and add the logic to insert an account:

-- name: CreateAccountWithPassword :one INSERT INTO "accounts" ( "first_name", "last_name", "username", "email", "password" ) VALUES ( $1, $2, $3, $4, $5 ) RETURNING *;
Enter fullscreen mode Exit fullscreen mode

SQLC knows what to return and what to call to the go method by the code comment on top of the SQL code.

Then in the terminal run the following command:

$ sqlc generate
Enter fullscreen mode Exit fullscreen mode

This will generate the following files:

  • models.go
  • db.go
  • accounts.sql.go

These files are auto-generated and auto-updated when you run the sqlc generate command, hence on a new database.go create the logic to connect to the SQLC generated code:

package database import ( "context" "github.com/jackc/pgx/v5" "github.com/jackc/pgx/v5/pgxpool" "github.com/tugascript/devlogs/idp/internal/exceptions" ) type Database struct { connPool *pgxpool.Pool *Queries } func NewDatabase(connPool *pgxpool.Pool) *Database { return &Database{ connPool: connPool, Queries: New(connPool), } } func (d *Database) BeginTx(ctx context.Context) (*Queries, pgx.Tx, error) { txn, err := d.connPool.BeginTx(ctx, pgx.TxOptions{ DeferrableMode: pgx.Deferrable, IsoLevel: pgx.ReadCommitted, AccessMode: pgx.ReadWrite, }) if err != nil { return nil, nil, err } return d.WithTx(txn), txn, nil } func (d *Database) FinalizeTx(ctx context.Context, txn pgx.Tx, err error, serviceErr *exceptions.ServiceError) { if serviceErr != nil || err != nil { if err := txn.Rollback(ctx); err != nil { panic(err) } return } if commitErr := txn.Commit(ctx); commitErr != nil { panic(commitErr) } if p := recover(); p != nil { if err := txn.Rollback(ctx); err != nil { panic(err) } panic(p) } } func (d *Database) RawQuery(ctx context.Context, sql string, args []interface{}) (pgx.Rows, error) { return d.connPool.Query(ctx, sql, args...) } func (d *Database) RawQueryRow(ctx context.Context, sql string, args []interface{}) pgx.Row { return d.connPool.QueryRow(ctx, sql, args...) } func (d *Database) Ping(ctx context.Context) error { return d.connPool.Ping(ctx) }
Enter fullscreen mode Exit fullscreen mode

Encryption

Encryption is less of a provider, but more of a logic isolation, hence we'll still load it as a provider.

Utilities

For each secret (or key), we will need to derive a KID (Key ID), this is common logic, so in the internal directory lets create a utils package and on a jwk.go folder add the following files:

  • encoders.go:

    package utils import ( "math/big" ) func Base62Encode(bytes []byte) string { var codeBig big.Int codeBig.SetBytes(bytes) return codeBig.Text(62) }

  • jwk.go:

    package utils import ( "crypto/sha256" ) func ExtractKeyID(keyBytes []byte) string { hash := sha256.Sum256(keyBytes) return Base62Encode(hash[:12]) }

  • secrets.go:

 package utils import ( "crypto/rand" "encoding/base32" "encoding/base64" "encoding/hex" "math/big" ) func generateRandomBytes(byteLen int) ([]byte, error) { b := make([]byte, byteLen) if _, err := rand.Read(b); err != nil { return nil, err } return b, nil } func GenerateBase64Secret(byteLen int) (string, error) { randomBytes, err := generateRandomBytes(byteLen) if err != nil { return "", err } return base64.RawURLEncoding.EncodeToString(randomBytes), nil } func DecodeBase64Secret(secret string) ([]byte, error) { decoded, err := base64.RawURLEncoding.DecodeString(secret) if err != nil { return nil, err } return decoded, nil } func GenerateBase62Secret(byteLen int) (string, error) { randomBytes, err := generateRandomBytes(byteLen) if err != nil { return "", err } randomInt := new(big.Int).SetBytes(randomBytes) return randomInt.Text(62), nil } func GenerateBase32Secret(byteLen int) (string, error) { randomBytes, err := generateRandomBytes(byteLen) if err != nil { return "", err } return base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(randomBytes), nil } func GenerateHexSecret(byteLen int) (string, error) { randomBytes, err := generateRandomBytes(byteLen) if err != nil { return "", err } return hex.EncodeToString(randomBytes), nil }
Enter fullscreen mode Exit fullscreen mode

Provider

Now with the utilites done, create the following providers/encryption package:

package encryption import ( "encoding/base64" "log/slog" "github.com/tugascript/devlogs/idp/internal/config" "github.com/tugascript/devlogs/idp/internal/utils" ) const logLayer string = "encryption" type Secret struct { kid string key []byte } type Encryption struct { logger *slog.Logger accountSecretKey Secret appSecretKey Secret userSecretKey Secret oldSecrets map[string][]byte backendDomain string } func decodeSecret(secret string) Secret { // Decode the Base64 code to bytes decodedKey, err := base64.StdEncoding.DecodeString(secret) if err != nil { panic(err) } return Secret{ // Generate a Key ID per secret kid: utils.ExtractKeyID(decodedKey), key: decodedKey, } } func NewEncryption( logger *slog.Logger, cfg config.EncryptionConfig, backendDomain string, ) *Encryption { oldSecretsMap := make(map[string][]byte) for _, s := range cfg.OldSecrets() { ds := decodeSecret(s) oldSecretsMap[ds.kid] = ds.key } return &Encryption{ logger: logger, accountSecretKey: decodeSecret(cfg.AccountSecret()), appSecretKey: decodeSecret(cfg.AppSecret()), userSecretKey: decodeSecret(cfg.UserSecret()), oldSecrets: oldSecretsMap, backendDomain: backendDomain, } }
Enter fullscreen mode Exit fullscreen mode

DEK Generation

DEKs (Date Encryption Keys) need to be generated by the system, and are saved in the database, for this reason, create a file called dek.go and add the following logic:

  • DEK generation & Encryption: 
package encryption import ( "fmt" "github.com/tugascript/devlogs/idp/internal/utils" ) const dekLocation string = "dek" func generateDEK(keyID string, key []byte) ([]byte, string, error) { base64DEK, err := utils.GenerateBase64Secret(32) if err != nil { return nil, "", err } encryptedDEK, err := utils.Encrypt(base64DEK, key) if err != nil { return nil, "", err } dek, err := utils.DecodeBase64Secret(base64DEK) if err != nil { return nil, "", err } return dek, fmt.Sprintf("%s:%s", keyID, encryptedDEK), nil } // ... func reEncryptDEK(isOldKey bool, dek, key []byte) (string, error) { if !isOldKey { return "", nil } return utils.Encrypt(base64.RawURLEncoding.EncodeToString(dek), key) }
Enter fullscreen mode Exit fullscreen mode
  • DEK decryption: 
package encryption import ( "context" "encoding/base64" "errors" "fmt" "log/slog" "strings" "github.com/tugascript/devlogs/idp/internal/utils" ) // ... type decryptDEKOptions struct { storedDEK string secret *Secret oldSecrets map[string][]byte } func decryptDEK( logger *slog.Logger, ctx context.Context, opts decryptDEKOptions, ) ([]byte, bool, error) { dekID, encryptedDEK, err := splitDEK(opts.storedDEK) if err != nil { logger.ErrorContext(ctx, "Failed to split DEK", "error", err) return nil, false, err } key := opts.secret.key oldKey := dekID != opts.secret.kid if oldKey { var ok bool key, ok = opts.oldSecrets[dekID] if !ok { logger.ErrorContext(ctx, "DEK key ID not found") return nil, false, errors.New("secret key not found") } } base64DEK, err := utils.Decrypt(encryptedDEK, key) if err != nil { logger.ErrorContext(ctx, "Failed to decrypt DEK", "error", err) return nil, false, err } dek, err := utils.DecodeBase64Secret(base64DEK) if err != nil { logger.ErrorContext(ctx, "Failed to decode DEK", "error", err) return nil, false, err } return dek, oldKey, nil }
Enter fullscreen mode Exit fullscreen mode
  • Methods to decrypt for Account, App & User: 
// ... func (e *Encryption) decryptAccountDEK(ctx context.Context, requestID, storedDEK string) ([]byte, bool, error) { logger := utils.BuildLogger(e.logger, utils.LoggerOptions{ Layer: logLayer, Location: dekLocation, Method: "decryptAccountDEK", RequestID: requestID, }) logger.DebugContext(ctx, "Decrypting Account DEK...") return decryptDEK(logger, ctx, decryptDEKOptions{ storedDEK: storedDEK, secret: &e.accountSecretKey, oldSecrets: e.oldSecrets, }) } func (e *Encryption) decryptAppDEK(ctx context.Context, requestID, storedDEK string) ([]byte, bool, error) { logger := utils.BuildLogger(e.logger, utils.LoggerOptions{ Layer: logLayer, Location: dekLocation, Method: "decryptAppDEK", RequestID: requestID, }) logger.DebugContext(ctx, "Decrypting App DEK...") return decryptDEK(logger, ctx, decryptDEKOptions{ storedDEK: storedDEK, secret: &e.appSecretKey, oldSecrets: e.oldSecrets, }) } func (e *Encryption) decryptUserDEK(ctx context.Context, requestID, storedDEK string) ([]byte, bool, error) { logger := utils.BuildLogger(e.logger, utils.LoggerOptions{ Layer: logLayer, Location: dekLocation, Method: "decryptUserDEK", RequestID: requestID, }) logger.DebugContext(ctx, "Decrypting User DEK...") return decryptDEK(logger, ctx, decryptDEKOptions{ storedDEK: storedDEK, secret: &e.userSecretKey, oldSecrets: e.oldSecrets, }) }
Enter fullscreen mode Exit fullscreen mode

Mailer

The mailer will just be a redis publisher, that will publish messages to the email service, hence create a mailer package with the following mailer.go file:

package mailer import ( "context" "encoding/json" "log/slog" "github.com/tugascript/devlogs/idp/internal/utils" "github.com/redis/go-redis/v9" ) const logLayer string = "mailer" type email struct { To string `json:"to"` Subject string `json:"subject"` Body string `json:"body"` } type EmailPublisher struct { client redis.UniversalClient pubChannel string frontendDomain string logger *slog.Logger } func NewEmailPublisher( client redis.UniversalClient, pubChannel, frontendDomain string, logger *slog.Logger, ) *EmailPublisher { return &EmailPublisher{ client: client, pubChannel: pubChannel, frontendDomain: frontendDomain, logger: logger, } } type PublishEmailOptions struct { To string Subject string Body string RequestID string } func (e *EmailPublisher) publishEmail(ctx context.Context, opts PublishEmailOptions) error { logger := utils.BuildLogger(e.logger, utils.LoggerOptions{ Layer: logLayer, Location: "mailer", Method: "PublishEmail", RequestID: opts.RequestID, }) logger.DebugContext(ctx, "Publishing email...") message, err := json.Marshal(email{ To: opts.To, Subject: opts.Subject, Body: opts.Body, }) if err != nil { logger.ErrorContext(ctx, "Failed to marshal email", "error", err) return err } return e.client.Publish(ctx, e.pubChannel, string(message)).Err() }
Enter fullscreen mode Exit fullscreen mode

It will consume the UniversalClient from the cache, and publish to a given channel.

OAuth

For OAuth 2.0 we can just use the golang.org/x/oauth2 package, just run:

$ go get golang.org/x/oauth2
Enter fullscreen mode Exit fullscreen mode

Scopes

Each external provider has their own scopes, lets start by creating a scopes struct:

package oauth // ... type oauthScopes struct { email string profile string birthday string location string gender string }
Enter fullscreen mode Exit fullscreen mode

And for each provider lets create a global scope variable:

  • apple.go:

    package oauth // ... var appleScopes = oauthScopes{ email: "email", profile: "name", }

  • facebook.go:

    package oauth // ... var facebookScopes = oauthScopes{ email: "email", profile: "public_profile", birthday: "user_birthday", location: "user_location", gender: "gender", }

  • github.go:

    package oauth // ... var gitHubScopes = oauthScopes{ email: "user:email", profile: "read:user", }

  • google.go:

    package oauth // ... var googleScopes = oauthScopes{ email: "https://www.googleapis.com/auth/userinfo.email", profile: "https://www.googleapis.com/auth/userinfo.profile", birthday: "https://www.googleapis.com/auth/user.birthday.read", location: "https://www.googleapis.com/auth/user.addresses.read", gender: "https://www.googleapis.com/auth/user.gender.read", }

  • microsoft.go:

    package oauth // ... var microsoftScopes = oauthScopes{ email: "User.Read", profile: "User.ReadBasic.All", }

Provider

Start by creating an oauth2.Config struct for each external provider, using existing defaults when possible:

package oauth import ( "log/slog" "golang.org/x/oauth2" "golang.org/x/oauth2/facebook" "golang.org/x/oauth2/github" "golang.org/x/oauth2/google" "golang.org/x/oauth2/microsoft" "github.com/tugascript/devlogs/idp/internal/config" ) // ... type Config struct { Enabled bool oauth2.Config } type Providers struct { gitHub Config google Config facebook Config apple Config microsoft Config logger *slog.Logger } func NewProviders( log *slog.Logger, githubCfg, googleCfg, facebookCfg, appleCfg, microsoftCfg config.OAuthProviderConfig, ) *Providers { return &Providers{ gitHub: Config{ Config: oauth2.Config{ ClientID: githubCfg.ClientID(), ClientSecret: githubCfg.ClientSecret(), Endpoint: github.Endpoint, Scopes: []string{gitHubScopes.email}, }, Enabled: githubCfg.Enabled(), }, google: Config{ Config: oauth2.Config{ ClientID: googleCfg.ClientID(), ClientSecret: googleCfg.ClientSecret(), Endpoint: google.Endpoint, Scopes: []string{googleScopes.email}, }, Enabled: googleCfg.Enabled(), }, facebook: Config{ Config: oauth2.Config{ ClientID: facebookCfg.ClientID(), ClientSecret: facebookCfg.ClientSecret(), Endpoint: facebook.Endpoint, Scopes: []string{facebookScopes.email}, }, Enabled: facebookCfg.Enabled(), }, apple: Config{ Config: oauth2.Config{ ClientID: appleCfg.ClientID(), ClientSecret: appleCfg.ClientSecret(), Endpoint: oauth2.Endpoint{ AuthURL: "https://appleid.apple.com/auth/authorize", TokenURL: "https://appleid.apple.com/auth/token", }, Scopes: []string{appleScopes.email}, }, Enabled: appleCfg.Enabled(), }, microsoft: Config{ Config: oauth2.Config{ ClientID: microsoftCfg.ClientID(), ClientSecret: microsoftCfg.ClientSecret(), Endpoint: microsoft.AzureADEndpoint("common"), Scopes: []string{microsoftScopes.email}, }, Enabled: microsoftCfg.Enabled(), }, logger: log, } }
Enter fullscreen mode Exit fullscreen mode

For getting the token we will need to pass the correct Scopes, this touches on a concept in go that is passing a value as a value (or copy) of the underlying resource. Since we dynamically add scopes we create a copy of the config each time.

Getting the access token:

package oauth import ( "context" // ... "github.com/tugascript/devlogs/idp/internal/exceptions" ) // ... func mapScopes(scopes []Scope, oas oauthScopes) []string { scopeMapper := make(map[string]bool) for _, s := range scopes { switch s { case ScopeBirthday: scopeMapper[oas.birthday] = true case ScopeGender: scopeMapper[oas.gender] = true case ScopeLocation: scopeMapper[oas.location] = true case ScopeProfile: scopeMapper[oas.location] = true } } mappedScopes := make([]string, 0, len(scopeMapper)) for k := range scopeMapper { if k != "" { mappedScopes = append(mappedScopes, k) } } return mappedScopes } // Here we pass by value so we don't update the base configuration func appendScopes(cfg Config, scopes []string) Config { cfg.Scopes = append(cfg.Scopes, scopes...) return cfg } // Here we pass by value so we don't update the base configuration func getConfig(cfg Config, redirectURL string, oas oauthScopes, scopes []Scope) Config { cfg.RedirectURL = redirectURL if scopes != nil { return appendScopes(cfg, mapScopes(scopes, oas)) } return cfg } type getAccessTokenOptions struct { logger *slog.Logger cfg Config redirectURL string oas oauthScopes scopes []Scope code string } func getAccessToken(ctx context.Context, opts getAccessTokenOptions) (string, *exceptions.ServiceError) { opts.logger.DebugContext(ctx, "Getting access token...") if !opts.cfg.Enabled { opts.logger.DebugContext(ctx, "OAuth config is disabled") return "", exceptions.NewNotFoundError() } cfg := getConfig(opts.cfg, opts.redirectURL, opts.oas, opts.scopes) token, err := cfg.Exchange(ctx, opts.code) if err != nil { opts.logger.ErrorContext(ctx, "Failed to exchange the code for a token", "error", err) return "", exceptions.NewUnauthorizedError() } opts.logger.DebugContext(ctx, "Access token exchanged successfully") return token.AccessToken, nil }
Enter fullscreen mode Exit fullscreen mode

Getting the authorization URL:

package oauth import ( "context" // ... "github.com/tugascript/devlogs/idp/internal/utils" ) type getAuthorizationURLOptions struct { logger *slog.Logger redirectURL string cfg Config oas oauthScopes scopes []Scope } func getAuthorizationURL( ctx context.Context, opts getAuthorizationURLOptions, ) (string, string, *exceptions.ServiceError) { opts.logger.DebugContext(ctx, "Getting authorization url...") if !opts.cfg.Enabled { opts.logger.DebugContext(ctx, "OAuth config is disabled") return "", "", exceptions.NewNotFoundError() } state, err := utils.GenerateHexSecret(16) if err != nil { opts.logger.ErrorContext(ctx, "Failed to generate state", "error", err) return "", "", exceptions.NewServerError() } cfg := getConfig(opts.cfg, opts.redirectURL, opts.oas, opts.scopes) url := cfg.AuthCodeURL(state) opts.logger.DebugContext(ctx, "Authorization url generated successfully") return url, state, nil }
Enter fullscreen mode Exit fullscreen mode

Getting the user data:

package oauth import ( "context" "errors" "io" "log/slog" "net/http" // ... "github.com/tugascript/devlogs/idp/internal/config" "github.com/tugascript/devlogs/idp/internal/exceptions" "github.com/tugascript/devlogs/idp/internal/utils" ) func getUserResponse(logger *slog.Logger, ctx context.Context, url, token string) ([]byte, int, error) { logger.DebugContext(ctx, "Getting user data...", "url", url) logger.DebugContext(ctx, "Building user data request") req, err := http.NewRequest(http.MethodGet, url, nil) if err != nil { logger.ErrorContext(ctx, "Failed to build user data request") return nil, 0, err } req.Header.Set("Accept", "application/json") req.Header.Set("Authorization", "Bearer "+token) logger.DebugContext(ctx, "Requesting user data...") res, err := http.DefaultClient.Do(req) if err != nil { logger.ErrorContext(ctx, "Failed to request the user data") return nil, 0, err } if res.StatusCode != http.StatusOK { logger.ErrorContext(ctx, "Responded with a non 200 OK status", "status", res.StatusCode) return nil, res.StatusCode, errors.New("status code is not 200 OK") } logger.DebugContext(ctx, "Reading the body") body, err := io.ReadAll(res.Body) if err != nil { logger.ErrorContext(ctx, "Failed to read the body", "error", err) return nil, 0, err } defer func() { if err := res.Body.Close(); err != nil { logger.ErrorContext(ctx, "Failed to close response body", "error", err) } }() return body, res.StatusCode, nil } type UserLocation struct { City string Region string Country string } type UserData struct { Name string FirstName string LastName string Username string Picture string Email string Gender string Location UserLocation BirthDate string IsVerified bool } type ToUserData interface { ToUserData() UserData } type extraParams struct { params string } func (p *extraParams) addParam(prm string) { if p.params != "" { p.params = p.params + "," + prm return } p.params += prm } func (p *extraParams) isEmpty() bool { return p.params == "" }
Enter fullscreen mode Exit fullscreen mode

Common options between all providers:

package oauth // ... type AccessTokenOptions struct { RequestID string Code string RedirectURL string Scopes []Scope } type AuthorizationURLOptions struct { RequestID string RedirectURL string Scopes []Scope } type UserDataOptions struct { RequestID string Token string Scopes []Scope }
Enter fullscreen mode Exit fullscreen mode

Tokens

Keys Algorithms

When choosing the algorithm of the key pairs to sign and verify JWTs we need to choose the best ones with the best efficient and security ratio in mind.

By my own research the most balanced algorithm would be EdDSC (Edwards-curve Digital Signature Algorithm) with the Ed25519 signature scheme, however this algorithm is not part of the base JWT RFC-7519 standard, only the RFC-8037 which is not widly supported.

Hence for simplicity and compatibility sake for all tokens that require their public keys to be distributed will use the recommended ECDSA (Elliptic Curve Digital Signature Algorithm) algorithm with the P-256 signature scheme.

Utilities

For sharing and saving keys we need to convert them to JWKs so create an encode and decode functions for both Ed25519 and P256 on the utils/jwk.go file:

package utils import ( "crypto/ecdsa" "crypto/ed25519" "crypto/elliptic" "crypto/sha256" "encoding/base64" "fmt" "math/big" ) type Ed25519JWK struct { Kty string `json:"kty"` // Key Type (OKP for Ed25519) Crv string `json:"crv"` // Curve (Ed25519) X string `json:"x"` // Public Key Use string `json:"use"` // Usage (e.g., "sig" for signing) Alg string `json:"alg"` // Algorithm (EdDSA for Ed25519) Kid string `json:"kid"` // Key AccountID KeyOps []string `json:"key_ops"` // Key Operations } type P256JWK struct { Kty string `json:"kty"` // Key Type (EC for Elliptic Curve) Crv string `json:"crv"` // Curve (P-256) X string `json:"x"` // X Coordinate Y string `json:"y"` // Y Coordinate Use string `json:"use"` // Usage (e.g., "sig" for signing) Alg string `json:"alg"` // Algorithm (ES256 for P-256) Kid string `json:"kid"` // Key AccountID KeyOps []string `json:"key_ops"` // Key Operations } // Because of Apple type RS256JWK struct { Kty string `json:"kty"` Kid string `json:"kid"` Use string `json:"use"` Alg string `json:"alg"` N string `json:"n"` E string `json:"e"` KeyOps []string `json:"key_ops,omitempty"` } const ( kty string = "OKP" crv string = "Ed25519" use string = "sig" alg string = "EdDSA" verify string = "verify" p256Kty string = "EC" p256Crv string = "P-256" ) // ... func EncodeEd25519Jwk(publicKey ed25519.PublicKey, kid string) Ed25519JWK { return Ed25519JWK{ Kty: kty, Crv: crv, X: base64.RawURLEncoding.EncodeToString(publicKey), Use: use, Alg: alg, Kid: kid, KeyOps: []string{verify}, } } func DecodeEd25519Jwk(jwk Ed25519JWK) (ed25519.PublicKey, error) { publicKey, err := base64.RawURLEncoding.DecodeString(jwk.X) if err != nil { return nil, err } return publicKey, nil } func EncodeP256Jwk(publicKey *ecdsa.PublicKey, kid string) P256JWK { return P256JWK{ Kty: p256Kty, Crv: p256Crv, X: base64.RawURLEncoding.EncodeToString(publicKey.X.Bytes()), Y: base64.RawURLEncoding.EncodeToString(publicKey.Y.Bytes()), Use: use, Alg: alg, Kid: kid, KeyOps: []string{verify}, } } func DecodeP256Jwk(jwk P256JWK) (ecdsa.PublicKey, error) { x, err := base64.RawURLEncoding.DecodeString(jwk.X) if err != nil { return ecdsa.PublicKey{}, err } y, err := base64.RawURLEncoding.DecodeString(jwk.Y) if err != nil { return ecdsa.PublicKey{}, err } return ecdsa.PublicKey{ Curve: elliptic.P256(), X: new(big.Int).SetBytes(x), Y: new(big.Int).SetBytes(y), }, nil } func DecodeRS256Jwk(jwk RS256JWK) (*rsa.PublicKey, error) { nBytes, err := base64.RawURLEncoding.DecodeString(jwk.N) if err != nil { return nil, err } n := new(big.Int).SetBytes(nBytes) eBytes, err := base64.RawURLEncoding.DecodeString(jwk.E) if err != nil { return nil, err } e := big.NewInt(0).SetBytes(eBytes).Int64() if e <= 0 { return nil, fmt.Errorf("invalid RSA exponent") } return &rsa.PublicKey{N: n, E: int(e)}, nil }
Enter fullscreen mode Exit fullscreen mode

Provider

For each key we need a key pair, and a reference to a previous public key for keys rotation, create the tokens/tokens.go package:

package tokens import ( "crypto/ecdsa" "crypto/ed25519" "crypto/x509" "encoding/pem" "github.com/golang-jwt/jwt/v5" "github.com/tugascript/devlogs/idp/internal/config" "github.com/tugascript/devlogs/idp/internal/utils" ) type PreviousPublicKey struct { publicKey ed25519.PublicKey kid string } type TokenKeyPair struct { publicKey ed25519.PublicKey privateKey ed25519.PrivateKey kid string } type TokenSecretData struct { curKeyPair TokenKeyPair prevPubKey *PreviousPublicKey ttlSec int64 } // ... type PreviousEs256PublicKey struct { publicKey *ecdsa.PublicKey kid string } type Es256TokenKeyPair struct { privateKey *ecdsa.PrivateKey publicKey *ecdsa.PublicKey kid string } type Es256TokenSecretData struct { curKeyPair Es256TokenKeyPair prevPubKey *PreviousEs256PublicKey ttlSec int64 }
Enter fullscreen mode Exit fullscreen mode

Now each key is encoded as a PEM in the environment, hence we need to decode the x509 certificates for:

  • Ed25519 keys: 
package tokens // ... func extractEd25519PublicKey(publicKey string) (ed25519.PublicKey, string) { publicKeyBlock, _ := pem.Decode([]byte(publicKey)) if publicKeyBlock == nil || publicKeyBlock.Type != "PUBLIC KEY" { panic("Invalid public key") } publicKeyData, err := x509.ParsePKIXPublicKey(publicKeyBlock.Bytes) if err != nil { panic(err) } publicKeyValue, ok := publicKeyData.(ed25519.PublicKey) if !ok { panic("Invalid public key") } return publicKeyValue, utils.ExtractKeyID(publicKeyValue) } func extractEd25519PrivateKey(privateKey string) ed25519.PrivateKey { privateKeyBlock, _ := pem.Decode([]byte(privateKey)) if privateKeyBlock == nil || privateKeyBlock.Type != "PRIVATE KEY" { panic("Invalid private key") } privateKeyData, err := x509.ParsePKCS8PrivateKey(privateKeyBlock.Bytes) if err != nil { panic(err) } privateKeyValue, ok := privateKeyData.(ed25519.PrivateKey) if !ok { panic("Invalid private key") } return privateKeyValue } func extractEd25519PublicPrivateKeyPair(publicKey, privateKey string) TokenKeyPair { pubKey, kid := extractEd25519PublicKey(publicKey) return TokenKeyPair{ publicKey: pubKey, privateKey: extractEd25519PrivateKey(privateKey), kid: kid, } } func newTokenSecretData( publicKey, privateKey, previousPublicKey string, ttlSec int64, ) TokenSecretData { curKeyPair := extractEd25519PublicPrivateKeyPair(publicKey, privateKey) if previousPublicKey != "" { pubKey, kid := extractEd25519PublicKey(previousPublicKey) return TokenSecretData{ curKeyPair: curKeyPair, prevPubKey: &PreviousPublicKey{publicKey: pubKey, kid: kid}, ttlSec: ttlSec, } } return TokenSecretData{ curKeyPair: curKeyPair, ttlSec: ttlSec, } }
Enter fullscreen mode Exit fullscreen mode
  • Es256 keys: 
package tokens // ... func extractEs256KeyPair(privateKey string) Es256TokenKeyPair { privateKeyBlock, _ := pem.Decode([]byte(privateKey)) if privateKeyBlock == nil || privateKeyBlock.Type != "PRIVATE KEY" { panic("Invalid private key") } privateKeyData, err := x509.ParsePKCS8PrivateKey(privateKeyBlock.Bytes) if err != nil { privateKeyData, err = x509.ParseECPrivateKey(privateKeyBlock.Bytes) if err != nil { panic(err) } } privateKeyValue, ok := privateKeyData.(*ecdsa.PrivateKey) if !ok { panic("Invalid private key") } publicKeyValue, err := x509.MarshalPKIXPublicKey(&privateKeyValue.PublicKey) if err != nil { panic(err) } return Es256TokenKeyPair{ privateKey: privateKeyValue, publicKey: &privateKeyValue.PublicKey, kid: utils.ExtractKeyID(publicKeyValue), } } func extractEs256PublicKey(publicKey string) (*ecdsa.PublicKey, string) { publicKeyBlock, _ := pem.Decode([]byte(publicKey)) if publicKeyBlock == nil || publicKeyBlock.Type != "PUBLIC KEY" { panic("Invalid public key") } publicKeyData, err := x509.ParsePKIXPublicKey(publicKeyBlock.Bytes) if err != nil { panic(err) } pubKey, ok := publicKeyData.(*ecdsa.PublicKey) if !ok { panic("Invalid public key") } publicKeyValue, err := x509.MarshalPKIXPublicKey(pubKey) if err != nil { panic(err) } return pubKey, utils.ExtractKeyID(publicKeyValue) } // ... func newEs256TokenSecretData(privateKey, previousPublicKey string, ttlSec int64) Es256TokenSecretData { curKeyPair := extractEs256KeyPair(privateKey) if previousPublicKey != "" { prevPubKey, kid := extractEs256PublicKey(previousPublicKey) return Es256TokenSecretData{ curKeyPair: curKeyPair, prevPubKey: &PreviousEs256PublicKey{publicKey: prevPubKey, kid: kid}, ttlSec: ttlSec, } } return Es256TokenSecretData{ curKeyPair: curKeyPair, ttlSec: ttlSec, } }
Enter fullscreen mode Exit fullscreen mode

Finally finish by creating the full provider:

package tokens // ... type Tokens struct { frontendDomain string backendDomain string accessData Es256TokenSecretData accountCredentialsData Es256TokenSecretData refreshData TokenSecretData confirmationData TokenSecretData resetData TokenSecretData oauthData TokenSecretData twoFAData TokenSecretData jwks []utils.P256JWK } func NewTokens( accessCfg, accountCredentialsCfg, refreshCfg, confirmationCfg, resetCfg, oauthCfg, twoFACfg config.SingleJwtConfig, frontendDomain, backendDomain string, ) *Tokens { accessData := newEs256TokenSecretData( accessCfg.PrivateKey(), accessCfg.PreviousPublicKey(), accessCfg.TtlSec(), ) accountKeysData := newEs256TokenSecretData( accountCredentialsCfg.PrivateKey(), accountCredentialsCfg.PreviousPublicKey(), accountCredentialsCfg.TtlSec(), ) jwks := []utils.P256JWK{ utils.EncodeP256Jwk(accountKeysData.curKeyPair.publicKey, accountKeysData.curKeyPair.kid), utils.EncodeP256Jwk(accessData.curKeyPair.publicKey, accessData.curKeyPair.kid), } if accountKeysData.prevPubKey != nil { jwks = append(jwks, utils.EncodeP256Jwk( accountKeysData.prevPubKey.publicKey, accessData.prevPubKey.kid, )) } if accessData.prevPubKey != nil { jwks = append(jwks, utils.EncodeP256Jwk( accessData.prevPubKey.publicKey, accessData.prevPubKey.kid, )) } return &Tokens{ accessData: accessData, accountCredentialsData: accountKeysData, refreshData: newTokenSecretData( refreshCfg.PublicKey(), refreshCfg.PrivateKey(), refreshCfg.PreviousPublicKey(), refreshCfg.TtlSec(), ), confirmationData: newTokenSecretData( confirmationCfg.PublicKey(), confirmationCfg.PrivateKey(), confirmationCfg.PreviousPublicKey(), confirmationCfg.TtlSec(), ), resetData: newTokenSecretData( resetCfg.PublicKey(), resetCfg.PrivateKey(), resetCfg.PreviousPublicKey(), resetCfg.TtlSec(), ), oauthData: newTokenSecretData( oauthCfg.PublicKey(), oauthCfg.PrivateKey(), oauthCfg.PreviousPublicKey(), oauthCfg.TtlSec(), ), twoFAData: newTokenSecretData( twoFACfg.PublicKey(), twoFACfg.PrivateKey(), twoFACfg.PreviousPublicKey(), twoFACfg.TtlSec(), ), frontendDomain: frontendDomain, backendDomain: backendDomain, jwks: jwks, } } func (t *Tokens) JWKs() []utils.P256JWK { return t.jwks }
Enter fullscreen mode Exit fullscreen mode

Error Handling

Error handling in Go works differently from most languages, instead of throwing and catching exceptions, values are return as a pointer (error as values) to the error.

This means for ease of use, we need to map/coerce the errors to a known value on the service layer, and map them to an error response on the controller layer.

Service Errors

Create an expections directory on the internal folder for the errors mapping.

Format

The service error is gonna have a standard error class, with a type/code, and a message, but for simplicity we won't add a details slice field.

Create the exceptions/services.go file for the exceptions package:

package exceptions type ServiceError struct { Code string Message string } func NewError(code string, message string) *ServiceError { return &ServiceError{ Code: code, Message: message, } } // To make `ServiceError` an error type interface func (e *ServiceError) Error() string { return e.Message }
Enter fullscreen mode Exit fullscreen mode

Codes and messages

We need to standerdize the code and message based on HTTP status codes:

package exceptions // ... const ( CodeValidation string = "VALIDATION" CodeConflict string = "CONFLICT" CodeInvalidEnum string = "INVALID_ENUM" CodeNotFound string = "NOT_FOUND" CodeUnknown string = "UNKNOWN" CodeServerError string = "SERVER_ERROR" CodeUnauthorized string = "UNAUTHORIZED" CodeForbidden string = "FORBIDDEN" CodeUnsupportedMediaType string = "UNSUPPORTED_MEDIA_TYPE" ) const ( MessageDuplicateKey string = "Resource already exists" MessageNotFound string = "Resource not found" MessageUnknown string = "Something went wrong" MessageUnauthorized string = "Unauthorized" MessageForbidden string = "Forbidden" ) func NewNotFoundError() *ServiceError { return NewError(CodeNotFound, MessageNotFound) } func NewValidationError(message string) *ServiceError { return NewError(CodeValidation, message) } func NewServerError() *ServiceError { return NewError(CodeServerError, MessageUnknown) } func NewConflictError(message string) *ServiceError { return NewError(CodeConflict, message) } func NewUnsupportedMediaTypeError(message string) *ServiceError { return NewError(CodeUnsupportedMediaType, message) } func NewUnauthorizedError() *ServiceError { return NewError(CodeUnauthorized, MessageUnauthorized) } func NewForbiddenError() *ServiceError { return NewError(CodeForbidden, MessageForbidden) }
Enter fullscreen mode Exit fullscreen mode

Coercion

For the PostgreSQL errors we can coerce them using a mapper:

package exceptions import ( "errors" "github.com/jackc/pgx/v5" "github.com/jackc/pgx/v5/pgconn" ) // ... func FromDBError(err error) *ServiceError { if errors.Is(err, pgx.ErrNoRows) { return NewError(CodeNotFound, MessageNotFound) } var pgErr *pgconn.PgError if errors.As(err, &pgErr) { switch pgErr.Code { case "23505": return NewError(CodeConflict, MessageDuplicateKey) case "23514": return NewError(CodeInvalidEnum, pgErr.Message) case "23503": return NewError(CodeNotFound, MessageNotFound) default: return NewError(CodeUnknown, pgErr.Message) } } return NewError(CodeUnknown, MessageUnknown) }
Enter fullscreen mode Exit fullscreen mode

Controllers Error

ServiceError need to be mappend to a JSON Response, however there are also other types of error responses: body validation and oauth validation.

Error Response

Create the controllers.go and add the error response, which is the same as ServiceError but JSON parsable:

package exceptions // ... const ( StatusConflict string = "Conflict" StatusInvalidEnum string = "BadRequest" StatusNotFound string = "NotFound" StatusServerError string = "InternalServerError" StatusUnknown string = "InternalServerError" StatusUnauthorized string = "Unauthorized" StatusForbidden string = "Forbidden" StatusValidation string = "Validation" ) type ErrorResponse struct { Code string `json:"code"` Message string `json:"message"` } func NewErrorResponse(err *ServiceError) ErrorResponse { switch err.Code { case CodeServerError: return ErrorResponse{ Code: StatusServerError, Message: err.Message, } case CodeConflict: return ErrorResponse{ Code: StatusConflict, Message: err.Message, } case CodeInvalidEnum: return ErrorResponse{ Code: StatusInvalidEnum, Message: err.Message, } case CodeNotFound: return ErrorResponse{ Code: StatusNotFound, Message: err.Message, } case CodeValidation: return ErrorResponse{ Code: StatusValidation, Message: err.Message, } case CodeUnknown: return ErrorResponse{ Code: StatusUnknown, Message: err.Message, } case CodeUnauthorized: return ErrorResponse{ Code: StatusUnauthorized, Message: StatusUnauthorized, } case CodeForbidden: return ErrorResponse{ Code: StatusForbidden, Message: StatusForbidden, } default: return ErrorResponse{ Code: StatusUnknown, Message: err.Message, } } }
Enter fullscreen mode Exit fullscreen mode

Validation Response

To validation the body inputs we will use the go validator package, start by installing it:

$ go get github.com/go-playground/validator/v10
Enter fullscreen mode Exit fullscreen mode

Services

Services is where most of the server business logic is located, lets create the services/services.go package, the Services struct needs to encapsulate all providers, and be the only layer talking directly to them.

package services import ( "log/slog" "github.com/tugascript/devlogs/idp/internal/providers/cache" "github.com/tugascript/devlogs/idp/internal/providers/database" "github.com/tugascript/devlogs/idp/internal/providers/encryption" "github.com/tugascript/devlogs/idp/internal/providers/mailer" "github.com/tugascript/devlogs/idp/internal/providers/oauth" "github.com/tugascript/devlogs/idp/internal/providers/tokens" ) type Services struct { logger *slog.Logger database *database.Database cache *cache.Cache mail *mailer.EmailPublisher jwt *tokens.Tokens encrypt *encryption.Encryption oauthProviders *oauth.Providers } func NewServices( logger *slog.Logger, database *database.Database, cache *cache.Cache, mail *mailer.EmailPublisher, jwt *tokens.Tokens, encrypt *encryption.Encryption, oauthProv *oauth.Providers, ) *Services { return &Services{ logger: logger, database: database, cache: cache, mail: mail, jwt: jwt, encrypt: encrypt, oauthProviders: oauthProv, } }
Enter fullscreen mode Exit fullscreen mode

We also need a helper to build the logger, and some common constant, create a helpers.go file:

 import ( "log/slog" "github.com/tugascript/devlogs/idp/internal/utils" ) const ( AuthProviderEmail string = "email" AuthProviderGoogle string = "google" AuthProviderGitHub string = "github" AuthProviderApple string = "apple" AuthProviderMicrosoft string = "microsoft" AuthProviderFacebook string = "facebook" TwoFactorNone string = "none" TwoFactorEmail string = "email" TwoFactorTotp string = "totp" ) func (s *Services) buildLogger(requestID, location, function string) *slog.Logger { return utils.BuildLogger(s.logger, utils.LoggerOptions{ Layer: utils.ServicesLogLayer, Location: location, Method: function, RequestID: requestID, }) }
Enter fullscreen mode Exit fullscreen mode

Health

For our first service, we will create the health endpoint for our API, this service only needs to ping PostgreSQL and ValKey.

Create the health.go file:

package services import ( "context" "github.com/tugascript/devlogs/idp/internal/exceptions" ) const healthLocation string = "health" func (s *Services) HealthCheck(ctx context.Context, requestID string) *exceptions.ServiceError { logger := s.buildLogger(requestID, healthLocation, "HealthCheck") logger.InfoContext(ctx, "Performing health check...") if err := s.database.Ping(ctx); err != nil { logger.ErrorContext(ctx, "Failed to ping database", "error", err) return exceptions.NewServerError() } if err := s.cache.Ping(ctx); err != nil { logger.ErrorContext(ctx, "Failed to ping cache", "error", err) return exceptions.NewServerError() } logger.InfoContext(ctx, "Service is healthy") return nil }
Enter fullscreen mode Exit fullscreen mode

Controllers

Controllers are where we map our services to the correct HTTP status response.

Helpers

For the controllor, error handling and logging is the same for all routes, hence create a logger builder and error handling functions on a helpers.go file.

package controllers import ( "errors" "fmt" "log/slog" "github.com/go-playground/validator/v10" "github.com/gofiber/fiber/v2" "github.com/google/uuid" "github.com/tugascript/devlogs/idp/internal/exceptions" "github.com/tugascript/devlogs/idp/internal/utils" ) func (c *Controllers) buildLogger( requestID, location, method string, ) *slog.Logger { return utils.BuildLogger(c.logger, utils.LoggerOptions{ Layer: utils.ControllersLogLayer, Location: location, Method: method, RequestID: requestID, }) } func logRequest(logger *slog.Logger, ctx *fiber.Ctx) { logger.InfoContext( ctx.UserContext(), fmt.Sprintf("Request: %s %s", ctx.Method(), ctx.Path()), ) } func getRequestID(ctx *fiber.Ctx) string { return ctx.Get("requestid", uuid.NewString()) } func logResponse(logger *slog.Logger, ctx *fiber.Ctx, status int) { logger.InfoContext( ctx.UserContext(), fmt.Sprintf("Response: %s %s", ctx.Method(), ctx.Path()), "status", status, ) } func validateErrorResponse(logger *slog.Logger, ctx *fiber.Ctx, location string, err error) error { logger.WarnContext(ctx.UserContext(), "Failed to validate request", "error", err) logResponse(logger, ctx, fiber.StatusBadRequest) var errs validator.ValidationErrors ok := errors.As(err, &errs) if !ok { return ctx. Status(fiber.StatusBadRequest). JSON(exceptions.NewEmptyValidationErrorResponse(location)) } return ctx. Status(fiber.StatusBadRequest). JSON(exceptions.ValidationErrorResponseFromErr(&errs, location)) } func validateBodyErrorResponse(logger *slog.Logger, ctx *fiber.Ctx, err error) error { return validateErrorResponse(logger, ctx, exceptions.ValidationResponseLocationBody, err) } func validateURLParamsErrorResponse(logger *slog.Logger, ctx *fiber.Ctx, err error) error { return validateErrorResponse(logger, ctx, exceptions.ValidationResponseLocationParams, err) } func validateQueryParamsErrorResponse(logger *slog.Logger, ctx *fiber.Ctx, err error) error { return validateErrorResponse(logger, ctx, exceptions.ValidationResponseLocationQuery, err) } func serviceErrorResponse(logger *slog.Logger, ctx *fiber.Ctx, serviceErr *exceptions.ServiceError) error { status := exceptions.NewRequestErrorStatus(serviceErr.Code) resErr := exceptions.NewErrorResponse(serviceErr) logResponse(logger, ctx, status) return ctx.Status(status).JSON(&resErr) } func oauthErrorResponse(logger *slog.Logger, ctx *fiber.Ctx, message string) error { resErr := exceptions.NewOAuthError(message) logResponse(logger, ctx, fiber.StatusBadRequest) return ctx.Status(fiber.StatusBadRequest).JSON(&resErr) } func parseRequestErrorResponse(logger *slog.Logger, ctx *fiber.Ctx, err error) error { logger.WarnContext(ctx.UserContext(), "Failed to parse request", "error", err) logResponse(logger, ctx, fiber.StatusBadRequest) return ctx. Status(fiber.StatusBadRequest). JSON(exceptions.NewEmptyValidationErrorResponse(exceptions.ValidationResponseLocationBody)) }
Enter fullscreen mode Exit fullscreen mode

Health

The health controller is pretty simple, since it will only consume the service HealtCheck method directly:

package controllers import "github.com/gofiber/fiber/v2" func (c *Controllers) HealthCheck(ctx *fiber.Ctx) error { requestID := getRequestID(ctx) logger := c.buildLogger(requestID, "health", "HealthCheck") logRequest(logger, ctx) if serviceErr := c.services.HealthCheck(ctx.UserContext(), requestID); serviceErr != nil { return serviceErrorResponse(logger, ctx, serviceErr) } return ctx.SendStatus(fiber.StatusOK) }
Enter fullscreen mode Exit fullscreen mode

Paths

On the controllers folder create a paths package with the health.go path:

package paths const Health string = "/health"
Enter fullscreen mode Exit fullscreen mode

Server

The server is where we will initialize the Fiber App, build Services, Controllers and load common middleware.

Routes

Start by creating a server/routes and add routes.go file with the Routes struct:

package routes import ( "github.com/tugascript/devlogs/idp/internal/controllers" ) type Routes struct { controllers *controllers.Controllers } func NewRoutes(ctrls *controllers.Controllers) *Routes { return &Routes{controllers: ctrls} }
Enter fullscreen mode Exit fullscreen mode

Now create a router method for the health endpoint on health.go:

package routes import ( "github.com/gofiber/fiber/v2" "github.com/tugascript/devlogs/idp/internal/controllers/paths" ) func (r *Routes) HealthRoutes(app *fiber.App) { app.Get(paths.Health, r.controllers.HealthCheck) }
Enter fullscreen mode Exit fullscreen mode

Plus also create a common.go with the route Group for API version 1:

package routes import "github.com/gofiber/fiber/v2" const V1Path string = "/v1" func v1PathRouter(app *fiber.App) fiber.Router { return app.Group(V1Path) }
Enter fullscreen mode Exit fullscreen mode

Server Instance

Server instance is where we hook up most of our logic, hence it will be a big method where you take the config as a parameter and initialize everything.

On the server.go file inside the server directory, and create the FiberServer instance:

package server import ( // ... // ... "github.com/tugascript/devlogs/idp/internal/server/routes" // ... ) type FiberServer struct { *fiber.App routes *routes.Routes }
Enter fullscreen mode Exit fullscreen mode

And load everything in the New function that takes the context.Context, the struter logger (*slog.Logger) and the configuration (config.Config):

package server import ( "context" "log/slog" "time" "github.com/gofiber/fiber/v2" "github.com/gofiber/fiber/v2/middleware/cors" "github.com/gofiber/fiber/v2/middleware/encryptcookie" "github.com/gofiber/fiber/v2/middleware/helmet" "github.com/gofiber/fiber/v2/middleware/limiter" "github.com/gofiber/fiber/v2/middleware/requestid" fiberRedis "github.com/gofiber/storage/redis/v3" "github.com/google/uuid" "github.com/jackc/pgx/v5/pgxpool" "github.com/tugascript/devlogs/idp/internal/config" "github.com/tugascript/devlogs/idp/internal/controllers" "github.com/tugascript/devlogs/idp/internal/providers/cache" "github.com/tugascript/devlogs/idp/internal/providers/database" "github.com/tugascript/devlogs/idp/internal/providers/encryption" "github.com/tugascript/devlogs/idp/internal/providers/mailer" "github.com/tugascript/devlogs/idp/internal/providers/oauth" "github.com/tugascript/devlogs/idp/internal/providers/tokens" "github.com/tugascript/devlogs/idp/internal/server/routes" "github.com/tugascript/devlogs/idp/internal/server/validations" "github.com/tugascript/devlogs/idp/internal/services" ) // ... func New( ctx context.Context, logger *slog.Logger, cfg config.Config, ) *FiberServer { logger.InfoContext(ctx, "Building redis storage...") cacheStorage := fiberRedis.New(fiberRedis.Config{ URL: cfg.RedisURL(), }) cc := cache.NewCache( logger, cacheStorage, ) logger.InfoContext(ctx, "Finished building redis storage") logger.InfoContext(ctx, "Building database connection pool...") dbConnPool, err := pgxpool.New(ctx, cfg.DatabaseURL()) if err != nil { logger.ErrorContext(ctx, "Failed to connect to database", "error", err) panic(err) } db := database.NewDatabase(dbConnPool) logger.InfoContext(ctx, "Finished building database connection pool") logger.InfoContext(ctx, "Building mailer...") mail := mailer.NewEmailPublisher( cc.Client(), cfg.EmailPubChannel(), cfg.FrontendDomain(), logger, ) logger.InfoContext(ctx, "Finished building mailer") logger.InfoContext(ctx, "Building JWT token keys...") tokensCfg := cfg.TokensConfig() jwts := tokens.NewTokens( tokensCfg.Access(), tokensCfg.AccountCredentials(), tokensCfg.Refresh(), tokensCfg.Confirm(), tokensCfg.Reset(), tokensCfg.OAuth(), tokensCfg.TwoFA(), cfg.FrontendDomain(), cfg.BackendDomain(), ) logger.InfoContext(ctx, "Finished building JWT tokens keys") logger.InfoContext(ctx, "Building encryption...") encryp := encryption.NewEncryption(logger, cfg.EncryptionConfig(), cfg.BackendDomain()) logger.InfoContext(ctx, "Finished encryption") logger.InfoContext(ctx, "Building OAuth provider...") oauthProvidersCfg := cfg.OAuthProvidersConfig() oauthProviders := oauth.NewProviders( logger, oauthProvidersCfg.GitHub(), oauthProvidersCfg.Google(), oauthProvidersCfg.Facebook(), oauthProvidersCfg.Apple(), oauthProvidersCfg.Microsoft(), ) logger.InfoContext(ctx, "Finished building OAuth provider") logger.InfoContext(ctx, "Building services...") newServices := services.NewServices( logger, db, cc, mail, jwts, encryp, oauthProviders, ) logger.InfoContext(ctx, "Finished building services") logger.InfoContext(ctx, "Loading validators...") vld := validations.NewValidator(logger) logger.InfoContext(ctx, "Finished loading validators") server := &FiberServer{ App: fiber.New(fiber.Config{ ServerHeader: "idp", AppName: "idp", }), routes: routes.NewRoutes(controllers.NewControllers( logger, newServices, vld, cfg.FrontendDomain(), cfg.BackendDomain(), cfg.CookieName(), )), } logger.InfoContext(ctx, "Loading middleware...") server.Use(helmet.New()) server.Use(requestid.New(requestid.Config{ Header: fiber.HeaderXRequestID, Generator: func() string { return uuid.NewString() }, })) rateLimitCfg := cfg.RateLimiterConfig() server.Use(limiter.New(limiter.Config{ Max: int(rateLimitCfg.Max()), Expiration: time.Duration(rateLimitCfg.ExpSec()) * time.Second, LimiterMiddleware: limiter.SlidingWindow{}, Storage: cacheStorage, })) server.Use(encryptcookie.New(encryptcookie.Config{ Key: cfg.CookieSecret(), })) server.App.Use(cors.New(cors.Config{ AllowOrigins: "*", AllowMethods: "GET,POST,PUT,DELETE,OPTIONS,PATCH,HEAD", AllowHeaders: "Accept,Authorization,Content-Type", AllowCredentials: false, // credentials require explicit origins MaxAge: 300, })) logger.Info("Finished loading common middlewares") return server }
Enter fullscreen mode Exit fullscreen mode

Registering the routes

Just create a routes.go file inside the server directory with a RegisterFiberRoutes method:

package server func (s *FiberServer) RegisterFiberRoutes() { s.routes.HealthRoutes(s.App) }
Enter fullscreen mode Exit fullscreen mode

Logging

For logging we will need to create a default logger initially then add the configuration when they are loaded on startup, on a logger.go file:

package server import ( "log/slog" "os" "github.com/tugascript/devlogs/idp/internal/config" ) func DefaultLogger() *slog.Logger { return slog.New(slog.NewJSONHandler( os.Stdout, &slog.HandlerOptions{ Level: slog.LevelInfo, }, )) } func ConfigLogger(cfg config.LoggerConfig) *slog.Logger { logLevel := slog.LevelInfo if cfg.IsDebug() { logLevel = slog.LevelDebug } logger := slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{ Level: logLevel, })) if cfg.Env() == "production" { logger = slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{ Level: logLevel, })) } return logger.With("service", cfg.ServiceName()) }
Enter fullscreen mode Exit fullscreen mode

Running the app

On the cmd/api folder's main.go file just set-up everything and register the routes:

package main import ( "context" "fmt" "log/slog" "os/signal" "runtime" "syscall" "time" "github.com/tugascript/devlogs/idp/internal/config" "github.com/tugascript/devlogs/idp/internal/server" ) func gracefulShutdown( logger *slog.Logger, fiberServer *server.FiberServer, done chan bool, ) { // Create context that listens for the interrupt signal from the OS. ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) defer stop() // Listen for the interrupt signal. <-ctx.Done() logger.InfoContext(ctx, "shutting down gracefully, press Ctrl+C again to force") // The context is used to inform the server it has 5 seconds to finish // the request it is currently handling ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel() if err := fiberServer.ShutdownWithContext(ctx); err != nil { logger.ErrorContext(ctx, "Server forced to shutdown with error", "error", err) } logger.InfoContext(ctx, "Server exiting") // Notify the main goroutine that the shutdown is complete done <- true } func main() { logger := server.DefaultLogger() ctx := context.Background() logger.InfoContext(ctx, "Loading configuration...") cfg := config.NewConfig(logger, "./.env") logger = server.ConfigLogger(cfg.LoggerConfig()) logger.InfoContext(ctx, "Setting GOMAXPROCS...", "maxProcs", cfg.MaxProcs()) runtime.GOMAXPROCS(int(cfg.MaxProcs())) logger.InfoContext(ctx, "Finished setting GOMAXPROCS") logger.InfoContext(ctx, "Building server...") server := server.New(ctx, logger, cfg) logger.InfoContext(ctx, "Server built") server.RegisterFiberRoutes() // Create a done channel to signal when the shutdown is complete done := make(chan bool, 1) go func() { err := server.Listen(fmt.Sprintf(":%d", cfg.Port())) if err != nil { logger.ErrorContext(ctx, "http server error", "error", err) panic(fmt.Sprintf("http server error: %s", err)) } }() // Run graceful shutdown in a separate goroutine go gracefulShutdown(logger, server, done) // Wait for the graceful shutdown to complete <-done logger.InfoContext(ctx, "Graceful shutdown complete.") }
Enter fullscreen mode Exit fullscreen mode

Conclusion

In this first article, we introduced the fiber framework, a express inspired go library, as well as how to set up our API with a MSC (model, service, controller) architecture with a centralized configuration.

This code is based on the current ongoing project found on the devlogs repository.

About the Author

Hey there! I am Afonso Barracha, your go-to econometrician who found his way into the world of back-end development with a soft spot for GraphQL. If you enjoyed reading this article, why not show some love by buying me a coffee?

Lately, I have been diving deep into more advanced subjects. As a result, I have switched from sharing my thoughts every week to posting once or twice a month. This way, I can make sure to bring you the highest quality content possible.

Do not miss out on any of my latest articles – follow me here on dev, LinkedIn or Instagram to stay updated. I would be thrilled to welcome you to our ever-growing community! See you around!

Top comments (1)

Collapse
 
ezep02 profile image
Ezep02 •

This post is insane, Thanks!!