feat!: Add CRD maintainer

[Techassi]

Part of stackabletech/secret-operator#634 and https://github.com/stackabletech/decisions/issues/62.

This PR adds a new CRD maintainer which reconciles the CRDs when the conversion webhook server certificate is rotated. It additionally provides a oneshot channel to indicate that the initial CRD rollout is done and default custom resources can be applied.

This is a crude first implementation of the CRD and default CR maintenance mechanism. This PR will be used as a test-balloon in stackabletech/secret-operator#634.

In the future, this mechanism should be implemented in a more generic way, similar to kube::Controller.

[sbernauer]

Is there any example code in secret-op I can look at?
At a first glance it looks like stackabletech/secret-operator#634 is not using it yet

[sbernauer]

@Techassi pushed his code, so we have a working example in secret-op.

What I don't like is that the webhook and the maintainer are not integrated into each other and we basically need to configure everything twice, which is annoying and error-prone:

pub async fn conversion_webhook( operator_environment: &OperatorEnvironmentOptions, ) -> anyhow::Result<(ConversionWebhookServer, mpsc::Receiver<Certificate>)> { let crds_and_handlers = [ ( SecretClass::merged_crd(SecretClassVersion::V1Alpha2)?, SecretClass::try_convert as fn(_) -> _, ), ( TrustStore::merged_crd(TrustStoreVersion::V1Alpha1)?, TrustStore::try_convert as fn(_) -> _, ), ]; let options = ConversionWebhookOptions { socket_addr: ConversionWebhookServer::DEFAULT_SOCKET_ADDRESS, namespace: operator_environment.operator_namespace.clone(), service_name: operator_environment.operator_service_name.clone(), }; Ok(ConversionWebhookServer::new(crds_and_handlers, options).await?) }

and further down

 let (maintainer, initial_reconcile_rx) = CustomResourceDefinitionMaintainer::new( client.as_kube_client(), certificate_rx, [ SecretClass::merged_crd(SecretClassVersion::V1Alpha2).unwrap(), TrustStore::merged_crd(TrustStoreVersion::V1Alpha1).unwrap(), ], CustomResourceDefinitionMaintainerOptions { operator_service_name: operator_environment.operator_service_name, operator_namespace: operator_environment.operator_namespace, field_manager: OPERATOR_NAME.to_owned(), webhook_https_port: WebhookServer::DEFAULT_HTTPS_PORT, disabled: maintenance.disable_crd_maintenance, }, );

We duplicate:

• The listen port
• operator namespace
• operator service name
• The list of CRDS. What happens when I add a CRD to one but not hte other?
• What stored version the CRDs should use. What happens if we use different onces?
• We need to know that there is a channel of CA certs in between and need to wire it up

I'd say all of this can be gathered from the webhook.
I'm thinking of a API similar to this (one code place instead of two - keep main.rs clean!)

pub async fn conversion_webhook_and_crd_maintainer( client: Client, operator_environment: &OperatorEnvironmentOptions, maintenance: &MaintenanceOptions, ) -> anyhow::Result<( ConversionWebhookServer, CustomResourceDefinitionMaintainer, oneshot::Receiver<()>, )> { let crds_and_handlers = [ ( SecretClass::merged_crd(SecretClassVersion::V1Alpha2)?, SecretClass::try_convert as fn(_) -> _, ), ( TrustStore::merged_crd(TrustStoreVersion::V1Alpha1)?, TrustStore::try_convert as fn(_) -> _, ), ]; let options = ConversionWebhookOptions { listen_addr: ConversionWebhookServer::DEFAULT_LISTEN_ADDRESS, listen_port: ConversionWebhookServer::DEFAULT_HTTPS_PORT, namespace: operator_environment.operator_namespace.clone(), service_name: operator_environment.operator_service_name.clone(), }; let conversion_webhook = ConversionWebhookServer::new(crds_and_handlers, options).await?; let (crd_maintainer, initial_reconcile_rx) = CustomResourceDefinitionMaintainer::for_conversion_webhook( &conversion_webhook, client, OPERATOR_NAME.to_owned(), maintenance.disable_crd_maintenance, ); Ok((conversion_webhook, crd_maintainer, initial_reconcile_rx)) }

We only specify everything once.

[Techassi]

I had similar ideas to what is mentioned in #1099 (review). Commit 99eae00 implements a simplified API to construct a webhook server in combination with a CRD maintainer.

This follow-up commit uses the improved code: stackabletech/secret-operator@788dd08 (#634)

[sbernauer]

I think that was it. Calling code in stackabletech/secret-operator#634 looks very nice and clean

[NickLarsenNZ]

Approving on behalf of @sbernauer (given that the review comments are adequately resolved)

All requested changes have been implemented.