Enable templating for secret operator #55

nightkr · 2022-01-26T12:59:27Z

Enable templating for secret-operator
Fix -operator-operator in templates
Fix -operator-operator in folder names
Deploy as DaemonSet rather than Deployment
Set CSI_ENDPOINT env variable
CRD harmonization

This ensures that the templates also work for non-product operators.

Fixes #25 For now I applied the templating manually (from stackabletech/operator-templating#55, commit stackabletech/operator-templating@4fd238d) and adapted it for secret-operator by hand. These changes are contained in 878c4a6, and basically mean: - Deploying as a `DaemonSet` rather than a `Deployment` - Adding CSI sidecar containers - Running as root (at some point we might be able to get away with "just" adding a bunch of capabilities, but secret-operator will probably always require relatively elevated privileges) - Disabling product-config - Extracting the CRD by running `cargo run crd` rather than reading the file that `build.rs` creates (Nix doesn't allow access to files outside of the source tree) - Adapting to the `ClusterRole` rules that secret-operator requires Currently this branch contains both the `Dockerfile`- and Nix-based workflows for building the operator, but we should probably pick one at some point. Co-authored-by: Stacky McStackface <stackable-bot@users.noreply.github.com>

nightkr · 2022-03-14T09:43:21Z

The remaining question here is harmonizing CRD YAML generation. The Makefile currently assumes that the operator's build.rs script will write the CRD YAML, which secret-op currently doesn't because Nix doesn't let it write outside of its build sandbox.

We could make this conditional ("only write the CRD when built outside of Nix"), but I'm disinclined to let them diverge too much, and this kind of code mutation is discouraged upstream (rule 1 of https://doc.rust-lang.org/cargo/reference/build-scripts.html#outputs-of-the-build-script is "Scripts should not modify any files outside of that [$OUT_DIR] directory.").

An alternative used by secret-op is to use the output of cargo run crd. This is already supported by all operators, but is somewhat slower (since it causes the check-manifests task to also do a cargo build). Most of this could be mitigated with caching, however.

nightkr · 2022-03-14T09:58:58Z

After (so-far) unanimous agreement in Slack (https://stackable-workspace.slack.com/archives/C02FZ581UCD/p1647251421170359) I'm burning deploy/crd entirely.

…ackabletech/operator-templating repo. Triggered by: Manual run triggered by: teozkr with message [Enabled secret operator templating (stackabletech/operator-templating#55)]

sbernauer · 2022-03-17T07:05:56Z

FYI @teozkr this only works with non-snap version of yq. Snap version is silly and throws (file exists though)

Only FYI, fine for me ;)

make regenerate-charts rm -rf deploy/helm/superset-operator/configs rm -rf deploy/helm/superset-operator/crds mkdir -p deploy/manifests rm -rf $(find deploy/manifests -maxdepth 1 -mindepth 1 -not -name Kustomization) yq eval -i '.version = "0.4.0-nightly" | .appVersion = "0.4.0-nightly"' deploy/helm/superset-operator/Chart.yaml Error: stat deploy/helm/superset-operator/Chart.yaml: no such file or directory make: *** [Makefile:39: version] Error 1

sbernauer · 2022-03-17T07:07:04Z

See https://github.com/mikefarah/yq/#snap-notes

Automatically created PR based on commit 9d3387ff20d362a8c78301c3993da29edd0e3f4b in stackabletech/operator-templating repo. Triggered by: Manual run triggered by: teozkr with message [Enabled secret operator templating (stackabletech/operator-templating#55)]

Fixes #25 For now I applied the templating manually (from stackabletech/operator-templating#55, commit stackabletech/operator-templating@4fd238d) and adapted it for secret-operator by hand. These changes are contained in 878c4a6176782450c068669638c92700ebcb65d7, and basically mean: - Deploying as a `DaemonSet` rather than a `Deployment` - Adding CSI sidecar containers - Running as root (at some point we might be able to get away with "just" adding a bunch of capabilities, but secret-operator will probably always require relatively elevated privileges) - Disabling product-config - Extracting the CRD by running `cargo run crd` rather than reading the file that `build.rs` creates (Nix doesn't allow access to files outside of the source tree) - Adapting to the `ClusterRole` rules that secret-operator requires Currently this branch contains both the `Dockerfile`- and Nix-based workflows for building the operator, but we should probably pick one at some point. Co-authored-by: Stacky McStackface <stackable-bot@users.noreply.github.com>

nightkr added 3 commits January 26, 2022 13:48

Use operator.name rather than {operator.product}-operator

0cedc44

This ensures that the templates also work for non-product operators.

Enable for secret-operator

6407ef6

Use operator.name over operator.product_name for helm chart folder

4fd238d

nightkr mentioned this pull request Jan 26, 2022

[Merged by Bors] - Helm chart stackabletech/secret-operator#27

Closed

nightkr added 2 commits January 27, 2022 10:59

Fix another case of -operator-operator in the manifest regen script

77ccf3a

Fix -operator-operator in Helm upload scripts

19ea5b6

lfrancke assigned nightkr Feb 22, 2022

nightkr added 3 commits March 8, 2022 14:59

Merge branch 'main' into feature/secret-operator

4f3dc7e

Merge remote-tracking branch 'origin/main' into feature/secret-operator

9d318ea

Allow operators to opt out of the automatic Deployment

6d178e9

nightkr added 3 commits March 14, 2022 10:45

Make linters happier (where relevant)

b35ec87

Generate CRD YAML as needed, rather than reading deploy/crd

e381ece

Burn all references to deploy/crd

8ba60d4

nightkr requested a review from a team March 14, 2022 09:59

stackable-bot mentioned this pull request Mar 16, 2022

[Merged by Bors] - Update templated files to rev 9d3387f stackabletech/trino-operator#169

Closed

stackable-bot mentioned this pull request Mar 16, 2022

[Merged by Bors] - Update templated files to rev 9d3387f stackabletech/spark-operator#311

Closed

stackable-bot mentioned this pull request Mar 16, 2022

[Merged by Bors] - Update templated files to rev 9d3387f stackabletech/nifi-operator#237

Closed

stackable-bot mentioned this pull request Mar 16, 2022

[Merged by Bors] - Update templated files to rev 9d3387f stackabletech/hbase-operator#144

Closed

stackable-bot mentioned this pull request Mar 16, 2022

[Merged by Bors] - Update templated files to rev 9d3387f stackabletech/hdfs-operator#142

Closed

stackable-bot mentioned this pull request Mar 16, 2022

[Merged by Bors] - Update templated files to rev 9d3387f stackabletech/superset-operator#155

Closed

stackable-bot mentioned this pull request Mar 16, 2022

[Merged by Bors] - Update templated files to rev 9d3387f stackabletech/airflow-operator#65

Closed

stackable-bot mentioned this pull request Mar 16, 2022

[Merged by Bors] - [Merged by Bors] - Update templated files to rev 9d3387f stackabletech/druid-operator#201

Closed

stackable-bot mentioned this pull request Mar 16, 2022

Update templated files to rev 9d3387f stackabletech/secret-operator#79

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Enable templating for secret operator #55

Enable templating for secret operator #55

Uh oh!

nightkr commented Jan 26, 2022 •

edited

Loading

nightkr commented Mar 14, 2022

nightkr commented Mar 14, 2022

sbernauer commented Mar 17, 2022

sbernauer commented Mar 17, 2022

Labels

4 participants

Uh oh!

Enable templating for secret operator #55

Enable templating for secret operator #55

Uh oh!

Conversation

nightkr commented Jan 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

nightkr commented Mar 14, 2022

nightkr commented Mar 14, 2022

sbernauer commented Mar 17, 2022

sbernauer commented Mar 17, 2022

Labels

4 participants

nightkr commented Jan 26, 2022 •

edited

Loading