stackablectl operator

The stackable operator command allows the listing, installation and removal of Stackable operators. Operators manage the individual data products of the Stackable Data Platform.

This command manages individual operators. It is intended for people with experience working on the Stackable Data Platform. If you want an easy way to get started or don’t know which products and versions to install, using the stackablectl release command is recommended. This command will install a bundle of operators from an official Stackable release.

General Usage

Interact with single operator instead of the full platform Usage: stackablectl operator [OPTIONS] <COMMAND> Commands: list List available operators describe Print out detailed operator information install Install one or more operators uninstall Uninstall one or more operators installed List installed operators help Print this message or the help of the given subcommand(s) Options: -l, --log-level <LOG_LEVEL> Log level this application uses --no-cache Do not cache the remote (default) demo, stack and release files Cached files are saved at '$XDG_CACHE_HOME/stackablectl', which is usually '$HOME/.cache/stackablectl' when not explicitly set. --offline Do not request any remote files via the network -h, --help Print help (see a summary with '-h') -V, --version Print version File options: -d, --demo-file <DEMO_FILE> Provide one or more additional (custom) demo file(s) Demos are loaded in the following order: Remote (default) demo file, custom demo files provided via the 'STACKABLE_DEMO_FILES' environment variable, and lastly demo files provided via the '-d/--demo-file' argument(s). If there are demos with the same name, the last demo definition will be used. Use "stackablectl [OPTIONS] <COMMAND> -d path/to/demos1.yaml -d path/to/demos2.yaml" to provide multiple additional demo files. -s, --stack-file <STACK_FILE> Provide one or more additional (custom) stack file(s) Stacks are loaded in the following order: Remote (default) stack file, custom stack files provided via the 'STACKABLE_STACK_FILES' environment variable, and lastly demo files provided via the '-s/--stack-file' argument(s). If there are stacks with the same name, the last stack definition will be used. Use "stackablectl [OPTIONS] <COMMAND> -s path/to/stacks1.yaml -s path/to/stacks2.yaml" to provide multiple additional stack files. -r, --release-file <RELEASE_FILE> Provide one or more additional (custom) release file(s) Releases are loaded in the following order: Remote (default) release file, custom release files provided via the 'STACKABLE_RELEASE_FILES' environment variable, and lastly release files provided via the '-r/--release-file' argument(s). If there are releases with the same name, the last release definition will be used. Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml" to provide multiple additional release files. Helm repository options: --helm-repo-stable <URL> Provide a custom Helm stable repository URL [default: https://repo.stackable.tech/repository/helm-stable/] --helm-repo-test <URL> Provide a custom Helm test repository URL [default: https://repo.stackable.tech/repository/helm-test/] --helm-repo-dev <URL> Provide a custom Helm dev repository URL [default: https://repo.stackable.tech/repository/helm-dev/]

Listing Available Operators

To list the operators that are part of the Stackable Data Platform and their stable versions, run the following command:

$ stackablectl operator list ┌────┬───────────┬────────────────────────────────────────────────────────────────────────────────────────┐ │ # ┆ OPERATOR ┆ STABLE VERSIONS │ ╞════╪═══════════╪════════════════════════════════════════════════════════════════════════════════════════╡ │ 1 ┆ airflow ┆ 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 23.1.0, 23.4.0, 23.4.1, 23.7.0 │ ├╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ │ 2 ┆ ... ┆ ... │ ├╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ │ 15 ┆ zookeeper ┆ 0.10.0, 0.11.0, 0.12.0, 0.6.0, 0.7.0, 0.8.0, 0.9.0, 23.1.0, 23.4.0, 23.4.1, 23.7.0 │ └────┴───────────┴────────────────────────────────────────────────────────────────────────────────────────┘

This command only includes the stable versions of every operator for clarity. If you’re interested in a particular version of an operator, you can use the describe command to get more details for a specific operator as follows:

$ stackablectl operator describe airflow OPERATOR airflow STABLE VERSIONS 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 23.1.0, 23.4.0, 23.4.1, 23.7.0 TEST VERSIONS 0.0.0-pr303, 0.0.0-pr304, 0.0.0-pr305, 0.0.0-pr307, 0.0.0-pr308, 0.0.0-pr310, 0.0.0-pr311, 0.0.0-pr312, 0.0.0-pr314, 0.0.0-pr315, 0.0.0-pr316, 0.0.0-pr317 DEV VERSIONS 0.0.0-dev

Installing Operators

If you want to access a Kubernetes cluster, make sure your kubectl Kubernetes client is configured to interact with the Kubernetes cluster. After that, run the following command, which will install the operators in their latest nightly version - built from the main branch of the operators.

$ stackablectl operator install airflow commons secret listener Installing 4 operators Installed airflow operator Installed commons operator Installed secret operator Installed listener operator Installed 4 operators

If you don’t have a Kubernetes cluster available, stackablectl can spin up a kind or minikube Kubernetes cluster for you. Based on the type of local cluster you want to use, ensure you have either kind or minikube installed on your system. See here for more information.

With this command, we installed the operator for Apache Airflow and three operators needed internally by the Stackable Data Platform (commons, secret and listener). As we didn’t specify a specific version to install, the operators were installed in the latest nightly version - built from the main branch of the operators. If you want to install a specific version, you can add the version to each operator to install as follows:

$ stackablectl operator install airflow=23.7 commons=23.7 secret=23.7 listener=23.7 Installing 4 operators Installed airflow=23.7 operator Installed commons=23.7 operator Installed secret=23.7 operator Installed listener=23.7 operator Installed 4 operators

As you can see, the four operators were installed in the requested version.

Remember: If you want to install a recommended and tested set of operator versions, look at the stackablectl release command.

Listing Installed Operators

After installing some operators, you can list which operators are installed in your Kubernetes cluster:

$ stackablectl operator installed ┌────────────────────┬─────────┬─────────────────────┬──────────┬──────────────────────────────────────────┐ │ OPERATOR ┆ VERSION ┆ NAMESPACE ┆ STATUS ┆ LAST UPDATED │ ╞════════════════════╪═════════╪═════════════════════╪══════════╪══════════════════════════════════════════╡ │ airflow-operator ┆ 23.7.0 ┆ stackable-operators ┆ deployed ┆ 2023-08-23 17:33:01.509777626 +0200 CEST │ ├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ │ commons-operator ┆ 23.7.0 ┆ stackable-operators ┆ deployed ┆ 2023-08-23 17:33:04.012698515 +0200 CEST │ ├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ │ listener-operator ┆ 23.7.0 ┆ stackable-operators ┆ deployed ┆ 2023-08-23 17:33:07.217309791 +0200 CEST │ ├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ │ secret-operator ┆ 23.7.0 ┆ stackable-operators ┆ deployed ┆ 2023-08-23 17:33:06.328410802 +0200 CEST │ └────────────────────┴─────────┴─────────────────────┴──────────┴──────────────────────────────────────────┘

Uninstalling Operators

You can use the stackablectl operator uninstall command to uninstall the operators again.

$ stackablectl operator uninstall airflow commons secret listener The release airflow-operator was successfully uninstalled. The release commons-operator was successfully uninstalled. The release secret-operator was successfully uninstalled. The release listener-operator was successfully uninstalled. Uninstalled 4 operators