Merge branch 'service-discovery-refactoring' of github.com:stackabletech/documentation into service-discovery-refactoring

Author: Felix Hennig

antora-playbook.yml

@@ -14,6 +14,11 @@ content:
  - url: https://github.com/stackabletech/documentation.git
  branches: main
 
+ - url: https://github.com/stackabletech/stackablectl.git
+ branches: main
+ tags: docs/*
+ start_path: docs
+
  - url: https://github.com/stackabletech/commons-operator.git
  branches: main
  tags: docs/*
@@ -55,10 +60,6 @@ content:
  branches: main
  tags: docs/*
  start_path: docs
- - url: https://github.com/stackabletech/spark-operator.git
- branches: main
- tags: docs/*
- start_path: docs
  - url: https://github.com/stackabletech/spark-k8s-operator.git
  branches: main
  tags: docs/*

gitpod-antora-playbook.yml

@@ -4,56 +4,77 @@ site:
  url: https://docs.stackable.tech
  robots: allow
 
+urls:
+ # This replaces the component version in the URL of the latest stable version with 'stable'
+ # i.e. /commons-operator/stable/index.html instead of /commons-operator/0.3/index.html
+ latest_version_segment: stable
+
 content:
  sources: 
  - url: /workspace/documentation
  branches: HEAD
 
+ - url: https://github.com/stackabletech/stackablectl.git
+ branches: main
+ tags: docs/*
+ start_path: docs
+
  - url: https://github.com/stackabletech/commons-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/secret-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
 
  - url: https://github.com/stackabletech/airflow-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/druid-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/hbase-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/hdfs-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/hive-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/kafka-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/nifi-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/opa-operator.git
  branches: main
- start_path: docs
- - url: https://github.com/stackabletech/spark-operator.git
- branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/spark-k8s-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/superset-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/trino-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
  - url: https://github.com/stackabletech/zookeeper-operator.git
  branches: main
+ tags: docs/*
  start_path: docs
 
 ui:
@@ -64,6 +85,7 @@ ui:
 antora:
  extensions:
  - '@antora/lunr-extension'
+ - asciidoctor-kroki
 
 asciidoc:
  attributes:

local-antora-playbook.yml

@@ -15,6 +15,11 @@ content:
  - url: ./
  branches: HEAD
 
+ - url: https://github.com/stackabletech/stackablectl.git
+ branches: main
+ tags: docs/*
+ start_path: docs
+
  - url: https://github.com/stackabletech/commons-operator.git
  branches: main
  tags: docs/*
@@ -56,10 +61,6 @@ content:
  branches: main
  tags: docs/*
  start_path: docs
- - url: https://github.com/stackabletech/spark-operator.git
- branches: main
- tags: docs/*
- start_path: docs
  - url: https://github.com/stackabletech/spark-k8s-operator.git
  branches: main
  tags: docs/*

modules/ROOT/pages/getting_started.adoc

@@ -74,30 +74,32 @@ CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/servi
 Metrics-server is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy
 ----
 
-=== Installing Helm
-Stackable uses https://helm.sh/[Helm] as the package manager for its Kubernetes operators. This greatly simplifies the deployment and management of Kubernetes operators and CRDs. The quickest way to install Helm is to run the following command:
+== Installing Stackable
+=== Install stackablectl
+
+Install the Stackable command line utility xref:stackablectl::index.adoc[stackablectl] by following the installation steps for your platform on the xref:stackablectl::installation.adoc[installation] page.
+
+=== Installing Stackable Operators
+The Stackable operators are components that translate the service definitions deployed via Kubernetes into deploy services on the worker nodes. These can be installed on any node that has access to the Kubernetes control plane. In this example we will install them on the controller node.
+
+Stackable operators are installed using stackablectl. Run the following commands to install the latest operator versions of ZooKeeper, Kafka and NiFi.
 
 [source,bash]
 ----
-curl -sfL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | /bin/bash -
+stackablectl operator install zookeeper secret nifi kafka
 ----
 
-
-== Installing Stackable
-=== Install Stackable Helm Repositories
-With Helm installed you can add the Stackable operator repo, where the helm charts to install Stackable operators can be found. There are development, test and a stable repositories available, we'll be using the stable repo in this guide.
+.Using Helm instead
+[%collapsible]
+====
+Add the stackable-stable Helm Chart repository:
 
 [source,bash]
 ----
 helm repo add stackable-stable https://repo.stackable.tech/repository/helm-stable/
 ----
 
-=== Installing Stackable Operators
-The Stackable operators are components that translate the service definitions deployed via Kubernetes into deploy services on the worker nodes. These can be installed on any node that has access to the Kubernetes control plane. In this example we will install them on the controller node.
-
-Stackable operators are installed using Helm charts. Run the following commands to install the operators for ZooKeeper, Kafka and NiFi using the repo configured earlier. The --devel flag will choose the latest available version; alternatively the --version flag can be used to deploy a specific version.
-
-==== Install operators from the Stackable repository
+Install the operators:
 
 [source,bash]
 ----
@@ -106,15 +108,16 @@ helm install kafka-operator stackable-stable/kafka-operator --version=0.5.0
 helm install secret-operator stackable-stable/secret-operator --version=0.2.0
 helm install nifi-operator stackable-stable/nifi-operator --version=0.5.0
 ----
+====
 
-You can check which operators are installed using `helm list`:
+You can check which operators are installed using `stackablectl operator installed`:
 
 ----
-NAME NAMESPACE	REVISION	UPDATED  STATUS CHART 	APP VERSION
-kafka-operator default	1 2022-02-15 08:17:26.84659409 +0000 UTC	deployed	kafka-operator-0.5.0 	0.5.0
-nifi-operator default	1 2022-02-15 08:17:37.93720808 +0000 UTC	deployed	nifi-operator-0.5.0 	0.5.0
-secret-operator default 1  2022-02-15 08:17:32.700301793 +0000 UTC	deployed	secret-operator-0.2.0	0.2.0
-zookeeper-operatordefault 1  2022-02-15 08:17:17.893844595 +0000 UTC	deployed	zookeeper-operator-0.9.0	0.9.0
+OPERATOR VERSION NAMESPACE STATUS LAST UPDATED
+kafka    0.6.0-nightly default deployed 2022-06-16 13:51:05.661282811 +0200 CEST
+nifi    0.6.0-nightly default deployed 2022-06-20 16:22:45.985504689 +0200 CEST
+secret 0.5.0-nightly default deployed 2022-06-20 16:07:40.248027167 +0200 CEST
+zookeeper 0.10.0-nightly default deployed 2022-06-16 13:50:51.497813301 +0200 CEST
 ----
 
 == Deploying Stackable Services

modules/ROOT/pages/index.adoc

@@ -112,7 +112,7 @@ xref:druid::index.adoc[Read more]
 <h3>Apache HBase</h3>
 ++++
 
-HBase is a distributed, scalable, big data store
+HBase is a distributed, scalable, big data store.
 
 xref:hbase::index.adoc[Read more]
 
@@ -194,11 +194,7 @@ xref:nifi::index.adoc[Read more]
 
 Apache Spark is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters.
 
-Stackable offers two different operators for managing Apache Spark clusters. One for multi-tenancy or standalone clusters and one for isolated workloads.
-
-xref:spark::index.adoc[Read more] for more information on how to run a multi-tenancy Spark clusters with Stackable.
-
-For more information on how to run isolated Spark workloads with Stackable operators xref:spark-k8s::index.adoc[see here].
+xref:spark-k8s::index.adoc[Read more]
 
 ++++
 </div>
@@ -212,7 +208,7 @@ For more information on how to run isolated Spark workloads with Stackable opera
 <h3>Apache Superset</h3>
 ++++
 
-Apache Superset is a modern data exploration and visualization platform
+Apache Superset is a modern data exploration and visualization platform.
 
 xref:superset::index.adoc[Read more]
 

modules/concepts/pages/s3.adoc

@@ -51,7 +51,7 @@ include::partial$diagrams/S3ResourceOverview.excalidraw[]
 The diagram above shows three examples of how the objects can be
  structured.
 // Option 1
-In option 1 all objects are separate from each other. This provides maximum reusability because the same connection or bucket object can be referenced by multiple resources. It also allows for separation of concerts across team members. Cluster administrators can define S3 connection objects that developers reference in their applications.
+In option 1 all objects are separate from each other. This provides maximum re-usability because the same connection or bucket object can be referenced by multiple resources. It also allows for separation of concerns across team members. Cluster administrators can define S3 connection objects that developers reference in their applications.
 // Option 2
 In option 2 the bucket is inlined in the cluster definition. This makes sense if you have a dedicated bucket for a specific purpose, if it is only used in this one cluster instance, in this single product.
 // Option 3
@@ -83,7 +83,7 @@ The inline definition is variant 3 in the figure above.
 include::partial$diagrams/S3ResourcesInlined.excalidraw[]
 ----
 
-This variant as the advantage that everything is defined in a single file, right where it is going to be used:
+This variant has the advantage that everything is defined in a single file, right where it is going to be used:
 
 [source,yaml]
 ----
@@ -179,7 +179,7 @@ metadata:
  secrets.stackable.tech/class: s3-credentials-class # <1>
 stringData:
  accessKey: YOUR_VALID_ACCESS_KEY_ID_HERE
- secretKey: YOUR_SECRET_ACCES_KEY_THATBELONGS_TO_THE_KEY_ID_HERE
+ secretKey: YOUR_SECRET_ACCESS_KEY_THAT_BELONGS_TO_THE_KEY_ID_HERE
 ----
 
 <1> This label connects the `Secret` to the `SecretClass`.

modules/contributor/pages/development_dashboard.adoc

@@ -68,15 +68,6 @@ This page contains information about our products and relevant links you might n
 |https://docs.stackable.tech/nifi/index.html
 |
 
-|Apache Spark (standalone)
-|https://spark.apache.org/
-|https://github.com/apache/spark
-|https://github.com/stackabletech/spark-operator
-|https://github.com/stackabletech/docker-images/tree/main/spark
-|https://ci.stackable.tech/job/Spark%20Operator%20Integration%20Tests/
-|https://docs.stackable.tech/spark/index.html
-|
-
 |Apache Spark on Kubernetes
 |https://spark.apache.org/
 |https://github.com/apache/spark

modules/operators/pages/supported_versions.adoc

@@ -34,10 +34,6 @@ include::nifi::partial$supported-versions.adoc[]
 
 include::opa::partial$supported-versions.adoc[]
 
-== Apache Spark (standalone)
-
-include::spark::partial$supported-versions.adoc[]
-
 == Apache Spark on Kubernetes
 
 include::spark-k8s::partial$supported-versions.adoc[]

modules/operators/partials/operator_doc_links.adoc

@@ -5,7 +5,6 @@
 ** xref:hive::index.adoc[Apache Hive]
 ** xref:kafka::index.adoc[Apache Kafka]
 ** xref:nifi::index.adoc[Apache NiFi]
-** xref:spark::index.adoc[Apache Spark (standalone)]
 ** xref:spark-k8s::index.adoc[Apache Spark on K8S]
 ** xref:superset::index.adoc[Apache Superset]
 ** xref:trino::index.adoc[Trino]

modules/reference/pages/s3.adoc

@@ -1,6 +1,6 @@
 = S3 resources
 
-This page contains the reference information for th S3Bucket and S3Connection resources. For guidance on usage, see xref:concepts:s3.adoc[S3 resources concept].
+This page contains the reference information for the S3Bucket and S3Connection resources. For guidance on usage, see xref:concepts:s3.adoc[S3 resources concept].
 
 == S3Bucket
 
@@ -9,9 +9,9 @@ A bucket consists of a bucket name and the connection to the object store where
 * `name`: `String`, the name of the Bucket.
 * `connection`: can either be `inline` or `reference`.
 ** `inline`: See the properties below for <<S3Connection>>.
-** `reference`: `String`, the name of the referenced S3Connection resource. Must be in the same namespace as the S3Bucket resource.
+** `reference`: `String`, the name of the referenced S3Connection resource, which must be in the same namespace as the S3Bucket resource.
 
-=== Full Examle
+=== Full Example
 
 Using an inline connection: