Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions modules/ROOT/pages/release-notes.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ The certificate lifetime is also jittered to avoid all pods restarting around th
This is also the first release we're publishing SBOMs (Software Bill of Materials) in the https://cyclonedx.org/[CycloneDX] format for.
We publish these for both our operators and our product images.
The work on this was https://stackable.tech/en/empowering-rust-projects/[partially funded] by the https://www.sovereigntechfund.de/[Sovereign Tech Fund].
These SBOMS are published to our OCI registry as signed in-toto attestations and we have written a https://docs.stackable.tech/home/24.3/tutorials/viewing-and-verifying-sboms.html[tutorial] on how you can use it.
These SBOMS are published to our OCI registry as signed in-toto attestations and we have written a xref:guides:viewing-and-verifying-sboms.adoc[guide] on how you can use it.
To make it easier to use we have written a https://sboms.stackable.tech[SBOM browser] which allows you to download the raw CycloneDX JSON files as well.

[IMPORTANT]
Expand Down Expand Up @@ -543,7 +543,7 @@ documentation] for more details.

Signed SDP product images::
As of this release all Stackable product images are signed (the signing of operator images was delivered in SDP 23.7). Please see this
xref:tutorials:enabling-verification-of-image-signatures.adoc[tutorial] for more information.
xref:guides:enabling-verification-of-image-signatures.adoc[tutorial] for more information.

Airflow KubernetesExecutor::
Airflow clusters can now be configured to use Kubernetes executors, whereby pods are spun up for job tasks and terminated when complete, thus offering an alternative way to use resources without the need for job queuing.
Expand Down Expand Up @@ -1352,7 +1352,7 @@ Signed SDP operator images::

As of this release all Stackable operator images are signed (this feature will be added to product images in a
subsequent release). More information about this, including how to verify the image signatures, can be found in this
xref:tutorials:enabling-verification-of-image-signatures.adoc[tutorial].
xref:guides:enabling-verification-of-image-signatures.adoc[guide].

New Versions::

Expand Down
1 change: 0 additions & 1 deletion modules/concepts/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
** Resources
*** xref:resources.adoc[]
*** xref:s3.adoc[]
*** xref:pvc.adoc[]
** Connectivity
*** xref:service_discovery.adoc[]
*** xref:service-exposition.adoc[]
Expand Down
8 changes: 6 additions & 2 deletions modules/guides/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,2 +1,6 @@
* Guides
** xref:custom-images.adoc[]
* xref:index.adoc[]
** xref:custom-images.adoc[]
** xref:providing-resources-with-pvcs.adoc[]
** xref:running-stackable-in-an-airgapped-environment.adoc[]
** xref:viewing-and-verifying-sboms.adoc[]
** xref:enabling-verification-of-image-signatures.adoc[]
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Enabling verification of image signatures
:page-aliases: tutorials:enabling-verification-of-image-signatures.adoc

Image signing is a security measure that helps ensure the authenticity and integrity of container images. Starting with SDP 23.11, all our images are signed https://docs.sigstore.dev/cosign/openid_signing/["keyless"]. By verifying these signatures, cluster administrators can ensure that the images pulled from Stackable's container registry are authentic and have not been tampered with.
Since Kubernetes does not have native support for verifying image signatures yet, we will use Sigstore's https://docs.sigstore.dev/policy-controller/overview/[Policy Controller] in this tutorial.
Expand Down
3 changes: 3 additions & 0 deletions modules/guides/pages/index.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
= Guides

The guides in this section are useful to complete particular tasks.
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
= PersistentVolumeClaim usage
= Providing external resources to Stacklets with PersistentVolumeClaims
:page-aliases: concepts:pvc.adoc
:pvcs: https://kubernetes.io/docs/concepts/storage/persistent-volumes/
:pvc-capacity: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#capacity
:pvc-access-modes: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes

Several of the tools on the Stackable platform can use external resources that the cluster administrator makes available via a PersistentVolume. Airflow users can access DAG jobs this way, and Spark users can do the same for data or other job dependencies, to name just two examples.
Several of the tools on the Stackable platform can use external resources that the cluster administrator makes available via a PersistentVolume.
Airflow users can access DAG jobs this way, and Spark users can do the same for data or other job dependencies, to name just two examples.

A PersistentVolume will usually be provisioned by the Kubernetes Container Storage Interface (CSI) on behalf of the cluster administrator, who will take into account the type of storage that is required. This will include, for example, an https://kubernetes.io/docs/concepts/storage/persistent-volumes/#capacity[appropriate sizing], and relevant access modes (which in turn are dependent on the StorageClass chosen to back the PersistentVolume).
A PersistentVolume will usually be provisioned by the Kubernetes Container Storage Interface (CSI) on behalf of the cluster administrator, who will take into account the type of storage that is required.
This will include, for example, an {pvc-capacity}[appropriate sizing], and relevant access modes (which in turn are dependent on the StorageClass chosen to back the PersistentVolume).

The relationship between a PersistentVolume and a PersistentVolumeClaim can be illustrated by these https://kubernetes.io/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolume[two] https://kubernetes.io/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolumeclaim[examples]:

Expand Down Expand Up @@ -48,11 +54,14 @@ spec:

== Access modes and the StorageClass

Not all storage classes support all https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes[access modes]. The supported access modes also depend on the Kubernetes implementation, see for example the compatiblity table https://docs.openshift.com/container-platform/4.8/storage/understanding-persistent-storage.html#pv-access-modes_understanding-persistent-storage[Supported access modes for PVs] in the OpenShift documentation. Other managed Kubernetes implementations will be similar, albeit with different default storage class names. The important point is that the default StorageClass only supports `ReadWriteOnce`, which limits access to the PersistentVolumeClaim to a single node. A strategy governing PersistentVolumeClaim resources will thus address the following:
Not all storage classes support all {pvc-access-modes}[access modes].
The supported access modes also depend on the Kubernetes implementation, see for example the compatiblity table https://docs.openshift.com/container-platform/4.8/storage/understanding-persistent-storage.html#pv-access-modes_understanding-persistent-storage[Supported access modes for PVs] in the OpenShift documentation. Other managed Kubernetes implementations will be similar, albeit with different default storage class names.
The important point is that the default StorageClass only supports `ReadWriteOnce`, which limits access to the PersistentVolumeClaim to a single node.
A strategy governing PersistentVolumeClaim resources will thus address the following:

- what access mode is appropriate? Will the resources be accessed by multiple pods and/or modes?
- does the Kubernetes cluster have a default implementation for these access modes?
- if access modes are restricted (e.g. `ReadWriteOnce`), does the cluster prioritise available resources over implicit application dependencies (in other words, is the PersistentVolumeClaim treated as a soft- or hard-dependency)?
* what access mode is appropriate? Will the resources be accessed by multiple pods and/or modes?
* does the Kubernetes cluster have a default implementation for these access modes?
* if access modes are restricted (e.g. `ReadWriteOnce`), does the cluster prioritise available resources over implicit application dependencies (in other words, is the PersistentVolumeClaim treated as a soft- or hard-dependency)?

If a PersistentVolumeClaim should be mounted on a single node for the application and its components that use it, this can be specified explicitly (see the next section).

Expand Down Expand Up @@ -85,7 +94,6 @@ Users of the xref:spark-k8s:index.adoc[Spark-k8s operator] have a variety of way

== Further reading

- https://kubernetes.io/docs/concepts/storage/persistent-volumes/[Persistent Volumes]
- https://kubernetes.io/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolumeclaim[PV/PVC example]
- https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/[Labels and selectors
]
* {pvcs}[Persistent Volumes]
* https://kubernetes.io/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolumeclaim[PV/PVC example]
* https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/[Labels and selectors]
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Running Stackable in an air-gapped environment
:page-aliases: tutorials:running-stackable-in-an-airgapped-environment.adoc

The main challenge with running Stackable in an air-gapped environment is how to get the artifacts (container images and Helm charts) into the environment. There are a few ways to do this:

Expand Down Expand Up @@ -32,4 +33,4 @@ Then restart the `containerd` service. Now `containerd` will fetch all images th

* Add an alias for `oci.stackable.tech` to the `/etc/hosts` file on every node (like `10.7.228.12 oci.stackable.tech`), issue a self-signed certificate for `oci.stackable.tech` to your registry and add the certificate to the trusted certificates on every node. Note that if you also want to enforce signature checks for Stackable's images via a policy controller, you will need to add this host alias to the Pod of the policy controller as well (and make it trust the certificate).

If you want to know how to verify image signatures in an air-gapped environment, check out our documentation about xref:tutorials:enabling-verification-of-image-signatures.adoc[image signature verification].
If you want to know how to verify image signatures in an air-gapped environment, check out our documentation about xref:enabling-verification-of-image-signatures.adoc[image signature verification].
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Viewing and verifying SBOMs of the Stackable Data Platform
:page-aliases: tutorials:viewing-and-verifying-sboms.adoc

With release 24.3 of SDP, we started providing SBOMs (Software Bill of Materials) for our container images. Please note that they currently are in a draft stage and we are continually working on improving them. As a first step, we aim to provide a list of all primary (top level) components and their versions included in each container image. Our SBOMs follow the https://cyclonedx.org/[CycloneDX] standard and are available in JSON format.

Expand All @@ -7,7 +8,7 @@ You can browse through our SBOMs at https://sboms.stackable.tech/.
You will find a simple hierarchical structure, one directory per release, containing a list of all container images included in that release. For each container image, one SBOM per version of the image is listed.

This page is a simple wrapper on top of the Stackable OCI registry, where the SBOMs are attached as signed https://github.com/in-toto/attestation[attestations] to the container images. When you click on a link in the SBOM browser, the SBOM is validated, extracted from the container registry, and then downloaded to your device.
The next step of this tutorial explains the single steps happening under the hood when a link is clicked, and how to do them manually.
The next step of this guide explains the single steps happening under the hood when a link is clicked, and how to do them manually.

== Verifying and extracting an SBOM manually with cosign
To verify and extract the SBOM, a tool called https://github.com/sigstore/cosign[cosign] is needed. Please have a look at the https://docs.sigstore.dev/system_config/installation/[installation instructions] in the cosign documentation and choose your preferred installation method. Additionally, https://github.com/jqlang/jq[jq] is used to parse the JSON output of cosign.
Expand Down Expand Up @@ -37,7 +38,7 @@ If the identity of the signer matches, you can be sure the contents of the attes
You can now be sure that the SBOM was attested to the container image you're interested in by a Stackable Github Action workflow, it's even possible to look at the workflow to see how exactly this happened.

== Enabling automatic verification of SBOMs
Similar to our xref:tutorials:enabling-verification-of-image-signatures.adoc[image signature verification] tutorial, it's possible to enforce that only container images with SBOMs that are signed by Stackable are allowed to run in your cluster. Sigstore's https://docs.sigstore.dev/policy-controller/overview/[Policy Controller] can be used to achieve this.
Similar to our xref:enabling-verification-of-image-signatures.adoc[image signature verification] guide, it's possible to enforce that only container images with SBOMs that are signed by Stackable are allowed to run in your cluster. Sigstore's https://docs.sigstore.dev/policy-controller/overview/[Policy Controller] can be used to achieve this.

IMPORTANT: Releases prior to SDP 24.3 do not have signed SBOMs. If you are using an older release and enforce SBOM verification, Pods with Stackable images will be prevented from starting.

Expand Down
3 changes: 0 additions & 3 deletions modules/tutorials/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
* xref:index.adoc[]
** xref:authentication_with_openldap.adoc[]
** xref:logging-vector-aggregator.adoc[]
** xref:enabling-verification-of-image-signatures.adoc[]
** xref:running-stackable-in-an-airgapped-environment.adoc[]
** xref:viewing-and-verifying-sboms.adoc[]
3 changes: 3 additions & 0 deletions supplemental-ui/partials/navbar.hbs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
<a class="navbar-item" href="{{{ relativize (versioned "home" page "concepts/index.html") }}}">Concepts</a>
<a class="navbar-item" href="{{{ relativize (versioned "home" page "demos/index.html") }}}">Demos</a>
<a class="navbar-item" href="{{{ relativize (versioned "home" page "tutorials/index.html") }}}">Tutorials</a>
{{#if (not (or (eq page.version "23.1") (eq page.version "23.4") (eq page.version "23.7") (eq page.version "23.11") (eq page.version "24.3")))}}
<a class="navbar-item" href="{{{ relativize (versioned "home" page "guides/index.html") }}}">Guides</a>
{{/if}}
<hr class="navbar-divider">
{{#if (or (eq page.version "23.1") (eq page.version "23.4") (eq page.version "23.7") (eq page.version "23.11"))}}
<a class="navbar-item" href="{{{ relativize (versioned "home" page "reference/index.html") }}}">CRD Reference</a>
Expand Down