-
- Notifications
You must be signed in to change notification settings - Fork 13
added a note on PVC usage #224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
13 commits Select commit Hold shift + click to select a range
fa27d2e added a note on PVC usage
adwk67 c7ddaf2 pvc usage: corrections
adwk67 7982a77 pvc usage: minor change
adwk67 0109bb9 added links and examples
adwk67 45f8dd9 added suggestions
adwk67 f0bfc9d Merge branch 'main' into pvc-concept
adwk67 e63b795 added some moer links to docs and examples
adwk67 e28c0c9 Merge branch 'main' into pvc-concept
fhennig d0a93db Added document to menu
e1ff855 Update modules/concepts/pages/pvc.adoc
adwk67 a23d1d5 Update modules/concepts/pages/pvc.adoc
adwk67 9a5268e Update modules/concepts/pages/pvc.adoc
adwk67 24d4d03 Update modules/concepts/pages/pvc.adoc
adwk67 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| = `PersistentVolumeClaim` usage | ||
| | ||
| Several of the tools on the Stackable platform can use external resources that the cluster administrator makes available via a `PersistentVolume`. Airflow users can access DAG jobs this way, and Spark users can do the same for data or other job dependencies, to name just two examples. | ||
| | ||
| A `PersistentVolume` will usually be provisioned by the cluster administrator, who will take into account the type of storage that is required. This will include, for example, an appropriate sizing, and relevant access modes (which in turn are dependent on the `StorageClass` chosen to back the `PersistentVolume`). | ||
adwk67 marked this conversation as resolved. Outdated Show resolved Hide resolved | ||
| | ||
| == Access modes and the `StorageClass` | ||
adwk67 marked this conversation as resolved. Outdated Show resolved Hide resolved | ||
| | ||
| Not all access modes are compatible with all storage classes. This https://docs.openshift.com/container-platform/4.8/storage/understanding-persistent-storage.html#pv-access-modes_understanding-persistent-storage[link] provides a helpful compatability matrix (in this case for OpenShift). Other managed Kubernetes implementations will be similar, albeit with different default storage class names. The important point is that the default `StorageClass` only supports `ReadWriteOnce`, which limits access to the `PersistentVolumeClaim` to a single node. | ||
| | ||
| == Node selection | ||
adwk67 marked this conversation as resolved. Show resolved Hide resolved | ||
| | ||
| The Kubernetes https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/[documentation] states the following with regard to assigning pods to specific nodes: | ||
| ____ | ||
| Generally such constraints are unnecessary, as the scheduler will automatically do a reasonable placement (for example, spreading your Pods across nodes so as not place Pods on a node with insufficient free resources). | ||
adwk67 marked this conversation as resolved. Outdated Show resolved Hide resolved | ||
| ____ | ||
| This suggests that resources are automatically considered when pods are assigned to nodes, but it is not clear if the same is true for implicit dependencies, such as `PersistentVolumeClaim` usage. The https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/[scheduler] will take various factors into account, such as | ||
adwk67 marked this conversation as resolved. Outdated Show resolved Hide resolved | ||
| | ||
| ____ | ||
| ...individual and collective resource requirements, hardware / software / policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference.... | ||
| ____ | ||
| | ||
| but implementations may vary in the way soft dependencies (e.g. optimal resource usage) and hard dependencies (e.g. access modes, that may prevent the job from running) are handled and prioritised. | ||
| | ||
| == Test considerations | ||
adwk67 marked this conversation as resolved. Show resolved Hide resolved | ||
| | ||
| For `PersistentVolumeClaim`-relevant tests in the Stackable operator repositories the backing `PersistentVolume` is usually omitted as this is an implementation/provisioning choice to made by the cluster administrator and mocking e.g. an NFS volume for tests is non-trivial. | ||
adwk67 marked this conversation as resolved. Outdated Show resolved Hide resolved | ||
| | ||
| The only viable access mode is thus `ReadWriteOnce` (see above) which means that all test steps dependent on a `PersistentVolumeClaim` should be run on the same node, and this assignment should be made explicitly with a declaration of either a node selector or pod-affinity. | ||
adwk67 marked this conversation as resolved. Outdated Show resolved Hide resolved | ||
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.