[Merged by Bors] - Deploy default and support custom affinities

[sbernauer]

Description

For stackabletech/issues#323

Review Checklist

- [x] Code contains useful comments - [x] CRD change approved (or not applicable) - [x] (Integration-)Test cases added (or not applicable) - [x] Documentation added (or not applicable) - [x] Changelog updated (or not applicable) - [x] Cargo.toml only contains references to git tags (not specific commits or branches) - [x] Helm chart can be installed and deployed operator works (or not applicable) 

Once the review is done, comment bors r+ (or bors merge) to merge. Further information

[razvan]

A changelog entry is missing and probably an integration test.

[razvan]

Reminder to update this file before merging.

[sbernauer]

Regarding kuttl test: All we would do is assert on some field (affinity) on the scheduled Pod.
I personally think it's not worth having a kuttl test wasting k8s resources and taking time when we can check the contents in a unit-test. I tried my best writing unit tests, they can be improved in the future with the mechanism @vsupalov described yesterday. The only "untested" part is throwing the StackableAffinity struct into the PodBuilder as of now.
What do you think?

[razvan]

I'm fine with only unit testing it but we are missing actual tests for user defined affinities. Only two cases are covered:

• default affinities
• legacy node selector

[sbernauer]

The merge mechanism with custom user provided affinities is tested here https://github.com/stackabletech/operator-rs/blob/5aab38f9cdd96b98f483ee3834a9cc74d002c0f0/src/commons/affinities.rs#L160
I would rather not copy/paste the same test in all operators, especially as the tested codepath lives within operator-rs

[sbernauer]

The defaults and legacy behavior is operator-specific

[razvan]

bors merge

[bors]

Pull request successfully merged into main.

Build succeeded:

• All tests passed

[lfrancke]

Added to the Feature Tracker

[sbernauer]

Thx for adding the docs!