Recreate kind cluster on every evg host reboot #32
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Summary
Evergreen hosts reboot every day or weekend (depending on your configuration). After every reboot, inter-cluster connectivity might be broken for some reason. Recreating the clusters is the only solution we have so far.
This is a systemd service that runs on every boot and recreates all clusters (including the kind-kind for single cluster tests).
Our tunnel command will now also get the kubeconfig from the host otherwise the tunnel won't open to the (new) ports of the recreated clusters.
The only action for you is to run
evg_host.sh configure
which will update the scripts and set up the systemd service.Proof of Work
Tested locally, but it would be nice if someone can checkout this branch and try it for themselves.
Checklist
Reminder (Please remove this when merging)