Add redshift check cloud cleanup

[mrodm]

Add checks into cloud cleanup script so it is sent an email notification if there is any redshift cluster stale related to CI (integrations or elastic-package).

To be replaced by cloud-reapder when there is support for this kind of resources.

Example of build with this cloud cleanup step:
https://buildkite.com/elastic/elastic-package/builds/2527

Added input steps when this pipeline is triggered from the UI.

Relates #1700

[jsoriano]

Do we have the region in some environment variable that we could use here?

[mrodm]

Currently, it is using the default region set in the terraform files, there is no value set in our scripts. For instance:

elastic-package/test/packages/parallel/aws/data_stream/ec2_metrics/_dev/deploy/tf/env.yml

Line 9 in e74c021

- AWS_REGION=${AWS_REGION:-us-east-1}

https://github.com/elastic/integrations/blob/4efd95037467d5e54270006bcd71c3b0b6982932/packages/cisco_umbrella/_dev/deploy/tf/env.yml#L9

Should we add a value there in our scripts for both repositories or rely on that default value?
Currently, all terraform tags are set in the pre-command hook:

elastic-package/.buildkite/hooks/pre-command

Line 14 in 346081a

# variables required for terraform

Maybe, we could add/set there the variable export AWS_REGION=us-east-1 too.

[jsoriano]

Do we want to exit here? We could continue and check other resources, right?

[mrodm]

Thanks, I forgot to remove this exit!
I've updated the code to allow running all steps and just exit with failure at the end.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: cfd359d

History

• 💚 Build #2530 succeeded fdebf94
• 💚 Build #2528 succeeded 50a7238
• 💚 Build #2527 succeeded 8a078c8
• 💚 Build #2526 succeeded 5a4485b
• 💔 Build #2524 failed 9e9bd14
• 💔 Build #2523 failed b23892b

cc @mrodm