Skip to content

Forceful deletion of remaining pods after cleanup #670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

Forceful deletion of remaining pods after cleanup #670

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4c92d5d
Added support to allow for forceful deletion of remaining pods that a…
metalcycling Oct 20, 2023
4eb56ea
Merge remote-tracking branch 'upstream/main' into fault-tolerance
metalcycling Oct 20, 2023
d3a929e
Only change the QueueJobState when forceful deletion is enabled
metalcycling Oct 20, 2023
277e9c0
Changed spec from 'ForcefulDeletionAfterSeconds' to 'ForceDeletionTim…
metalcycling Oct 20, 2023
347b9c5
Update to the CRD to support 'pauseTimeInSeconds'
metalcycling Oct 24, 2023
55fbef9
Support for 'PauseTimeInSeconds' so MCAD delays redispatching after a…
metalcycling Oct 24, 2023
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update to the CRD to support 'pauseTimeInSeconds'
  • Loading branch information
@metalcycling
metalcycling committed Oct 24, 2023
commit 347b9c5c711011df0e0876f2a654e8e2a651715d
6 changes: 6 additions & 0 deletions config/crd/bases/workload.codeflare.dev_appwrappers.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,12 @@ spec:
description: Field to keep track of how many times a requeuing
event has been triggered.
type: integer
pauseTimeInSeconds:
default: 0
description: When a job is ready to be redispatched after
it has been requeued due to preemption, MCAD will wait 'pauseTimeInSeconds'
before redispatching
type: integer
timeInSeconds:
default: 300
description: Initial waiting time before requeuing conditions
Expand Down
6 changes: 6 additions & 0 deletions config/crd/bases/workload.codeflare.dev_schedulingspecs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,12 @@ spec:
description: Field to keep track of how many times a requeuing
event has been triggered.
type: integer
pauseTimeInSeconds:
default: 0
description: When a job is ready to be redispatched after it has
been requeued due to preemption, MCAD will wait 'pauseTimeInSeconds'
before redispatching
type: integer
timeInSeconds:
default: 300
description: Initial waiting time before requeuing conditions
Expand Down
6 changes: 6 additions & 0 deletions deployment/mcad-controller/crds/workload.codeflare.dev_appwrappers.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,12 @@ spec:
description: Field to keep track of how many times a requeuing
event has been triggered.
type: integer
pauseTimeInSeconds:
default: 0
description: When a job is ready to be redispatched after
it has been requeued due to preemption, MCAD will wait 'pauseTimeInSeconds'
before redispatching
type: integer
timeInSeconds:
default: 300
description: Initial waiting time before requeuing conditions
Expand Down
6 changes: 6 additions & 0 deletions deployment/mcad-controller/crds/workload.codeflare.dev_schedulingspecs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,12 @@ spec:
description: Field to keep track of how many times a requeuing
event has been triggered.
type: integer
pauseTimeInSeconds:
default: 0
description: When a job is ready to be redispatched after it has
been requeued due to preemption, MCAD will wait 'pauseTimeInSeconds'
before redispatching
type: integer
timeInSeconds:
default: 300
description: Initial waiting time before requeuing conditions
Expand Down
4 changes: 4 additions & 0 deletions pkg/apis/controller/v1beta1/schedulingspec.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ type RequeuingTemplate struct {
// This may be necesary to prevent redeployment of generic items that create pods that were not correctly deleted.
// +kubebuilder:default=0
ForceDeletionTimeInSeconds int `json:"forceDeletionTimeInSeconds,omitempty" protobuf:"bytes,7,rep,name=forceDeletionTimeInSeconds"`
// When a job is ready to be redispatched after it has been requeued due to preemption, MCAD will
// wait 'pauseTimeInSeconds' before redispatching
// +kubebuilder:default=0
PauseTimeInSeconds int `json:"pauseTimeInSeconds,omitempty" protobuf:"bytes,8,rep,name=pauseTimeInSeconds"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
Expand Down