Skip to content

Custom autoscaler #815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 79 commits into from
Feb 16, 2020
Merged

Custom autoscaler #815

Changes from 1 commit
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
d82eb2b
Add gunicorn and sidecar for queue length
vishalbollu Feb 6, 2020
c717863
Merge branch 'master' of github.com:cortexlabs/cortex into custom-aut…
deliahu Feb 6, 2020
4cc4a3c
Add ss to image
vishalbollu Feb 6, 2020
62bfe9d
Merge branch 'custom-autoscaler' of github.com:cortexlabs/cortex into…
vishalbollu Feb 6, 2020
686fec9
Add autoscaler algorithm
deliahu Feb 7, 2020
de4efea
Update lib_watch.go
deliahu Feb 7, 2020
3bd5b00
Create autoscaler crons
deliahu Feb 7, 2020
17fc700
Fix math.Min()
deliahu Feb 7, 2020
2d3c3a2
Delete cron on API deletion
deliahu Feb 7, 2020
f53418b
Autoscaler based on cloudwatch
vishalbollu Feb 7, 2020
30cc616
Add initial delay, respect min/max replicas
deliahu Feb 7, 2020
ac9851b
Add autoscaler logging
deliahu Feb 7, 2020
3e5eb8b
Tick on second 1 every 10 seconds
vishalbollu Feb 7, 2020
fd68479
Remove debug time
vishalbollu Feb 7, 2020
78f8475
Update autoscaler.go
deliahu Feb 7, 2020
1c7de6b
Update autoscaler.go
deliahu Feb 7, 2020
499de67
Sum total inflight requests
vishalbollu Feb 8, 2020
990dcb8
Update python.md
deliahu Feb 8, 2020
ea7c6eb
Add upscale and downscale tolerance
deliahu Feb 8, 2020
857e004
Add duration validator and parser
deliahu Feb 8, 2020
995b569
Add autoscaling config and annotations
deliahu Feb 10, 2020
0ef2eeb
Add TODO
deliahu Feb 10, 2020
4739cd0
Format lib_watch.go
deliahu Feb 10, 2020
4e8b6b0
Update autoscaler.go
deliahu Feb 10, 2020
6d4b662
Add license to ss.py
deliahu Feb 10, 2020
959ee97
Add Request monitor image
vishalbollu Feb 11, 2020
617aff0
Allow users to configure replica_parallelism and request_backlog length
vishalbollu Feb 12, 2020
b832a09
Add request-monitor to registry.sh
deliahu Feb 12, 2020
34555db
Add kublet config schema to generate_eks.py
deliahu Feb 12, 2020
5ed6690
Make _tickInterval a global const
deliahu Feb 12, 2020
85146c0
Add sysctl commands
deliahu Feb 12, 2020
8bd8252
Fix lint
deliahu Feb 12, 2020
267eafa
Remove kubectl from operator image
deliahu Feb 12, 2020
b9a6e39
Use builder for request-monitor
deliahu Feb 12, 2020
bf4d29d
Update versions.md
deliahu Feb 12, 2020
37fa4c4
Update all predictor types to use request monitor and accept args via…
vishalbollu Feb 12, 2020
9932a36
Allow users to configure workers per replica and threads per worker
vishalbollu Feb 12, 2020
3a423eb
Fix lint
deliahu Feb 13, 2020
70f0555
Update CORTEX_OS_MAX_CONN calculation
deliahu Feb 13, 2020
0a0f6b1
Remove TODOs
deliahu Feb 13, 2020
3eab28d
Update run.sh
vishalbollu Feb 13, 2020
d8bd44f
Merge branch 'master' of github.com:cortexlabs/cortex into custom-aut…
deliahu Feb 14, 2020
151525d
Update request-monitor log
deliahu Feb 14, 2020
44ca0ca
Update missing metrics message
vishalbollu Feb 14, 2020
7edbbb8
Blacklist request monitor logs from cortex logs
vishalbollu Feb 14, 2020
6a42154
Remove ss.py
vishalbollu Feb 14, 2020
4b32480
Remove iproute2 from python-serve image
vishalbollu Feb 14, 2020
9ea92dc
Fix typo in error
vishalbollu Feb 14, 2020
3ed8263
Revert security context delete
vishalbollu Feb 14, 2020
f849ab6
Update autoscaler.go
vishalbollu Feb 14, 2020
15c1cd8
Only count established connection when monitoring requests
vishalbollu Feb 14, 2020
4160ac4
Give request-monitor a resource request
deliahu Feb 14, 2020
df223e7
Merge branch 'custom-autoscaler' of github.com:cortexlabs/cortex into…
deliahu Feb 14, 2020
0568798
Remove request backlog from configuration
vishalbollu Feb 14, 2020
6c17aee
Merge branch 'custom-autoscaler' of github.com:cortexlabs/cortex into…
vishalbollu Feb 14, 2020
47277fa
Add request-monitor to Makefile build
vishalbollu Feb 14, 2020
aa62d05
User defer for unlocks in request-monitor
vishalbollu Feb 14, 2020
f288e65
Add sleep to health_check in request-monitor
vishalbollu Feb 14, 2020
fb88c10
Check request counts is non empty
vishalbollu Feb 14, 2020
100430e
Add hpa back to debug
vishalbollu Feb 14, 2020
a5fd6a2
Move _autoscalingTickInterval
deliahu Feb 14, 2020
4645863
Update autoscaler.go
deliahu Feb 14, 2020
387413f
Rename error message from ErrorMustBeMultiple to ErrorIsNotMultiple
vishalbollu Feb 14, 2020
7723581
Merge branch 'custom-autoscaler' of github.com:cortexlabs/cortex into…
vishalbollu Feb 14, 2020
06b3fef
Update gunicor log format
deliahu Feb 14, 2020
d47061d
Hide sysctl output
deliahu Feb 15, 2020
feee834
Move gunicorn health check to post_worker_init()
deliahu Feb 15, 2020
3f2c5b1
Show pid in api logs
deliahu Feb 15, 2020
bbf7964
Set gunicorn logger
deliahu Feb 15, 2020
08b4e53
Add missing license
deliahu Feb 15, 2020
2f1eaec
Move duration multupe validation to duration parser
deliahu Feb 16, 2020
d49fd42
Update api.go
deliahu Feb 16, 2020
6586ed2
Update log.py
deliahu Feb 16, 2020
cf631c8
Update gunicorn_config.py
deliahu Feb 16, 2020
5767b9e
Update serve.py
deliahu Feb 16, 2020
3d2ed21
Remove extract_waitress_params
deliahu Feb 16, 2020
6c8692c
Rename variables, update function signature
deliahu Feb 16, 2020
f76b3d3
Update autoscaler.go
deliahu Feb 16, 2020
6e366d4
Merge branch 'master' of github.com:cortexlabs/cortex into custom-aut…
deliahu Feb 16, 2020
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update serve.py
  • Loading branch information
@deliahu
deliahu committed Feb 16, 2020
commit 5767b9e6dd80e7ddaebde614364b04f8046b2f0c
2 changes: 1 addition & 1 deletion pkg/workloads/cortex/serve/serve.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
from cortex import consts
from cortex.lib import util
from cortex.lib.type import API
from cortex.lib.log import cx_logger, debug_obj, refresh_logger
from cortex.lib.log import cx_logger, debug_obj
from cortex.lib.storage import S3
from cortex.lib.exceptions import UserRuntimeException

Expand Down