Skip to content

Conversation

@abuchanan-airbyte
Copy link
Collaborator

@abuchanan-airbyte abuchanan-airbyte commented Sep 12, 2024

Hopefully this fixes: https://github.com/airbytehq/airbyte-internal-issues/issues/9712

I think the issue was that we were setting env vars like JOB_MAIN_CONTAINER_CPU_LIMIT, but the charts set them via config maps via global values.

I can now run install multiple times, changing between --low-resource-mode and not

Existing cluster:

docker exec -it airbyte-abctl-control-plane kubectl -n airbyte-abctl get configmap -oyaml --field-selector metadata.name=airbyte-abctl-airbyte-env | grep JOB_ JOB_MAIN_CONTAINER_CPU_LIMIT: "3" JOB_MAIN_CONTAINER_CPU_REQUEST: "" JOB_MAIN_CONTAINER_MEMORY_LIMIT: 4Gi JOB_MAIN_CONTAINER_MEMORY_REQUEST: "" 

After running abctl local install --low-resource-mode:

docker exec -it airbyte-abctl-control-plane kubectl -n airbyte-abctl get configmap -oyaml --field-selector metadata.name=airbyte-abctl-airbyte-env | grep JOB_ JOB_MAIN_CONTAINER_CPU_LIMIT: "0" JOB_MAIN_CONTAINER_CPU_REQUEST: "0" JOB_MAIN_CONTAINER_MEMORY_LIMIT: "0" JOB_MAIN_CONTAINER_MEMORY_REQUEST: "0" 

After running abctl local install:

docker exec -it airbyte-abctl-control-plane kubectl -n airbyte-abctl get configmap -oyaml --field-selector metadata.name=airbyte-abctl-airbyte-env | grep JOB_ JOB_MAIN_CONTAINER_CPU_LIMIT: "3" JOB_MAIN_CONTAINER_CPU_REQUEST: "" JOB_MAIN_CONTAINER_MEMORY_LIMIT: 4Gi JOB_MAIN_CONTAINER_MEMORY_REQUEST: "" 
@abuchanan-airbyte abuchanan-airbyte requested a review from a team as a code owner September 12, 2024 21:57
@jomach
Copy link

jomach commented Dec 5, 2024

Airbyte completely ignores these values on k8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

4 participants