feat: add support for hugepages_config #2223

DrFaust92 · 2024-12-21T18:18:58Z

Closes #2320

apeabody · 2024-12-23T17:44:52Z

/gcbrun

apeabody

Thanks for the contribution @DrFaust92!

Can you please add to an example for test coverage, otherwise other than some extra spaces, this LGTM.

NOTE: hugepages_config added in v6.5.0

autogen/main/cluster.tf.tmpl

DrFaust92 · 2025-01-02T20:01:51Z

@apeabody Tried my best with tests here as well 😊 . LMK if something needs fixing

apeabody · 2025-01-02T23:14:53Z

/gcbrun

apeabody · 2025-01-02T23:49:53Z

FYI @DrFaust92 - A few of the pools do use n1, but the rest are currently e2.

TestNodePool 2025-01-02T23:41:57Z command.go:185: Error: error creating NodePool: googleapi: Error 400: 1G hugepages was set but the nodepool machine type(e2-medium) does not support this configuration. TestNodePool 2025-01-02T23:41:57Z command.go:185: Details: TestNodePool 2025-01-02T23:41:57Z command.go:185: [ TestNodePool 2025-01-02T23:41:57Z command.go:185: { TestNodePool 2025-01-02T23:41:57Z command.go:185: "@type": "type.googleapis.com/google.rpc.RequestInfo", TestNodePool 2025-01-02T23:41:57Z command.go:185: "requestId": "0x7b26367c30d761e2" TestNodePool 2025-01-02T23:41:57Z command.go:185: } TestNodePool 2025-01-02T23:41:57Z command.go:185: ] TestNodePool 2025-01-02T23:41:57Z command.go:185: , badRequest TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: with module.example.module.gke.google_container_node_pool.pools["pool-04"], TestNodePool 2025-01-02T23:41:57Z command.go:185: on ../../../modules/beta-public-cluster/cluster.tf line 597, in resource "google_container_node_pool" "pools": TestNodePool 2025-01-02T23:41:57Z command.go:185: 597: resource "google_container_node_pool" "pools" { TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: Error: error creating NodePool: googleapi: Error 400: 1G hugepages was set but the nodepool machine type(e2-medium) does not support this configuration. TestNodePool 2025-01-02T23:41:57Z command.go:185: Details: TestNodePool 2025-01-02T23:41:57Z command.go:185: [ TestNodePool 2025-01-02T23:41:57Z command.go:185: { TestNodePool 2025-01-02T23:41:57Z command.go:185: "@type": "type.googleapis.com/google.rpc.RequestInfo", TestNodePool 2025-01-02T23:41:57Z command.go:185: "requestId": "0x48e2d757411a4297" TestNodePool 2025-01-02T23:41:57Z command.go:185: } TestNodePool 2025-01-02T23:41:57Z command.go:185: ] TestNodePool 2025-01-02T23:41:57Z command.go:185: , badRequest TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: with module.example.module.gke.google_container_node_pool.pools["pool-01"], TestNodePool 2025-01-02T23:41:57Z command.go:185: on ../../../modules/beta-public-cluster/cluster.tf line 597, in resource "google_container_node_pool" "pools": TestNodePool 2025-01-02T23:41:57Z command.go:185: 597: resource "google_container_node_pool" "pools" { TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: Error: error creating NodePool: googleapi: Error 400: 1G hugepages was set but the nodepool machine type(n1-standard-2) does not support this configuration. TestNodePool 2025-01-02T23:41:57Z command.go:185: Details: TestNodePool 2025-01-02T23:41:57Z command.go:185: [ TestNodePool 2025-01-02T23:41:57Z command.go:185: { TestNodePool 2025-01-02T23:41:57Z command.go:185: "@type": "type.googleapis.com/google.rpc.RequestInfo", TestNodePool 2025-01-02T23:41:57Z command.go:185: "requestId": "0x16e561c8b65549ca" TestNodePool 2025-01-02T23:41:57Z command.go:185: } TestNodePool 2025-01-02T23:41:57Z command.go:185: ] TestNodePool 2025-01-02T23:41:57Z command.go:185: , badRequest TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: with module.example.module.gke.google_container_node_pool.pools["pool-03"], TestNodePool 2025-01-02T23:41:57Z command.go:185: on ../../../modules/beta-public-cluster/cluster.tf line 597, in resource "google_container_node_pool" "pools": TestNodePool 2025-01-02T23:41:57Z command.go:185: 597: resource "google_container_node_pool" "pools" { TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: Error: error creating NodePool: googleapi: Error 400: 1G hugepages was set but the nodepool machine type(n1-standard-2) does not support this configuration. TestNodePool 2025-01-02T23:41:57Z command.go:185: Details: TestNodePool 2025-01-02T23:41:57Z command.go:185: [ TestNodePool 2025-01-02T23:41:57Z command.go:185: { TestNodePool 2025-01-02T23:41:57Z command.go:185: "@type": "type.googleapis.com/google.rpc.RequestInfo", TestNodePool 2025-01-02T23:41:57Z command.go:185: "requestId": "0x40383fdeb6fa524a" TestNodePool 2025-01-02T23:41:57Z command.go:185: } TestNodePool 2025-01-02T23:41:57Z command.go:185: ] TestNodePool 2025-01-02T23:41:57Z command.go:185: , badRequest TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: with module.example.module.gke.google_container_node_pool.pools["pool-05"], TestNodePool 2025-01-02T23:41:57Z command.go:185: on ../../../modules/beta-public-cluster/cluster.tf line 597, in resource "google_container_node_pool" "pools": TestNodePool 2025-01-02T23:41:57Z command.go:185: 597: resource "google_container_node_pool" "pools" { TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: Error: error creating NodePool: googleapi: Error 400: 1G hugepages was set but the nodepool machine type(n1-standard-2) does not support this configuration. TestNodePool 2025-01-02T23:41:57Z command.go:185: Details: TestNodePool 2025-01-02T23:41:57Z command.go:185: [ TestNodePool 2025-01-02T23:41:57Z command.go:185: { TestNodePool 2025-01-02T23:41:57Z command.go:185: "@type": "type.googleapis.com/google.rpc.RequestInfo", TestNodePool 2025-01-02T23:41:57Z command.go:185: "requestId": "0x37ecaeacf733a8ea" TestNodePool 2025-01-02T23:41:57Z command.go:185: } TestNodePool 2025-01-02T23:41:57Z command.go:185: ] TestNodePool 2025-01-02T23:41:57Z command.go:185: , badRequest TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z command.go:185: with module.example.module.gke.google_container_node_pool.pools["pool-02"], TestNodePool 2025-01-02T23:41:57Z command.go:185: on ../../../modules/beta-public-cluster/cluster.tf line 597, in resource "google_container_node_pool" "pools": TestNodePool 2025-01-02T23:41:57Z command.go:185: 597: resource "google_container_node_pool" "pools" { TestNodePool 2025-01-02T23:41:57Z command.go:185: TestNodePool 2025-01-02T23:41:57Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1;

DrFaust92 · 2025-01-03T00:23:22Z

@apeabody should i change the node pools types to support the test case? or how should I approach this?

apeabody · 2025-01-03T00:33:13Z

@apeabody should i change the node pools types to support the test case? or how should I approach this?

The 2-m huge pages should be fine as-is, however for the 1-g perhaps just test a single node pool which has a supported machine type. Maybe change "pool-05" to c3-standard-4?

1-gigabtye-sized huge pages are only available on the A3, C2D, C3, C3A, C3D, C4, CT5E, CT5L, CT5LP, CT6E, H3, M2, M3, or Z3 [machine types](https://cloud.google.com/compute/docs/machine-resource).

DrFaust92 · 2025-01-04T23:04:01Z

apeabody lets try now

apeabody · 2025-01-06T16:24:44Z

/gcbrun

autogen/main/cluster.tf.tmpl

apeabody · 2025-04-03T15:25:03Z

/gcbrun

DrFaust92 · 2025-04-07T03:18:27Z

apeabody added try as requested

rawanbadawi · 2025-04-07T17:10:10Z

@apeabody My customer needs this feature, is it possible to approve the latest?

Thanks

apeabody · 2025-04-07T17:17:18Z

/gcbrun

apeabody · 2025-04-07T19:04:23Z

INT test results:

	Error: Error in function call on ../../../modules/beta-public-cluster/cluster.tf line 938, in resource "google_container_node_pool" "pools": 938: hugepage_size_1g = try(coalesce(local.node_pools_hugepage_size_1g[each.value["name"]], local.node_pools_hugepage_size_1g["all"], null)) ├──────────────── │ each.value["name"] is "pool-03" │ local.node_pools_hugepage_size_1g is object with 8 attributes │ local.node_pools_hugepage_size_1g["all"] is ""	Call to function "try" failed: no expression succeeded:	- Error in function call (at	../../../modules/beta-public-cluster/cluster.tf:938,36-45) Call to function "coalesce" failed: no non-null, non-empty-string arguments.	At least one expression must produce a successful result

Looks like try() can't be used in this situation, probably need to use a less elegant ternary, etc.

apeabody · 2025-04-29T15:59:15Z

apeabody any details?

Looks like need to adjust the test:

Step #54 - "apply node-pool-local":	Error Trace:	/builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.48.2/modules/terraform/apply.go:34 Step #54 - "apply node-pool-local":	/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.17.6/pkg/tft/terraform.go:571 Step #54 - "apply node-pool-local":	/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.17.6/pkg/tft/terraform.go:630 Step #54 - "apply node-pool-local":	/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.17.6/pkg/tft/terraform.go:669 Step #54 - "apply node-pool-local":	/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.17.6/pkg/utils/stages.go:31 Step #54 - "apply node-pool-local":	/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.17.6/pkg/tft/terraform.go:669 Step #54 - "apply node-pool-local":	Error:	Received unexpected error: Step #54 - "apply node-pool-local":	FatalError{Underlying: error while running command: exit status 1; Step #54 - "apply node-pool-local":	Error: error creating NodePool: googleapi: Error 400: 'pd-standard' with confidential storage mode: false is not compatible with the machine type c3-standard-4, please review the GCP online documentation for available persistent disk options. Step #54 - "apply node-pool-local":	Details: Step #54 - "apply node-pool-local":	[ Step #54 - "apply node-pool-local": { Step #54 - "apply node-pool-local": "@type": "type.googleapis.com/google.rpc.RequestInfo", Step #54 - "apply node-pool-local": "requestId": "0x16c806feae5789fb" Step #54 - "apply node-pool-local": } Step #54 - "apply node-pool-local":	] Step #54 - "apply node-pool-local":	, badRequest Step #54 - "apply node-pool-local": Step #54 - "apply node-pool-local": with module.example.module.gke.google_container_node_pool.pools["pool-05"], Step #54 - "apply node-pool-local": on ../../../modules/beta-public-cluster/cluster.tf line 644, in resource "google_container_node_pool" "pools": Step #54 - "apply node-pool-local": 644: resource "google_container_node_pool" "pools" { Step #54 - "apply node-pool-local":	}

Signed-off-by: drfaust92 <ilia.lazebnik@gmail.com>

DrFaust92 · 2025-05-05T23:34:39Z

apeabody Ive attempted to make changes to accommodate this node type

apeabody · 2025-05-16T22:05:15Z

/gcbrun

apeabody · 2025-05-16T23:58:15Z

Thanks @DrFaust92 - Current INT test result:

Step #55 - "verify node-pool-local":	Error:	Not equal: Step #55 - "verify node-pool-local":	expected: "1" Step #55 - "verify node-pool-local":	actual : "2" Step #55 - "verify node-pool-local": Step #55 - "verify node-pool-local":	Diff: Step #55 - "verify node-pool-local":	--- Expected Step #55 - "verify node-pool-local":	+++ Actual Step #55 - "verify node-pool-local":	@@ -1 +1 @@ Step #55 - "verify node-pool-local":	-1 Step #55 - "verify node-pool-local":	+2 Step #55 - "verify node-pool-local":	Test:	TestNodePool Step #55 - "verify node-pool-local":	Messages:	For node "pool-05" path "config.linuxNodeConfig.hugepages.hugepageSize1g" expected "2" to match fixture "1"

Signed-off-by: drfaust92 <ilia.lazebnik@gmail.com>

DrFaust92 · 2025-05-17T12:27:53Z

fail is correct, fixed test json. hopefully this is the last of it

apeabody · 2025-05-19T16:25:18Z

/gcbrun

apeabody · 2025-05-19T16:36:18Z

/gcbrun

apeabody · 2025-05-29T16:40:41Z

/gcbrun

apeabody · 2025-05-29T17:58:18Z

/gcbrun

apeabody · 2025-05-29T18:08:58Z

Hi @DrFaust92 - Can you please run make docker_generate_docs and update the PR. Thanks!

Signed-off-by: drfaust92 <ilia.lazebnik@gmail.com>

DrFaust92 · 2025-05-30T03:00:23Z

apeabody done

apeabody · 2025-05-30T16:24:03Z

/gcbrun

apeabody · 2025-05-30T21:04:00Z

/gcbrun

apeabody

Thanks for the contribution @DrFaust92!

DrFaust92 requested review from a team and ericyz as code owners December 21, 2024 18:18

DrFaust92 changed the title ~~feat: add support for hugepages_config + fix setting cgroup_name via "all""~~ feat: add support for hugepages_config Dec 21, 2024

apeabody requested changes Dec 24, 2024

View reviewed changes

autogen/main/cluster.tf.tmpl Outdated Show resolved Hide resolved

autogen/main/cluster.tf.tmpl Outdated Show resolved Hide resolved

apeabody self-assigned this Dec 24, 2024

DrFaust92 requested a review from apeabody January 2, 2025 15:19

DrFaust92 force-pushed the huge_pages branch from dcca780 to 580a3fb Compare January 2, 2025 20:42

DrFaust92 force-pushed the huge_pages branch 2 times, most recently from 99c20b9 to 51a780f Compare January 4, 2025 23:03

apeabody reviewed Jan 6, 2025

View reviewed changes

autogen/main/cluster.tf.tmpl Outdated Show resolved Hide resolved

apeabody removed their assignment Feb 3, 2025

DrFaust92 force-pushed the huge_pages branch from 51a780f to c629490 Compare February 27, 2025 02:50

DrFaust92 force-pushed the huge_pages branch from d380ab4 to f38654d Compare April 7, 2025 03:06

DrFaust92 requested a review from apeabody April 7, 2025 03:18

DrFaust92 mentioned this pull request Apr 7, 2025

feat(Hugepages_config): Support hugepages configuration in linux nodepools #2322

Closed

DrFaust92 marked this pull request as draft April 9, 2025 02:39

DrFaust92 force-pushed the huge_pages branch from 13c39f5 to 6ccbeb5 Compare April 25, 2025 23:24

DrFaust92 added 6 commits May 5, 2025 19:19

support setting hugepages config

8f38b8f

Signed-off-by: drfaust92 <ilia.lazebnik@gmail.com>

align tests

65a6cb0

Signed-off-by: drfaust92 <ilia.lazebnik@gmail.com>

CR comments

2a3fcfa

Signed-off-by: drfaust92 <ilia.lazebnik@gmail.com>

huge-pages

e6e7e27

Signed-off-by: drfaust92 <ilia.lazebnik@gmail.com>

try try()

9039d8c

Signed-off-by: drfaust92 <ilia.lazebnik@gmail.com>

attempt to fix test

9a4a35f

Signed-off-by: drfaust92 <ilia.lazebnik@gmail.com>

DrFaust92 force-pushed the huge_pages branch from 6ccbeb5 to 9a4a35f Compare May 5, 2025 23:34

Merge branch 'main' into huge_pages

ee3e75c

fix test result

5a1d5e7

Signed-off-by: drfaust92 <ilia.lazebnik@gmail.com>

Merge branch 'main' into huge_pages

186df1b

Merge branch 'main' into huge_pages

48fd2d0

regenerate docs

0b1e05f

Signed-off-by: drfaust92 <ilia.lazebnik@gmail.com>

apeabody self-assigned this May 30, 2025

Merge branch 'main' into huge_pages

8034bb9

apeabody approved these changes May 30, 2025

View reviewed changes

apeabody merged commit cf71718 into terraform-google-modules:main May 30, 2025
4 checks passed

release-please bot mentioned this pull request May 30, 2025

chore(main): release 37.0.0 #2335

Merged

feat: add support for hugepages_config #2223

feat: add support for hugepages_config #2223

Uh oh!

Conversation

DrFaust92 commented Dec 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

apeabody commented Dec 23, 2024

apeabody left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

DrFaust92 commented Jan 2, 2025

apeabody commented Jan 2, 2025

apeabody commented Jan 2, 2025

DrFaust92 commented Jan 3, 2025

apeabody commented Jan 3, 2025

DrFaust92 commented Jan 4, 2025

apeabody commented Jan 6, 2025

Uh oh!

apeabody commented Apr 3, 2025

DrFaust92 commented Apr 7, 2025

rawanbadawi commented Apr 7, 2025

apeabody commented Apr 7, 2025

apeabody commented Apr 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

apeabody commented Apr 29, 2025

DrFaust92 commented May 5, 2025

apeabody commented May 16, 2025

apeabody commented May 16, 2025

DrFaust92 commented May 17, 2025

apeabody commented May 19, 2025

apeabody commented May 19, 2025

apeabody commented May 29, 2025

apeabody commented May 29, 2025

apeabody commented May 29, 2025

DrFaust92 commented May 30, 2025

apeabody commented May 30, 2025

apeabody commented May 30, 2025

apeabody left a comment

Choose a reason for hiding this comment

Uh oh!

Labels

3 participants

DrFaust92 commented Dec 21, 2024 •

edited

Loading

apeabody commented Apr 7, 2025 •

edited

Loading