Skip to content

Conversation

@mohitjha-elastic
Copy link
Collaborator

Proposed Commit Message

Add the support of Chrome Audit Events through Chrome Data Stream.
Update ECS version to 8.16 in all the pipeline.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

Clone integrations repo.
Install the elastic package locally.
Start the elastic stack using the elastic package.
Move to integrations/packages/google_workspace directory.
Run the following command to run tests.
elastic-package test -v

Related issues

Automated Test

elastic-package test system -d chrome -v 2024/12/20 11:12:51 DEBUG Enable verbose logging 2024/12/20 11:12:51 DEBUG latest version (cached): v0.108.0. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.108.0 (Timestamp 2024-12-20 11:12:29.576823167 +0530 IST) Run system tests for the package 2024/12/20 11:12:51 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}} 2024/12/20 11:12:51 DEBUG output command: /usr/bin/docker inspect 117984acac5e 2eb45b4a3faf 9cc47a043541 ff4d0d2bd957 c10b28ec9c72 cbad2987d1a0 e4f62b7b9f7a 659f7ed367b0 48daf08b3934 ae28ad8a8677 2024/12/20 11:12:51 DEBUG Connecting with Kibana host from current profile (profile: default, host: "https://127.0.0.1:5601") 2024/12/20 11:12:51 DEBUG GET https://127.0.0.1:5601/api/status 2024/12/20 11:12:52 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}} 2024/12/20 11:12:52 DEBUG output command: /usr/bin/docker inspect 117984acac5e 2eb45b4a3faf 9cc47a043541 ff4d0d2bd957 c10b28ec9c72 cbad2987d1a0 e4f62b7b9f7a 659f7ed367b0 48daf08b3934 ae28ad8a8677 2024/12/20 11:12:52 DEBUG Connecting with Elasticsearch host from current profile (profile: default, host: "https://127.0.0.1:9200") 2024/12/20 11:12:52 DEBUG Running suite... 2024/12/20 11:12:52 DEBUG Running system tests for data stream "chrome" 2024/12/20 11:12:52 DEBUG System runner: data stream "chrome" config file "test-default-config.yml" variant "" 2024/12/20 11:12:52 DEBUG Installing package... 2024/12/20 11:12:52 DEBUG GET https://127.0.0.1:5601/api/fleet/epm/packages/google_workspace 2024/12/20 11:12:52 DEBUG Build directory: /home/devuser/integrations/build/packages/google_workspace/2.29.0 2024/12/20 11:12:52 DEBUG Clear target directory (path: /home/devuser/integrations/build/packages/google_workspace/2.29.0) 2024/12/20 11:12:52 DEBUG Copy package content (source: /home/devuser/integrations/packages/google_workspace) 2024/12/20 11:12:52 DEBUG Copy license file if needed 2024/12/20 11:12:52 INFO License text found in "/home/devuser/integrations/LICENSE.txt" will be included in package 2024/12/20 11:12:52 DEBUG Encode dashboards 2024/12/20 11:12:52 DEBUG Resolve external fields 2024/12/20 11:12:52 DEBUG Package has external dependencies defined 2024/12/20 11:12:52 DEBUG data_stream/access_transparency/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/access_transparency/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/access_transparency/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/access_transparency/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/admin/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/admin/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/admin/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/admin/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/alert/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/alert/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/alert/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/chrome/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/chrome/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/chrome/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/context_aware_access/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/context_aware_access/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/context_aware_access/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/context_aware_access/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/device/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/device/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/device/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/device/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/drive/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/drive/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/drive/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/drive/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/gcp/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/gcp/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/gcp/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/gcp/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/group_enterprise/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/group_enterprise/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/group_enterprise/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/group_enterprise/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/groups/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/groups/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/groups/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/groups/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/login/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/login/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/login/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/login/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/rules/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/rules/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/rules/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/rules/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/saml/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/saml/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/saml/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/saml/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/token/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/token/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/token/fields/fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/token/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/user_accounts/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/user_accounts/fields/beats.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG data_stream/user_accounts/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:12:52 DEBUG Package doesn't have to import ECS mappings 2024/12/20 11:12:52 DEBUG Build zipped package 2024/12/20 11:12:52 DEBUG Compress using archiver.Zip (destination: /home/devuser/integrations/build/packages/google_workspace-2.29.0.zip) 2024/12/20 11:12:52 DEBUG Create work directory for archiving: /tmp/elastic-package-1714770766/google_workspace-2.29.0 2024/12/20 11:12:52 DEBUG Skip validation of the built .zip package 2024/12/20 11:12:52 DEBUG POST https://127.0.0.1:5601/api/fleet/epm/packages 2024/12/20 11:12:56 DEBUG Running tests sequentially 2024/12/20 11:12:56 DEBUG Using config: "default" 2024/12/20 11:12:56 DEBUG running test with configuration 'default' 2024/12/20 11:12:56 DEBUG creating enroll policy... 2024/12/20 11:12:56 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies 2024/12/20 11:13:00 DEBUG creating test policy... 2024/12/20 11:13:00 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies 2024/12/20 11:13:03 DEBUG setting up independent Elastic Agent... 2024/12/20 11:13:03 DEBUG setting up agent using Docker Compose agent deployer 2024/12/20 11:13:03 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:03 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:03 DEBUG output command: /usr/bin/docker network inspect elastic-package-stack_default 2024/12/20 11:13:03 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/deployer/docker-agent-google_workspace-chrome-89583/docker-agent-base.yml -p elastic-package-agent-google_workspace-chrome-89583 up --build -d [+] Running 2/2 ✔ Network elastic-package-agent-google_workspace-chrome-89583_default Created 0.1s ✔ Container elastic-package-agent-google_workspace-chrome-89583-elastic-agent-1 Started 0.3s 2024/12/20 11:13:04 DEBUG run command: /usr/bin/docker network connect elastic-package-stack_default elastic-package-agent-google_workspace-chrome-89583-elastic-agent-1 2024/12/20 11:13:04 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/deployer/docker-agent-google_workspace-chrome-89583/docker-agent-base.yml -p elastic-package-agent-google_workspace-chrome-89583 ps -a -q 2024/12/20 11:13:04 DEBUG Wait for healthy containers: de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:04 DEBUG output command: /usr/bin/docker inspect de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:04 DEBUG Container elastic-agent (de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751) status: unhealthy 2024/12/20 11:13:05 DEBUG Wait for healthy containers: de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:05 DEBUG output command: /usr/bin/docker inspect de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:05 DEBUG Container elastic-agent (de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751) status: unhealthy 2024/12/20 11:13:06 DEBUG Wait for healthy containers: de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:06 DEBUG output command: /usr/bin/docker inspect de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:06 DEBUG Container elastic-agent (de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751) status: unhealthy 2024/12/20 11:13:07 DEBUG Wait for healthy containers: de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:07 DEBUG output command: /usr/bin/docker inspect de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:07 DEBUG Container elastic-agent (de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751) status: unhealthy 2024/12/20 11:13:08 DEBUG Wait for healthy containers: de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:08 DEBUG output command: /usr/bin/docker inspect de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:08 DEBUG Container elastic-agent (de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751) status: unhealthy 2024/12/20 11:13:09 DEBUG Wait for healthy containers: de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:09 DEBUG output command: /usr/bin/docker inspect de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751 2024/12/20 11:13:09 DEBUG Container elastic-agent (de74cc24f3643f03f00354d386d4de37d11b3b8e4f7ada4688d6d878f3871751) status: running (health: healthy) 2024/12/20 11:13:09 DEBUG adding service container elastic-package-agent-google_workspace-chrome-89583-elastic-agent-1 internal ports to context 2024/12/20 11:13:09 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/deployer/docker-agent-google_workspace-chrome-89583/docker-agent-base.yml -p elastic-package-agent-google_workspace-chrome-89583 config 2024/12/20 11:13:09 DEBUG setting up service... 2024/12/20 11:13:09 DEBUG setting up service using Docker Compose service deployer 2024/12/20 11:13:09 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:09 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:09 DEBUG output command: /usr/bin/docker network inspect elastic-package-stack_default 2024/12/20 11:13:09 DEBUG running command: /usr/bin/docker compose -f /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-31916 up --build -d WARN[0000] /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion [+] Running 3/3 ✔ Network elastic-package-service-31916_default Created 0.1s ✔ Container elastic-package-service-31916-creds-1 Started 0.3s ✔ Container elastic-package-service-31916-google_workspace-1 Started 0.6s 2024/12/20 11:13:10 DEBUG running command: /usr/bin/docker compose -f /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-31916 ps -a -q WARN[0000] /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 2024/12/20 11:13:10 DEBUG Wait for healthy containers: 8c88e3a07cd2ad9d6d9023e6b7164370f492313e60d2c43772045e15b7f51e1a,7d7f8921610e69191002d6abcac39fe77e5bb92c5223e08c64a120c58e624abe 2024/12/20 11:13:10 DEBUG output command: /usr/bin/docker inspect 8c88e3a07cd2ad9d6d9023e6b7164370f492313e60d2c43772045e15b7f51e1a 7d7f8921610e69191002d6abcac39fe77e5bb92c5223e08c64a120c58e624abe 2024/12/20 11:13:10 DEBUG Container creds (8c88e3a07cd2ad9d6d9023e6b7164370f492313e60d2c43772045e15b7f51e1a) status: running (no health status) 2024/12/20 11:13:10 DEBUG Container google_workspace (7d7f8921610e69191002d6abcac39fe77e5bb92c5223e08c64a120c58e624abe) status: running (no health status) 2024/12/20 11:13:10 DEBUG run command: /usr/bin/docker network connect elastic-package-agent-google_workspace-chrome-89583_default elastic-package-service-31916-google_workspace-1 --alias svc-google_workspace 2024/12/20 11:13:10 DEBUG adding service container elastic-package-service-31916-google_workspace-1 internal ports to context 2024/12/20 11:13:10 DEBUG running command: /usr/bin/docker compose -f /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-31916 config WARN[0000] /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 2024/12/20 11:13:10 DEBUG adding package data stream to test policy... 2024/12/20 11:13:10 DEBUG POST https://127.0.0.1:5601/api/fleet/package_policies 2024/12/20 11:13:13 DEBUG deleting old data in data stream... 2024/12/20 11:13:13 DEBUG GET https://127.0.0.1:5601/api/fleet/agents 2024/12/20 11:13:13 DEBUG found 0 enrolled agent(s) 2024/12/20 11:13:14 DEBUG GET https://127.0.0.1:5601/api/fleet/agents 2024/12/20 11:13:14 DEBUG found 0 enrolled agent(s) 2024/12/20 11:13:15 DEBUG GET https://127.0.0.1:5601/api/fleet/agents 2024/12/20 11:13:15 DEBUG found 1 enrolled agent(s) 2024/12/20 11:13:15 DEBUG Selected enrolled agent "16b29b17-7973-4530-89a3-df25f0606117" 2024/12/20 11:13:15 DEBUG Set Debug log level to agent 2024/12/20 11:13:15 DEBUG POST https://127.0.0.1:5601/api/fleet/agents/16b29b17-7973-4530-89a3-df25f0606117/actions 2024/12/20 11:13:15 DEBUG GET https://127.0.0.1:5601/api/fleet/agent_policies/420fa235-6bf8-4ed3-a3a1-876bc3869633 2024/12/20 11:13:15 DEBUG assigning package data stream to agent... 2024/12/20 11:13:15 DEBUG PUT https://127.0.0.1:5601/api/fleet/agents/16b29b17-7973-4530-89a3-df25f0606117/reassign 2024/12/20 11:13:16 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/16b29b17-7973-4530-89a3-df25f0606117 2024/12/20 11:13:16 DEBUG Agent 16b29b17-7973-4530-89a3-df25f0606117 (Host: elastic-agent-89583): Policy ID 420fa235-6bf8-4ed3-a3a1-876bc3869633 LogLevel: info Status: updating 2024/12/20 11:13:16 DEBUG Wait until the policy (ID: 420fa235-6bf8-4ed3-a3a1-876bc3869633, revision: 2) is assigned to the agent (ID: 16b29b17-7973-4530-89a3-df25f0606117)... 2024/12/20 11:13:18 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/16b29b17-7973-4530-89a3-df25f0606117 2024/12/20 11:13:18 DEBUG Agent 16b29b17-7973-4530-89a3-df25f0606117 (Host: elastic-agent-89583): Policy ID 420fa235-6bf8-4ed3-a3a1-876bc3869633 LogLevel: info Status: updating 2024/12/20 11:13:18 DEBUG Wait until the policy (ID: 420fa235-6bf8-4ed3-a3a1-876bc3869633, revision: 2) is assigned to the agent (ID: 16b29b17-7973-4530-89a3-df25f0606117)... 2024/12/20 11:13:20 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/16b29b17-7973-4530-89a3-df25f0606117 2024/12/20 11:13:20 DEBUG Agent 16b29b17-7973-4530-89a3-df25f0606117 (Host: elastic-agent-89583): Policy ID 420fa235-6bf8-4ed3-a3a1-876bc3869633 LogLevel: info Status: updating 2024/12/20 11:13:20 DEBUG Wait until the policy (ID: 420fa235-6bf8-4ed3-a3a1-876bc3869633, revision: 2) is assigned to the agent (ID: 16b29b17-7973-4530-89a3-df25f0606117)... 2024/12/20 11:13:22 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/16b29b17-7973-4530-89a3-df25f0606117 2024/12/20 11:13:22 DEBUG Agent 16b29b17-7973-4530-89a3-df25f0606117 (Host: elastic-agent-89583): Policy ID 420fa235-6bf8-4ed3-a3a1-876bc3869633 LogLevel: info Status: online 2024/12/20 11:13:22 DEBUG Policy revision assigned to the agent (ID: 16b29b17-7973-4530-89a3-df25f0606117)... 2024/12/20 11:13:22 DEBUG checking for expected data in data stream (10m0s)... 2024/12/20 11:13:22 DEBUG found 0 hits in logs-google_workspace.chrome-62033 data stream 2024/12/20 11:13:23 DEBUG found 0 hits in logs-google_workspace.chrome-62033 data stream 2024/12/20 11:13:24 DEBUG found 0 hits in logs-google_workspace.chrome-62033 data stream 2024/12/20 11:13:25 DEBUG found 0 hits in logs-google_workspace.chrome-62033 data stream 2024/12/20 11:13:26 DEBUG found 2 hits in logs-google_workspace.chrome-62033 data stream 2024/12/20 11:13:30 DEBUG found 2 hits in logs-google_workspace.chrome-62033 data stream 2024/12/20 11:13:30 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:30 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:30 DEBUG running command: /usr/bin/docker compose -f /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-31916 ps -a -q google_workspace WARN[0000] /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 2024/12/20 11:13:30 DEBUG output command: /usr/bin/docker inspect 7d7f8921610e69191002d6abcac39fe77e5bb92c5223e08c64a120c58e624abe 2024/12/20 11:13:30 DEBUG Check whether or not synthetic source mode is enabled (data stream logs-google_workspace.chrome-62033)... 2024/12/20 11:13:30 DEBUG Data stream logs-google_workspace.chrome-62033 has synthetic source mode enabled: false 2024/12/20 11:13:30 DEBUG Checking failure store for data stream logs-google_workspace.chrome-62033 2024/12/20 11:13:30 DEBUG Found 0 docs in failure store for data stream logs-google_workspace.chrome-62033 2024/12/20 11:13:30 DEBUG Imported ECS fields definition from external schema for validation (embedded in package: false, stack uses ecs@mappings template: true) 2024/12/20 11:13:30 DEBUG assert hit count expected 2, observed 2 2024/12/20 11:13:31 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:31 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:31 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/deployer/docker-agent-google_workspace-chrome-89583/docker-agent-base.yml -p elastic-package-agent-google_workspace-chrome-89583 logs 2024/12/20 11:13:31 DEBUG tearing down service... 2024/12/20 11:13:31 DEBUG tearing down service using Docker Compose runner 2024/12/20 11:13:31 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:31 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:31 DEBUG running command: /usr/bin/docker compose -f /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-31916 stop WARN[0000] /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion [+] Stopping 2/2 ✔ Container elastic-package-service-31916-google_workspace-1 Stopped 0.4s ✔ Container elastic-package-service-31916-creds-1 Stopped 10.2s 2024/12/20 11:13:41 DEBUG running command: /usr/bin/docker compose -f /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-31916 logs WARN[0000] /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 2024/12/20 11:13:41 INFO Write container logs to file: /home/devuser/integrations/build/container-logs/google_workspace-1734673421967912033.log 2024/12/20 11:13:41 DEBUG running command: /usr/bin/docker compose -f /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-31916 down --volumes WARN[0000] /home/devuser/integrations/packages/google_workspace/_dev/deploy/docker/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion [+] Running 3/3 ✔ Container elastic-package-service-31916-google_workspace-1 Removed 0.0s ✔ Container elastic-package-service-31916-creds-1 Removed 0.0s ✔ Network elastic-package-service-31916_default Removed 0.3s 2024/12/20 11:13:42 DEBUG Deleting data stream for testing logs-google_workspace.chrome-62033 2024/12/20 11:13:42 DEBUG removing agent... 2024/12/20 11:13:42 DEBUG POST https://127.0.0.1:5601/api/fleet/agents/16b29b17-7973-4530-89a3-df25f0606117/unenroll 2024/12/20 11:13:44 DEBUG deleting test policies... 2024/12/20 11:13:44 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies/delete 2024/12/20 11:13:46 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies/delete 2024/12/20 11:13:48 DEBUG tearing down agent... 2024/12/20 11:13:48 DEBUG tearing down agent using Docker Compose runner 2024/12/20 11:13:48 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:48 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:48 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/deployer/docker-agent-google_workspace-chrome-89583/docker-agent-base.yml -p elastic-package-agent-google_workspace-chrome-89583 logs 2024/12/20 11:13:49 INFO Write container logs to file: /home/devuser/integrations/build/container-logs/elastic-agent-1734673429047397971.log 2024/12/20 11:13:49 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/deployer/docker-agent-google_workspace-chrome-89583/docker-agent-base.yml -p elastic-package-agent-google_workspace-chrome-89583 down --volumes [+] Running 2/2 ✔ Container elastic-package-agent-google_workspace-chrome-89583-elastic-agent-1 Removed 1.5s ✔ Network elastic-package-agent-google_workspace-chrome-89583_default Removed 0.3s 2024/12/20 11:13:50 DEBUG Dump Elastic stack data 2024/12/20 11:13:50 DEBUG Dump stack logs (location: /tmp/test-system-4110821416) 2024/12/20 11:13:50 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}} 2024/12/20 11:13:50 DEBUG output command: /usr/bin/docker inspect 117984acac5e 2eb45b4a3faf 9cc47a043541 ff4d0d2bd957 c10b28ec9c72 cbad2987d1a0 e4f62b7b9f7a 659f7ed367b0 48daf08b3934 ae28ad8a8677 2024/12/20 11:13:50 DEBUG Dump stack logs for elastic-agent 2024/12/20 11:13:50 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:51 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:51 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs elastic-agent 2024/12/20 11:13:51 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:51 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:51 DEBUG run command: /usr/bin/docker cp elastic-package-stack-elastic-agent-1:/usr/share/elastic-agent/state/data/logs/ /tmp/test-system-4110821416/logs/elastic-agent-internal 2024/12/20 11:13:51 DEBUG Dump stack logs for fleet-server 2024/12/20 11:13:51 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:51 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:51 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs fleet-server 2024/12/20 11:13:51 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:51 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:51 DEBUG run command: /usr/bin/docker cp elastic-package-stack-fleet-server-1:/usr/share/elastic-agent/state/data/logs/ /tmp/test-system-4110821416/logs/fleet-server-internal 2024/12/20 11:13:51 DEBUG Dump stack logs for kibana 2024/12/20 11:13:51 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:51 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:51 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs kibana 2024/12/20 11:13:51 DEBUG Dump stack logs for elasticsearch 2024/12/20 11:13:51 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:51 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:51 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs elasticsearch 2024/12/20 11:13:51 DEBUG Dump stack logs for package-registry 2024/12/20 11:13:52 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:13:52 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:13:52 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs package-registry 2024/12/20 11:13:52 DEBUG Uninstalling package... 2024/12/20 11:13:52 DEBUG GET https://127.0.0.1:5601/api/fleet/epm/packages/google_workspace 2024/12/20 11:13:52 DEBUG DELETE https://127.0.0.1:5601/api/fleet/epm/packages/google_workspace/2.29.0 --- Test results for package: google_workspace - START --- ╭──────────────────┬─────────────┬───────────┬───────────┬────────┬───────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├──────────────────┼─────────────┼───────────┼───────────┼────────┼───────────────┤ │ google_workspace │ chrome │ system │ default │ PASS │ 34.299160431s │ ╰──────────────────┴─────────────┴───────────┴───────────┴────────┴───────────────╯ --- Test results for package: google_workspace - END --- Done elastic-package test pipeline -d chrome -v 2024/12/20 11:15:53 DEBUG Enable verbose logging 2024/12/20 11:15:53 DEBUG latest version (cached): v0.108.0. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.108.0 (Timestamp 2024-12-20 11:12:29.576823167 +0530 IST) Run pipeline tests for the package 2024/12/20 11:15:53 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}} 2024/12/20 11:15:53 DEBUG output command: /usr/bin/docker inspect 117984acac5e 2eb45b4a3faf 9cc47a043541 ff4d0d2bd957 c10b28ec9c72 cbad2987d1a0 e4f62b7b9f7a 659f7ed367b0 48daf08b3934 ae28ad8a8677 2024/12/20 11:15:53 DEBUG Connecting with Elasticsearch host from current profile (profile: default, host: "https://127.0.0.1:9200") 2024/12/20 11:15:53 DEBUG Running tests sequentially 2024/12/20 11:15:53 DEBUG Imported ECS fields definition from external schema for validation (embedded in package: false, stack uses ecs@mappings template: true) 2024/12/20 11:15:53 DEBUG Dump Elastic stack data 2024/12/20 11:15:53 DEBUG Dump stack logs (location: ) 2024/12/20 11:15:53 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}} 2024/12/20 11:15:53 DEBUG output command: /usr/bin/docker inspect 117984acac5e 2eb45b4a3faf 9cc47a043541 ff4d0d2bd957 c10b28ec9c72 cbad2987d1a0 e4f62b7b9f7a 659f7ed367b0 48daf08b3934 ae28ad8a8677 2024/12/20 11:15:53 DEBUG Dump stack logs for elasticsearch 2024/12/20 11:15:53 DEBUG running command: /usr/bin/docker compose version --short 2024/12/20 11:15:54 DEBUG Determined Docker Compose version: 2.29.7 2024/12/20 11:15:54 DEBUG running command: /usr/bin/docker compose -f /home/devuser/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs --since 2024-12-20T05:45:53Z elasticsearch --- Test results for package: google_workspace - START --- ╭──────────────────┬─────────────┬───────────┬────────────────────────────────────────────┬────────┬──────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├──────────────────┼─────────────┼───────────┼────────────────────────────────────────────┼────────┼──────────────┤ │ google_workspace │ chrome │ pipeline │ (ingest pipeline warnings test-chrome.log) │ PASS │ 333.430271ms │ │ google_workspace │ chrome │ pipeline │ test-chrome.log │ PASS │ 177.642944ms │ ╰──────────────────┴─────────────┴───────────┴────────────────────────────────────────────┴────────┴──────────────╯ --- Test results for package: google_workspace - END --- Done elastic-package test static -d chrome -v 2024/12/20 11:16:18 DEBUG Enable verbose logging 2024/12/20 11:16:18 DEBUG latest version (cached): v0.108.0. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.108.0 (Timestamp 2024-12-20 11:12:29.576823167 +0530 IST) Run static tests for the package 2024/12/20 11:16:18 DEBUG Running tests sequentially 2024/12/20 11:16:19 DEBUG Imported ECS fields definition from external schema for validation (embedded in package: false, stack uses ecs@mappings template: true) --- Test results for package: google_workspace - START --- ╭──────────────────┬─────────────┬───────────┬──────────────────────────┬────────┬──────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├──────────────────┼─────────────┼───────────┼──────────────────────────┼────────┼──────────────┤ │ google_workspace │ chrome │ static │ Verify sample_event.json │ PASS │ 158.448615ms │ ╰──────────────────┴─────────────┴───────────┴──────────────────────────┴────────┴──────────────╯ --- Test results for package: google_workspace - END --- Done elastic-package test asset -v 2024/12/20 11:16:50 DEBUG Enable verbose logging 2024/12/20 11:16:50 DEBUG latest version (cached): v0.108.0. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.108.0 (Timestamp 2024-12-20 11:12:29.576823167 +0530 IST) Run asset tests for the package 2024/12/20 11:16:50 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}} 2024/12/20 11:16:50 DEBUG output command: /usr/bin/docker inspect 117984acac5e 2eb45b4a3faf 9cc47a043541 ff4d0d2bd957 c10b28ec9c72 cbad2987d1a0 e4f62b7b9f7a 659f7ed367b0 48daf08b3934 ae28ad8a8677 2024/12/20 11:16:50 DEBUG Connecting with Kibana host from current profile (profile: default, host: "https://127.0.0.1:5601") 2024/12/20 11:16:50 DEBUG GET https://127.0.0.1:5601/api/status 2024/12/20 11:16:50 DEBUG Running tests sequentially 2024/12/20 11:16:50 DEBUG installing package... 2024/12/20 11:16:50 DEBUG GET https://127.0.0.1:5601/api/fleet/epm/packages/google_workspace 2024/12/20 11:16:50 DEBUG Build directory: /home/devuser/integrations/build/packages/google_workspace/2.29.0 2024/12/20 11:16:50 DEBUG Clear target directory (path: /home/devuser/integrations/build/packages/google_workspace/2.29.0) 2024/12/20 11:16:50 DEBUG Copy package content (source: /home/devuser/integrations/packages/google_workspace) 2024/12/20 11:16:50 DEBUG Copy license file if needed 2024/12/20 11:16:50 INFO License text found in "/home/devuser/integrations/LICENSE.txt" will be included in package 2024/12/20 11:16:50 DEBUG Encode dashboards 2024/12/20 11:16:50 DEBUG Resolve external fields 2024/12/20 11:16:50 DEBUG Package has external dependencies defined 2024/12/20 11:16:50 DEBUG data_stream/access_transparency/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/access_transparency/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/access_transparency/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/access_transparency/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/admin/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/admin/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/admin/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/admin/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/alert/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/alert/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/alert/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/chrome/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/chrome/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/chrome/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/context_aware_access/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/context_aware_access/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/context_aware_access/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/context_aware_access/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/device/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/device/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/device/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/device/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/drive/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/drive/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/drive/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/drive/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/gcp/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/gcp/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/gcp/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/gcp/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/group_enterprise/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/group_enterprise/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/group_enterprise/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/group_enterprise/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/groups/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/groups/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/groups/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/groups/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/login/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/login/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/login/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/login/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/rules/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/rules/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/rules/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/rules/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/saml/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/saml/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/saml/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/saml/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/token/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/token/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/token/fields/fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/token/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/user_accounts/fields/base-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/user_accounts/fields/beats.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG data_stream/user_accounts/fields/package-fields.yml: source file hasn't been changed 2024/12/20 11:16:50 DEBUG Package doesn't have to import ECS mappings 2024/12/20 11:16:50 DEBUG Build zipped package 2024/12/20 11:16:50 DEBUG Compress using archiver.Zip (destination: /home/devuser/integrations/build/packages/google_workspace-2.29.0.zip) 2024/12/20 11:16:50 DEBUG Create work directory for archiving: /tmp/elastic-package-3752645195/google_workspace-2.29.0 2024/12/20 11:16:50 DEBUG Skip validation of the built .zip package 2024/12/20 11:16:50 DEBUG POST https://127.0.0.1:5601/api/fleet/epm/packages 2024/12/20 11:16:54 DEBUG GET https://127.0.0.1:5601/api/fleet/epm/packages/google_workspace 2024/12/20 11:16:54 DEBUG removing package... 2024/12/20 11:16:54 DEBUG GET https://127.0.0.1:5601/api/fleet/epm/packages/google_workspace 2024/12/20 11:16:54 DEBUG DELETE https://127.0.0.1:5601/api/fleet/epm/packages/google_workspace/2.29.0 --- Test results for package: google_workspace - START --- ╭──────────────────┬──────────────────────┬───────────┬─────────────────────────────────────────────────────────────────────────────┬────────┬──────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├──────────────────┼──────────────────────┼───────────┼─────────────────────────────────────────────────────────────────────────────┼────────┼──────────────┤ │ google_workspace │ │ asset │ dashboard google_workspace-26c10e40-8cbc-11ed-add3-0fec96545f1c is loaded │ PASS │ 2.387µs │ │ google_workspace │ │ asset │ dashboard google_workspace-3be0b490-3430-11ed-9f31-c9178ccae8cd is loaded │ PASS │ 541ns │ │ google_workspace │ │ asset │ dashboard google_workspace-3fb94480-8cbc-11ed-add3-0fec96545f1c is loaded │ PASS │ 505ns │ │ google_workspace │ │ asset │ dashboard google_workspace-4c5a4cc0-8cbc-11ed-add3-0fec96545f1c is loaded │ PASS │ 473ns │ │ google_workspace │ │ asset │ dashboard google_workspace-7b55f304-7a6b-4131-bc36-591e35732394 is loaded │ PASS │ 479ns │ │ google_workspace │ │ asset │ dashboard google_workspace-8925d900-3b43-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 499ns │ │ google_workspace │ │ asset │ dashboard google_workspace-ca3ff140-3b3f-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 540ns │ │ google_workspace │ │ asset │ dashboard google_workspace-d3cf6d50-3bfb-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 514ns │ │ google_workspace │ │ asset │ dashboard google_workspace-d6287d50-0107-11ed-825d-df764a9c0c57 is loaded │ PASS │ 605ns │ │ google_workspace │ │ asset │ dashboard google_workspace-d79f1730-9585-11ed-82ba-c3ec829933e4 is loaded │ PASS │ 549ns │ │ google_workspace │ │ asset │ dashboard google_workspace-e9a62e70-9583-11ed-82ba-c3ec829933e4 is loaded │ PASS │ 561ns │ │ google_workspace │ │ asset │ dashboard google_workspace-ec193fd0-3ab6-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 575ns │ │ google_workspace │ │ asset │ dashboard google_workspace-f163f270-3b13-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 595ns │ │ google_workspace │ │ asset │ dashboard google_workspace-f8210e80-3b28-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 606ns │ │ google_workspace │ │ asset │ search google_workspace-10b37c00-3c03-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 501ns │ │ google_workspace │ │ asset │ search google_workspace-1cac9ed0-3b2f-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 529ns │ │ google_workspace │ │ asset │ search google_workspace-2c0d5bc0-3b0d-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 534ns │ │ google_workspace │ │ asset │ search google_workspace-2c40f770-3b24-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 551ns │ │ google_workspace │ │ asset │ search google_workspace-3ceeeba0-3c04-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 590ns │ │ google_workspace │ │ asset │ search google_workspace-676e6980-3bfc-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 572ns │ │ google_workspace │ │ asset │ search google_workspace-7ab25b80-3b13-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 603ns │ │ google_workspace │ │ asset │ search google_workspace-8817b016-61c1-4d10-bdc2-e30e9fd93d4c is loaded │ PASS │ 577ns │ │ google_workspace │ │ asset │ search google_workspace-8e8f98d0-3c02-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 592ns │ │ google_workspace │ │ asset │ search google_workspace-8ec40930-0110-11ed-825d-df764a9c0c57 is loaded │ PASS │ 606ns │ │ google_workspace │ │ asset │ search google_workspace-b95cf166-2f93-42c0-bf69-6ce3e2309a5b is loaded │ PASS │ 620ns │ │ google_workspace │ │ asset │ search google_workspace-c3960ae0-9586-11ed-82ba-c3ec829933e4 is loaded │ PASS │ 653ns │ │ google_workspace │ │ asset │ search google_workspace-d542c8e0-3bfa-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 646ns │ │ google_workspace │ │ asset │ search google_workspace-e013b790-010b-11ed-825d-df764a9c0c57 is loaded │ PASS │ 675ns │ │ google_workspace │ │ asset │ search google_workspace-e3d44490-3bfc-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 677ns │ │ google_workspace │ │ asset │ search google_workspace-ebb44680-3bf5-11ed-8bdd-f5c5df6c1370 is loaded │ PASS │ 697ns │ │ google_workspace │ access_transparency │ asset │ index_template logs-google_workspace.access_transparency is loaded │ PASS │ 335ns │ │ google_workspace │ access_transparency │ asset │ ingest_pipeline logs-google_workspace.access_transparency-2.29.0 is loaded │ PASS │ 223ns │ │ google_workspace │ admin │ asset │ index_template logs-google_workspace.admin is loaded │ PASS │ 348ns │ │ google_workspace │ admin │ asset │ ingest_pipeline logs-google_workspace.admin-2.29.0 is loaded │ PASS │ 223ns │ │ google_workspace │ alert │ asset │ index_template logs-google_workspace.alert is loaded │ PASS │ 280ns │ │ google_workspace │ alert │ asset │ ingest_pipeline logs-google_workspace.alert-2.29.0 is loaded │ PASS │ 210ns │ │ google_workspace │ chrome │ asset │ index_template logs-google_workspace.chrome is loaded │ PASS │ 280ns │ │ google_workspace │ chrome │ asset │ ingest_pipeline logs-google_workspace.chrome-2.29.0 is loaded │ PASS │ 209ns │ │ google_workspace │ context_aware_access │ asset │ index_template logs-google_workspace.context_aware_access is loaded │ PASS │ 584ns │ │ google_workspace │ context_aware_access │ asset │ ingest_pipeline logs-google_workspace.context_aware_access-2.29.0 is loaded │ PASS │ 208ns │ │ google_workspace │ device │ asset │ index_template logs-google_workspace.device is loaded │ PASS │ 353ns │ │ google_workspace │ device │ asset │ ingest_pipeline logs-google_workspace.device-2.29.0 is loaded │ PASS │ 253ns │ │ google_workspace │ drive │ asset │ index_template logs-google_workspace.drive is loaded │ PASS │ 404ns │ │ google_workspace │ drive │ asset │ ingest_pipeline logs-google_workspace.drive-2.29.0 is loaded │ PASS │ 282ns │ │ google_workspace │ gcp │ asset │ index_template logs-google_workspace.gcp is loaded │ PASS │ 340ns │ │ google_workspace │ gcp │ asset │ ingest_pipeline logs-google_workspace.gcp-2.29.0 is loaded │ PASS │ 222ns │ │ google_workspace │ group_enterprise │ asset │ index_template logs-google_workspace.group_enterprise is loaded │ PASS │ 830ns │ │ google_workspace │ group_enterprise │ asset │ ingest_pipeline logs-google_workspace.group_enterprise-2.29.0 is loaded │ PASS │ 250ns │ │ google_workspace │ groups │ asset │ index_template logs-google_workspace.groups is loaded │ PASS │ 457ns │ │ google_workspace │ groups │ asset │ ingest_pipeline logs-google_workspace.groups-2.29.0 is loaded │ PASS │ 294ns │ │ google_workspace │ login │ asset │ index_template logs-google_workspace.login is loaded │ PASS │ 419ns │ │ google_workspace │ login │ asset │ ingest_pipeline logs-google_workspace.login-2.29.0 is loaded │ PASS │ 276ns │ │ google_workspace │ rules │ asset │ index_template logs-google_workspace.rules is loaded │ PASS │ 515ns │ │ google_workspace │ rules │ asset │ ingest_pipeline logs-google_workspace.rules-2.29.0 is loaded │ PASS │ 294ns │ │ google_workspace │ saml │ asset │ index_template logs-google_workspace.saml is loaded │ PASS │ 438ns │ │ google_workspace │ saml │ asset │ ingest_pipeline logs-google_workspace.saml-2.29.0 is loaded │ PASS │ 338ns │ │ google_workspace │ token │ asset │ index_template logs-google_workspace.token is loaded │ PASS │ 515ns │ │ google_workspace │ token │ asset │ ingest_pipeline logs-google_workspace.token-2.29.0 is loaded │ PASS │ 357ns │ │ google_workspace │ user_accounts │ asset │ index_template logs-google_workspace.user_accounts is loaded │ PASS │ 558ns │ │ google_workspace │ user_accounts │ asset │ ingest_pipeline logs-google_workspace.user_accounts-2.29.0 is loaded │ PASS │ 285ns │ ╰──────────────────┴──────────────────────┴───────────┴─────────────────────────────────────────────────────────────────────────────┴────────┴──────────────╯ --- Test results for package: google_workspace - END --- Done 
@mohitjha-elastic mohitjha-elastic requested a review from a team as a code owner December 20, 2024 07:28
@mohitjha-elastic mohitjha-elastic changed the title [Google Workspace] Add support of Chrome Audit Events [google_workspace] Add support of Chrome Audit Events Dec 20, 2024
@andrewkroh andrewkroh added Crest Contributions from Crest developement team. dashboard Relates to a Kibana dashboard bug, enhancement, or modification. enhancement New feature or request Integration:google_workspace Google Workspace Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Dec 20, 2024
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@kcreddy
Copy link
Contributor

kcreddy commented Dec 20, 2024

/test

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should rename the parameter to batch_size to be consistent with other integrations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this exception made?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mapping of Chrome events differs, which is why it is highlighted here. Specifically, our approach to breaking down the events is different; we are splitting them based on the events array and have identified alternative mappings that are more appropriate.

For example, we mapped the TIMESTAMP field to the ECS field @timestamp rather than using id.timestamp.

Copy link
Contributor

@kcreddy kcreddy Dec 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The httpjson split logic seems to be the same as this CEL program for all datastreams.
I think it makes sense to follow this standard mappings even for the Chrome events unless we know there is a bug.

If there is an additional field called TIMESTAMP, maybe we should create a custom field inside google_workspace.chrome. From the sample event, id.timestamp value is same as TIMESTAMP.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mohitjha-elastic Also the system tests are failing, please check. Possibly reference to page_size

@kcreddy
Copy link
Contributor

kcreddy commented Dec 23, 2024

/test

Copy link
Contributor

@kcreddy kcreddy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should match the ECS fields inside Chrome events just like the existing data-streams. #12171 (comment)

@kcreddy
Copy link
Contributor

kcreddy commented Dec 24, 2024

/test

@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link

💚 Build Succeeded

History

  • 💔 Build #19768 failed 33418bdfa6f55702cb316216e580947c4b837e19
  • 💚 Build #19743 succeeded cc21c320a8baff62d605dd30eb9122aa7480b27a
@kcreddy kcreddy merged commit 16bdfda into elastic:main Dec 24, 2024
5 checks passed
@elastic-vault-github-plugin-prod

Package google_workspace - 2.29.0 containing this change is available at https://epr.elastic.co/package/google_workspace/2.29.0/

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
Add the support of Chrome Audit Events through Chrome Data Stream. Update ECS version to 8.16 in all the pipeline.
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
Add the support of Chrome Audit Events through Chrome Data Stream. Update ECS version to 8.16 in all the pipeline.
@mohitjha-elastic mohitjha-elastic deleted the google_workspace-2.29.0 branch February 10, 2025 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Crest Contributions from Crest developement team. dashboard Relates to a Kibana dashboard bug, enhancement, or modification. enhancement New feature or request Integration:google_workspace Google Workspace Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

4 participants