Skip to content

[GCP] Add redis data stream #4325

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 13 commits into from
Oct 5, 2022
Merged

[GCP] Add redis data stream #4325

merged 13 commits into from
Oct 5, 2022

Conversation

@gpop63
Copy link
Contributor

@gpop63 gpop63 commented Sep 27, 2022

What does this PR do?

Adds redis data stream.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots
@elasticmachine
Copy link

elasticmachine commented Sep 27, 2022 •
edited
Loading

🚀 Benchmarks report

Package gcp 👍(4) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
dns 1908.4 1605.14 -303.26 (-15.89%) 💔

To see the full report comment with /test benchmark fullreport
@elasticmachine
Copy link

elasticmachine commented Sep 27, 2022 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-10-05T12:10:47.074+0000

  • Duration: 20 min 57 sec

Test stats 🧪

Test Results
Failed 0
Passed 84
Skipped 0
Total 84

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.
@elasticmachine
Copy link

elasticmachine commented Sep 27, 2022 •
edited
Loading

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (5/5) 💚
Files 100.0% (5/5) 💚 2.53
Classes 100.0% (5/5) 💚 2.53
Methods 90.099% (91/101) 👎 -0.135
Lines 95.752% (1375/1436) 👍 4.15
Conditionals 100.0% (0/0) 💚
@gpop63 gpop63 marked this pull request as ready for review September 28, 2022 14:36
@gpop63 gpop63 requested review from a team as code owners September 28, 2022 14:36
@endorama endorama mentioned this pull request Sep 28, 2022
Merged
11 tasks
endorama
endorama requested changes Sep 28, 2022
View reviewed changes
packages/gcp/changelog.yml Outdated
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "2.12.6"
Copy link
Contributor

@endorama endorama Sep 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- version: "2.12.6"
- version: "2.12.0"
packages/gcp/data_stream/redis/elasticsearch/ingest_pipeline/default.yml
@@ -0,0 +1,131 @@
---
description: Pipeline for parsing GCP Redis metrics.
processors:
Copy link
Contributor

@endorama endorama Sep 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the fields here should have proper suffixes, like mentioned in https://www.elastic.co/guide/en/beats/devguide/current/event-conventions.html
packages/gcp/data_stream/redis/fields/fields.yml
- name: gcp.redis
description: Google Cloud Redis metrics
type: group
fields:
Copy link
Contributor

@endorama endorama Sep 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add unit, metric_type and dimension where appropriate (see #4099 for more details)
packages/gcp/manifest.yml Outdated
name: gcp
title: Google Cloud Platform
version: "2.11.6"
version: "2.12.6"
Copy link
Contributor

@endorama endorama Sep 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
version: "2.12.6"
version: "2.12.0"
@gpop63 gpop63 requested a review from endorama September 28, 2022 19:24
@endorama endorama added the Integration:gcp Google Cloud Platform label Sep 29, 2022
@endorama endorama mentioned this pull request Sep 29, 2022
Merged
4 tasks
@endorama
Copy link
Contributor

endorama commented Sep 29, 2022

@gpop63 please fix the CI (needs a build)
endorama
endorama approved these changes Sep 29, 2022
View reviewed changes
Copy link
Contributor

@endorama endorama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from the red CI looks good to me!
@endorama
Copy link
Contributor

endorama commented Oct 5, 2022

/test
@gpop63
Copy link
Contributor Author

gpop63 commented Oct 5, 2022

Tested data stream with new field units etc and works as expected.
@endorama endorama merged commit 2891ec1 into elastic:main Oct 5, 2022
efd6
efd6 reviewed Oct 6, 2022
View reviewed changes
packages/gcp/data_stream/redis/fields/agent.yml
group: 2
description: Fields related to the cloud or infrastructure the events are coming from.
footnote: 'Examples: If Metricbeat is running on an GCP Compute VM and fetches data from its host, the cloud info contains the data about this machine. If Metricbeat runs on a remote machine outside the cloud and fetches data from a service running in the cloud, the field contains cloud data from the machine the service is running on.'
type: group
Copy link
Contributor

@efd6 efd6 Oct 6, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of these fields are duplicates of fields that are defined in ecs.yml. Fixed in #4397. You can check for duplicates in a package if you are running elastic-package v0.65.0 with format version 2.0.0 (do this temporarily before pushing).
@andrewkroh andrewkroh mentioned this pull request Oct 20, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Integration:gcp Google Cloud Platform

4 participants

@gpop63 @elasticmachine @endorama @efd6