[O11y][Salesforce] Add custom field for _id and update dashboard #6041

rajvi-patel-22 · 2023-05-01T05:20:58Z

Enhancement

What does this PR do?

Create one custom field salesforce.login.document_id and copy the value of _id field into salesforce.login.document_id.
Replace _id field with salesforce.login.document_id in [Logs Salesforce] Login dashboard.
Replace _id field with event_id in [Logs Salesforce] Setup Audit Trail dashboard.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Screenshots

Related Issues

Note When user will upgrade to the newer version, salesforce.login.document_id field will be added only to the new incoming documents. The data that is already present in indices will not get updated. Hence, the Changes made in the setup [Logs Salesforce] visualization will show documents which have salesforce.login.document_id field present (documents that have been ingested after upgrading to the latest version). Reindexing can be recommended in case the user wish to see the old information following the change (adoption of document_id) as the new field to store the values presently stored under _id. Please refer to the troubleshooting section of README.

elasticmachine · 2023-05-01T05:37:58Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-05-05T09:13:43.564+0000
Duration: 17 min 4 sec

Test stats 🧪

Test	Results
Failed	0
Passed	34
Skipped	0
Total	34

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2023-05-01T05:38:10Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`6/6`)	💚
Files	100.0% (`6/6`)	💚 50.0
Classes	100.0% (`6/6`)	💚 50.0
Methods	98.649% (`73/74`)	👍 28.649
Lines	96.281% (`1372/1425`)	👍 26.047
Conditionals	100.0% (`0/0`)	💚

agithomas · 2023-05-01T13:27:12Z

packages/salesforce/_dev/build/docs/README.md

 2. Click on the Connected App name created by the user to generate the client id and client secret (Refer to Client Key and Client Secret for Authentication) under the Master Label.
 3. Click on Edit Policies, and select `Relax IP restrictions` from the dropdown for IP Relaxation.

+- If **Login events table [Logs Salesforce]** does not display older documents after upgrading to ``0.7.0`` or later versions, then this issue can be solved by reindexing the ``login_rest`` data stream's indices.


I believe, there exist there are two cases under the troubleshooting section.

Please separate out two cases . Currently it is difficult to find out the 2 cases of troubleshooting . Formatting must be adjusted too.

agithomas · 2023-05-01T13:27:40Z

packages/salesforce/_dev/build/docs/README.md

+}
+```
+
+5. Verify data is reindexed completely.


How to you verify this?

This is the example response of reindex API. If there would be any failure it would be verified via failures parameter. Otherwise total should be equal to addition of created and updated documents.

{ "took": 308, "timed_out": false, "total": 100, "updated": 0, "created": 100, "deleted": 0, "batches": 1, "version_conflicts": 0, "noops": 0, "retries": { "bulk": 0, "search": 0 }, "throttled_millis": 0, "requests_per_second": -1, "throttled_until_millis": 0, "failures": [] }

agithomas · 2023-05-01T13:29:56Z

packages/salesforce/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: 0.7.0


Shouldn't this be a bugfix with version number 0.6.1?

The meta fields are supported in dashboards till 8.4.0. But after 8.4.0, the dashboard shows an error. Should we consider it bugfix because this issue occurred due to functional changes of meta fields?

agithomas · 2023-05-01T13:30:58Z

packages/salesforce/docs/README.md

 2. Click on the Connected App name created by the user to generate the client id and client secret (Refer to Client Key and Client Secret for Authentication) under the Master Label.
 3. Click on Edit Policies, and select `Relax IP restrictions` from the dropdown for IP Relaxation.

+- If **Login events table [Logs Salesforce]** does not display older documents after upgrading to ``0.7.0`` or later versions, then this issue can be solved by reindexing the ``login_rest`` data stream's indices.


This depends on the discussion whether the package version is 0.6.1 or 0.7.0. TBD

agithomas · 2023-05-01T13:33:37Z

packages/salesforce/changelog.yml

 # newer versions go on top
+- version: 0.7.0
+ changes:
+ - description: Add custom field for _id and update dashboards.


The description is too technical and around some low level details.

I believe the core of the issue is the PR indent to make the dashboard compatible with certain version of Lens. It is what must be put under the changelog

description: Make dashboard compatible with meta fields for Kibana 8.6.0 and above

Let me know if this looks good to you?

agithomas · 2023-05-01T13:33:52Z

packages/salesforce/manifest.yml

 name: salesforce
 title: Salesforce
-version: 0.6.0
+version: 0.7.0


…y-salesforce-update-id-field-in-dashboard

agithomas · 2023-05-05T05:34:42Z

packages/salesforce/docs/README.md


+### Missing old events in **Login events table [Logs Salesforce]** panel
+
+If **Login events table [Logs Salesforce]** does not display older documents after upgrading to ``0.8.0`` or later versions, then this issue can be solved by reindexing the ``login_rest`` data stream's indices.


Please remove this phrase - [Logs Salesforce]

agithomas · 2023-05-05T05:35:31Z

packages/salesforce/docs/README.md

 2. Click on the Connected App name created by the user to generate the client id and client secret (Refer to Client Key and Client Secret for Authentication) under the Master Label.
 3. Click on Edit Policies, and select `Relax IP restrictions` from the dropdown for IP Relaxation.

+### Missing old events in **Login events table [Logs Salesforce]** panel


Please remove this phrase - [Logs Salesforce]

agithomas · 2023-05-05T05:39:09Z

packages/salesforce/docs/README.md

+```
+
+```
+DELETE /_data_stream/<data_stream>


Quiet a lot of things happening here.
You can have delete datastream and index template as a next step.

Also, Keep the examples of deleting datastream and index template together , keep templates of delete datastream & index template together

I have updated the PR.

agikthomas

LGTM!

agikthomas

LGTM

agithomas

LGTM!

elasticmachine · 2023-05-08T05:54:56Z

Package salesforce - 0.8.0 containing this change is available at https://epr.elastic.co/search?package=salesforce

Add custom field for _id and update dashboards

bf58225

rajvi-patel-22 added 2 commits May 1, 2023 12:15

Update troubleshooting section of README

3503cb7

Update changelog

1870938

rajvi-patel-22 marked this pull request as ready for review May 1, 2023 11:11

rajvi-patel-22 requested a review from a team as a code owner May 1, 2023 11:11

kush-elastic added the Integration:salesforce Salesforce label May 1, 2023

kush-elastic requested a review from agithomas May 1, 2023 11:25

agithomas requested a review from lalit-satapathy May 1, 2023 11:31

milan-elastic approved these changes May 1, 2023

View reviewed changes

agithomas reviewed May 1, 2023

View reviewed changes

Merge branch 'main' of github.com:rajvi-elastic/integrations into oll…

ae533f7

…y-salesforce-update-id-field-in-dashboard

rajvi-patel-22 requested a review from agithomas May 3, 2023 11:19

agithomas reviewed May 5, 2023

View reviewed changes

Update troubleshooting steps in README

9fcbecc

rajvi-patel-22 requested a review from agithomas May 5, 2023 09:16

agikthomas approved these changes May 8, 2023

View reviewed changes

agithomas approved these changes May 8, 2023

View reviewed changes

kush-elastic merged commit cc0ea78 into elastic:main May 8, 2023

rajvi-patel-22 mentioned this pull request May 8, 2023

[O11y][Salesforce] Update _id field in Salesforce dashboard #5876

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[O11y][Salesforce] Add custom field for _id and update dashboard #6041

[O11y][Salesforce] Add custom field for _id and update dashboard #6041

Uh oh!

rajvi-patel-22 commented May 1, 2023 •

edited

Loading

elasticmachine commented May 1, 2023 •

edited

Loading

Build stats

Test stats 🧪

elasticmachine commented May 1, 2023 •

edited

Loading

agithomas May 1, 2023

agithomas May 1, 2023

rajvi-patel-22 May 2, 2023

agithomas May 1, 2023

rajvi-patel-22 May 2, 2023

agithomas May 1, 2023

agithomas May 1, 2023

rajvi-patel-22 May 2, 2023

agithomas May 1, 2023

agithomas May 5, 2023

agithomas May 5, 2023

agithomas May 5, 2023

rajvi-patel-22 May 5, 2023

agikthomas left a comment

agikthomas left a comment

agithomas left a comment

elasticmachine commented May 8, 2023

Labels

6 participants


		### Missing old events in Login events table [Logs Salesforce] panel

		If Login events table [Logs Salesforce] does not display older documents after upgrading to ``0.8.0`` or later versions, then this issue can be solved by reindexing the ``login_rest`` data stream's indices.

[O11y][Salesforce] Add custom field for _id and update dashboard #6041

[O11y][Salesforce] Add custom field for _id and update dashboard #6041

Uh oh!

Conversation

rajvi-patel-22 commented May 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist

Screenshots

Related Issues

elasticmachine commented May 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

elasticmachine commented May 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agikthomas left a comment

Choose a reason for hiding this comment

agikthomas left a comment

Choose a reason for hiding this comment

agithomas left a comment

Choose a reason for hiding this comment

elasticmachine commented May 8, 2023

Labels

6 participants

rajvi-patel-22 commented May 1, 2023 •

edited

Loading

elasticmachine commented May 1, 2023 •

edited

Loading

elasticmachine commented May 1, 2023 •

edited

Loading