box_events: add limit parameter and pagination

[chemamartinez]

Proposed commit message

Following changes added:

• Add limit configuration for API queries, defaults to 500 as it is the maximum allowed value for the API.
• Add pagination settings to httpjson input.

API reference: https://developer.box.com/reference/get-events/

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

System test output

box-http-1 | {"level":"debug","ts":"2025-09-10T15:04:57.218Z","caller":"httpserver/httpserver.go:151","msg":"Rule #1 matched: request #1 => Request path: GET /2.0/events?limit=2&stream_position=0&stream_type=all, Request Headers: 'User-Agent: [Elastic-Filebeat/9.1.2 (linux; arm64; b036c1c565cf24c9b720605632234d20cb9dba60; 2025-08-11 14:01:10 +0000 UTC)]' 'Accept: [application/json]' 'Authorization: [Bearer c3FIOG9vSGV4VHo4QzAyg5T1JvNnJoZ3ExaVNyQWw6WjRsanRKZG5lQk9qUE1BVQ]' 'Accept-Encoding: [gzip]' 'Connection: [close]' , Request Body: ","address":":8080"} box-http-1 | {"level":"debug","ts":"2025-09-10T15:04:59.203Z","caller":"httpserver/httpserver.go:151","msg":"Rule #2 matched: request #1 => Request path: GET /2.0/events?limit=2&stream_position=1152922976252290816&stream_type=all, Request Headers: 'User-Agent: [Elastic-Filebeat/9.1.2 (linux; arm64; b036c1c565cf24c9b720605632234d20cb9dba60; 2025-08-11 14:01:10 +0000 UTC)]' 'Accept: [application/json]' 'Authorization: [Bearer c3FIOG9vSGV4VHo4QzAyg5T1JvNnJoZ3ExaVNyQWw6WjRsanRKZG5lQk9qUE1BVQ]' 'Accept-Encoding: [gzip]' 'Connection: [close]' , Request Body: ","address":":8080"} box-http-1 | {"level":"debug","ts":"2025-09-10T15:05:09.209Z","caller":"httpserver/httpserver.go:151","msg":"Rule #3 matched: request #1 => Request path: GET /2.0/events?limit=2&stream_position=3&stream_type=all, Request Headers: 'Accept-Encoding: [gzip]' 'Connection: [close]' 'User-Agent: [Elastic-Filebeat/9.1.2 (linux; arm64; b036c1c565cf24c9b720605632234d20cb9dba60; 2025-08-11 14:01:10 +0000 UTC)]' 'Accept: [application/json]' 'Authorization: [Bearer c3FIOG9vSGV4VHo4QzAyg5T1JvNnJoZ3ExaVNyQWw6WjRsanRKZG5lQk9qUE1BVQ]' , Request Body: ","address":":8080"} 

As interval is 10s, we can see the first two API calls are made during the first interval, and it wait another 10s to make the third one as the limit is higher than the count of returned events.

Request trace logs:

{"log.level":"debug","@timestamp":"2025-09-10T18:12:36.122Z","message":"HTTP request","transaction.id":"0PSN6GQTVPHHG-1","url.original":"http://svc-box-http:8080/oauth2/token","url.scheme":"http","url.path":"/oauth2/token","url.domain":"svc-box-http","url.port":"8080","url.query":"","http.request.method":"POST","http.request.header":{"Authorization":["Basic OGprNTEydzM5c2hvM3k5MjgybjBqNXZkeGw0eDZwbzg6OENONEoxVUx5NnB5UjZYQTZVOHBBWW0xQ0JVTk9OYzc="],"Content-Type":["application/x-www-form-urlencoded"]},"user_agent.original":"","http.request.body.content":"box_subject_id=19630872360&box_subject_type=user&grant_type=client_credentials","http.request.body.truncated":false,"http.request.body.bytes":78,"http.request.mime_type":"application/x-www-form-urlencoded","ecs.version":"1.6.0"} {"log.level":"debug","@timestamp":"2025-09-10T18:12:36.123Z","message":"HTTP response","transaction.id":"0PSN6GQTVPHHG-1","http.response.status_code":404,"http.response.body.content":"","http.response.body.truncated":false,"http.response.body.bytes":0,"http.response.mime_type":"","http.response.header":{"Content-Length":["0"],"Date":["Wed, 10 Sep 2025 18:12:36 GMT"]},"ecs.version":"1.6.0"} {"log.level":"debug","@timestamp":"2025-09-10T18:12:36.123Z","message":"HTTP request","transaction.id":"0PSN6GQTVPHHG-2","url.original":"http://svc-box-http:8080/oauth2/token","url.scheme":"http","url.path":"/oauth2/token","url.domain":"svc-box-http","url.port":"8080","url.query":"","http.request.method":"POST","http.request.header":{"Content-Type":["application/x-www-form-urlencoded"]},"user_agent.original":"","http.request.body.content":"box_subject_id=19630872360&box_subject_type=user&client_id=8jk512w39sho3y9282n0j5vdxl4x6po8&client_secret=8CN4J1ULy6pyR6XA6U8pAYm1CBUNONc7&grant_type=client_credentials","http.request.body.truncated":false,"http.request.body.bytes":168,"http.request.mime_type":"application/x-www-form-urlencoded","ecs.version":"1.6.0"} {"log.level":"debug","@timestamp":"2025-09-10T18:12:36.124Z","message":"HTTP response","transaction.id":"0PSN6GQTVPHHG-2","http.response.status_code":200,"http.response.body.content":"{\"access_token\":\"c3FIOG9vSGV4VHo4QzAyg5T1JvNnJoZ3ExaVNyQWw6WjRsanRKZG5lQk9qUE1BVQ\",\"expires_in\":3600,\"token_type\":\"bearer\",\"refresh_token\":\"c3FIOG9vSGV4VHo4QzAyg5T1JvNnJoZ3ExaVNyQWw6WjRsanRKZG5lQk9qUE1BVQ\",\"issued_token_type\":\"urn:ietf:params:oauth:token-type:access_token\"}","http.response.body.truncated":false,"http.response.body.bytes":274,"http.response.mime_type":"application/json","http.response.header":{"Content-Length":["274"],"Content-Type":["application/json"],"Date":["Wed, 10 Sep 2025 18:12:36 GMT"]},"ecs.version":"1.6.0"} {"log.level":"debug","@timestamp":"2025-09-10T18:12:36.124Z","message":"HTTP request","transaction.id":"0PSN6GQTVPHHG-3","url.original":"http://svc-box-http:8080/2.0/events?limit=2&stream_position=0&stream_type=all","url.scheme":"http","url.path":"/2.0/events","url.domain":"svc-box-http","url.port":"8080","url.query":"limit=2&stream_position=0&stream_type=all","http.request.method":"GET","http.request.header":{"Accept":["application/json"],"Authorization":["Bearer c3FIOG9vSGV4VHo4QzAyg5T1JvNnJoZ3ExaVNyQWw6WjRsanRKZG5lQk9qUE1BVQ"],"User-Agent":["Elastic-Filebeat/9.1.3 (linux; arm64; d9d2860c7593868e25d1b2da7da43793fe12c99e; 2025-08-23 00:02:13 +0000 UTC)"]},"user_agent.original":"Elastic-Filebeat/9.1.3 (linux; arm64; d9d2860c7593868e25d1b2da7da43793fe12c99e; 2025-08-23 00:02:13 +0000 UTC)","http.request.body.content":"","http.request.body.truncated":false,"http.request.body.bytes":0,"http.request.mime_type":"","ecs.version":"1.6.0"} {"log.level":"debug","@timestamp":"2025-09-10T18:12:36.125Z","message":"HTTP response","transaction.id":"0PSN6GQTVPHHG-3","http.response.status_code":200,"http.response.body.content":"{\"chunk_size\":2,\"entries\":[{\"source\":null,\"created_by\":{\"type\":\"user\",\"id\":\"2\",\"name\":\"Unknown User\",\"login\":\"\"},\"action_by\":null,\"created_at\":\"2019-12-20T11:38:56-08:00\",\"event_id\":\"97f1b31f-f143-4777-81f8-000000000001\",\"event_type\":\"SHIELD_ALERT\",\"ip_address\":\"10.1.2.3\",\"type\":\"event\",\"session_id\":null,\"additional_details\":{\"shield_alert\":{\"rule_category\":\"Anomalous Download\",\"rule_id\":123,\"rule_name\":\"Anomalous Download Rule\",\"risk_score\":77,\"alert_summary\":{\"description\":\"Significant increase in download content week over week, 9200% (25.04 MB) more than last week 12 additional files downloaded week over week)\",\"download_delta_size\":\"25 Mb\",\"download_delta_percent\":9200,\"historical_period\":{\"date_range\":{\"start_date\":\"2019-12-01T01:01:00-08:00\",\"end_date\":\"2019-12-08T01:01:00-08:00\"},\"download_size\":\"0 Mb\",\"downloaded_files_count\":1},\"anomaly_period\":{\"date_range\":{\"start_date\":\"2019-12-08T01:01:00-08:00\",\"end_date\":\"2019-12-15T01:01:00-08:00\"},\"download_size\":\"25 Mb\",\"downloaded_files_count\":13},\"download_ips\":[{\"ip\":\"1.128.0.0\"},{\"ip\":\"175.16.199.0\"}]},\"alert_id\":444,\"priority\":\"medium\",\"user\":{\"id\":567,\"name\":\"Some user\",\"email\":\"some@user.com\"},\"link\":\"https://cloud.app.box.com/master/shield/alerts/444\",\"created_at\":\"2019-12-20T11:38:16-08:00\"}}},{\"created_at\":\"2022-06-27T05:09:40-07:00\",\"created_by\":{\"id\":\"19530772260\",\"login\":\"info@elastic.co\",\"name\":\"Elastic Integrations\",\"type\":\"user\"},\"event_id\":\"97f1b31f-f143-4777-81f8-000000000002\",\"event_type\":\"ITEM_CREATE\",\"recorded_at\":\"2022-06-27T05:09:41-07:00\",\"session_id\":\"rzraadh3n273zc5f\",\"source\":{\"content_created_at\":\"2022-06-27T05:09:40-07:00\",\"content_modified_at\":\"2022-06-27T05:09:40-07:00\",\"created_at\":\"2022-06-27T05:09:40-07:00\",\"created_by\":{\"id\":\"19530772260\",\"login\":\"info@elastic.co\",\"name\":\"Elastic Integrations\",\"type\":\"user\"},\"description\":\"\",\"etag\":\"0\",\"folder_upload_email\":null,\"id\":\"166233012413\",\"item_status\":\"active\",\"modified_at\":\"2022-06-27T05:09:40-07:00\",\"modified_by\":{\"id\":\"19530772260\",\"login\":\"info@elastic.co\",\"name\":\"Elastic Integrations\",\"type\":\"user\"},\"name\":\"Platform App Diagnostics run on 2022-06-27 05-09-38 PDT\",\"owned_by\":{\"id\":\"19530772260\",\"login\":\"info@elastic.co\",\"name\":\"Elastic Integrations\",\"type\":\"user\"},\"parent\":{\"etag\":\"0\",\"id\":\"166232910591\",\"name\":\"Box Reports\",\"sequence_id\":\"0\",\"type\":\"folder\"},\"path_collection\":{\"entries\":[{\"etag\":null,\"id\":\"0\",\"name\":\"All Files\",\"sequence_id\":null,\"type\":\"folder\"},{\"etag\":\"0\",\"id\":\"166232910591\",\"name\":\"Box Reports\",\"sequence_id\":\"0\",\"type\":\"folder\"}],\"total_count\":2},\"purged_at\":null,\"sequence_id\":\"0\",\"shared_link\":null,\"size\":0,\"synced\":false,\"trashed_at\":null,\"type\":\"folder\"},\"type\":\"event\"}],\"next_stream_position\":1152922976252290800}","http.response.body.truncated":false,"http.response.body.bytes":2721,"http.response.mime_type":"application/json; charset=utf-8","http.response.header":{"Content-Type":["application/json; charset=utf-8"],"Date":["Wed, 10 Sep 2025 18:12:36 GMT"]},"ecs.version":"1.6.0"} {"log.level":"debug","@timestamp":"2025-09-10T18:12:38.115Z","message":"HTTP request","transaction.id":"0PSN6GQTVPHHG-4","url.original":"http://svc-box-http:8080/2.0/events?limit=2&stream_position=1152922976252290816&stream_type=all","url.scheme":"http","url.path":"/2.0/events","url.domain":"svc-box-http","url.port":"8080","url.query":"limit=2&stream_position=1152922976252290816&stream_type=all","http.request.method":"GET","http.request.header":{"Accept":["application/json"],"Authorization":["Bearer c3FIOG9vSGV4VHo4QzAyg5T1JvNnJoZ3ExaVNyQWw6WjRsanRKZG5lQk9qUE1BVQ"],"User-Agent":["Elastic-Filebeat/9.1.3 (linux; arm64; d9d2860c7593868e25d1b2da7da43793fe12c99e; 2025-08-23 00:02:13 +0000 UTC)"]},"user_agent.original":"Elastic-Filebeat/9.1.3 (linux; arm64; d9d2860c7593868e25d1b2da7da43793fe12c99e; 2025-08-23 00:02:13 +0000 UTC)","http.request.body.content":"","http.request.body.truncated":false,"http.request.body.bytes":0,"http.request.mime_type":"","ecs.version":"1.6.0"} {"log.level":"debug","@timestamp":"2025-09-10T18:12:38.122Z","message":"HTTP response","transaction.id":"0PSN6GQTVPHHG-4","http.response.status_code":200,"http.response.body.content":"{\"chunk_size\":1,\"entries\":[{\"source\":null,\"created_by\":{\"type\":\"user\",\"id\":\"2\",\"name\":\"Unknown User\",\"login\":\"\"},\"action_by\":null,\"created_at\":\"2019-12-20T11:38:56-08:00\",\"event_id\":\"97f1b31f-f143-4777-81f8-000000000003\",\"event_type\":\"SHIELD_ALERT\",\"ip_address\":\"10.1.2.3\",\"type\":\"event\",\"session_id\":null,\"additional_details\":{\"shield_alert\":{\"rule_category\":\"Anomalous Download\",\"rule_id\":123,\"rule_name\":\"Anomalous Download Rule\",\"risk_score\":77,\"alert_summary\":{\"description\":\"Significant increase in download content week over week, 9200% (25.04 MB) more than last week 12 additional files downloaded week over week)\",\"download_delta_size\":\"25 Mb\",\"download_delta_percent\":9200,\"historical_period\":{\"date_range\":{\"start_date\":\"2019-12-01T01:01:00-08:00\",\"end_date\":\"2019-12-08T01:01:00-08:00\"},\"download_size\":\"0 Mb\",\"downloaded_files_count\":1},\"anomaly_period\":{\"date_range\":{\"start_date\":\"2019-12-08T01:01:00-08:00\",\"end_date\":\"2019-12-15T01:01:00-08:00\"},\"download_size\":\"25 Mb\",\"downloaded_files_count\":13},\"download_ips\":[{\"ip\":\"1.128.0.0\"},{\"ip\":\"175.16.199.0\"}]},\"alert_id\":444,\"priority\":\"medium\",\"user\":{\"id\":567,\"name\":\"Some user\",\"email\":\"some@user.com\"},\"link\":\"https://cloud.app.box.com/master/shield/alerts/444\",\"created_at\":\"2019-12-20T11:38:16-08:00\"}}}],\"next_stream_position\":3}","http.response.body.truncated":false,"http.response.body.bytes":1301,"http.response.mime_type":"application/json; charset=utf-8","http.response.header":{"Content-Length":["1301"],"Content-Type":["application/json; charset=utf-8"],"Date":["Wed, 10 Sep 2025 18:12:38 GMT"]},"ecs.version":"1.6.0"} {"log.level":"debug","@timestamp":"2025-09-10T18:12:48.126Z","message":"HTTP request","transaction.id":"0PSN6GQTVPHHG-5","url.original":"http://svc-box-http:8080/2.0/events?limit=2&stream_position=3&stream_type=all","url.scheme":"http","url.path":"/2.0/events","url.domain":"svc-box-http","url.port":"8080","url.query":"limit=2&stream_position=3&stream_type=all","http.request.method":"GET","http.request.header":{"Accept":["application/json"],"Authorization":["Bearer c3FIOG9vSGV4VHo4QzAyg5T1JvNnJoZ3ExaVNyQWw6WjRsanRKZG5lQk9qUE1BVQ"],"User-Agent":["Elastic-Filebeat/9.1.3 (linux; arm64; d9d2860c7593868e25d1b2da7da43793fe12c99e; 2025-08-23 00:02:13 +0000 UTC)"]},"user_agent.original":"Elastic-Filebeat/9.1.3 (linux; arm64; d9d2860c7593868e25d1b2da7da43793fe12c99e; 2025-08-23 00:02:13 +0000 UTC)","http.request.body.content":"","http.request.body.truncated":false,"http.request.body.bytes":0,"http.request.mime_type":"","ecs.version":"1.6.0"} {"log.level":"debug","@timestamp":"2025-09-10T18:12:48.127Z","message":"HTTP response","transaction.id":"0PSN6GQTVPHHG-5","http.response.status_code":200,"http.response.body.content":"{\"chunk_size\":0,\"entries\":[{\"source\":null,\"created_by\":{\"type\":\"user\",\"id\":\"4\",\"name\":\"Unknown User\",\"login\":\"\"},\"action_by\":null,\"created_at\":\"2019-12-20T11:38:56-08:00\",\"event_id\":\"97f1b31f-f143-4777-81f8-000000000004\",\"event_type\":\"SHIELD_ALERT\",\"ip_address\":\"10.1.2.3\",\"type\":\"event\",\"session_id\":null,\"additional_details\":{\"shield_alert\":{\"rule_category\":\"Anomalous Download\",\"rule_id\":123,\"rule_name\":\"Anomalous Download Rule\",\"risk_score\":77,\"alert_summary\":{\"description\":\"Significant increase in download content week over week, 9200% (25.04 MB) more than last week 12 additional files downloaded week over week)\",\"download_delta_size\":\"25 Mb\",\"download_delta_percent\":9200,\"historical_period\":{\"date_range\":{\"start_date\":\"2019-12-01T01:01:00-08:00\",\"end_date\":\"2019-12-08T01:01:00-08:00\"},\"download_size\":\"0 Mb\",\"downloaded_files_count\":1},\"anomaly_period\":{\"date_range\":{\"start_date\":\"2019-12-08T01:01:00-08:00\",\"end_date\":\"2019-12-15T01:01:00-08:00\"},\"download_size\":\"25 Mb\",\"downloaded_files_count\":13},\"download_ips\":[{\"ip\":\"1.128.0.0\"},{\"ip\":\"175.16.199.0\"}]},\"alert_id\":444,\"priority\":\"medium\",\"user\":{\"id\":567,\"name\":\"Some user\",\"email\":\"some@user.com\"},\"link\":\"https://cloud.app.box.com/master/shield/alerts/444\",\"created_at\":\"2025-12-20T11:38:16-08:00\"}}}],\"next_stream_position\":4}","http.response.body.truncated":false,"http.response.body.bytes":1301,"http.response.mime_type":"application/json; charset=utf-8","http.response.header":{"Content-Length":["1301"],"Content-Type":["application/json; charset=utf-8"],"Date":["Wed, 10 Sep 2025 18:12:48 GMT"]},"ecs.version":"1.6.0"}

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[andrewkroh]

Part of the reason we missed this problem was that the test case had a low interval so it looked like pagination worked based on assert.hit_count. Now that pagination is working, let's make sure that the test is also covering:

• Expand the test to verify that it makes no more requests when the returned entries count is less than the limit.
• Expand the test to verify that the second periodic interval executes with the previously persisted cursor stream position.

---

Here's a rough sketch of what I was thinking: 45b02c3

[efd6]

This needs some field definitions:

test case failed: one or more errors found in document: [0] field "related.location.coordinates" is undefined, could be a multifield [1] field "related.location.type" is undefined, could be a multifield [2] field "threat.enrichments.indicator.geo.location.coordinates" is undefined, could be a multifield [3] field "threat.enrichments.indicator.geo.location.type" is undefined, could be a multifield 

[efd6]

I think this is also a bug fix against the absence of pagination.

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[efd6]

LGTM though nit and query. Also please wait for @andrewkroh

[efd6]

Is this because of geo's field reuse rules?

[chemamartinez]

Not sure what do you mean with the reuse rules. These fields are manually populated in the pipeline and mapping for them was missing, as they are not ECS.

[efd6]

The reuse rules are at the bottom of this page and they do not list geo as reusable under related.

[efd6]

This is not an ECS field, but it's been here almost since the package was originally published (0.3.0). It's unlikely to cause a conflict, but we may want to consider what we are going to do about it (not for now).

[andrewkroh]

These are all the non-ECS fields being added into ECS managed namespaces. There are three classes in my opinion.

Fields produced by beat processors that are always enabled (must keep).

• cloud.image.id
• host.containerized
• host.os.build
• host.os.codename

Never produced by beat processors or cel/httpjson inputs (can be removed).

• host.cpu.pct
• host.network.in.bytes
• host.network.in.packets
• host.network.out.bytes
• host.network.out.packets

New fields "created" in the box integration (should plan to remove).

• related.location.lat
• related.location.lon
• related.description
• related.indicator_typ

---

fydler analysis

ecsnamespace

Detect fields being added to namespaces controlled by ECS.

• packages/box_events/data_stream/events/fields/agent.yml:4 cloud.image.id is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/agent.yml:10 host.containerized is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/agent.yml:15 host.os.build is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/agent.yml:21 host.os.codename is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/agent.yml:27 host.cpu.pct is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/agent.yml:32 host.network.in.bytes is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/agent.yml:37 host.network.in.packets is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/agent.yml:42 host.network.out.bytes is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/agent.yml:47 host.network.out.packets is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/fields.yml:461 related.location.lat is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/fields.yml:464 related.location.lon is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/fields.yml:467 related.description is defined in an ECS managed namespace, custom fields must use the dataset's namespace

• packages/box_events/data_stream/events/fields/fields.yml:470 related.indicator_type is defined in an ECS managed namespace, custom fields must use the dataset's namespace

---

Generated by fydler v0.0.0-20250910175042-5a5ea22c4415 5a5ea22c44150397a503f84105948a6fc294214d

[efd6]

Agree

[andrewkroh]

@chemamartinez Can you please create an issue to follow-up on this problem.

[chemamartinez]

#15338

[efd6]

Final new line.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 88ec443

History

• 💚 Build #31081 succeeded 9bcfc8b
• 💔 Build #31038 failed ce8a9be

cc @chemamartinez

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[andrewkroh]

@chemamartinez Can you please post the request trace JSON logs from the system test to the PR description for review.

[chemamartinez]

@andrewkroh couldn't find the request trace logs but I added the debug logs from the test that I think it can be enough to review its behaviour.

[andrewkroh]

The request trace logs are in the state/data/logs/httpjson/*.ndjson on the agent that spins up for the system test. If you run test system -v --defer-cleanup=10m you'll have time to grab the log data.

[andrewkroh]

The request trace logs indicate that it's doing what I would expect. The one unvalidated assumption is that the API response when there is no more data is like

{ "chunk_size": 0, "entries": [], "next_stream_position": <big int> }

[chemamartinez]

The request trace logs are in the state/data/logs/httpjson/*.ndjson on the agent that spins up for the system test. If you run test system -v --defer-cleanup=10m you'll have time to grab the log data.

I looked there but forgot to defer the cleanup, I thought they are kept after the test finishes. Thanks!

[elastic-vault-github-plugin-prod]

Package box_events - 2.15.0 containing this change is available at https://epr.elastic.co/package/box_events/2.15.0/