Add cribl integration package #8487
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
kgeller added enhancement kgeller changed the title cribl integration package 
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
🤖 GitHub commentsExpand to view the GitHub comments
To re-run your PR in the CI, just comment with:
|
🌐 Coverage report
|
| Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
Contributor
taylor-swanson left a comment
taylor-swanson left a comment There was a problem hiding this comment.
Just a comment about the ECS version, otherwise LGTM
packages/cribl/data_stream/logs/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved
taylor-swanson approved these changes Dec 4, 2023
| Package cribl - 0.1.0 containing this change is available at https://epr.elastic.co/search?package=cribl |
| Package cribl - 0.1.1 containing this change is available at https://epr.elastic.co/search?package=cribl |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Proposed commit message
This PR adds the first phase of the new Cribl integration.
This integration is unique in that it 1) has no input to run, since Cribl pushes data directly to elasticsearch via the _bulk API via their elasticsearch output, and 2) it doesn't really have its own pipeline, since the goal is to utilize existing integration's pipelines. It enables users to utilize the
rerouteprocessor to 'forward' logs from Cribl to a specific integration's datastream.This first phase contains mostly documentation on how to set everything up using a custom pipeline. The second phase will contain a more elegant UI so that users can have a simpler experience configuring.
Checklist
I have verified that all data streams collect metrics or logs.changelog.ymlfile.Author's Checklist
How to test this PR locally
The steps to set everything up are documented in the integration's README
Then I just utilized the _bulk API (since that's what the cribl output uses) to verify the routing works
Related issues
Screenshots
Integration main view
Instructions