[New Integration] Goflow2 integration #10561
Conversation
| 💚 CLA has been signed |
andrewkroh added New Integration 
jamiehynds added the Team:Security-Deployment and Devices 
| Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices) |
andrewkroh removed the needs CLA 
| /test |
| /test |
…grations into goflow2-integration
.github/CODEOWNERS Outdated
| /packages/github @elastic/security-service-integrations | ||
| /packages/gitlab @elastic/security-service-integrations | ||
| /packages/golang @elastic/obs-infraobs-integrations | ||
| /packages/goflow2 @elastic/security-service-integrations |
There was a problem hiding this comment.
this one should be @elastic/sec-deployment-and-devices
| @HaveSec @marioschaefer could you please add some tests for the |
on my list for today. |
@pkoutsovasilis took a bit longer, but now finished. all tests pass. |
| /test |
@taylor-swanson codeowner fixed |
packages/goflow2/data_stream/sflow/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved
packages/goflow2/data_stream/sflow/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved
packages/goflow2/data_stream/sflow/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved
packages/goflow2/data_stream/sflow/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved
taylor-swanson left a comment
taylor-swanson left a comment There was a problem hiding this comment.
One other note, I'm checking to see if we need a fields/ecs.yml file or not. In the past, we did, otherwise elastic-package would complain about the ECS fields, but that doesn't seem like the case now.
packages/goflow2/data_stream/sflow/_dev/test/system/test-logfile-config.yml Show resolved Hide resolved
| @taylor-swanson could you pls check? |
| /test |
🚀 Benchmarks reportTo see the full report comment with |
taylor-swanson left a comment
taylor-swanson left a comment There was a problem hiding this comment.
Do we anticipate this list of fields ever to change? Specifically, could any of these fields become optional in the future?
# File: /etc/goflow2/mapping.yaml formatter: fields: # list of fields to format in JSON - type - time_flow_start_ns - sampler_address - sequence_num - in_if - out_if - src_addr - dst_addr - etype - proto - src_port - dst_port - src_vlan - dst_vlan - sampling_rate - bytes If yes, then I would advocate for null checks/ignore_missing on every (applicable) processor. Otherwise I'm fine with leaving those off for now. Since we've specified this in the documentation, we know they should be there and it isn't as critical to add null checks and ignore_missing everywhere.
packages/goflow2/data_stream/sflow/elasticsearch/ingest_pipeline/default.yml Show resolved Hide resolved
…rocessors, added more testsamples
| /test |
|
💚 Build Succeeded
History
|
| Package goflow2 - 0.1.0 containing this change is available at https://epr.elastic.co/search?package=goflow2 |
- Add the GoFlow2 integration to monitor goflow2 logs - Add initial sflow data stream to ingest sflow logs from goflow2 - Add system and pipeline tests --------- Co-authored-by: Christian Hilgers <christian@hilgers.ag> Co-authored-by: Mario Schäfer <mario.schaefer@indevis.de>
- Add the GoFlow2 integration to monitor goflow2 logs - Add initial sflow data stream to ingest sflow logs from goflow2 - Add system and pipeline tests --------- Co-authored-by: Christian Hilgers <christian@hilgers.ag> Co-authored-by: Mario Schäfer <mario.schaefer@indevis.de>
8 participants
The GoFlow2 integration allows you to monitor goflow2 logs. At the moment only goflow2 sflow logs are supported.
Checklist
changelog.ymlfile.How to test this PR locally
goflow2 -format json -listen "sflow://:6343" -mapping /root/sflow/goflow2/mapping.yaml -transport.file /var/log/sflow/goflow2/goflow2.log