Skip to content

Convert logs integration to type: input #5347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hop-dev merged 5 commits into from
Apr 3, 2023
Merged

Convert logs integration to type: input #5347

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e43802b
Move logs integration to type input
hop-dev Feb 21, 2023
640a3ba
update changelog PR link
hop-dev Feb 21, 2023
2886940
pass linting
hop-dev Feb 22, 2023
6a14810
remove default dataset and update spec version
hop-dev Mar 28, 2023
a01a7b7
remove invalid fields
hop-dev Mar 28, 2023
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
0 ...ta_stream/log/agent/stream/stream.yml.hbs → packages/log/agent/input/input.yml.hbs
View file
Open in desktop
File renamed without changes.
5 changes: 5 additions & 0 deletions packages/log/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "2.0.0"
changes:
- description: Change from type "integration" to "input"
type: enhancement
link: https://github.com/elastic/integrations/pull/5347
- version: "1.1.2"
changes:
- description: Added categories and/or subcategories.
Expand Down
41 changes: 0 additions & 41 deletions packages/log/data_stream/log/manifest.yml
View file
Open in desktop

This file was deleted.

0 ...ages/log/data_stream/log/fields/agent.yml → packages/log/fields/agent.yml
View file
Open in desktop
File renamed without changes.
0 ...og/data_stream/log/fields/base-fields.yml → packages/log/fields/base-fields.yml
View file
Open in desktop
File renamed without changes.
54 changes: 44 additions & 10 deletions packages/log/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,57 @@
format_version: 1.0.0
format_version: 2.6.0
name: log
title: Custom Logs
description: >-
Collect custom logs with Elastic Agent.
type: integration
version: 1.1.2
release: ga
license: basic
type: input
version: 2.0.0
categories:
- custom
- custom_logs
conditions:
kibana.version: ^8.8.0
policy_templates:
- name: logs
title: Custom logs
type: logs
title: Custom log file
description: Collect your custom log files.
inputs:
- type: logfile
title: Custom log file
description: Collect your custom log files.
input: logfile
Copy link
Contributor

@BenB196 BenB196 Mar 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I saw this PR and was wondering (and this could be the wrong place to ask this). But I believe that the logfile input is deprecated. With this PR introducing a major version increase and a minimum version of 8.8.0. Would it make sense to migrate this to the filestream input and leverage the new take_over functionality added in 8.7.0?
Copy link
Contributor Author

@hop-dev hop-dev Mar 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @BenB196 sorry about the delay in replying.

I spoke to @rdner last week about this last week and his view was that as the takeover mode is still considered beta and it isn't ready for us to move the custom logs integration over just yet.
Copy link
Member

@rdner rdner Mar 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're going to release a beta package with the filestream+take_over first and let people to try it out.
template_path: input.yml.hbs
vars:
- name: paths
required: true
title: Log file path
description: Path to log files to be collected
type: text
multi: true
- name: data_stream.dataset
required: true
title: Dataset name
description: >
Set the name for your dataset. Changing the dataset will send the data to a different index. You can't use `-` in the name of a dataset and only valid characters for [Elasticsearch index names](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html).

type: text
- name: tags
type: text
title: Tags
description: Tags to include in the published event
multi: true
show_user: false
- name: processors
type: yaml
title: Processors
multi: false
required: false
show_user: false
description: >-
Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details.
- name: custom
title: Custom configurations
description: >
Here YAML configuration options can be used to be added to your configuration. Be careful using this as it might break your configuration file.

type: yaml
default: ""
icons:
- src: "/img/icon.svg"
type: "image/svg+xml"
Expand Down