2

I'm trying to configure rsyslog to send logs to logstash and then forward them to elasticsearch.

I have create a config file /etc/rsyslog.d/60-output.conf with the following content:

*.* @localhost:10514;json-template

and a template file /etc/rsyslog.d/01-json-template.conf with the following content:

template(name="json-template" type="list") { constant(value="{") constant(value="\"@timestamp\":\"") property(name="timereported" dateFormat="rfc3339") constant(value="\",\"@version\":\"1") constant(value="\",\"message\":\"") property(name="msg" format="json") constant(value="\",\"sysloghost\":\"") property(name="hostname") constant(value="\",\"severity\":\"") property(name="syslogseverity-text") constant(value="\",\"facility\":\"") property(name="syslogfacility-text") constant(value="\",\"programname\":\"") property(name="programname") constant(value="\",\"procid\":\"") property(name="procid") constant(value="\"}\n") }

Then I restarted rsyslog service. And for logstash I created a config file /etc/logstash/conf.d/logstash.conf with the following content:

input { udp { port => 10514 codec => "json" type => "rsyslog" } } filter { } output { if [type] == "rsyslog" { elasticsearch { hosts => [ "localhost:9200" ] } } }

Then I restarted logstash.

When I run sudo netstat -tulpn | grep 10514 I get this:

user@rsyslog-server:/var/log$ sudo netstat -tulpn | grep 10514 udp 0 0 0.0.0.0:10514 0.0.0.0:* 5327/java

so Logstash is listening on port 10514.

To verify the elasticsearch input I run curl -XGET 'http://localhost:9200/logstash-*/_search?q=*&pretty' but this doesn't return any results:

{ "took" : 0, "timed_out" : false, "_shards" : { "total" : 0, "successful" : 0, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 0, "relation" : "eq" }, "max_score" : 0.0, "hits" : [ ] } }

I'm using:

  • rsyslogd 8.2208.0 (aka 2022.08)
  • logstash 7.17.8
  • elastisearch 7.17.8

How can I solve this ?

Edit:

I run logstash as following to see the logs:

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf --verbose

and this is the output:

sing bundled JDK: /usr/share/logstash/jdk OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console [WARN ] 2023-01-01 18:50:48.417 [main] runner - DEPRECATION WARNING: The flag ["--verbose"] has been deprecated, please use "--log.level=info" instead. [INFO ] 2023-01-01 18:50:48.423 [main] runner - Starting Logstash {"logstash.version"=>"7.17.8", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.17+8 on 11.0.17+8 +indy +jit [linux-x86_64]"} [INFO ] 2023-01-01 18:50:48.426 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djdk.io.File.enableADS=true, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true] [WARN ] 2023-01-01 18:50:48.689 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified [INFO ] 2023-01-01 18:50:49.682 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false} [INFO ] 2023-01-01 18:50:50.739 [Converge PipelineAction::Create<main>] Reflections - Reflections took 50 ms to scan 1 urls, producing 119 keys and 419 values [WARN ] 2023-01-01 18:50:51.350 [Converge PipelineAction::Create<main>] plain - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode. [WARN ] 2023-01-01 18:50:51.396 [Converge PipelineAction::Create<main>] udp - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode. [WARN ] 2023-01-01 18:50:51.438 [Converge PipelineAction::Create<main>] plain - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode. [WARN ] 2023-01-01 18:50:51.479 [Converge PipelineAction::Create<main>] elasticsearch - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode. [INFO ] 2023-01-01 18:50:51.658 [[main]-pipeline-manager] elasticsearch - New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]} [INFO ] 2023-01-01 18:50:51.953 [[main]-pipeline-manager] elasticsearch - Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}} [WARN ] 2023-01-01 18:50:52.174 [[main]-pipeline-manager] elasticsearch - Restored connection to ES instance {:url=>"http://localhost:9200/"} [INFO ] 2023-01-01 18:50:52.199 [[main]-pipeline-manager] elasticsearch - Elasticsearch version determined (7.17.8) {:es_version=>7} [WARN ] 2023-01-01 18:50:52.200 [[main]-pipeline-manager] elasticsearch - Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7} [INFO ] 2023-01-01 18:50:52.276 [[main]-pipeline-manager] elasticsearch - Config is not compliant with data streams. `data_stream => auto` resolved to `false` [INFO ] 2023-01-01 18:50:52.362 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/logstash.conf"], :thread=>"#<Thread:0x332001c0 run>"} [INFO ] 2023-01-01 18:50:52.380 [Ruby-0-Thread-10: :1] elasticsearch - Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled} [INFO ] 2023-01-01 18:50:53.026 [[main]-pipeline-manager] javapipeline - Pipeline Java execution initialization time {"seconds"=>0.66} [INFO ] 2023-01-01 18:50:53.093 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"} [WARN ] 2023-01-01 18:50:53.158 [[main]<udp] plain - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode. [WARN ] 2023-01-01 18:50:53.186 [[main]<udp] plain - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode. [INFO ] 2023-01-01 18:50:53.188 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]} [INFO ] 2023-01-01 18:50:53.206 [[main]<udp] udp - Starting UDP listener {:address=>"0.0.0.0:10514"} [INFO ] 2023-01-01 18:50:53.244 [[main]<udp] udp - UDP listener started {:address=>"0.0.0.0:10514", :receive_buffer_bytes=>"106496", :queue_size=>"2000"} ^C[WARN ] 2023-01-01 19:02:34.721 [SIGINT handler] runner - SIGINT received. Shutting down. [INFO ] 2023-01-01 19:02:35.721 [[main]-pipeline-manager] javapipeline - Pipeline terminated {"pipeline.id"=>"main"} [INFO ] 2023-01-01 19:02:35.798 [Converge PipelineAction::StopAndDelete<main>] pipelinesregistry - Removed pipeline from registry successfully {:pipeline_id=>:main} [INFO ] 2023-01-01 19:02:35.873 [LogStash::Runner] runner - Logstash shut down.

sudo journalctl -u logstash -f :

janv. 01 18:50:32 rsyslog-server systemd[1]: logstash.service: Deactivated successfully. janv. 01 18:50:32 rsyslog-server systemd[1]: Stopped logstash. janv. 01 18:50:32 rsyslog-server systemd[1]: logstash.service: Consumed 1min 31.847s CPU time. janv. 01 19:04:10 rsyslog-server systemd[1]: Started logstash. janv. 01 19:04:10 rsyslog-server logstash[16111]: Using bundled JDK: /usr/share/logstash/jdk janv. 01 19:04:10 rsyslog-server logstash[16111]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. janv. 01 19:04:22 rsyslog-server logstash[16111]: Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties janv. 01 19:04:22 rsyslog-server logstash[16111]: [2023-01-01T19:04:22,731][INFO ][logstash.runner ] Log4j configuration path used is: /etc/logstash/log4j2.properties janv. 01 19:04:22 rsyslog-server logstash[16111]: [2023-01-01T19:04:22,740][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.17.8", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.17+8 on 11.0.17+8 +indy +jit [linux-x86_64]"} janv. 01 19:04:22 rsyslog-server logstash[16111]: [2023-01-01T19:04:22,741][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djdk.io.File.enableADS=true, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true] janv. 01 19:04:24 rsyslog-server logstash[16111]: [2023-01-01T19:04:24,002][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false} janv. 01 19:04:24 rsyslog-server logstash[16111]: [2023-01-01T19:04:24,940][INFO ][org.reflections.Reflections] Reflections took 56 ms to scan 1 urls, producing 119 keys and 419 values janv. 01 19:04:25 rsyslog-server logstash[16111]: [2023-01-01T19:04:25,731][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]} janv. 01 19:04:25 rsyslog-server logstash[16111]: [2023-01-01T19:04:25,965][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}} janv. 01 19:04:26 rsyslog-server logstash[16111]: [2023-01-01T19:04:26,138][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://localhost:9200/"} janv. 01 19:04:26 rsyslog-server logstash[16111]: [2023-01-01T19:04:26,148][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch version determined (7.17.8) {:es_version=>7} janv. 01 19:04:26 rsyslog-server logstash[16111]: [2023-01-01T19:04:26,153][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7} janv. 01 19:04:26 rsyslog-server logstash[16111]: [2023-01-01T19:04:26,206][INFO ][logstash.outputs.elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false` janv. 01 19:04:26 rsyslog-server logstash[16111]: [2023-01-01T19:04:26,256][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled} janv. 01 19:04:26 rsyslog-server logstash[16111]: [2023-01-01T19:04:26,276][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/logstash.conf"], :thread=>"#<Thread:0x53d2107b run>"} janv. 01 19:04:26 rsyslog-server logstash[16111]: [2023-01-01T19:04:26,466][INFO ][logstash.outputs.elasticsearch][main] Created rollover alias {:name=>"<logstash-{now/d}-000001>"} janv. 01 19:04:27 rsyslog-server logstash[16111]: [2023-01-01T19:04:27,008][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>0.73} janv. 01 19:04:27 rsyslog-server logstash[16111]: [2023-01-01T19:04:27,069][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"} janv. 01 19:04:27 rsyslog-server logstash[16111]: [2023-01-01T19:04:27,127][INFO ][logstash.inputs.udp ][main][9e1b2cf1672d2dd22a032525fa695bdf0d4d43163b1fd297f3b99f01f316ae38] Starting UDP listener {:address=>"0.0.0.0:10514"} janv. 01 19:04:27 rsyslog-server logstash[16111]: [2023-01-01T19:04:27,157][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]} janv. 01 19:04:27 rsyslog-server logstash[16111]: [2023-01-01T19:04:27,160][INFO ][logstash.inputs.udp ][main][9e1b2cf1672d2dd22a032525fa695bdf0d4d43163b1fd297f3b99f01f316ae38] UDP listener started {:address=>"0.0.0.0:10514", :receive_buffer_bytes=>"106496", :queue_size=>"2000"} ^C
Improve this question
2
  • Please add logs as well. Commented Jan 1, 2023 at 15:56
  • @asktyagi please check my updates Commented Jan 1, 2023 at 18:05

0

Reset to default

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.