I am building a tiny system for log analysis, and not able to install additional agent, so using already working rsyslog to send logs to elasticsearch.
My rsyslog and elasticsearch are separate VM's. These VM's have 1 common network: 10.8.4.0/24 .
rsyslog server has installed rsyslog-elasticsearch.x86_64 and configured to receive error logs from all servers in all infrastructure.
My ElasticSearch is running on localhost:9200, I have setup nginx as a proxy to it.
here are my configs:
[root@all-logs rsyslog.d]# cat 00-Send2Elastic.conf module(load="omelasticsearch") template(name="testTemplate" type="list" option.json="on") { constant(value="{") constant(value="\"timestamp\":\"") property(name="timereported" dateFormat="rfc3339") constant(value="\",\"message\":\"") property(name="msg") constant(value="\",\"host\":\"") property(name="hostname") constant(value="\",\"severity\":\"") property(name="syslogseverity-text") constant(value="\",\"facility\":\"") property(name="syslogfacility-text") constant(value="\",\"syslogtag\":\"") property(name="syslogtag") constant(value="\"}") } action(type="omelasticsearch" server="10.8.4.21" serverport="29200" template="testTemplate" searchIndex="test-index" searchType="test-type" bulkmode="on" maxbytes="100m" queue.type="linkedlist" queue.size="5000" queue.dequeuebatchsize="300" action.resumeretrycount="-1") # http://www.rsyslog.com/doc/v8-stable/configuration/modules/omelasticsearch.html # https://sematext.com/blog/recipe-rsyslog-elasticsearch-kibana/ And Nginx:
[root@test-elk centos]# cat /etc/nginx/conf.d/elastic.conf server { listen 10.8.4.21:29200; server_name elastic.host; allow 10.8.4.0/24; deny all; location / { proxy_pass http://localhost:9200; # proxy_http_version 1.1; # proxy_set_header Upgrade $http_upgrade; # proxy_set_header Connection 'upgrade'; # proxy_set_header Host $host; # proxy_cache_bypass $http_upgrade; } } Everything else seems have no issue :slight_smile: at first look.
Error log from rsyslogd -nd: https://pastebin.com/YDrTi2q5
error line:
0478.493507244:action 0 queue:Reg/w0: omelasticsearch: pWrkrData reply: '{"error":"Content-Type header [text/json; charset=utf-8] is not supported","status":406}' 
0481.499106700:action 0 queue:Reg/w0: omelasticsearch: error in elasticsearch reply: bulkmode insert does not return array, reply is: {"error":"Content-Type header [text/json; charset=utf-8] is not supported","status":406}