Questions tagged [elasticsearch]
Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.
387 questions
0 votes
1 answer
91 views
Monitor backend of httpd's load balancer
I have a httpd load balancer with the following configuration (extracted from the example Clusters and Balancers): <Proxy balancer://myset> BalancerMember http://www2.example.com:8080 ...
- 153
0 votes
1 answer
19 views
ELK login instead of token
I'am trying to learn ELK and I launched binary on windows with all default config I have launched Elastic with elastic.bat and in second Kibana with kibana.dat. I had authentitication with token and ...
- 35
0 votes
0 answers
36 views
Fluent Bit v4 and Elasticsearch v8
I'm running a Fluent Bit v3.0.2 instance using opentelemetry as input. It filters and modifies logs written by another service and outputs it to an elasticsearch instance. I noticed v3.0.2 is quite ...
0 votes
1 answer
132 views
Uninstall Winlogbeat (ElasticSearch) Windows Service
How can I uninstall winlogbeat Windows Service? A sample script can be find below: https://github.com/elastic/beats/blob/master/dev-tools/packaging/templates/windows/uninstall-service.ps1.tmpl But how ...
- 286
0 votes
0 answers
86 views
Elasticsearch losing logs during DDoS
Here are my setup details: I have Elasticsearch, Kibana, Filebeat, and the Apache module for Filebeat running on a single server along with my website (only one website is hosted). The stack is ...
0 votes
0 answers
150 views
How can I configure Filebeat to add the `orchestrator.*` fields?
I'm using Filebeat in Kubernetes to ship the logs to Elasticsearch. I've noticed that the log messages are missing the orchestrator.cluster.name field. None of the orchestrator fields are being set ...
0 votes
0 answers
107 views
aws ssh tunnel to valkey cluster via EC2 ubuntu not working
I have a Valkey cluster I am trying to connect to from my local Windows machine via an SSH tunnel and I would appreciate some help troubleshooting since I am unable to reach the cluster. What I have ...
- 1
3 votes
1 answer
288 views
ElasticSearch cluster master data deleted
I have an Elastic cluster running on Kubernetes with 3 master nodes (2 with persistent storage and 1 voting-only node without persistent storage) and 20 data nodes. I accidentally deleted the data on ...
- 31
0 votes
1 answer
244 views
ELK Stack with Docker and multiple server for UDP logging
I'm trying to setup an ELK stack (kibana, logstash and elastic search) in portainer which should receive loggings from pc's around the world. The thing I'm not sure is how the proper setup should look ...
- 101
0 votes
0 answers
101 views
Error deploying ELK on kubernetes
I was following the Elasticsearch documentation (Deploy an Elasticsearch cluster | Elastic Cloud on Kubernetes [2.14] | Elastic) I am getting the below error while deploying the elasticsearch cluster ...
0 votes
1 answer
1k views
"SSL routines::wrong version number" after elasticsearch upgrade
This is a pretty new ELK installation. I originally installed v7.x because I mistakenly thought it was required; but then upgraded in place by editing the source URL for the Elastic PPA. The upgrade ...
- 506
1 vote
1 answer
171 views
How to Configure hideAnnouncements in kibana.yml?
In Kibana's GUI settings page, there is an entry called "Hide announcements" with the field named hideAnnouncements. The URL path for this page is /app/management/kibana/settings Then in the ...
- 398
2 votes
1 answer
834 views
Intermittent High CPU Spikes and Network Latency on Proxmox VMs Running Kubernetes
Summary: I recently migrated a Kubernetes cluster from bare metal to a Proxmox VM on the same physical host. Since the migration, the VM experiences intermittent high CPU spikes and network latency, ...
- 183
1 vote
0 answers
510 views
Issue between Fleet-managed Elastic agent on external server and Fleet in k8s
My fleet-managed agent can't send logs (they are always dropped) to ES, but does send the metrics. The error message seen in the agent's status is a 504 Gateway Time-out. Additionally, we see this ...
- 11
0 votes
1 answer
110 views
EFK Data is getting lost on pod restart
I am facing the data persistence issue for EFK setup when pod gets restarted. I have setup the master, data and client pod. Please have a look at below configurations. master.yaml apiVersion: apps/v1 ...
- 101
0 votes
1 answer
128 views
Connecting multiple PHP containers to a single shared Elasticsearch container
14 PHP containers php_1, ..., php_14 and 14 NGINX containers nginx_1, ..., nginx_14 are running on a server. The PHP container php_i is only connected to nginx_i and all NGINX containers are connected ...
- 101
2 votes
2 answers
125 views
How to set kubernetes secrets outside a container section
I have a generated kubernetes secret like this as it is shown e.g. in argocd: apiVersion: v1 data: ES_GUEST_PASSWORD: ++++++++ ES_GUEST_USERNAME: ++++++++ kind: Secret I now want to use it as an ...
- 123
0 votes
1 answer
238 views
How to gather application logs which are generating in custom path using EFK
I created the elasticsearch and kibina setup which is running outside the kubernetes cluster and i am using fluentd to gather kubernetes logs and to send data to elasticsearch. I am running fluentd as ...
- 111
0 votes
1 answer
2k views
Fluent-bit in Kubernetes cannot input timestamp data properly
At my company, I built a K8s cluster with Terraform and configured a logging system with EFK (Elasticsearch, Fluent-bit, Kibana). k8s and Elasticsearch use AWS's EKS and Opensearch Servcie (ES 7.10), ...
- 1
1 vote
1 answer
2k views
Could not contact DNS servers (fluent bit)
I am configuring the fluent bit connection with elasticsearch, I have 3 elasticsearch nodes and I want to have high availability, according to the following documentation it is done like this text, ...
0 votes
0 answers
419 views
Fluentd logs not sent to Elasticsearch
I have a cluster in VirtualBox to learn kubernetes. I have a deployment that contains MySQL and phpMyAdmin. I created a DemonSet that has the fluentd image and collects the logs to transmit them to ...
- 1
0 votes
0 answers
1k views
Running Opensearch behind a proxy
when I configure nginx to redirect /monitor to opensearch, opensearch receives the requests and redirects the user to /app/something which is not under /monitor and thus fails (error 404). My nginx ...
- 113
0 votes
0 answers
462 views
How to have multiple instances of filebeat load balance Netflow input?
I have a very high volume Netflow input stream, and I was hoping that I could run multiple instances of Filebeat and load-balance the Netflow traffic over the Filebeat instances, and then write to a ...
- 211
0 votes
1 answer
919 views
Can I use keycloak as an Idp for kibana installed using the ElasticSearch Operator in kubernetes?
I have setup a kubernetes cluster in a private network and I'm using a gateway machine for accessing the cluster. In the kubernetes cluster I have installed the elasticsearch operator and through that ...
0 votes
1 answer
17k views
Elasticsearch: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors (ca.crt matches on all nodes)
Note that this entire cluster was working fine up 'til I updated certificates. I'm on Elasticsearch 7.5. I updated my certificates using the certificate-util - created a server.yml with all my servers ...
- 11
2 votes
0 answers
793 views
rsyslog is not forwarding logs to elasticsearch
I'm trying to configure rsyslog to send logs to logstash and then forward them to elasticsearch. I have create a config file /etc/rsyslog.d/60-output.conf with the following content: *.* @localhost:...
- 141
0 votes
1 answer
1k views
SonarQube is not starting on SynologyNAS due to CONFIG_SECCOMP
SonarQube is unable to start on Synology NAS Docker due to the CONFIG_SECCOMP not being present on Synology NAS. Is there an option to ignore SECCOMP for SonarQube while it's starting ElasticSearch? ...
- 103
0 votes
1 answer
485 views
Restricting Kibana to TLSv1.2
What is the setting that needs to be set in the kibana.yml file (or other file) to restrict Kibana to TLSv1.2 over HTTP? I believe that it uses TLSv1.0, TLSv1.1, and TLSv1.2 by default. I need to ...
- 1
1 vote
0 answers
99 views
Suricata / Filebeat / ELK - iptables tee - Create virtual hosts
I have an IDS setup as follow: Hardware / interfaces WAN <----(brwan)> ROUTER / AP <(br0)----> LAN \ -----(eth1)>...
1 vote
1 answer
123 views
Handle 150 req/s and 2TB of logs for 7 days of retention available by API
I would like to get advise from experienced people to build a HA infrastructure to log 2To of data in JSON format every week. I need to have a retention time of 7 days and need to be able to requests ...
- 11
0 votes
1 answer
4k views
curl -X GET http://localhost:9200 return Access Denied
I am trying to install and setup Elasticsearch 8 on Centos 7 server, the service started but when I do curl -x GET HTTP://localhost:9200, or curl -x GET HTTP://127.0.0.1:9200 or curl -x GET HTTP://(...
- 1
0 votes
1 answer
569 views
Graylog - data from newest indice not returned on search
I have a Graylog server (newest version) collecting data from nginx. It´s been running for a while. I retrieve the collected data by querying my Elasticsearch cluster (v7 newest version) which ...
- 71
1 vote
0 answers
174 views
Amazon OpenSearch Service stably or less downtime?
Can anyone point out, What is the best way for the elastic search to run stably? We frequently get messages about it requiring updates and that an update requires some downtime. Could we have 2 ...
1 vote
0 answers
516 views
On Debian system, keepalived is consuming full cpu usage on both kibana nodes
I have master/slave kibana machines on Debian 10. From few days, cpu usage of keepalived daemon on both nodes are reaching to 100%. After keepalived restart it goes to normal, and this loops forever. ...
- 55
0 votes
0 answers
44 views
How to fetch Individual output of different match_phrase inside one request
I am able to run below query which is giving response as total of two match_phrase. Inserting dummy data as below. POST /mod1/_bulk { "index" : { } } { "msg": "BA1" } { &...
- 11
0 votes
1 answer
1k views
How to pass values to query externally
I have below similar logs. I have created dummy index and created mapping like below in dev-tools PUT new { "mappings": { "properties": { "@timestamp": { ...
- 11
1 vote
1 answer
190 views
Can logstash "pull" data?
I have two servers. Server A is running Elasticsearch and Logstash. Server B is running filebeat and is also the server which contains all the logs I'm trying to analyse. Server A is behind a firewall,...
- 993
0 votes
0 answers
906 views
Finding source of high IOWait
I have a 5 node elasticsearch cluster. One host has had consistently high IOWait% (40+) for several weeks. The others seem fine (<10%). Notable: Host in question is not the cluster master New ...
- 2,361
1 vote
1 answer
478 views
graylog 4.0 - Injector could not be created
Trying to set up graylog with official graylog2.graylog ansible role. Installed Elasticsearch (v7.10.2) cluster on three separate servers with official elasticsearch ansible role (node_1 - master, ...
- 123
1 vote
0 answers
1k views
My elastic search is not showing any data using elastic-agent
Surprisingly my elastic version 7.15 with basic license have installed the fleet server and then configured windows-policy with elastic-endpoint and windows integration. Have enabled all the rules; ...
- 39
1 vote
1 answer
714 views
Running Enterprise Search as a service
Elastic's Enterprise Search Installation Guide goes as far as starting the process locally - which is obviously not very stable. When installed from a .deb or .rpm, a service is actually set up and ...
- 131
1 vote
1 answer
667 views
Is there a way to use the ingest attachment plugin with Elastic App Search
I'm working on a portal that hosts multiple types of documentation (HTML, PDF, PPTx, DocX) and makes them all searchable in one place. We could achieve this using the "standard" out of the ...
- 131
1 vote
1 answer
913 views
Kibana does not trust Let's Encrypt CA in Kubernetes Helm Setup
I have a k8s cluster configured with calico as networking system. I'm running slightly customized versions of helm elastic/elasticsearch and elastic/kibana with security enabled. For security, I use ...
- 31
1 vote
1 answer
3k views
When to use a new index in Graylog (Elasticsearch)?
I have been searching for days now to find a good explanation on how indices are used by Graylog/Elasticsearch and when to create a new one. There is a lot of info about sharding indices but not much ...
- 123
0 votes
1 answer
313 views
Filebeat on ECK with AWS Module Fails Due To Metadata Error
We are running an Elastic Stack with ECK in EKS (7.8). We noticed that our filebeat daemonset and the AWS module were not processing logs from S3 and our SQS queues backing up. Looking at the logs on ...
- 1
0 votes
1 answer
12k views
Elasticsearch - Can I delete a data-stream backing index?
I have a few data-streams configured in my Elasticsearch cluster (with ILM policy). Suppose I run into storage problems and etc. and I want to prematurely delete a few backing indices of the data-...
- 535
0 votes
1 answer
12k views
Kibana cannot be accessed through port 5601 in browser [duplicate]
I cannot get to kibana through web UI. I'm set the necessary ports in firewall-cmd, I've configured kibana.yml and elasticsearch.yml. I'm not seeing any relevant errors, or at least don't know if they ...
- 121
0 votes
0 answers
76 views
Multiple Web Application - Cache Layer Design
I have multiple client e-commerce web applications running on a single VM, each web application is a node.js express application. The express web application communicates with the back end via API's ...
- 111
0 votes
1 answer
811 views
How to stop ElasticSearch background task
According to the CLI help of ElasticSearch, you can start ElasticSearch in the background with elasticsearch -d or elasticsearch --daemonize. This is what I use all the time locally, as it seems to ...
- 295
0 votes
0 answers
43 views
Ubuntu server 'partially' responsive
I have an Ubuntu 18.04 server that has periodically unresponsive - I can ping it, and it appears that some https services are running (albeit v. slowly), however it hangs if I try to log in (on the ...
- 11