Skip to content

parseablehq/parseable

Repository files navigation

Parseable Parseable

commits activity monthly join slack Github stars Twitter

Parseable is a cloud native, log storage and observability platform. Parseable is written in Rust and index free by design. Parseable is available as a single binary / Docker image and can be deployed on a wide range of platforms.

It ingests log data via HTTP POST calls and exposes a query API to search and analyze logs. It is compatible with all the standard logging agents via HTTP output plugins.

Features

  • Highly compressed log data storage with Parquet.
  • Use standard SQL for querying log data.
  • Auto inferred schema for log streams.
  • Dashboard to query the log data.
  • Compatible with existing logging agents.
  • Scale with scaling up the node.
  • Configurable alerting.
  • Distributed, multi-node cluster.
  • Log data visualization via Parseable UI.
  • Access control for users and groups.
  • Log data retention and export.
  • Kafka plugin to ingest log data.
  • Grafana plugin to visualize log data.
  • Anomaly detection within log data.

Getting Started

Docker

Parseable docker image is available on Docker hub. Please change the environment variables as relevant.

cat << EOF > parseable-env P_S3_URL=https://minio.parseable.io:9000 P_S3_ACCESS_KEY=minioadmin P_S3_SECRET_KEY=minioadmin P_S3_REGION=us-east-1 P_S3_BUCKET=parseable P_LOCAL_STORAGE=/data P_USERNAME=parseable P_PASSWORD=parseable EOF mkdir -p /tmp/data docker run \ -p 8000:8000 \ --env-file parseable-env \ -v /tmp/data:/data \ parseable/parseable:latest

Kubernetes

Parseable helm chart is available at Parseable charts repository.

helm repo add parseable https://charts.parseable.io/ kubectl create namespace parseable helm install parseable parseable/parseable --namespace parseable

Binary

Parseable binary is available on Github releases. Please download the latest release for your platform, also make sure to change the environment variables as relevant.

export P_S3_URL="https://minio.parseable.io:9000" export P_S3_ACCESS_KEY="minioadmin" export P_S3_SECRET_KEY="minioadmin" export P_S3_REGION="us-east-1" export P_S3_BUCKET="parseable" export P_LOCAL_STORAGE="./data" export P_USERNAME="parseable" export P_PASSWORD="parseable" chmod +x parseable ./parseable

Parseable dashboard is available at http://localhost:8000. Credentials to login to the dashboard are the values you set in the environment variables.

📝 Parseable is in alpha stage and will evolve over time. There may be breaking changes between releases. Please give us your feedback in Slack, or Issues.

Using Parseable

<stream-name> is the name of the stream you want to create. <basic-auth-header> is the basic auth header value generated from username & password of the user you created in the environment variables. You can generate the basic auth header value using the following command.

echo -n '<user-name>:<password>' | base64

Create a stream

curl --location --request PUT 'http://localhost:8000/api/v1/logstream/<stream-name>' \ --header 'Authorization: Basic <basic-auth-header>'

Send events to the stream

curl --location --request POST 'http://localhost:8000/api/v1/logstream/<stream-name>' \ --header 'X-P-META-meta1: value1' \ --header 'X-P-TAG-tag1: value1' \ --header 'Authorization: Basic <basic-auth-header>' \ --header 'Content-Type: application/json' \ --data-raw '[  {  "id": "434a5f5e-2f5f-11ed-a261-0242ac120002",  "datetime": "24/Jun/2022:14:12:15 +0000",  "host": "153.10.110.81",   "user-identifier": "Mozilla/5.0 Gecko/20100101 Firefox/64.0",   "method": "PUT",   "status": 500,   "referrer": "http://www.google.com/"  } ]'

For complete Parseable API documentation, refer to Parseable API Ref Docs.

Live Demo

You can also try out Parseable on our https://demo.parseable.io. Credentials to login to the dashboard are parseable / parseable.

Contributing

Refer to the contributing guide here.

License

Licensed under the GNU Affero General Public License, Version 3 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.gnu.org/licenses/agpl-3.0.txt

About

Parseable is an observability platform built for the modern - cloud native, AI era.

Topics

Resources

License

Code of conduct

Contributing

Stars

Watchers

Forks

Sponsor this project

Contributors 58