Merge pull request #52 from VictoriaMetrics-Community/docs-update-2025-12-15

updated embedded docs and blog posts

Author: Amper

cmd/mcp-victorialogs/resources/vm/docs/victorialogs/CHANGELOG.md

@@ -243,7 +243,37 @@ or [Grafana plugin for VictoriaLogs](https://docs.victoriametrics.com/victorialo
 then make sure the selected time range covers the last day. Otherwise, the query above returns
 results on the intersection of the last day and the selected time range.
 
-See [why the log field occupies a lot of disk space](https://docs.victoriametrics.com/victorialogs/faq/#why-the-log-field-occupies-a-lot-of-disk-space).
+See also:
+
+- [Why the log field occupies a lot of disk space](https://docs.victoriametrics.com/victorialogs/faq/#why-the-log-field-occupies-a-lot-of-disk-space).
+- [How to detect log streams, which occupy the most of disk space](https://docs.victoriametrics.com/victorialogs/faq/#how-to-determine-which-log-streams-occupy-the-most-of-disk-space).
+
+## How to determine which log streams occupy the most of disk space?
+
+[Run](https://docs.victoriametrics.com/victorialogs/querying/) the following [LogsQL](https://docs.victoriametrics.com/victorialogs/logsql/) query
+based on [`block_stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#block_stats-pipe):
+
+```logsql
+_time:1d
+ | block_stats
+ | stats by (_stream)
+ sum(values_bytes) as values_bytes,
+ sum(bloom_bytes) as bloom_bytes
+ | math
+ (values_bytes+bloom_bytes) as total_bytes
+ | first 10 (total_bytes desc)
+```
+
+This query returns top 10 [log streams](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields),
+which occupy the most of disk space across the logs ingested during the last day. The occupied disk space
+is returned in the `total_bytes` field.
+
+If you use [VictoriaLogs web UI](https://docs.victoriametrics.com/victorialogs/querying/#web-ui)
+or [Grafana plugin for VictoriaLogs](https://docs.victoriametrics.com/victorialogs/integrations/grafana/),
+then make sure the selected time range covers the last day. Otherwise, the query above returns
+results on the intersection of the last day and the selected time range.
+
+See also [how to detect log fields, which occupy the most of disk space](https://docs.victoriametrics.com/victorialogs/faq/#how-to-determine-which-log-fields-occupy-the-most-of-disk-space).
 
 ## Why the log field occupies a lot of disk space?
 

cmd/mcp-victorialogs/resources/vm/docs/victorialogs/FAQ.md

@@ -1666,7 +1666,8 @@ LogsQL supports the following pipes:
 
 ### block_stats pipe
 
-`<q> | block_stats` [pipe](https://docs.victoriametrics.com/victorialogs/logsql/#pipes) returns the following stats for each block processed by `<q>` [query](https://docs.victoriametrics.com/victorialogs/logsql/#query-syntax):
+`<q> | block_stats` [pipe](https://docs.victoriametrics.com/victorialogs/logsql/#pipes) returns the following stats for each field in every data block
+processed by `<q>` [query](https://docs.victoriametrics.com/victorialogs/logsql/#query-syntax):
 
 - `field` - [field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model) name
 - `rows` - the number of rows at the given `field`
@@ -1675,9 +1676,12 @@ LogsQL supports the following pipes:
 - `bloom_bytes` - on-disk size of bloom filter data for the given `field`
 - `dict_bytes` - on-disk size of the dictionary data for the given `field`
 - `dict_items` - the number of unique values in the dictionary for the given `field`
-- `part_path` - the path to the data part where the block is stored
+- `_stream` - the [log stream](https://docs.victoriametrics.com/victorialogs/keyconcepts/#stream-fields) for the given `field`
+- `part_path` - the path to the data part where the field data is stored
 
 The `block_stats` pipe is needed mostly for debugging purposes.
+See, for example, [how to detect which log field occupies the most of the disk space](https://docs.victoriametrics.com/victorialogs/faq/#how-to-determine-which-log-fields-occupy-the-most-of-disk-space),
+or [how to detect which log stream occupies the most of the disk space](https://docs.victoriametrics.com/victorialogs/faq/#how-to-determine-which-log-streams-occupy-the-most-of-disk-space).
 
 See also:
 
@@ -4532,6 +4536,11 @@ over logs for the last 5 minutes:
 _time:5m | stats max(duration) max_duration
 ```
 
+The `max(some_field)` function works with string values for the `some_field`, so it returns an empty string value if `some_field`
+is missing in some of the processed logs according to [VictoriaLogs data model](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
+Use `max(some_field) if (some_field:*) as min_value_without_empty_string` syntax for filtering out empty string values.
+See [conditional stats docs](https://docs.victoriametrics.com/victorialogs/logsql/#stats-with-additional-filters) for more details.
+
 It is possible to calculate the maximum value across all the fields with common prefix via `max(prefix*)` syntax.
 
 [`row_max`](https://docs.victoriametrics.com/victorialogs/logsql/#row_max-stats) function can be used for obtaining other fields with the maximum duration.
@@ -4555,6 +4564,11 @@ over logs for the last 5 minutes:
 _time:5m | stats median(duration) median_duration
 ```
 
+The `median(some_field)` function works with string values for the `some_field`, so it returns an empty string value if `some_field`
+is missing in some of the processed logs according to [VictoriaLogs data model](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
+Use `median(some_field) if (some_field:*) as min_value_without_empty_string` syntax for filtering out empty string values.
+See [conditional stats docs](https://docs.victoriametrics.com/victorialogs/logsql/#stats-with-additional-filters) for more details.
+
 It is possible to calculate the median across all the fields with common prefix via `median(prefix*)` syntax.
 
 See also:
@@ -4574,6 +4588,11 @@ over logs for the last 5 minutes:
 _time:5m | stats min(duration) min_duration
 ```
 
+The `min(some_field)` function works with string values for the `some_field`, so it returns an empty string value if `some_field`
+is missing in some of the processed logs according to [VictoriaLogs data model](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
+Use `min(some_field) if (some_field:*) as min_value_without_empty_string` syntax for filtering out empty string values.
+See [conditional stats docs](https://docs.victoriametrics.com/victorialogs/logsql/#stats-with-additional-filters) for more details.
+
 It is possible to find the minimum across all the fields with common prefix via `min(prefix*)` syntax.
 
 [`row_min`](https://docs.victoriametrics.com/victorialogs/logsql/#row_min-stats) function can be used for obtaining other fields with the minimum duration.
@@ -4601,6 +4620,11 @@ _time:5m | stats
  quantile(0.99, request_duration_seconds) p99
 ```
 
+The `quantile(phi, some_field)` function works with string values for the `some_field`, so it returns an empty string value if `some_field`
+is missing in some of the processed logs according to [VictoriaLogs data model](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model).
+Use `quantile(phi, some_field) if (some_field:*) as min_value_without_empty_string` syntax for filtering out empty string values.
+See [conditional stats docs](https://docs.victoriametrics.com/victorialogs/logsql/#stats-with-additional-filters) for more details.
+
 It is possible to calculate the quantile across all the fields with common prefix via `quantile(phi, prefix*)` syntax.
 
 See also:

cmd/mcp-victorialogs/resources/vm/docs/victorialogs/LogsQL.md

@@ -37,8 +37,8 @@ Just download the archive for the needed operating system and architecture, unpa
 For example, the following commands download VictoriaLogs archive for Linux/amd64, unpack and run it:
 
 ```sh
-curl -L -O https://github.com/VictoriaMetrics/VictoriaLogs/releases/download/v1.38.0/victoria-logs-linux-amd64-v1.38.0.tar.gz
-tar xzf victoria-logs-linux-amd64-v1.38.0.tar.gz
+curl -L -O https://github.com/VictoriaMetrics/VictoriaLogs/releases/download/v1.41.0/victoria-logs-linux-amd64-v1.41.0.tar.gz
+tar xzf victoria-logs-linux-amd64-v1.41.0.tar.gz
 ./victoria-logs-prod -storageDataPath=victoria-logs-data
 ```
 
@@ -61,7 +61,7 @@ Here is the command to run VictoriaLogs in a Docker container:
 
 ```sh
 docker run --rm -it -p 9428:9428 -v ./victoria-logs-data:/victoria-logs-data \
- docker.io/victoriametrics/victoria-logs:v1.38.0 -storageDataPath=victoria-logs-data
+ docker.io/victoriametrics/victoria-logs:v1.41.0 -storageDataPath=victoria-logs-data
 ```
 
 See also:

cmd/mcp-victorialogs/resources/vm/docs/victorialogs/QuickStart.md

@@ -60,8 +60,7 @@ to [Grafana plugin playground for VictoriaLogs](https://play-grafana.victoriamet
  The only option is increasing the limit on [the number of open files in the OS](https://medium.com/@muhammadtriwibowo/set-permanently-ulimit-n-open-files-in-ubuntu-4d61064429a).
 - The recommended filesystem is `ext4`, the recommended persistent storage is [persistent HDD-based disk on GCP](https://cloud.google.com/compute/docs/disks/#pdspecs),
  since it is protected from hardware failures via internal replication and it can be [resized on the fly](https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd).
- If you plan to store more than 1TB of data on `ext4` partition or plan extending it to more than 16TB,
- then the following options are recommended to pass to `mkfs.ext4`:
+ If you plan to store more than 1TB of data on `ext4` partition, then the following options are recommended to pass to `mkfs.ext4`:
 
  ```sh
  mkfs.ext4 ... -O 64bit,huge_file,extent -T huge
@@ -195,6 +194,11 @@ VictoriaLogs accepts logs with timestamps in the time range `[now-retentionPerio
 where `retentionPeriod` is the value for the `-retentionPeriod` command-line flag and `futureRetention` is the value for the `-futureRetention` command-line flag.
 Sometimes it is needed to reject logs older than the given age. This can be achieved by passing `-maxBackfillAge=d` command-line flag to VictoriaLogs,
 where `d` is the maximum age of logs to be accepted. Older logs are rejected and a sample of these logs is put into VictoriaLogs output logs, so they could be investigated.
+For example, the following command starts VictoriaLogs, which rejects logs older than 1 hour:
+
+```sh
+/path/to/victoria-logs -maxBackfillAge=1h
+```
 
 ## Storage
 

cmd/mcp-victorialogs/resources/vm/docs/victorialogs/README.md

@@ -352,8 +352,8 @@ If you want running VictoriaLogs cluster in Kubernetes, then please read [these
 Download and unpack the latest VictoriaLogs release:
 
 ```sh
-curl -L -O https://github.com/VictoriaMetrics/VictoriaLogs/releases/download/v1.38.0/victoria-logs-linux-amd64-v1.38.0.tar.gz
-tar xzf victoria-logs-linux-amd64-v1.38.0.tar.gz
+curl -L -O https://github.com/VictoriaMetrics/VictoriaLogs/releases/download/v1.41.0/victoria-logs-linux-amd64-v1.41.0.tar.gz
+tar xzf victoria-logs-linux-amd64-v1.41.0.tar.gz
 ```
 
 Start the first [`vlstorage` node](https://docs.victoriametrics.com/victorialogs/cluster/#architecture), which accepts incoming requests at the port `9491` and stores the ingested logs in the `victoria-logs-data-1` directory:

cmd/mcp-victorialogs/resources/vm/docs/victorialogs/cluster.md

@@ -452,11 +452,17 @@ VictoriaLogs provides `/select/logsql/stats_query?query=<query>&time=<t>` HTTP e
 for the given [`query`](https://docs.victoriametrics.com/victorialogs/logsql/) at the given timestamp `t`
 in the format compatible with [Prometheus querying API](https://prometheus.io/docs/prometheus/latest/querying/api/#instant-queries).
 
+The `<t>` arg can contain values in [any supported format](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#timestamp-formats).
+If `<t>` is missing, then it equals to the current time.
+
 The `<query>` must contain [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe). The calculated stats is converted into metrics
 with labels from `by(...)` clause of the `| stats by(...)` pipe.
 
-The `<t>` arg can contain values in [any supported format](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#timestamp-formats).
-If `<t>` is missing, then it equals to the current time.
+The [`row_any`](https://docs.victoriametrics.com/victorialogs/logsql/#row_any-stats), [`row_min`](https://docs.victoriametrics.com/victorialogs/logsql/#row_min-stats)
+and [`row_max`](https://docs.victoriametrics.com/victorialogs/logsql/#row_max-stats) stats functions create labels instead of metrics.
+
+Additional labels can be created from metrics via [`format` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#format-pipe).
+Additional metrics can be created from the existing metrics via [`math` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#math-pipe).
 
 For example, the following command returns the number of logs per each `level` [field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model)
 across logs over `2024-01-01` day by UTC:
@@ -509,6 +515,7 @@ Below is an example JSON output returned from this endpoint:
 ```
 
 The `/select/logsql/stats_query` API is useful for generating Prometheus-compatible alerts and calculating recording rules results.
+It is used by [vmalert](https://docs.victoriametrics.com/victorialogs/vmalert/).
 
 The `/select/logsql/stats_query` returns the following additional HTTP response headers:
 
@@ -529,9 +536,6 @@ VictoriaLogs provides `/select/logsql/stats_query_range?query=<query>&start=<sta
 for the given [`query`](https://docs.victoriametrics.com/victorialogs/logsql/) on the given `[start ... end)` time range with the given `step` interval.
 The stats is returned in the format compatible with [Prometheus querying API](https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries).
 
-The `<query>` must contain [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe). The calculated stats is converted into metrics
-with labels from `by(...)` clause of the `| stats by(...)` pipe.
-
 The `<start>` and `<end>` args can contain values in [any supported format](https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#timestamp-formats).
 If `<start>` is missing, then it equals to the minimum timestamp across logs stored in VictoriaLogs.
 If `<end>` is missing, then it equals to the maximum timestamp across logs stored in VictoriaLogs.
@@ -543,6 +547,19 @@ Note: The `/select/logsql/stats_query_range` endpoint
 relies on `_time` field for time bucketing and therefore does not allow any pipe to change or remove the `_time` before the `| stats ...` pipe.
 In contrast, queries passed to [`/select/logsql/stats_query`](https://docs.victoriametrics.com/victorialogs/querying/#querying-log-stats) can include any pipes before the `| stats ...` pipe, including pipes that modify or remove the `_time` field.
 
+The `<query>` must contain [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe). The calculated stats is converted into metrics
+with labels from `by(...)` clause of the `| stats by(...)` pipe.
+
+The [`row_any`](https://docs.victoriametrics.com/victorialogs/logsql/#row_any-stats), [`row_min`](https://docs.victoriametrics.com/victorialogs/logsql/#row_min-stats)
+and [`row_max`](https://docs.victoriametrics.com/victorialogs/logsql/#row_max-stats) stats functions create labels instead of metrics.
+
+Additional labels can be created from metrics via [`format` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#format-pipe).
+Additional metrics can be created from the existing metrics via [`math` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#math-pipe).
+
+It may be useful to use [`running_stats`](https://docs.victoriametrics.com/victorialogs/logsql/#running_stats-pipe)
+and [`total_stats`](https://docs.victoriametrics.com/victorialogs/logsql/#total_stats-pipe) pipes for calculating running and total stats over the stats
+returned by the [`stats` pipe](https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe).
+
 For example, the following command returns the number of logs per each `level` [field](https://docs.victoriametrics.com/victorialogs/keyconcepts/#data-model)
 across logs over `2024-01-01` day by UTC with 6-hour granularity:
 
@@ -612,6 +629,7 @@ Below is an example JSON output returned from this endpoint:
 ```
 
 The `/select/logsql/stats_query_range` API is useful for generating Prometheus-compatible graphs in Grafana.
+It is used by [Grafana plugin for VictoriaLogs](https://docs.victoriametrics.com/victorialogs/integrations/grafana/).
 
 The `/select/logsql/stats_query_range` returns the following additional HTTP response headers:
 
@@ -1098,7 +1116,7 @@ VictoriaLogs provides the following options to limit resource usage by the execu
 ## Web UI
 
 VictoriaLogs provides Web UI for logs [querying](https://docs.victoriametrics.com/victorialogs/logsql/) and exploration
-at `http://localhost:9428/select/vmui`. Try [VictoriaLogs web UI demo playground](https://play-vmlogs.victoriametrics.com/).
+at `http://localhost:9428/select/vmui/`. Try [VictoriaLogs web UI demo playground](https://play-vmlogs.victoriametrics.com/).
 
 There are three modes of displaying query results:
 

cmd/mcp-victorialogs/resources/vm/docs/victorialogs/querying/README.md

@@ -25,8 +25,8 @@ or from docker images at [Docker Hub](https://hub.docker.com/r/victoriametrics/v
 ### Running `vlogscli` from release binary
 
 ```sh
-curl -L -O https://github.com/VictoriaMetrics/VictoriaLogs/releases/download/v1.38.0/vlutils-linux-amd64-v1.38.0.tar.gz
-tar xzf vlutils-linux-amd64-v1.38.0.tar.gz
+curl -L -O https://github.com/VictoriaMetrics/VictoriaLogs/releases/download/v1.41.0/vlutils-linux-amd64-v1.41.0.tar.gz
+tar xzf vlutils-linux-amd64-v1.41.0.tar.gz
 ./vlogscli-prod
 ```
 

cmd/mcp-victorialogs/resources/vm/docs/victorialogs/querying/vlogscli.md

@@ -229,7 +229,7 @@ However, user `admin` needs to set the required `AccountID` or `ProjectID` heade
 
 In Grafana, you need to create a separate data source for each tenant and user, an example of such an address is: `http://vmauth:8427/my-account/mobile-logs`.
 Using the configuration above, you do not need to set the tenant in the data source settings because vmauth will set it.
-Each tenant will have `vmui` at the address `/select/vmui`, for example: `http://vmauth:8427/my-account/mobile-logs/select/vmui`.
+Each tenant will have `vmui` at the address `/select/vmui/`, for example: `http://vmauth:8427/my-account/mobile-logs/select/vmui/`.
 
 If you want to restrict users by only one of the fields `AccountID` or `ProjectID`,
 it is enough to not specify the corresponding field in the `headers` section.