Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
02047a5
Updated text and image
fhennig Oct 18, 2023
2707338
Updated formatting
fhennig Oct 18, 2023
61c828e
Update modules/concepts/pages/logging.adoc
fhennig Oct 19, 2023
89c3145
more info
fhennig Oct 19, 2023
84fee5d
Merge branch 'main' into logging-sinks
fhennig Oct 25, 2023
ca31eeb
Removed HDFS mentions
fhennig Nov 1, 2023
00dd180
Added tutorial
fhennig Nov 1, 2023
30c205a
link to tutorial from the concepts page
fhennig Nov 1, 2023
15102db
Merge branch 'main' into logging-sinks
fhennig Nov 3, 2023
7d85edb
Update modules/concepts/pages/logging.adoc
fhennig Nov 13, 2023
11cf8e5
Update modules/tutorials/examples/logging-aggregator/main.sh
fhennig Nov 13, 2023
6370f0f
Update modules/tutorials/examples/logging-aggregator/main.sh
fhennig Nov 13, 2023
9ae2bec
Updated ZK manifest
fhennig Nov 13, 2023
3464e80
Made main.sh executable
fhennig Nov 13, 2023
402f843
Removed WebHDFS from the diagram
fhennig Nov 13, 2023
332f86b
fixes for nightly
fhennig Nov 13, 2023
26ae240
Merge branch 'main' into logging-sinks
fhennig Nov 13, 2023
cfe552e
Update modules/tutorials/pages/logging-vector-aggregator.adoc
fhennig Nov 22, 2023
f270655
Update modules/tutorials/pages/logging-vector-aggregator.adoc
fhennig Nov 22, 2023
74aa6d0
Update modules/tutorials/examples/logging-aggregator/main.sh
fhennig Nov 22, 2023
2d03125
Update modules/tutorials/examples/logging-aggregator/main.sh
fhennig Nov 22, 2023
956e728
Changed install command
fhennig Nov 22, 2023
c1a8bbe
removed sleep in favor of server-side apply
fhennig Nov 22, 2023
87628df
Install listener
fhennig Nov 23, 2023
1eb2cc1
shell script improvements
fhennig Nov 23, 2023
ed39252
fixed typo
fhennig Nov 23, 2023
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixes for nightly
  • Loading branch information
fhennig committed Nov 13, 2023
commit 332f86bffaa84131c78a51d44e233c13f06d3c6e
8 changes: 5 additions & 3 deletions modules/tutorials/examples/logging-aggregator/main.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@

echo "Installing ZooKeeper Operator"
# tag::zk-op[]
stackablectl release install -i secret -i commons -i zookeeper 23.7
#stackablectl release install -i secret -i commons -i zookeeper 23.7
stackablectl op in secret commons zookeeper
# end::zk-op[]

# tag::vector-agg[]
Expand All @@ -17,17 +18,18 @@ kubectl apply -f vector-aggregator-discovery.yaml
kubectl apply -f zookeeper.yaml
# end::zk[]

sleep 10
kubectl rollout status statefulset simple-zk-server-default --timeout=5m
kubectl wait \
--for=jsonpath='.status.readyReplicas'=3 \
--timeout=5m \
statefulsets.apps/simple-zk-server-default

# tag::grep[]
kubectl logs vector-aggregator-0 | grep "zookeeper.version="
kubectl logs vector-aggregator-0 | grep "zookeeper.version=" | jq
# end::grep[]

if [ "$?" -eq 0 ]
if [ "${PIPESTATUS[1]}" -eq 0 ]
then
echo "it worked"
else
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ customConfig:
inputs:
- vector
encoding:
codec: text
codec: json
target: stderr
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ metadata:
spec:
image:
productVersion: 3.8.0
stackableVersion: "23.7"
stackableVersion: "0.0.0-dev"
clusterConfig:
vectorAggregatorConfigMapName: vector-aggregator-discovery # <1>
servers:
Expand Down
27 changes: 25 additions & 2 deletions modules/tutorials/pages/logging-vector-aggregator.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,33 @@ Vector was configured to print the aggregated logs to `stderr`, so if you look a
[source,bash]
include::example$logging-aggregator/main.sh[tag=grep]

You should see a line printed that looks like
You should see a JSON oject per ZooKeeper replica printed that look like

[source,json]
{
"cluster": "simple-zk",
"container": "zookeeper",
"file": "zookeeper.log4j.xml",
"level": "INFO",
"logger": "org.apache.zookeeper.server.ZooKeeperServer",
"message": "Server environment:zookeeper.version=3.8.0-5a02a05eddb59aee6ac762f7ea82e92a68eb9c0f, built on 2022-02-25 08:49 UTC",
"namespace": "default",
"pod": "simple-zk-server-default-0",
"role": "server",
"roleGroup": "default",
"source_type": "vector",
"timestamp": "2023-11-06T10:30:40.223Z"
}

The JSON object contains a timestamp, the log message, log level and some additional information.

You can see the same log line in the log output of the ZooKeeper container:

[source]
Server environment:zookeeper.version=3.8.0-5a02a05eddb59aee6ac762f7ea82e92a68eb9c0f, built on 2022-02-25 08:49 UTC
kubectl logs --container=zookeeper simple-zk-server-default-0 | grep "zookeeper.version="

[source]
2023-11-06 10:30:40,223 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:zookeeper.version=3.8.0-5a02a05eddb59aee6ac762f7ea82e92a68eb9c0f, built on 2022-02-25 08:49 UTC

Congratulations, this concludes the tutorial!

Expand Down