tests: add "test" documenting surprising shell behavior

Shell just looks like a logical language, it's much more tricky in practice. This patch adds a test that shows how we work around a particular oddity, specifically the interaction of "set -e" and shell negation operator "!". Signed-off-by: Zygmunt Krynicki <me@zygoon.pl> 

5 files changed, 79 insertions, 0 deletions

diff --git a/tests/unit/shell-traps/set-e-pipe-chain-with-negation.sh b/tests/unit/shell-traps/set-e-pipe-chain-with-negation.sh
new file mode 100755
index 0000000000..9063c201dc
--- /dev/null
+++ b/tests/unit/shell-traps/set-e-pipe-chain-with-negation.sh
@@ -0,0 +1,18 @@
+#!/bin/sh -x
+# When "set -e" is in effect it is natural to expect that failing commands
+# cause execution of the script to fail with an error code. This logic is
+# obviously not applied to all the possible cases as if-then-else expressions
+# must be allowed to fail to execute correctly.
+#
+# We've learned that negating a shell command is treated like an if-then-else
+# expression, in that it doesn't cause the script to fail to execute.
+#
+# When applied to pipe expressions the result of the last element of the pipe
+# determines the result of the expression but the use of "!" is causing set -e
+# not to matter.
+set -e
+# NOTE: disable shellcheck warning about this gotcha, since this test
+# explicitly documents and measures the behavior.
+# shellcheck disable=SC2251
+! echo foo bar | grep "foo"
+echo "surprise, last error: $?"
diff --git a/tests/unit/shell-traps/set-e-pipe-chain-with-not.sh b/tests/unit/shell-traps/set-e-pipe-chain-with-not.sh
new file mode 100755
index 0000000000..ea90d3bd5a
--- /dev/null
+++ b/tests/unit/shell-traps/set-e-pipe-chain-with-not.sh
@@ -0,0 +1,15 @@
+#!/bin/sh -x
+# When "set -e" is in effect it is natural to expect that failing commands
+# cause execution of the script to fail with an error code. This logic is
+# obviously not applied to all the possible cases as if-then-else expressions
+# must be allowed to fail to execute correctly.
+#
+# We've learned that negating a shell command is treated like an if-then-else
+# expression, in that it doesn't cause the script to fail to execute.
+#
+# When applied to pipe expressions the result of the last element of the pipe
+# determines the result of the expression and the use of "not" instead of "!"
+# makes the non-zero result fatal.
+set -e
+echo foo bar | not grep "foo"
+echo "not reached"
diff --git a/tests/unit/shell-traps/set-e-simple-cmd-with-negation.sh b/tests/unit/shell-traps/set-e-simple-cmd-with-negation.sh
new file mode 100755
index 0000000000..42a7a6a852
--- /dev/null
+++ b/tests/unit/shell-traps/set-e-simple-cmd-with-negation.sh
@@ -0,0 +1,14 @@
+#!/bin/sh -x
+# When "set -e" is in effect it is natural to expect that failing commands
+# cause execution of the script to fail with an error code. This logic is
+# obviously not applied to all the possible cases as if-then-else expressions
+# must be allowed to fail to execute correctly.
+#
+# We've learned that negating a shell command is treated like an if-then-else
+# expression, in that it doesn't cause the script to fail to execute.
+set -e
+# NOTE: disable shellcheck warning about this gotcha, since this test
+# explicitly documents and measures the behavior. 
+# shellcheck disable=SC2251
+! true
+echo "surprise, last error: $?"
diff --git a/tests/unit/shell-traps/set-e-simple-cmd-with-not.sh b/tests/unit/shell-traps/set-e-simple-cmd-with-not.sh
new file mode 100755
index 0000000000..684140020a
--- /dev/null
+++ b/tests/unit/shell-traps/set-e-simple-cmd-with-not.sh
@@ -0,0 +1,11 @@
+#!/bin/sh -x
+# When "set -e" is in effect it is natural to expect that failing commands
+# cause execution of the script to fail with an error code. This logic is
+# obviously not applied to all the possible cases as if-then-else expressions
+# must be allowed to fail to execute correctly.
+#
+# We've learned that negating a shell command is treated like an if-then-else
+# expression, in that it doesn't cause the script to fail to execute.
+set -e
+not true
+echo "not reached"
diff --git a/tests/unit/shell-traps/task.yaml b/tests/unit/shell-traps/task.yaml
new file mode 100644
index 0000000000..df58406817
--- /dev/null
+++ b/tests/unit/shell-traps/task.yaml
@@ -0,0 +1,21 @@
+summary: shell is tricky
+details: |
+ shell can be surprisingly tricky, this test captures some of the things
+ we've learned and now guard against. The test is expected to pass all the
+ time, it simply contains "executable documentation" that is meant to
+ illustrate how non-obvious some behavior is.
+# 1: increment if you had to read this or edit this
+execute: |
+ # NOTE: Disable set -e that was implicitly provided by spread and check for
+ # errors explicitly. This allows us to to be verify the exit code of each
+ # test *without* falling into one of the traps of shell negation.
+ set +e
+ ./set-e-pipe-chain-with-negation.sh
+ test $? -eq 0 || exit 1
+ ./set-e-pipe-chain-with-not.sh
+ test $? -eq 1 || exit 1
+ ./set-e-simple-cmd-with-negation.sh
+ test $? -eq 0 || exit 1
+ ./set-e-simple-cmd-with-not.sh
+ test $? -eq 1 || exit 1
+