diff options
| author | Sylvain Pineau <sylvain.pineau@canonical.com> | 2015-09-09 00:04:46 +0200 |
|---|---|---|
| committer | Sylvain Pineau <sylvain.pineau@canonical.com> | 2015-09-09 00:04:46 +0200 |
| commit | a3a0546226fb56b8da899a67556f5b67366eee7a (patch) | |
| tree | daecd5eab6291813506a988f78dc633bd973a8e7 /jobs | |
| parent | b15705168431e6f8f07fc2d89f59934b484e7ecb (diff) | |
providers:checkbox: Adds category ids to all jobs (except local jobs)
This patch adds category ids to all jobs in the checkbox provider (almost all jobs) so that users can select the category they want to test in checkbox-converged. local jobs are not getting a category id as those will be run during bootstrapping. Legacy jobs in install, panel-reboot, panel-control and server-services are also kept uncategorised.
Diffstat (limited to 'jobs')
42 files changed, 695 insertions, 0 deletions
diff --git a/jobs/audio.txt.in b/jobs/audio.txt.in index ea316e8..81f3837 100644 --- a/jobs/audio.txt.in +++ b/jobs/audio.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::audio id: audio/list_devices estimated_duration: 1.0 requires: @@ -8,6 +9,7 @@ command: cat /proc/asound/cards _description: Test to detect audio devices plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::audio id: audio/playback_auto estimated_duration: 5.0 depends: audio/list_devices @@ -48,6 +50,7 @@ command: id: audio/`echo ${index}`_playback_hdmi_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: display.hdmi == 'supported' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::audio estimated_duration: 30.0 command: audio_settings store --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings @@ -80,6 +83,7 @@ command: id: audio/`echo ${index}`_playback_displayport_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: display.dp == 'supported' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::audio estimated_duration: 30.0 command: audio_settings store --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings @@ -113,6 +117,7 @@ command: imports: from 2013.com.canonical.plainbox import manifest requires: manifest.has_thunderbolt == 'True' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::audio estimated_duration: 5.0 command: audio_settings store --verbose --file=$PLAINBOX_SESSION_SHARE/pulseaudio_settings @@ -132,6 +137,7 @@ command: EOF plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::audio id: audio/playback_headphones estimated_duration: 20.0 depends: audio/list_devices @@ -157,6 +163,7 @@ _description: Did you hear a sound through the headphones and did the sound play without any distortion, clicks or other strange noises from your headphones? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::audio id: audio/alsa_record_playback_internal estimated_duration: 20.0 depends: audio/playback_auto @@ -183,6 +190,7 @@ _description: Did you hear your speech played back? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::audio id: audio/alsa_record_playback_external estimated_duration: 20.0 depends: audio/playback_headphones @@ -209,6 +217,7 @@ _description: Did you hear your speech played back? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::audio id: audio/alsa_record_playback_usb estimated_duration: 120.0 requires: @@ -234,6 +243,7 @@ _description: Did you hear your speech played back through the USB headphones? plugin: shell +category_id: 2013.com.canonical.plainbox::audio id: audio/alsa_record_playback_automated estimated_duration: 10.0 requires: @@ -250,6 +260,7 @@ _description: default input. plugin: shell +category_id: 2013.com.canonical.plainbox::audio id: audio/alsa_info_collect estimated_duration: 2.0 command: alsa_info --no-dialog --no-upload --output ${PLAINBOX_SESSION_SHARE}/alsa_info.log @@ -259,6 +270,7 @@ _description: under a controlled environment. plugin: attachment +category_id: 2013.com.canonical.plainbox::audio id: audio/alsa_info_attachment depends: audio/alsa_info_collect estimated_duration: 1.0 @@ -267,6 +279,7 @@ _description: Attaches the audio hardware data collection log to the results. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::audio id: audio/channels estimated_duration: 20.0 command: speaker-test -c 2 -l 1 -t wav @@ -279,6 +292,7 @@ _description: You should clearly hear a voice from the different audio channels plugin: shell +category_id: 2013.com.canonical.plainbox::audio id: audio/check_volume estimated_duration: 1.0 requires: @@ -294,6 +308,7 @@ _description: volume or mute before running this test. plugin: manual +category_id: 2013.com.canonical.plainbox::audio id: audio/external-lineout estimated_duration: 30.0 _description: @@ -308,6 +323,7 @@ _description: 2. Do you hear the sound coming out on the corresponding channel? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::audio id: audio/external-linein estimated_duration: 120.0 requires: @@ -333,6 +349,7 @@ _description: Did you hear your recording? plugin: user-interact +category_id: 2013.com.canonical.plainbox::audio id: audio/speaker-headphone-plug-detection estimated_duration: 60.0 requires: @@ -353,6 +370,7 @@ _description: The test times out after 30 seconds (and fails in that case). plugin: user-interact +category_id: 2013.com.canonical.plainbox::audio id: audio/microphone-plug-detection estimated_duration: 60.0 requires: @@ -374,6 +392,7 @@ _description: The test times out after 30 seconds (and fails in that case). plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::audio id: audio/balance_internal_speaker estimated_duration: 20.0 depends: audio/playback_auto @@ -401,6 +420,7 @@ _description: Does the slider move smoothly, as well as being followed by the setting by the actual audio output? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::audio id: audio/balance_headphones depends: audio/playback_headphones estimated_duration: 30.0 @@ -428,6 +448,7 @@ _description: Does the slider move smoothly, as well as being followed by the setting by the actual audio output? plugin: shell +category_id: 2013.com.canonical.plainbox::audio id: audio/list_devices_after_suspend_30_cycles estimated_duration: 1.0 depends: power-management/suspend_30_cycles @@ -438,6 +459,7 @@ command: cat /proc/asound/cards _description: Test to detect audio devices after suspending 30 times. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::audio id: audio/playback_auto_after_suspend_30_cycles estimated_duration: 5.0 depends: audio/list_devices power-management/suspend_30_cycles @@ -465,6 +487,7 @@ _description: Did you hear a tone? plugin: shell +category_id: 2013.com.canonical.plainbox::audio id: audio/alsa_record_playback_automated_after_suspend_30_cycles estimated_duration: 10.0 depends: power-management/suspend_30_cycles @@ -482,6 +505,7 @@ _description: default input, after suspending 30 times. plugin: shell +category_id: 2013.com.canonical.plainbox::audio id: audio/check_volume_after_suspend_30_cycles estimated_duration: 1.0 depends: power-management/suspend_30_cycles @@ -498,6 +522,7 @@ _description: volume or mute before running this test. plugin: shell +category_id: 2013.com.canonical.plainbox::audio id: audio/audio_after_suspend_30_cycles estimated_duration: 1.0 depends: power-management/suspend_30_cycles diff --git a/jobs/benchmarks.txt.in b/jobs/benchmarks.txt.in index d4abcbe..e71671f 100644 --- a/jobs/benchmarks.txt.in +++ b/jobs/benchmarks.txt.in @@ -7,6 +7,7 @@ _description: Benchmark for each disk command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"' plugin: shell + category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/disk/hdparm-read_`ls /sys$path/block` estimated_duration: 15.0 requires: device.path == "$path" @@ -24,6 +25,7 @@ _description: Benchmark for each disk command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"' plugin: shell + category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/disk/hdparm-cache-read_`ls /sys$path/block` estimated_duration: 10.0 requires: device.path == "$path" @@ -33,6 +35,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/gtkperf depends: graphics/xorg-version requires: package.name == 'gtkperf' @@ -42,6 +45,7 @@ _description: Run gtkperf to make sure that GTK based test cases work plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/render-bench requires: package.name == 'render-bench' command: /usr/bin/time -f "%e Seconds" render_bench 2>&1 >/dev/null @@ -49,6 +53,7 @@ estimated_duration: 52.000 _description: Run Render-Bench XRender/Imlib2 benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/qgears2-Xrender-gearsfancy requires: package.name == 'phoronix-test-suite' command: PRESET_OPTIONS="qgears2.render-backend=1; qgears2.test-mode=0" pts_run qgears2 @@ -56,6 +61,7 @@ estimated_duration: 180.000 _description: Run Qgears2 XRender Extension gearsfancy benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/qgears2-Xrender-compo requires: package.name == 'phoronix-test-suite' command: PRESET_OPTIONS="qgears2.render-backend=1; qgears2.test-mode=2" pts_run qgears2 @@ -63,6 +69,7 @@ estimated_duration: 31.500 _description: Run Qgears2 XRender Extension image scaling benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/qgears2-gl-gearsfancy requires: package.name == 'phoronix-test-suite' command: PRESET_OPTIONS="qgears2.render-backend=2; qgears2.test-mode=0" pts_run qgears2 @@ -70,6 +77,7 @@ estimated_duration: 52.000 _description: Run Qgears2 OpenGL gearsfancy benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/qgears2-gl-compo requires: package.name == 'phoronix-test-suite' command: PRESET_OPTIONS="qgears2.render-backend=2; qgears2.test-mode=2" pts_run qgears2 @@ -77,6 +85,7 @@ estimated_duration: 23.000 _description: Run Qgears2 OpenGL image scaling benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/glmark2-es2 requires: package.name == 'glmark2-es2' @@ -86,6 +95,7 @@ command: _description: Run GLmark2-ES2 benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/glmark2 requires: package.name == 'glmark2' @@ -96,30 +106,35 @@ estimated_duration: 306.000 _description: Run GLmark2 benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/unigine-sanctuary requires: package.name == 'phoronix-test-suite' command: pts_run unigine-sanctuary _description: Run Unigine Santuary benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/unigine-tropics requires: package.name == 'phoronix-test-suite' command: pts_run unigine-tropics _description: Run Unigine Tropics benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/unigine-heaven requires: package.name == 'phoronix-test-suite' command: pts_run unigine-heaven _description: Run Unigine Heaven benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/lightsmark requires: package.name == 'phoronix-test-suite' command: pts_run lightsmark _description: Run Lightsmark benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/memory/cachebench-read estimated_duration: 160.0 requires: package.name == 'phoronix-test-suite' @@ -127,6 +142,7 @@ command: PRESET_OPTIONS="cachebench.test=0" pts_run cachebench _description: Run Cachebench Read benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/memory/cachebench-write estimated_duration: 130.0 requires: package.name == 'phoronix-test-suite' @@ -134,6 +150,7 @@ command: PRESET_OPTIONS="cachebench.test=1" pts_run cachebench _description: Run Cachebench Write benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/memory/cachebench-read-modify-write estimated_duration: 140.0 requires: package.name == 'phoronix-test-suite' @@ -141,6 +158,7 @@ command: PRESET_OPTIONS="cachebench.test=2" pts_run cachebench _description: Run Cachebench Read / Modify / Write benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/memory/stream-copy estimated_duration: 6.0 requires: package.name == 'phoronix-test-suite' @@ -148,6 +166,7 @@ command: PRESET_OPTIONS="stream.run-type=0" pts_run stream _description: Run Stream Copy benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/memory/stream-scale estimated_duration: 6.0 requires: package.name == 'phoronix-test-suite' @@ -155,6 +174,7 @@ command: PRESET_OPTIONS="stream.run-type=1" pts_run stream _description: Run Stream Scale benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/memory/stream-add estimated_duration: 5.0 requires: package.name == 'phoronix-test-suite' @@ -162,6 +182,7 @@ command: PRESET_OPTIONS="stream.run-type=2" pts_run stream _description: Run Stream Add benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/memory/stream-triad estimated_duration: 4.0 requires: package.name == 'phoronix-test-suite' @@ -169,6 +190,7 @@ command: PRESET_OPTIONS="stream.run-type=3" pts_run stream _description: Run Stream Triad benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/network/network-loopback requires: package.name == 'phoronix-test-suite' command: pts_run network-loopback @@ -176,12 +198,14 @@ estimated_duration: 85.0 _description: Run Network Loopback benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/network/wifi_time_to_reconnect requires: device.category == 'WIRELESS' command: wifi_time2reconnect _description: Check the time needed to reconnect to a WIFI access point plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/processor/encode-mp3 estimated_duration: 140.0 requires: package.name == 'phoronix-test-suite' @@ -189,6 +213,7 @@ command: pts_run encode-mp3 _description: Run Encode MP3 benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/processor/x264 estimated_duration: 400.0 requires: package.name == 'phoronix-test-suite' @@ -196,12 +221,14 @@ command: pts_run x264 _description: Run x264 H.264/AVC encoder benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/processor/gnupg requires: package.name == 'phoronix-test-suite' command: pts_run gnupg _description: Run GnuPG benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/processor/compress-pbzip2 estimated_duration: 90.0 requires: package.name == 'phoronix-test-suite' @@ -209,6 +236,7 @@ command: pts_run compress-pbzip2 _description: Run Compress PBZIP2 benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/processor/compress-7zip estimated_duration: 1400 requires: package.name == 'phoronix-test-suite' @@ -216,6 +244,7 @@ command: pts_run compress-7zip _description: Run Compress 7ZIP benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/processor/n-queens estimated_duration: 330.0 requires: package.name == 'phoronix-test-suite' @@ -223,6 +252,7 @@ command: pts_run n-queens _description: Run N-Queens benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/processor/himeno estimated_duration: 75.0 requires: package.name == 'phoronix-test-suite' @@ -230,6 +260,7 @@ command: pts_run himeno _description: Run Himeno benchmark plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/system/cpu_on_idle estimated_duration: 10.0 requires: package.name == 'sysstat' @@ -237,6 +268,7 @@ command: iostat -x -m 1 10 | python3 -c 'import sys, re; lines="".join(sys.stdin _description: CPU utilization on an idle system. plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/system/disk_on_idle estimated_duration: 10.0 requires: package.name == 'sysstat' @@ -244,6 +276,7 @@ command: iostat -x -m 1 10 | python3 -c 'import sys, re; lines="".join(sys.stdin _description: Disk utilization on an idle system. plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/gputest_furmark_fullscreen_1920x1080 requires: package.name == 'gputest' @@ -255,6 +288,7 @@ _description: Fullscreen 1920x1080 no antialiasing plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/gputest_furmark_windowed_1024x640 requires: package.name == 'gputest' @@ -266,6 +300,7 @@ _description: Windowed 1024x640 no antialiasing plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/gputest_gimark_fullscreen_1920x1080 requires: package.name == 'gputest' @@ -277,6 +312,7 @@ _description: Fullscreen 1920x1080 no antialiasing plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/gputest_gimark_windowed_1024x640 requires: package.name == 'gputest' @@ -288,6 +324,7 @@ _description: Windowed 1024x640 no antialiasing plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/gputest_tessmark_fullscreen_1920x1080 requires: package.name == 'gputest' @@ -299,6 +336,7 @@ _description: Fullscreen 1920x1080 no antialiasing plugin: shell +category_id: 2013.com.canonical.plainbox::benchmarks id: benchmarks/graphics/gputest_tessmark_windowed_1024x640 requires: package.name == 'gputest' diff --git a/jobs/bluetooth.txt.in b/jobs/bluetooth.txt.in index a9eb508..619767a 100644 --- a/jobs/bluetooth.txt.in +++ b/jobs/bluetooth.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::bluetooth id: bluetooth/detect-output estimated_duration: 1.2 requires: @@ -23,6 +24,7 @@ _description: Automated test to store bluetooth device information in checkbox report plugin: manual +category_id: 2013.com.canonical.plainbox::bluetooth id: bluetooth/browse-files depends: bluetooth/detect-output estimated_duration: 120.0 @@ -43,6 +45,7 @@ _description: Did all the steps work? plugin: manual +category_id: 2013.com.canonical.plainbox::bluetooth id: bluetooth/file-transfer depends: bluetooth/browse-files bluetooth/detect-output estimated_duration: 120.0 @@ -57,6 +60,7 @@ _description: Were all files copied correctly? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::bluetooth id: bluetooth/audio-a2dp depends: bluetooth/detect-output estimated_duration: 120.0 @@ -82,6 +86,7 @@ _description: Did you hear the tone? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::bluetooth id: bluetooth/audio depends: bluetooth/detect-output estimated_duration: 120.0 @@ -103,6 +108,7 @@ _description: Did you hear the sound you recorded in the bluetooth with a slight intended delay? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::bluetooth id: bluetooth/HID depends: bluetooth/detect-output estimated_duration: 120.0 @@ -121,6 +127,7 @@ _description: Did the device work as expected? plugin: shell +category_id: 2013.com.canonical.plainbox::bluetooth id: bluetooth/file_transfer_stress estimated_duration: 300.0 requires: diff --git a/jobs/camera.txt.in b/jobs/camera.txt.in index a08b7f6..688b1e0 100644 --- a/jobs/camera.txt.in +++ b/jobs/camera.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::camera id: camera/detect estimated_duration: 1.2 requires: @@ -7,6 +8,7 @@ command: camera_test detect _description: This Automated test attempts to detect a camera. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::camera id: camera/display estimated_duration: 120.0 depends: camera/detect @@ -22,6 +24,7 @@ _description: Did you see the video capture? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::camera id: camera/still estimated_duration: 120.0 depends: camera/detect @@ -40,6 +43,7 @@ _description: Did you see the image? plugin: shell +category_id: 2013.com.canonical.plainbox::camera id: camera/multiple-resolution-images estimated_duration: 1.2 depends: camera/detect diff --git a/jobs/codecs.txt.in b/jobs/codecs.txt.in index c1d953f..cc3bdf7 100644 --- a/jobs/codecs.txt.in +++ b/jobs/codecs.txt.in @@ -1,4 +1,5 @@ plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::codecs id: codecs/ogg-vorbis requires: package.name == 'gstreamer0.10-plugins-base' or package.name == 'gstreamer1.0-plugins-base' @@ -15,6 +16,7 @@ _description: Did the sample play correctly? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::codecs id: codecs/wav requires: package.name == 'gstreamer0.10-plugins-good' or package.name == 'gstreamer1.0-plugins-good' diff --git a/jobs/cpu.txt.in b/jobs/cpu.txt.in index 3b8db4d..eef467c 100644 --- a/jobs/cpu.txt.in +++ b/jobs/cpu.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::cpu id: cpu/scaling_test estimated_duration: 150.0 requires: @@ -10,6 +11,7 @@ _description: Test the CPU scaling capabilities using Firmware Test Suite (fwts cpufreq). plugin: attachment +category_id: 2013.com.canonical.plainbox::cpu id: cpu/scaling_test-log-attach estimated_duration: 1.0 depends: cpu/scaling_test @@ -18,6 +20,7 @@ _description: Attaches the log generated by cpu/scaling_test to the results plugin: shell +category_id: 2013.com.canonical.plainbox::cpu id: cpu/maxfreq_test estimated_duration: 0.6 requires: @@ -28,6 +31,7 @@ _description: Test that the CPU can run at its max frequency using Firmware Test Suite (fwts cpufreq). plugin: attachment +category_id: 2013.com.canonical.plainbox::cpu id: cpu/maxfreq_test-log-attach estimated_duration: 1.0 depends: cpu/maxfreq_test @@ -36,6 +40,7 @@ _description: Attaches the log generated by cpu/maxfreq_test to the results plugin: shell +category_id: 2013.com.canonical.plainbox::cpu id: cpu/clocktest estimated_duration: 60.0 command: clocktest @@ -43,6 +48,7 @@ _description: Test for clock jitter. plugin: shell +category_id: 2013.com.canonical.plainbox::cpu id: cpu/offlining_test estimated_duration: 1.0 user: root @@ -51,6 +57,7 @@ _description: Test offlining CPUs in a multicore system. plugin: shell +category_id: 2013.com.canonical.plainbox::cpu id: cpu/topology estimated_duration: 1.0 requires: int(cpuinfo.count) > 1 and (cpuinfo.platform == 'i386' or cpuinfo.platform == 'x86_64') @@ -59,6 +66,7 @@ _description: This test checks cpu topology for accuracy plugin: shell +category_id: 2013.com.canonical.plainbox::cpu id: cpu/frequency_governors estimated_duration: 22.0 user: root @@ -67,6 +75,7 @@ _description: This test checks that CPU frequency governors are obeyed when set. plugin: shell +category_id: 2013.com.canonical.plainbox::cpu id: cpu/arm_vfp_support estimated_duration: 1.0 requires: 'arm' in cpuinfo.type @@ -87,6 +96,7 @@ command: fwts_test -l $PLAINBOX_SESSION_SHARE/fwts_cstates_results.log -t cstates plugin: attachment +category_id: 2013.com.canonical.plainbox::cpu estimated_duration: 0.5 id: cpu/cstates_results.log command: diff --git a/jobs/daemons.txt.in b/jobs/daemons.txt.in index 3aaceb6..80d57bf 100644 --- a/jobs/daemons.txt.in +++ b/jobs/daemons.txt.in @@ -1,58 +1,68 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::daemons id: daemons/atd requires: package.name == 'at' command: pgrep -f '/usr/sbin/atd' >/dev/null _description: Test if the atd daemon is running when the package is installed. plugin: shell +category_id: 2013.com.canonical.plainbox::daemons id: daemons/cron requires: package.name == 'cron' command: pgrep -f '/usr/sbin/cron' >/dev/null _description: Test if the cron daemon is running when the package is installed. plugin: shell +category_id: 2013.com.canonical.plainbox::daemons id: daemons/cupsd requires: package.name == 'cupsys' command: pgrep -f '/usr/sbin/cupsd' >/dev/null _description: Test if the cupsd daemon is running when the package is installed. plugin: shell +category_id: 2013.com.canonical.plainbox::daemons id: daemons/getty requires: package.name == 'util-linux' command: pgrep -f '/sbin/getty' >/dev/null _description: Test if the getty daemon is running when the package is installed. plugin: shell +category_id: 2013.com.canonical.plainbox::daemons id: daemons/init requires: package.name == 'upstart' command: pgrep -f '/sbin/init' >/dev/null _description: Test if the init daemon is running when the package is installed. plugin: shell +category_id: 2013.com.canonical.plainbox::daemons id: daemons/klogd requires: package.name == 'klogd' command: pgrep -f '/sbin/klogd' >/dev/null _description: Test if the klogd daemon is running when the package is installed. plugin: shell +category_id: 2013.com.canonical.plainbox::daemons id: daemons/nmbd requires: package.name == 'samba' command: pgrep -f '/usr/sbin/nmbd' >/dev/null _description: Test if the nmbd daemon is running when the package is installed. plugin: shell +category_id: 2013.com.canonical.plainbox::daemons id: daemons/smbd requires: package.name == 'samba' command: pgrep -f '/usr/sbin/smbd' >/dev/null _description: Test if the smbd daemon is running when the package is installed. plugin: shell +category_id: 2013.com.canonical.plainbox::daemons id: daemons/syslogd requires: package.name == 'syslogd' command: pgrep -f '/sbin/syslogd' >/dev/null _description: Test if the syslogd daemon is running when the package is installed. plugin: shell +category_id: 2013.com.canonical.plainbox::daemons id: daemons/udevd requires: package.name == 'udevd' @@ -60,6 +70,7 @@ command: pgrep -f '/sbin/udevd' >/dev/null _description: Test if the udevd daemon is running when the package is installed. plugin: shell +category_id: 2013.com.canonical.plainbox::daemons id: daemons/winbindd requires: package.name == 'winbind' command: pgrep -f '/usr/sbin/winbindd' >/dev/null diff --git a/jobs/disk.txt.in b/jobs/disk.txt.in index 33a6dbc..94b4e11 100644 --- a/jobs/disk.txt.in +++ b/jobs/disk.txt.in @@ -4,6 +4,7 @@ os-id: debian Depends: lshw plugin: shell +category_id: 2013.com.canonical.plainbox::disk id: disk/detect estimated_duration: 0.25 user: root @@ -16,6 +17,7 @@ template-imports: from 2013.com.canonical.certification import device template-resource: device template-filter: device.category == 'DISK' and device.name != '' plugin: shell +category_id: 2013.com.canonical.plainbox::disk id: staging/disk/stats_{name} requires: block_device.{name}_state != 'removable' @@ -42,6 +44,7 @@ _description: Check stats changes for each disk command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"' plugin: shell + category_id: 2013.com.canonical.plainbox::disk id: disk/stats_`ls /sys$path/block` flags: deprecated requires: @@ -63,6 +66,7 @@ _description: SMART test command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"' plugin: shell + category_id: 2013.com.canonical.plainbox::disk id: disk/smart_`ls /sys$path/block` requires: device.path == "$path" @@ -83,6 +87,7 @@ _description: Verify system storage performs at or above baseline performance command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"' plugin: shell + category_id: 2013.com.canonical.plainbox::disk id: disk/read_performance_`ls /sys$path/block` estimated_duration: 65.0 requires: @@ -103,6 +108,7 @@ _description: Verify that storage devices, such as Fibre Channel and RAID can be command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"' plugin: shell + category_id: 2013.com.canonical.plainbox::disk id: disk/storage_device_`ls /sys$path/block` estimated_duration: 375.0 user: root @@ -114,6 +120,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::disk id: disk/spindown estimated_duration: 875.0 requires: @@ -125,6 +132,7 @@ _description: Some new hard drives include a feature that parks the drive heads after a short period of inactivity. This is a power-saving feature, but it can have a bad interaction with the operating system that results in the drive constantly parked then activated. This produces excess wear on the drive, potentially leading to early failures. plugin: user-interact +category_id: 2013.com.canonical.plainbox::disk id: disk/hdd-parking estimated_duration: 60.0 requires: diff --git a/jobs/esata.txt.in b/jobs/esata.txt.in index e6ec175..402310f 100644 --- a/jobs/esata.txt.in +++ b/jobs/esata.txt.in @@ -1,4 +1,5 @@ plugin: user-interact +category_id: 2013.com.canonical.plainbox::esata id: esata/insert command: removable_storage_watcher insert ata_serial_esata _description: @@ -13,6 +14,7 @@ _description: selected result plugin: shell +category_id: 2013.com.canonical.plainbox::esata id: esata/storage-test user: root depends: esata/insert @@ -22,6 +24,7 @@ _description: eSATA HDD plugin: user-interact +category_id: 2013.com.canonical.plainbox::esata id: esata/remove depends: esata/insert command: removable_storage_watcher remove ata_serial_esata diff --git a/jobs/ethernet.txt.in b/jobs/ethernet.txt.in index e66ab6d..a542ed8 100644 --- a/jobs/ethernet.txt.in +++ b/jobs/ethernet.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::ethernet id: ethernet/detect requires: device.category == 'NETWORK' or device.category == 'WIRELESS' @@ -9,6 +10,7 @@ estimated_duration: 1.2 _description: Test to detect the available network controllers plugin: shell +category_id: 2013.com.canonical.plainbox::ethernet id: ethernet/info_automated requires: executable.name == 'ip' @@ -19,6 +21,7 @@ _description: This is an automated test to gather some info on the current state of your network devices. If no devices are found, the test will exit with an error. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::ethernet id: ethernet/wired_connection command: network_check estimated_duration: 1.2 @@ -40,6 +43,7 @@ _description: Automated test to walk multiple network cards and test each one in command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: shell + category_id: 2013.com.canonical.plainbox::ethernet id: ethernet/multi_nic_$2 requires: package.name == 'ethtool' @@ -60,6 +64,7 @@ requires: command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: shell + category_id: 2013.com.canonical.plainbox::ethernet id: ethernet/ethertool_check_$2 requires: device.path == "$1" command: ethtool $2 @@ -79,6 +84,7 @@ requires: command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::ethernet user: root id: ethernet/maximum_bandwidth_$2 requires: device.path == "$1" @@ -107,6 +113,7 @@ _description: Automated test that tests performance of each wired network device command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: shell + category_id: 2013.com.canonical.plainbox::ethernet id: ethernet/stress_performance_$2 requires: device.path == "$1" user: root diff --git a/jobs/expresscard.txt.in b/jobs/expresscard.txt.in index 4b2ac39..90d8b84 100644 --- a/jobs/expresscard.txt.in +++ b/jobs/expresscard.txt.in @@ -1,4 +1,5 @@ plugin: manual +category_id: 2013.com.canonical.plainbox::expresscard id: expresscard/verification _description: PURPOSE: diff --git a/jobs/fingerprint.txt.in b/jobs/fingerprint.txt.in index 45263db..24033f3 100644 --- a/jobs/fingerprint.txt.in +++ b/jobs/fingerprint.txt.in @@ -1,4 +1,5 @@ plugin: manual +category_id: 2013.com.canonical.plainbox::fingerprint id: fingerprint/login _description: PURPOSE: @@ -14,6 +15,7 @@ _description: Did the authentication procedure work correctly? plugin: manual +category_id: 2013.com.canonical.plainbox::fingerprint id: fingerprint/unlock _description: PURPOSE: diff --git a/jobs/firewire.txt.in b/jobs/firewire.txt.in index a94d749..31547e3 100644 --- a/jobs/firewire.txt.in +++ b/jobs/firewire.txt.in @@ -1,4 +1,5 @@ plugin: user-interact +category_id: 2013.com.canonical.plainbox::firewire id: firewire/insert require: device.category == 'FIREWIRE' command: removable_storage_watcher insert firewire @@ -14,6 +15,7 @@ _description: selected result plugin: shell +category_id: 2013.com.canonical.plainbox::firewire id: firewire/storage-test user: root require: device.category == 'FIREWIRE' @@ -24,6 +26,7 @@ _description: FireWire HDD plugin: user-interact +category_id: 2013.com.canonical.plainbox::firewire id: firewire/remove depends: firewire/insert require: device.category == 'FIREWIRE' diff --git a/jobs/firmware.txt.in b/jobs/firmware.txt.in index ac2188f..f9fec6b 100644 --- a/jobs/firmware.txt.in +++ b/jobs/firmware.txt.in @@ -8,6 +8,7 @@ command: cat << 'EOF' | run_templates -s 'fwts_test --list' estimated_duration: 1.2 plugin: shell + category_id: 2013.com.canonical.plainbox::firmware id: firmware/fwts_$1 requires: package.name == 'fwts' user: root @@ -26,6 +27,7 @@ command: cat << 'EOF' | run_templates -s 'fwts_test --list' estimated_duration: 1.2 plugin: attachment + category_id: 2013.com.canonical.plainbox::firmware id: firmware/fwts_$1.log requires: package.name == 'fwts' user: root @@ -61,6 +63,7 @@ command: fwts_test --hwe -l $PLAINBOX_SESSION_SHARE/fwts_desktop_diagnosis_results_hwe.log plugin: attachment +category_id: 2013.com.canonical.plainbox::firmware estimated_duration: 0.5 id: firmware/fwts_desktop_diagnosis_results.log.gz command: @@ -69,6 +72,7 @@ _description: Attaches the FWTS desktop diagnosis results log to the submission _summary: Attach FWTS desktop diagnosis log to submission plugin: attachment +category_id: 2013.com.canonical.plainbox::firmware estimated_duration: 0.5 id: firmware/fwts_desktop_diagnosis_results_hwe.log.gz command: @@ -77,6 +81,7 @@ _description: Attaches the FWTS desktop diagnosis results log to the submission _summary: Attach FWTS desktop diagnosis log to submission (to HWE) plugin: shell +category_id: 2013.com.canonical.plainbox::firmware id: firmware/no_ACPI_REV_interface _summary: No _REV interface in ACPI [DS]SDT tables user: root diff --git a/jobs/floppy.txt.in b/jobs/floppy.txt.in index e87febd..a741f07 100644 --- a/jobs/floppy.txt.in +++ b/jobs/floppy.txt.in @@ -6,6 +6,7 @@ _description: Floppy test command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "driver=floppy"' plugin: shell + category_id: 2013.com.canonical.plainbox::floppy id: floppy/check_`ls /sys$path/driver/*/*/*/block` requires: device.path == "$path" description: Floppy test for $product diff --git a/jobs/graphics-legacy.txt.in b/jobs/graphics-legacy.txt.in index 851e481..10ebb3b 100644 --- a/jobs/graphics-legacy.txt.in +++ b/jobs/graphics-legacy.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/driver_version command: graphics_driver estimated_duration: 0.500 @@ -6,6 +7,7 @@ _description: Parses Xorg.0.Log and discovers the running X driver and version _summary: Test X driver/version plugin: manual +category_id: 2013.com.canonical.plainbox::graphics id: graphics/resolution-change depends: graphics/xorg-version _summary: Test changing resolution @@ -22,6 +24,7 @@ _description: Did the resolution change as expected? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::graphics id: graphics/resolution requires: device.category == 'VIDEO' @@ -40,6 +43,7 @@ _description: Is this acceptable for your display? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::graphics id: graphics/screen-resolution requires: device.category == 'VIDEO' @@ -56,6 +60,7 @@ _description: Is this acceptable for your display? plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/minimum_resolution requires: device.category == 'VIDEO' @@ -69,6 +74,7 @@ _description: https://help.ubuntu.com/community/Installation/SystemRequirements plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::graphics id: graphics/maximum_resolution requires: device.category == 'VIDEO' @@ -87,6 +93,7 @@ _description: id: graphics/modes plugin: shell +category_id: 2013.com.canonical.plainbox::graphics command: graphics_modes_info estimated_duration: 0.250 _description: Collect info on graphics modes (screen resolution and refresh rate) @@ -94,6 +101,7 @@ _summary: Collect info on graphics modes id: graphics/color_depth plugin: shell +category_id: 2013.com.canonical.plainbox::graphics command: color_depth_info estimated_duration: 0.150 _description: Collect info on color depth and pixel format. @@ -101,17 +109,20 @@ _summary: Collect info on color depth and pixel format. id: graphics/fresh_rate plugin: shell +category_id: 2013.com.canonical.plainbox::graphics command: fresh_rate_info _description: Collect info on fresh rate. _summary: Collect info on fresh rate. id: graphics/graphic_memory plugin: shell +category_id: 2013.com.canonical.plainbox::graphics command: graphic_memory_info _description: Collect info on graphic memory. _summary: Collect info on graphic memory. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::graphics id: graphics/cycle_resolution requires: package.name == 'xorg' depends: graphics/VESA_drivers_not_in_use @@ -127,6 +138,7 @@ _description: Did the screen appear to be working for each mode? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::graphics id: graphics/rotation depends: graphics/xorg-version command: rotation_test @@ -142,6 +154,7 @@ _description: Did the display rotation take place without without permanent screen corruption? plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/compiz_check requires: package.name == 'nux-tools' command: ! /usr/lib/nux/unity_support_test -c -p 2>&1 | ansi_parser | grep -e ":\(\s\+\)no$" -ie "error" @@ -150,6 +163,7 @@ _summary: Test Compiz support _description: Check that hardware is able to run compiz plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/unity-support requires: package.name == 'nux-tools' command: ! /usr/lib/nux/unity_support_test -p 2>&1 | ansi_parser | grep -e ":\(\s\+\)no$" -ie "error" @@ -158,6 +172,7 @@ _description: Check that hardware is able to run Unity 3D _summary: Test Unity 3D support plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::graphics id: graphics/glxgears requires: package.name == 'mesa-utils' command: glxgears; true @@ -173,6 +188,7 @@ _description: 2. Was the animation free from slowness/jerkiness? plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/3d_window_open_close requires: package.name == 'mesa-utils' command: window_test -t open-close -i 10 @@ -181,6 +197,7 @@ _summary: Test 3D window open/close _description: Open and close a 3D window multiple times plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/3d_window_suspend_resume requires: package.name == 'mesa-utils' command: window_test -t suspend-resume -i 10 @@ -189,6 +206,7 @@ _description: Open, suspend resume and close a 3D window multiple times _summary: Test a 3D window with suspend/resume plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/multi_3d_windows_open_close requires: package.name == 'mesa-utils' command: window_test -t open-close-multi -i 10 -w 4 @@ -197,6 +215,7 @@ _description: Open and close 4 3D windows multiple times _summary: Test Multi 3D window open/close plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/3d_window_move requires: package.name == 'mesa-utils' command: window_test -t move @@ -205,6 +224,7 @@ _description: Move a 3D window around the screen _summary: Test 3D window movement plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/screenshot requires: package.name == 'fswebcam' command: set -o pipefail; camera_test still --device=/dev/external_webcam -f ${PLAINBOX_SESSION_SHARE}/screenshot.jpg -q 2>&1 | ansi_parser @@ -218,6 +238,7 @@ _description: Review attachment manually later plugin: attachment +category_id: 2013.com.canonical.plainbox::graphics id: screenshot.jpg depends: graphics/screenshot command: base64 ${PLAINBOX_SESSION_SHARE}/screenshot.jpg @@ -225,6 +246,7 @@ _description: Attaches the screenshot captured in graphics/screenshot. _summary: Attach results of screenshot test plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/screenshot_fullscreen_video requires: package.name == 'fswebcam' command: @@ -246,6 +268,7 @@ _description: Review attachment manually later plugin: attachment +category_id: 2013.com.canonical.plainbox::graphics id: screenshot_fullscreen_video.jpg depends: graphics/screenshot_fullscreen_video command: base64 ${PLAINBOX_SESSION_SHARE}/screenshot_fullscreen_video.jpg diff --git a/jobs/graphics.txt.in b/jobs/graphics.txt.in index 14ec041..487e0c2 100644 --- a/jobs/graphics.txt.in +++ b/jobs/graphics.txt.in @@ -6,6 +6,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_driver_version_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` command: graphics_driver estimated_duration: 0.500 @@ -14,6 +15,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/xorg-version requires: package.name == "x11-utils" command: xdpyinfo | grep "^X.Org version" | cut -d ':' -f 2 | tr -d ' ' @@ -29,6 +31,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: manual + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_resolution-change_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: graphics/xorg-version _summary: Test changing resolution for $product @@ -47,6 +50,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/xorg-process requires: package.name == 'xorg' @@ -57,6 +61,7 @@ _description: Test that the X process is running. _summary: Test that the X process is running. plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/xorg-failsafe requires: package.name == 'xorg' command: ! test -e /var/log/Xorg.failsafe.log @@ -72,6 +77,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_resolution_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: device.category == 'VIDEO' @@ -97,6 +103,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_screen_resolution_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: device.category == 'VIDEO' @@ -121,6 +128,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_minimum_resolution_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: device.category == 'VIDEO' @@ -142,6 +150,7 @@ command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' id: graphics/`echo ${index}`_maximum_resolution_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::graphics requires: device.category == 'VIDEO' package.name == 'zenity' @@ -172,6 +181,7 @@ command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' id: graphics/`echo ${index}`_modes_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` plugin: shell + category_id: 2013.com.canonical.plainbox::graphics command: graphics_modes_info estimated_duration: 0.250 _description: Collect info on graphics modes (screen resolution and refresh rate) for $product @@ -187,6 +197,7 @@ command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' id: graphics/`echo ${index}`_color_depth_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` plugin: shell + category_id: 2013.com.canonical.plainbox::graphics command: color_depth_info estimated_duration: 0.150 _description: Collect info on color depth and pixel format for $product @@ -202,6 +213,7 @@ command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' id: graphics/`echo ${index}`_fresh_rate_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` plugin: shell + category_id: 2013.com.canonical.plainbox::graphics command: fresh_rate_info _description: Collect info on fresh rate for $product. _summary: Test refresh rate for $product @@ -216,6 +228,7 @@ command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' id: graphics/`echo ${index}`_graphic_memory_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` plugin: shell + category_id: 2013.com.canonical.plainbox::graphics command: graphic_memory_info _description: Collect info on graphic memory for $product. _summary: Test graphic memory info for $product. @@ -230,6 +243,7 @@ command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' id: graphics/`echo ${index}`_video_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::graphics requires: package.name == 'xorg' package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0' @@ -245,6 +259,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/VESA_drivers_not_in_use command: cat /var/log/Xorg.0.log | perl -e '$a=0;while(<>){$a++ if /Loading.*vesa_drv\.so/;$a-- if /Unloading.*vesa/&&$a}exit 1 if $a' estimated_duration: 0.011 @@ -259,6 +274,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_cycle_resolution_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'xorg' depends: graphics/VESA_drivers_not_in_use @@ -282,6 +298,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_rotation_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: graphics/xorg-version command: rotation_test @@ -305,6 +322,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_compiz_check_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'nux-tools' command: ! /usr/lib/nux/unity_support_test -c -p 2>&1 | ansi_parser | grep -e ":\(\s\+\)no$" -ie "error" @@ -321,6 +339,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_unity_support_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'nux-tools' command: ! /usr/lib/nux/unity_support_test -p 2>&1 | ansi_parser | grep -e ":\(\s\+\)no$" -ie "error" @@ -337,6 +356,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_glxgears_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'mesa-utils' command: glxgears; true @@ -360,6 +380,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_3d_window_open_close_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'mesa-utils' command: window_test -t open-close -i 10 @@ -376,6 +397,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_3d_window_suspend_resume_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'mesa-utils' command: window_test -t suspend-resume -i 10 @@ -392,6 +414,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_multi_3d_windows_open_close_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'mesa-utils' command: window_test -t open-close-multi -i 10 -w 4 @@ -408,6 +431,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_3d_window_move_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'mesa-utils' @@ -426,6 +450,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_screenshot_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'fswebcam' command: set -o pipefail; camera_test still --device=/dev/external_webcam -f ${PLAINBOX_SESSION_SHARE}/screenshot_${index}.jpg -q 2>&1 | ansi_parser @@ -447,6 +472,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: attachment + category_id: 2013.com.canonical.plainbox::graphics id: `echo ${index}`_screenshot_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`.jpg depends: graphics/`echo ${index}`_screenshot_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` command: base64 ${PLAINBOX_SESSION_SHARE}/screenshot_${index}.jpg @@ -462,6 +488,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_screenshot_fullscreen_video_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'fswebcam' command: @@ -491,6 +518,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: manual + category_id: 2013.com.canonical.plainbox::graphics id: graphics/`echo ${index}`_switch_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` _summary: Test GPU switching for $product _description: @@ -513,6 +541,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: attachment + category_id: 2013.com.canonical.plainbox::graphics id: `echo ${index}`_screenshot_fullscreen_video_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`.jpg depends: graphics/`echo ${index}`_screenshot_fullscreen_video_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` command: base64 ${PLAINBOX_SESSION_SHARE}/screenshot_fullscreen_video_${index}.jpg @@ -521,6 +550,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/screenshot_opencv_validation requires: package.name == 'python-opencv' environ: EXTERNAL_WEBCAM_DEVICE @@ -535,6 +565,7 @@ _description: Check that the screenshot matches the original file using OpenCV ORB detection. plugin: attachment +category_id: 2013.com.canonical.plainbox::graphics id: screenshot_opencv_validation.jpg depends: graphics/screenshot_opencv_validation command: base64 ${PLAINBOX_SESSION_SHARE}/screenshot_opencv_validation.jpg @@ -542,6 +573,7 @@ _description: Attaches the screenshot captured in graphics/screenshot_opencv_val _summary: Attach the screenshot captured for the automatically validated screenshot test plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: graphics/screen-capture-internal _summary: Obtains a simple screen capture estimated_duration: 1.0 @@ -550,6 +582,7 @@ command: gnome-screenshot --file ${PLAINBOX_SESSION_SHARE}/screen-capture.png _description: Obtains a simple screen capture using gnome-screenshot if present plugin: attachment +category_id: 2013.com.canonical.plainbox::graphics id: screen-capture-internal.png _summary: Attaches a simple screen capture estimated_duration: 1.0 @@ -561,12 +594,14 @@ _description: Attaches the simple screen capture file unit: job id: graphics_card_resource plugin: resource +category_id: 2013.com.canonical.plainbox::graphics command: graphics_card_resource unit: template template-unit: job template-resource: graphics_card_resource plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: "graphics/{index}_screen-capture-internal_{product}" _summary: Obtains a simple screen capture of {product} estimated_duration: 1.0 @@ -578,6 +613,7 @@ unit: template template-unit: job template-resource: graphics_card_resource plugin: attachment +category_id: 2013.com.canonical.plainbox::graphics id: graphics/{index}_screen-capture-internal_{product}.png depends: "graphics/{index}_screen-capture-internal_{product}" _summary: Attaches a simple screen capture of {product} diff --git a/jobs/hibernate.txt.in b/jobs/hibernate.txt.in index 7fde5ef..d0dfb66 100644 --- a/jobs/hibernate.txt.in +++ b/jobs/hibernate.txt.in @@ -1,4 +1,5 @@ plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::hibernate id: power-management/hibernate_advanced requires: sleep.disk == 'supported' @@ -35,6 +36,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::hibernate id: power-management/`echo ${index}`_hibernate_advanced_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` user: root environ: PLAINBOX_SESSION_SHARE @@ -60,12 +62,14 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::hibernate id: power-management/hibernate-single-log-check command: [ -e $PLAINBOX_SESSION_SHARE/hibernate-single.log ] && sleep_test_log_check -v s4 $PLAINBOX_SESSION_SHARE/hibernate-single.log _description: Automated check of the hibernate log for errors discovered by fwts plugin: attachment +category_id: 2013.com.canonical.plainbox::hibernate id: power-management/hibernate-single-log-attach depends: power-management/hibernate_advanced command: [ -e $PLAINBOX_SESSION_SHARE/hibernate-single.log ] && cat $PLAINBOX_SESSION_SHARE/hibernate-single.log @@ -79,6 +83,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: attachment + category_id: 2013.com.canonical.plainbox::hibernate id: power-management/`echo ${index}`_hibernate-single-log-attach_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: power-management/`echo ${index}`_hibernate_advanced_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` estimated_duration: 0.5 diff --git a/jobs/info.txt.in b/jobs/info.txt.in index 22ae54f..0379122 100644 --- a/jobs/info.txt.in +++ b/jobs/info.txt.in @@ -5,6 +5,7 @@ Depends: python3-debian id: codecs_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info requires: device.driver == 'snd_hda_intel' command: cat /proc/asound/card*/codec#* estimated_duration: 0.023 @@ -12,6 +13,7 @@ _description: Attaches a report of installed codecs for Intel HDA id: cpuinfo_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: cat /proc/cpuinfo estimated_duration: 0.006 _description: Attaches a report of CPU information @@ -19,6 +21,7 @@ _summary: Attach a copy of /proc/cpuinfo id: dmesg_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: cat /var/log/dmesg | ansi_parser estimated_duration: 0.640 _description: Attaches a copy of /var/log/dmesg to the test results @@ -26,6 +29,7 @@ _summary: Attach a copy of /var/log/dmesg id: dmi_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: [ -d /sys/class/dmi/id/ ] && (grep -r . /sys/class/dmi/id/ 2>/dev/null || true) || false estimated_duration: 0.044 _description: Attaches info on DMI @@ -33,6 +37,7 @@ _summary: Attach a copy of /sys/class/dmi/id/* id: dmidecode_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info requires: package.name == 'dmidecode' user: root command: dmidecode | iconv -t 'utf-8' -c @@ -42,6 +47,7 @@ _summary: Attach a copy of of the output of dmidecode id: lshw_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info requires: package.name == 'lshw' user: root command: lshw | iconv -t 'utf-8' -c @@ -49,6 +55,7 @@ _description: Attaches lshw output id: efi_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info user: root command: [ -d /sys/firmware/efi ] && grep -m 1 -o --color=never 'EFI v.*' /var/log/kern.log* || true estimated_duration: 0.5 @@ -56,6 +63,7 @@ _description: Attaches the firmware version id: lspci_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: lspci -vvnn | iconv -t 'utf-8' -c estimated_duration: 0.042 _description: Attaches very verbose lspci output. @@ -63,6 +71,7 @@ _summary: Attach a list of PCI devices id: lspci_network_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: lspci -vvnnQ | iconv -t 'utf-8' -c estimated_duration: 1.322 _description: Attaches very verbose lspci output (with central database Query). @@ -70,6 +79,7 @@ _description: Attaches very verbose lspci output (with central database Query). id: lspci_standard_config_attachment _summary: Attach PCI configuration space hex dump plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: lspci -x estimated_duration: 0.1 _description: @@ -78,6 +88,7 @@ _description: id: lsusb_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info requires: package.name == 'usbutils' user: root command: lsusb -vv | iconv -t 'utf-8' -c @@ -86,11 +97,13 @@ _description: List USB devices id: meminfo_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: cat /proc/meminfo estimated_duration: 0.043 id: modprobe_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: find /etc/modprobe.* -name \*.conf | xargs cat estimated_duration: 0.015 _description: Attaches the contents of the various modprobe conf files. @@ -98,6 +111,7 @@ _summary: Attach the contents of modprobe configuration files id: modinfo_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info _summary: Attach modinfo information command: for mod in $(lsmod | cut -f 1 -d " ") @@ -112,6 +126,7 @@ _description: Attaches modinfo information for all currently loaded modules id: modules_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: cat /etc/modules estimated_duration: 0.004 _description: Attaches the contents of the /etc/modules file. @@ -119,6 +134,7 @@ _summary: Attach the contents of /etc/modules id: sysctl_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: find /etc/sysctl.* -name \*.conf | xargs cat estimated_duration: 0.014 _description: attaches the contents of various sysctl config files. @@ -126,6 +142,7 @@ _summary: Attach sysctl configuration files. id: sysfs_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info _description: Attaches a report of sysfs attributes. command: for i in `udevadm info --export-db | sed -n 's/^P: //p'`; do @@ -138,6 +155,7 @@ _summary: Attach detailed sysfs property output from udev id: udev_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: udevadm info --export-db | xml_sanitize estimated_duration: 1.465 _description: Attaches a dump of the udev database showing system hardware information. @@ -145,12 +163,14 @@ _summary: Attach hardware database (udev) id: udev_resource_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: udev_resource estimated_duration: 0.432 _description: Attaches the output of udev_resource, for debugging purposes id: gcov_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info requires: package.name == 'lcov' user: root command: gcov_tarball @@ -158,6 +178,7 @@ _description: Attaches a tarball of gcov data if present. id: lsmod_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: lsmod_info estimated_duration: 0.5 _description: Attaches a list of the currently running kernel modules. @@ -165,17 +186,20 @@ _summary: Attach a list of currently running kernel modules id: dkms_info_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: dkms_info --format json _description: Attaches json dumps of installed dkms package information. _summary: Attaches json dumps of installed dkms package information. plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: acpi_sleep_attachment command: [ -e /proc/acpi/sleep ] && cat /proc/acpi/sleep || echo "No /proc/acpi/sleep found" estimated_duration: 0.5 _description: Attaches the contents of /proc/acpi/sleep if it exists. plugin: shell +category_id: 2013.com.canonical.plainbox::info id: info/bootchart _description: Bootchart information. requires: package.name == 'bootchart' or package.name == 'pybootchartgui' @@ -195,6 +219,7 @@ requires: command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"' plugin: attachment + category_id: 2013.com.canonical.plainbox::info id: info/hdparm_`ls /sys$path/block`.txt estimated_duration: 1.0 requires: @@ -205,6 +230,7 @@ command: EOF plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: bootchart.png depends: info/bootchart requires: package.name == 'pybootchartgui' @@ -214,6 +240,7 @@ command: [ -e "$file" ] && cat "$file" plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: bootchart.tgz depends: info/bootchart _description: Attaches the bootchart log for bootchart test runs. @@ -222,11 +249,13 @@ command: [ -e "$file" ] && cat "$file" plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: installer_bootchart.tgz command: [ -f /var/log/installer/bootchart.tgz ] && base64 /var/log/installer/bootchart.tgz _description: installs the installer bootchart tarball if it exists. plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: installer_debug.gz user: root command: [ -f /var/log/installer/debug ] && gzip -9 -c /var/log/installer/debug | base64 @@ -234,6 +263,7 @@ estimated_duration: 0.1 _description: Attaches the installer debug log if it exists. plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: info/touchpad_driver requires: device.category == 'TOUCHPAD' command: touchpad_driver_info @@ -243,6 +273,7 @@ _description: the system. plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: info/audio_device_driver requires: package.name == 'pulseaudio-utils' @@ -253,6 +284,7 @@ estimated_duration: 0.177 _description: Lists the device driver and version for all audio devices. plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: info/network_devices requires: device.category == 'NETWORK' or device.category == 'WIRELESS' @@ -263,17 +295,20 @@ estimated_duration: 0.550 _description: Provides information about network devices plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: info/mobile_broadband_info command: broadband_info estimated_duration: 0.5 _description: Provides information about wwan/mobile broadband devices plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: info/xrandr command: xrandr -q --verbose _description: Provides information about displays attached to the system plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: info/disk_partitions estimated_duration: 1.0 user: root @@ -281,6 +316,7 @@ command: parted -l -s _description: Attaches information about disk partitions plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: info/buildstamp estimated_duration: 0.1 _description: Attaches the buildstamp identifier for the preinstalled OS @@ -288,6 +324,7 @@ _summary: Attach /etc/buildstamp command: [ -s /etc/buildstamp ] && cat /etc/buildstamp plugin: shell +category_id: 2013.com.canonical.plainbox::info id: info/detect_recovery requires: package.name == 'dell-recovery' or package.name == 'ubuntu-recovery' user: root @@ -297,6 +334,7 @@ _description: Check existence of recovery partition command: recovery_info plugin: shell +category_id: 2013.com.canonical.plainbox::info id: info/detect_dell_recovery depends: info/detect_recovery requires: package.name == 'dell-recovery' @@ -307,6 +345,7 @@ _description: Check the recovery type is dell or not command: recovery_info checktype DELL plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: recovery_info_attachment depends: info/detect_recovery requires: package.name == 'dell-recovery' or package.name == 'ubuntu-recovery' @@ -323,6 +362,7 @@ _description: command: recovery_info version plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: dell_bto_xml_attachment depends: info/detect_dell_recovery requires: package.name == 'dell-recovery' @@ -337,6 +377,7 @@ _description: command: recovery_info file bto.xml plugin: attachment +category_id: 2013.com.canonical.plainbox::info id: info/kvm_output estimated_duration: 0.1 _description: Attaches the debug log from the virtualization run @@ -344,6 +385,7 @@ command: [ -f $PLAINBOX_SESSION_SHARE/virt_debug ] && cat $PLAINBOX_SESSION_SHAR id: kernel_cmdline_attachment plugin: attachment +category_id: 2013.com.canonical.plainbox::info command: cat /proc/cmdline estimated_duration: 0.005 _description: Attaches the kernel command line used to boot diff --git a/jobs/input.txt.in b/jobs/input.txt.in index 08b7257..33751c6 100644 --- a/jobs/input.txt.in +++ b/jobs/input.txt.in @@ -8,6 +8,7 @@ _description: Pointing device tests. command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=MOUSE" -w "category=TOUCHPAD" -w "category=TOUCHSCREEN"' plugin: manual + category_id: 2013.com.canonical.plainbox::input id: input/pointing_`echo "${product}_${category}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`_`basename $path` requires: device.path == "$path" description: @@ -20,6 +21,7 @@ command: EOF plugin: manual +category_id: 2013.com.canonical.plainbox::input id: input/mouse estimated_duration: 30.0 _description: @@ -32,6 +34,7 @@ _description: Did the pointing device work as expected? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::input id: input/keyboard estimated_duration: 30.0 command: keyboard_test @@ -46,6 +49,7 @@ _description: Is your keyboard working properly? plugin: manual +category_id: 2013.com.canonical.plainbox::input id: input/accelerometer_verify estimated_duration: 30.0 _description: @@ -57,6 +61,7 @@ _description: Is this system supposed to have an accelerometer? plugin: user-interact +category_id: 2013.com.canonical.plainbox::input id: input/accelerometer estimated_duration: 60.0 user: root @@ -82,6 +87,7 @@ _description: Click tests for pointing devices. command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=MOUSE" -w "category=TOUCHPAD"' plugin: manual + category_id: 2013.com.canonical.plainbox::input estimated_duration: 30.0 id: input/clicking_`echo "${product}_${category}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`_`basename $path` requires: device.path == "$path" diff --git a/jobs/keys.txt.in b/jobs/keys.txt.in index 4e2ecbf..b74ad4c 100644 --- a/jobs/keys.txt.in +++ b/jobs/keys.txt.in @@ -1,4 +1,5 @@ plugin: user-interact +category_id: 2013.com.canonical.plainbox::keys id: keys/lock-screen estimated_duration: 30.0 requires: @@ -17,6 +18,7 @@ _description: automatically selected result. plugin: manual +category_id: 2013.com.canonical.plainbox::keys id: keys/brightness estimated_duration: 30.0 requires: dmi.product in ['Notebook','Laptop','Portable'] @@ -29,6 +31,7 @@ _description: Did the brightness change following to your key presses? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::keys id: keys/volume estimated_duration: 30.0 requires: @@ -46,6 +49,7 @@ _description: Do the keys work as expected? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::keys id: keys/mute estimated_duration: 30.0 requires: @@ -62,6 +66,7 @@ _description: Does the mute key work as expected? plugin: manual +category_id: 2013.com.canonical.plainbox::keys id: keys/sleep estimated_duration: 90.0 requires: @@ -76,6 +81,7 @@ _description: Did the system go to sleep after pressing the sleep key? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::keys id: keys/battery-info estimated_duration: 30.0 requires: dmi.product in ['Notebook','Laptop','Portable'] @@ -93,6 +99,7 @@ _description: Did the Battery Info key work as expected? plugin: manual +category_id: 2013.com.canonical.plainbox::keys id: keys/wireless estimated_duration: 90.0 requires: dmi.product in ['Notebook','Laptop','Portable'] @@ -112,6 +119,7 @@ _description: only consider the functionality of the wifi itself here.) plugin: user-interact +category_id: 2013.com.canonical.plainbox::keys id: keys/media-control estimated_duration: 30.0 requires: @@ -129,6 +137,7 @@ _description: Do the keys work as expected? plugin: user-interact +category_id: 2013.com.canonical.plainbox::keys id: keys/super estimated_duration: 30.0 requires: @@ -145,6 +154,7 @@ _description: Does the super key work as expected? plugin: manual +category_id: 2013.com.canonical.plainbox::keys id: keys/video-out estimated_duration: 120.0 requires: dmi.product in ['Notebook','Laptop','Portable'] @@ -158,6 +168,7 @@ _description: Check that the video signal can be mirrored, extended, displayed on external or onboard only. plugin: manual +category_id: 2013.com.canonical.plainbox::keys id: keys/touchpad estimated_duration: 30.0 requires: dmi.product in ['Notebook','Laptop','Portable'] @@ -172,6 +183,7 @@ _description: Verify the touchpad has been disabled and re-enabled. plugin: manual +category_id: 2013.com.canonical.plainbox::keys id: keys/keyboard-backlight estimated_duration: 30.0 requires: dmi.product in ['Notebook','Laptop','Portable'] @@ -187,6 +199,7 @@ _description: Did the keyboard backlight state change on each press? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::keys id: keys/microphone-mute estimated_duration: 60.0 requires: @@ -213,6 +226,7 @@ _description: Does the microphone mute key work as expected? plugin: manual +category_id: 2013.com.canonical.plainbox::keys id: keys/hibernate estimated_duration: 120.0 requires: dmi.product in ['Notebook','Laptop','Portable'] @@ -227,6 +241,7 @@ _description: Did the system go to hibernate after pressing the hibernate key? plugin: manual +category_id: 2013.com.canonical.plainbox::keys id: keys/keyboard-overhead-light estimated_duration: 30.0 requires: dmi.product in ['Notebook','Laptop','Portable'] diff --git a/jobs/led.txt.in b/jobs/led.txt.in index c863271..5dd216d 100644 --- a/jobs/led.txt.in +++ b/jobs/led.txt.in @@ -1,4 +1,5 @@ plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/power estimated_duration: 15.0 _description: @@ -10,6 +11,7 @@ _description: Does the power LED light as expected? plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/power-blink-suspend estimated_duration: 30.0 depends: suspend/suspend_advanced @@ -22,6 +24,7 @@ _description: Did the Power LED blink or change color while the system was suspended for the previous suspend test? plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/suspend estimated_duration: 30.0 depends: suspend/suspend_advanced @@ -36,6 +39,7 @@ _description: Did the Suspend LED blink or change color while the system was suspended? plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/battery-charging estimated_duration: 120.0 _description: @@ -48,6 +52,7 @@ _description: Did the battery indicator LED turn orange? plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/battery-charged estimated_duration: 240.0 _description: @@ -61,6 +66,7 @@ _description: Does the orange battery LED shut off when system is fully charged? plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/battery-low estimated_duration: 30.0 _description: @@ -73,6 +79,7 @@ _description: Does the LED light orange when battery is low? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::led id: led/hdd estimated_duration: 30.0 command: led_hdd_test @@ -86,6 +93,7 @@ _description: Did the HDD LED light? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::led id: led/numeric-keypad estimated_duration: 30.0 command: keyboard_test @@ -101,6 +109,7 @@ _description: 2. Numbers should only be entered in the keyboard verification window when the LED is on plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/caps-lock estimated_duration: 30.0 _description: @@ -113,6 +122,7 @@ _description: Did the Cap Keys LED light as expected? plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/wlan depends: keys/wireless estimated_duration: 60.0 @@ -127,6 +137,7 @@ _description: Did the WLAN LED turn on or change color as expected? plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/wlan-disabled estimated_duration: 30.0 depends: keys/wireless @@ -141,6 +152,7 @@ _description: Did the WLAN LED turn off or change color as expected? plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/bluetooth estimated_duration: 60.0 depends: bluetooth/detect-output @@ -156,6 +168,7 @@ _description: Did the bluetooth LED turn off and on twice? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::led id: led/camera estimated_duration: 30.0 depends: camera/detect @@ -170,6 +183,7 @@ _description: Did the camera LED light? plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/touchpad estimated_duration: 30.0 _description: @@ -184,6 +198,7 @@ _description: 3. When the LED is off, the mouse pointer should not move on touchpad usage plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/wireless estimated_duration: 120.0 _description: @@ -200,6 +215,7 @@ _description: Did the WLAN/Bluetooth LED light as expected? plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/mute depends: keys/mute estimated_duration: 30.0 @@ -214,6 +230,7 @@ _description: Did the Audio LED turn on and off or change color as expected? plugin: manual +category_id: 2013.com.canonical.plainbox::led id: led/microphone-mute estimated_duration: 30.0 depends: keys/microphone-mute diff --git a/jobs/mediacard.txt.in b/jobs/mediacard.txt.in index 71fb8ab..92042a9 100644 --- a/jobs/mediacard.txt.in +++ b/jobs/mediacard.txt.in @@ -1,4 +1,5 @@ plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/mmc-insert estimated_duration: 30.0 command: removable_storage_watcher --memorycard insert sdio usb scsi @@ -16,6 +17,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/mmc-storage estimated_duration: 30.0 depends: mediacard/mmc-insert @@ -26,6 +28,7 @@ _description: test is run. It tests reading and writing to the MMC card. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/mmc-remove estimated_duration: 30.0 depends: mediacard/mmc-insert @@ -42,6 +45,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sd-insert estimated_duration: 30.0 command: removable_storage_watcher --memorycard insert sdio usb scsi @@ -59,6 +63,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sd-storage estimated_duration: 30.0 depends: mediacard/sd-insert @@ -69,6 +74,7 @@ _description: test is run. It tests reading and writing to the SD card. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sd-remove estimated_duration: 30.0 depends: mediacard/sd-insert @@ -85,6 +91,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sd-preinserted estimated_duration: 30.0 user: root @@ -96,6 +103,7 @@ _description: It is intended for SRU automated testing. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sdhc-insert estimated_duration: 30.0 command: removable_storage_watcher --memorycard insert sdio usb scsi @@ -114,6 +122,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sdhc-storage estimated_duration: 30.0 depends: mediacard/sdhc-insert @@ -124,6 +133,7 @@ _description: test is run. It tests reading and writing to the SDHC card. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sdhc-remove estimated_duration: 30.0 depends: mediacard/sdhc-insert @@ -140,6 +150,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/cf-insert estimated_duration: 30.0 command: removable_storage_watcher --memorycard insert sdio usb scsi @@ -157,6 +168,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/cf-storage estimated_duration: 30.0 depends: mediacard/cf-insert @@ -167,6 +179,7 @@ _description: test is run. It tests reading and writing to the CF card. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/cf-remove depends: mediacard/cf-storage estimated_duration: 30.0 @@ -183,6 +196,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sdxc-insert estimated_duration: 30.0 command: removable_storage_watcher --memorycard insert sdio usb scsi @@ -200,6 +214,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sdxc-storage estimated_duration: 30.0 depends: mediacard/sdxc-insert @@ -210,6 +225,7 @@ _description: test is run. It tests reading and writing to the SDXC card. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sdxc-remove estimated_duration: 30.0 depends: mediacard/sdxc-insert @@ -226,6 +242,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/ms-insert estimated_duration: 30.0 command: removable_storage_watcher --memorycard insert sdio usb scsi @@ -243,6 +260,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/ms-storage estimated_duration: 30.0 depends: mediacard/ms-insert @@ -253,6 +271,7 @@ _description: test is run. It tests reading and writing to the MS card. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/ms-remove estimated_duration: 30.0 depends: mediacard/ms-insert @@ -269,6 +288,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/msp-insert estimated_duration: 30.0 command: removable_storage_watcher --memorycard insert sdio usb scsi @@ -286,6 +306,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/msp-storage estimated_duration: 30.0 depends: mediacard/msp-insert @@ -296,6 +317,7 @@ _description: test is run. It tests reading and writing to the MSP card. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/msp-remove estimated_duration: 30.0 depends: mediacard/msp-insert @@ -312,6 +334,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/xd-insert estimated_duration: 30.0 command: removable_storage_watcher --memorycard insert sdio usb scsi @@ -329,6 +352,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/xd-storage estimated_duration: 30.0 depends: mediacard/xd-insert @@ -339,6 +363,7 @@ _description: test is run. It tests reading and writing to the xD card. plugin: user-interact +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/xd-remove estimated_duration: 30.0 depends: mediacard/xd-insert @@ -355,6 +380,7 @@ _description: automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sd-performance-manual depends: mediacard/sd-insert estimated_duration: 120.0 @@ -370,6 +396,7 @@ _description: Did the results of the test match the expected performance of the inserted device? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sdhc-performance-manual depends: mediacard/sdhc-insert estimated_duration: 120.0 @@ -385,6 +412,7 @@ _description: Did the results of the test match the expected performance of the inserted device? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/mmc-performance-manual depends: mediacard/mmc-insert estimated_duration: 120.0 @@ -400,6 +428,7 @@ _description: Did the results of the test match the expected performance of the inserted device? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/cf-performance-manual depends: mediacard/cf-insert estimated_duration: 120.0 @@ -415,6 +444,7 @@ _description: Did the results of the test match the expected performance of the inserted device? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/sdxc-performance-manual depends: mediacard/sdxc-insert estimated_duration: 120.0 @@ -430,6 +460,7 @@ _description: Did the results of the test match the expected performance of the inserted device? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/ms-performance-manual depends: mediacard/ms-insert estimated_duration: 120.0 @@ -445,6 +476,7 @@ _description: Did the results of the test match the expected performance of the inserted device? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/msp-performance-manual depends: mediacard/msp-insert estimated_duration: 120.0 @@ -460,6 +492,7 @@ _description: Did the results of the test match the expected performance of the inserted device? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::mediacard id: mediacard/xd-performance-manual depends: mediacard/xd-insert estimated_duration: 120.0 diff --git a/jobs/memory.txt.in b/jobs/memory.txt.in index 136d984..0f66efd 100644 --- a/jobs/memory.txt.in +++ b/jobs/memory.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::memory id: memory/info estimated_duration: 5.0 user: root @@ -8,6 +9,7 @@ _description: the size of the memory modules detected by DMI. plugin: shell +category_id: 2013.com.canonical.plainbox::memory id: memory/check estimated_duration: 1000.0 user: root diff --git a/jobs/mir.txt.in b/jobs/mir.txt.in index e71089b..15aaa6d 100644 --- a/jobs/mir.txt.in +++ b/jobs/mir.txt.in @@ -7,6 +7,7 @@ command: cat << 'EOF' | run_templates -s "mir_integration_tests --gtest_list_tests | sed -n '/\.$/s/\.$//p'" estimated_duration: 0.5 plugin: shell + category_id: 2013.com.canonical.plainbox::mir id: mir/integration/$1 requires: package.name == 'mir-test-tools' command: mir_integration_tests --gtest_filter=$1* @@ -22,6 +23,7 @@ command: cat << 'EOF' | run_templates -s "mir_acceptance_tests --gtest_list_tests | sed -n '/\.$/s/\.$//p'" estimated_duration: 0.5 plugin: shell + category_id: 2013.com.canonical.plainbox::mir id: mir/acceptance/$1 requires: package.name == 'mir-test-tools' command: mir_acceptance_tests --gtest_filter=$1* diff --git a/jobs/miscellanea.txt.in b/jobs/miscellanea.txt.in index ac5dbcd..a4d174d 100644 --- a/jobs/miscellanea.txt.in +++ b/jobs/miscellanea.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/submission-resources depends: cpuinfo @@ -17,6 +18,7 @@ _description: A meta-job depending on the resources needed for a valid submission. plugin: manual +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/tester-info _description: PURPOSE: @@ -31,6 +33,7 @@ _description: Nothing to verify for this test plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/chvt requires: package.name == 'kbd' @@ -45,6 +48,7 @@ _description: Did your screen change temporarily to a text console and then switch back to your current session? plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/fwts_test estimated_duration: 1.2 requires: @@ -57,12 +61,14 @@ command: fwts_test -l $PLAINBOX_SESSION_SHARE/fwts_results.log plugin: attachment +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/fwts_results.log command: [[ -e ${PLAINBOX_SESSION_SHARE}/fwts_results.log ]] && cat ${PLAINBOX_SESSION_SHARE}/fwts_results.log _description: Attaches the FWTS results log to the submission plugin: attachment +category_id: 2013.com.canonical.plainbox::miscellanea estimated_duration: 0.5 id: miscellanea/fwts_results_hwe.log.gz command: @@ -71,6 +77,7 @@ _description: Attaches the FWTS results log to the submission (to HWE) _summary: Attach FWTS log to submission (to HWE) plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/ipmi_test requires: package.name == 'ipmitool' user: root @@ -79,6 +86,7 @@ _description: This will run some basic connectivity tests against a BMC, verifying that IPMI works. plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea estimated_duration: 0.5 id: miscellanea/boot_mode requires: @@ -94,6 +102,7 @@ command: fi plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/bmc_info requires: package.name == 'ipmitool' estimated_duration: 0.5 @@ -105,6 +114,7 @@ _summary: Gather BMC Identification info plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/cpus_are_not_samples requires: package.name == 'dmidecode' estimated_duration: 0.5 @@ -116,6 +126,7 @@ _summary: CPU sanity check plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/dmitest_server requires: package.name == 'dmidecode' estimated_duration: 0.5 @@ -127,6 +138,7 @@ _summary: DMI ID sanity check (servers) plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/dmitest_client requires: package.name == 'dmidecode' estimated_duration: 0.5 @@ -138,6 +150,7 @@ _summary: DMI ID sanity check (desktops/laptops) plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/is_laptop user: root _description: @@ -145,6 +158,7 @@ _description: command: check_is_laptop plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/apport-directory requires: package.name == 'apport' command: if [ -d /var/crash ]; then if [ $(find /var/crash -type f | grep -v .lock | wc -l) -eq 0 ]; then echo "/var/crash is empty"; else echo `ls /var/crash`; false; fi; else echo "/var/crash does not exist"; fi @@ -152,6 +166,7 @@ _description: Test that the /var/crash directory doesn't contain anything. Lists the files contained within if it does, or echoes the status of the directory (doesn't exist/is empty) plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/sources-list command: sources_test $SOURCES_LIST "$REPOSITORIES" _description: Checks that a specified sources list file contains the requested repositories @@ -164,6 +179,7 @@ _description: Automated job to generate the PXE verification test for each NIC. command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: manual + category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/pxe_boot_$2 description: PURPOSE: @@ -185,6 +201,7 @@ _description: Automated job to generate the Remote Shared IPMI verification test command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: manual + category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/remote_shared_ipmi_$2 description: PURPOSE: @@ -198,6 +215,7 @@ command: EOF plugin: manual +category_id: 2013.com.canonical.plainbox::miscellanea id: miscellanea/remote_dedicated_ipmi _description: PURPOSE: @@ -222,6 +240,7 @@ command: fwts_test -l $PLAINBOX_SESSION_SHARE/fwts_oops_results.log -t oops plugin: attachment +category_id: 2013.com.canonical.plainbox::miscellanea estimated_duration: 0.5 id: miscellanea/oops_results.log command: @@ -229,6 +248,7 @@ command: _description: Attaches the FWTS oops results log to the submission plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea estimated_duration: 0.1 id: miscellanea/get_maas_version command: @@ -242,6 +262,7 @@ _description: If system was installed via MAAS from a cert server, the MAAS vers _summary: Gets the version of MAAS used to deploy the SUT plugin: shell +category_id: 2013.com.canonical.plainbox::miscellanea estimated_duration: 30.0 id: miscellanea/get_make_and_model user: root diff --git a/jobs/mobilebroadband.txt.in b/jobs/mobilebroadband.txt.in index 2f76a9c..d28cd3a 100644 --- a/jobs/mobilebroadband.txt.in +++ b/jobs/mobilebroadband.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::mobilebroadband id: mobilebroadband/gsm_connection estimated_duration: 20.0 requires: @@ -28,6 +29,7 @@ command: _description: Creates a mobile broadband connection for a GSM based modem and checks the connection to ensure it's working. plugin: shell +category_id: 2013.com.canonical.plainbox::mobilebroadband id: mobilebroadband/cdma_connection estimated_duration: 20.0 requires: @@ -65,6 +67,7 @@ depends: mobilebroadband/gsm_connection command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::mobilebroadband user: root id: mobilebroadband/maximum_bandwidth_$2 requires: device.path == "$1" @@ -95,6 +98,7 @@ depends: mobilebroadband/cdma_connection command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::mobilebroadband user: root id: mobilebroadband/maximum_bandwidth_$2 requires: device.path == "$1" diff --git a/jobs/monitor.txt.in b/jobs/monitor.txt.in index 78ce114..e1dc7bb 100644 --- a/jobs/monitor.txt.in +++ b/jobs/monitor.txt.in @@ -7,6 +7,7 @@ command: id: monitor/`echo ${index}`_vga_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: display.vga == 'supported' or display.dp == 'supported' plugin: manual + category_id: 2013.com.canonical.plainbox::monitor _description: PURPOSE: This test will check your VGA port. @@ -26,6 +27,7 @@ command: id: monitor/`echo ${index}`_dvi_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: display.dvi == 'supported' plugin: manual + category_id: 2013.com.canonical.plainbox::monitor _description: PURPOSE: This test will check your DVI port. @@ -45,6 +47,7 @@ command: id: monitor/`echo ${index}`_displayport_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: display.dp == 'supported' plugin: manual + category_id: 2013.com.canonical.plainbox::monitor _description: PURPOSE: This test will check your DisplayPort port. @@ -64,6 +67,7 @@ command: id: monitor/`echo ${index}`_hdmi_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: display.hdmi == 'supported' plugin: manual + category_id: 2013.com.canonical.plainbox::monitor _description: PURPOSE: This test will check your HDMI port. @@ -83,6 +87,7 @@ command: id: monitor/`echo ${index}`_svideo_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: display.svideo == 'supported' plugin: manual + category_id: 2013.com.canonical.plainbox::monitor _description: PURPOSE: This test will check your S-VIDEO port. @@ -102,6 +107,7 @@ command: id: monitor/`echo ${index}`_rca_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: display.rca == 'supported' plugin: manual + category_id: 2013.com.canonical.plainbox::monitor _description: PURPOSE: This test will check your RCA port. @@ -121,6 +127,7 @@ command: id: monitor/`echo ${index}`_multi-head_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: dmi.product in ['Desktop','Low Profile Desktop','Tower','Mini Tower'] plugin: manual + category_id: 2013.com.canonical.plainbox::monitor _description: PURPOSE: This test verifies that multi-monitor output works on your desktop system. This is NOT the same test as the external monitor tests you would run on your laptop. You will need two monitors to perform this test. @@ -142,6 +149,7 @@ command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' id: monitor/`echo ${index}`_powersaving_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::monitor command: xset dpms force off _description: PURPOSE: @@ -162,6 +170,7 @@ command: id: monitor/`echo ${index}`_dim_brightness_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: dmi.product in ['Notebook','Laptop','Portable','All In One','All-In-One','AIO'] plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::monitor user: root command: brightness_test _description: @@ -186,6 +195,7 @@ command: requires: manifest.has_thunderbolt == 'True' estimated_duration: 15.0 plugin: manual + category_id: 2013.com.canonical.plainbox::monitor _summary: Display connected via Thunderbolt for $product _description: PURPOSE: diff --git a/jobs/networking.txt.in b/jobs/networking.txt.in index c98ded8..d5eab0f 100644 --- a/jobs/networking.txt.in +++ b/jobs/networking.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::networking id: networking/gateway_ping depends: ethernet/detect command: gateway_ping_test @@ -13,6 +14,7 @@ _description: Network Information command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::networking id: networking/info_$2 requires: device.path == "$1" command: network_info $2 | zenity --text-info --title="$2" @@ -26,6 +28,7 @@ command: EOF plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::networking id: networking/modem_connection command: network_check _description: @@ -43,18 +46,21 @@ _description: Did a notification show and was the connection correctly established? plugin: shell +category_id: 2013.com.canonical.plainbox::networking id: networking/ping command: gateway_ping_test $CHECKBOX_SERVER _description: Automated test case to verify availability of some system on the network using ICMP ECHO packets. plugin: shell +category_id: 2013.com.canonical.plainbox::networking id: networking/http command: wget -SO /dev/null http://$TRANSFER_SERVER _description: Automated test case to make sure that it's possible to download files through HTTP plugin: shell +category_id: 2013.com.canonical.plainbox::networking id: networking/ntp requires: package.name == 'ntpdate' user: root @@ -62,12 +68,14 @@ command: network_ntp_test _description: Test to see if we can sync local clock to an NTP server plugin: shell +category_id: 2013.com.canonical.plainbox::networking id: networking/ssh requires: package.name == 'openssh-client' command: if [ $CHECKBOX_SERVER ]; then ssh -q -o 'StrictHostKeyChecking=no' -o "UserKnownHostsFile=/tmp/ssh_test_$$" -l ubuntu $CHECKBOX_SERVER "uname -a" && rm /tmp/ssh_test_$$; fi _description: Verify that an installation of checkbox-server on the network can be reached over SSH. plugin: shell +category_id: 2013.com.canonical.plainbox::networking id: networking/printer requires: package.name == 'cups-client' command: network_printer_test -s $CHECKBOX_SERVER diff --git a/jobs/optical.txt.in b/jobs/optical.txt.in index b7be742..20ba986 100644 --- a/jobs/optical.txt.in +++ b/jobs/optical.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::optical id: optical/detect requires: device.category == 'CDROM' estimated_duration: 1.2 @@ -18,6 +19,7 @@ _description: Optical read test. command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=CDROM"' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::optical id: optical/read_`ls /sys$path/block` requires: device.path == "$path" estimated_duration: 120.0 @@ -44,6 +46,7 @@ _description: Automated optical read test. command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=CDROM"' plugin: shell + category_id: 2013.com.canonical.plainbox::optical id: optical/read-automated_`ls /sys$path/block` estimated_duration: 120.0 requires: device.path == "$path" @@ -63,6 +66,7 @@ _description: CD write test. command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=CDROM"' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::optical id: optical/cdrom-write_`ls /sys$path/block` estimated_duration: 120.0 requires: @@ -92,6 +96,7 @@ _description: Automated CD write test command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=CDROM"' plugin: shell + category_id: 2013.com.canonical.plainbox::optical id: optical/cdrom-write-automated_`ls /sys$path/block` estimated_duration: 120.0 requires: @@ -104,6 +109,7 @@ command: EOF plugin: manual +category_id: 2013.com.canonical.plainbox::optical id: optical/cdrom-audio-playback depends: optical/read estimated_duration: 120.0 @@ -133,6 +139,7 @@ _description: DVD write test. command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=CDROM"' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::optical id: optical/dvd-write_`ls /sys$path/block` requires: device.path == "$path" @@ -162,6 +169,7 @@ _description: Automated DVD write test. command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=CDROM"' plugin: shell + category_id: 2013.com.canonical.plainbox::optical id: optical/dvd-write-automated_`ls /sys$path/block` estimated_duration: 120.0 requires: @@ -174,6 +182,7 @@ command: EOF plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::optical id: optical/dvd_playback command: totem /media/cdrom estimated_duration: 120.0 @@ -198,6 +207,7 @@ _description: Automated Blu-Ray read test. command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=CDROM"' plugin: user-interact + category_id: 2013.com.canonical.plainbox::optical id: optical/bluray-read_`ls /sys$path/block` estimated_duration: 120.0 requires: @@ -225,6 +235,7 @@ _description: Automated Blu-Ray write test. command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=CDROM"' plugin: user-interact + category_id: 2013.com.canonical.plainbox::optical id: optical/bluray-write_`ls /sys$path/block` requires: device.path == "$path" diff --git a/jobs/peripheral.txt.in b/jobs/peripheral.txt.in index b21a345..a41e83e 100644 --- a/jobs/peripheral.txt.in +++ b/jobs/peripheral.txt.in @@ -1,4 +1,5 @@ plugin: manual +category_id: 2013.com.canonical.plainbox::peripheral id: peripheral/printer _description: PURPOSE: @@ -13,6 +14,7 @@ _description: Were you able to print a test page to the network printer? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::peripheral id: peripheral/external-usb-modem command: network_check _description: @@ -31,6 +33,7 @@ _description: Was the connection correctly established? plugin: shell +category_id: 2013.com.canonical.plainbox::peripheral id: peripheral/external-usb-modem-http depends: peripheral/external-usb-modem command: wget -SO /dev/null http://$TRANSFER_SERVER diff --git a/jobs/power-management.txt.in b/jobs/power-management.txt.in index af93f2c..26256af 100644 --- a/jobs/power-management.txt.in +++ b/jobs/power-management.txt.in @@ -1,4 +1,5 @@ plugin: manual +category_id: 2013.com.canonical.plainbox::power-management id: power-management/shutdown-boot estimated_duration: 300.0 _description: @@ -12,6 +13,7 @@ _description: Did the system shutdown and rebooted correctly? plugin: shell +category_id: 2013.com.canonical.plainbox::power-management id: power-management/fwts_wakealarm environ: PLAINBOX_SESSION_SHARE estimated_duration: 25.0 @@ -22,6 +24,7 @@ requires: command: fwts_test -f aborted -t wakealarm -l $PLAINBOX_SESSION_SHARE/fwts-wakealarm.log plugin: attachment +category_id: 2013.com.canonical.plainbox::power-management id: power-management/fwts_wakealarm-log-attach depends: power-management/fwts_wakealarm estimated_duration: 2.0 @@ -31,6 +34,7 @@ requires: command: [ -e ${PLAINBOX_SESSION_SHARE}/fwts-wakealarm.log ] && cat ${PLAINBOX_SESSION_SHARE}/fwts-wakealarm.log plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::power-management id: power-management/poweroff estimated_duration: 120.0 depends: power-management/fwts_wakealarm @@ -55,6 +59,7 @@ _description: select 'No'. plugin: attachment +category_id: 2013.com.canonical.plainbox::power-management id: power-management/poweroff-log-attach estimated_duration: 1.0 command: @@ -64,6 +69,7 @@ _description: This will attach any logs from the power-management/poweroff test to the results. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::power-management id: power-management/reboot estimated_duration: 120.0 user: root @@ -85,6 +91,7 @@ _description: If the machine successfully reboots, select Yes then select Next. plugin: attachment +category_id: 2013.com.canonical.plainbox::power-management id: power-management/reboot-log-attach estimated_duration: 1.0 command: @@ -94,6 +101,7 @@ _description: This will attach any logs from the power-management/reboot test to the results. plugin: manual +category_id: 2013.com.canonical.plainbox::power-management id: power-management/lid estimated_duration: 120.0 requires: dmi.product in ['Notebook','Laptop','Portable'] @@ -106,6 +114,7 @@ _description: Does closing your laptop lid cause your system to suspend? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::power-management id: power-management/lid_close estimated_duration: 60.0 requires: device.product == 'Lid Switch' @@ -125,6 +134,7 @@ _description: Did the screen turn off while the lid was closed? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::power-management id: power-management/lid_open estimated_duration: 60.0 requires: device.product == 'Lid Switch' @@ -146,6 +156,7 @@ _description: Did the system resume when the lid was opened? plugin: shell +category_id: 2013.com.canonical.plainbox::power-management id: power-management/rtc requires: rtc.state == 'supported' @@ -158,6 +169,7 @@ _description: Verify that the Real-time clock (RTC) device functions properly, if present plugin: shell +category_id: 2013.com.canonical.plainbox::power-management id: power-management/tickless_idle estimated_duration: 1.0 requires: cpuinfo.platform in ('i386', 'x86_64', 'ppc64el', 'pSeries') @@ -166,6 +178,7 @@ command: zgrep 'CONFIG_NO_HZ=y' /boot/config-`uname -r` >/dev/null 2>&1 || ( echo "WARNING: Tickless Idle is NOT set" >&2 && exit 1 ) plugin: manual +category_id: 2013.com.canonical.plainbox::power-management id: power-management/unplug_ac estimated_duration: 30.0 _description: @@ -177,6 +190,7 @@ _description: Was the laptop unplugged from AC? plugin: shell +category_id: 2013.com.canonical.plainbox::power-management id: power-management/battery_drain_idle estimated_duration: 90.0 requires: package.name == 'upower' @@ -187,6 +201,7 @@ command: battery_test -t 90 --idle plugin: shell +category_id: 2013.com.canonical.plainbox::power-management id: power-management/battery_drain_movie estimated_duration: 90.0 requires: @@ -198,6 +213,7 @@ command: battery_test -t 90 --movie $MOVIE_VAR plugin: shell +category_id: 2013.com.canonical.plainbox::power-management id: power-management/battery_drain_sleep estimated_duration: 140.0 user: root @@ -211,6 +227,7 @@ command: battery_test -t 120 --sleep plugin: manual +category_id: 2013.com.canonical.plainbox::power-management id: power-management/plug_ac estimated_duration: 30.0 depends: power-management/battery_drain_idle power-management/battery_drain_movie power-management/battery_drain_sleep @@ -224,6 +241,7 @@ _description: Was the laptop plugged into AC? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::power-management id: power-management/reboot_manual estimated_duration: 120.0 user:root @@ -243,6 +261,7 @@ _description: Did the system reboot correctly? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::power-management id: power-management/shutdown_manual estimated_duration: 120.0 user: root diff --git a/jobs/rendercheck.txt.in b/jobs/rendercheck.txt.in index e665d12..c739b8d 100644 --- a/jobs/rendercheck.txt.in +++ b/jobs/rendercheck.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::graphics id: rendercheck/tests requires: package.name == 'x11-apps' @@ -7,6 +8,7 @@ _description: Runs all of the rendercheck test suites. This test can take a few minutes. plugin: attachment +category_id: 2013.com.canonical.plainbox::graphics id: rendercheck/tarball depends: rendercheck/tests command: [ -e $PLAINBOX_SESSION_SHARE/rendercheck-results ] && tar cvfz $PLAINBOX_SESSION_SHARE/rendercheck-results.tar.gz $PLAINBOX_SESSION_SHARE/rendercheck-results && cat $PLAINBOX_SESSION_SHARE/rendercheck-results.tar.gz diff --git a/jobs/stress.txt.in b/jobs/stress.txt.in index 855dfe3..0b12618 100644 --- a/jobs/stress.txt.in +++ b/jobs/stress.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/cpu_stress_test estimated_duration: 7200.0 requires: @@ -10,6 +11,7 @@ _description: Create jobs that use the CPU as much as possible for two hours. The test is considered passed if the system does not freeze. plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: power-management/hibernate_30_cycles estimated_duration: 5400.00 depends: @@ -32,6 +34,7 @@ _description: This is an automated stress test that will force the system to hibernate/resume for 30 cycles plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: power-management/hibernate-30-cycles-log-check estimated_duration: 1.0 command: [ -e $PLAINBOX_SESSION_SHARE/hibernate_30_cycles.log ] && sleep_test_log_check -v s4 $PLAINBOX_SESSION_SHARE/hibernate_30_cycles.log @@ -39,6 +42,7 @@ _description: Automated check of the 30 cycle hibernate log for errors detected by fwts. plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: power-management/hibernate-30-cycle-log-attach estimated_duration: 1.0 command: [ -e $PLAINBOX_SESSION_SHARE/hibernate_30_cycles.log ] && cat $PLAINBOX_SESSION_SHARE/hibernate_30_cycles.log @@ -46,6 +50,7 @@ _description: Attaches the log from the 30 cycle Hibernate/Resume test if it exists plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: power-management/suspend_30_cycles_with_reboots estimated_duration: 7200 depends: @@ -58,6 +63,7 @@ _description: This is an automated stress test that will run a sequence of '30 suspend/resume cycles and one reboot' 3 times. plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: power-management/suspend_30_cycles estimated_duration: 2400.0 depends: @@ -78,6 +84,7 @@ _description: This is an automated stress test that will force the system to suspend/resume for 30 cycles. plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: power-management/suspend-30-cycles-log-check depends: power-management/suspend_30_cycles estimated_duration: 1.0 @@ -86,6 +93,7 @@ _description: Automated check of the 30 cycle hibernate log for errors detected by fwts. plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: power-management/suspend-30-cycle-log-attach estimated_duration: 1.0 depends: power-management/suspend_30_cycles @@ -94,6 +102,7 @@ _description: Attaches the log from the 30 cycle Suspend/Resume test if it exists plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: power-management/suspend-30-cycles-time-check estimated_duration: 1.0 depends: power-management/suspend_30_cycles @@ -102,6 +111,7 @@ _description: Checks the sleep times to ensure that a machine suspends and resumes within a given threshold plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/hibernate_250_cycles estimated_duration: 43400.0 depends: power-management/rtc @@ -120,6 +130,7 @@ _description: This is an automated stress test that will force the system to hibernate/resume for 250 cycles plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/hibernate-250-cycle-log-attach estimated_duration: 1.0 command: [ -e $PLAINBOX_SESSION_SHARE/hibernate_250_cycles.log ] && cat $PLAINBOX_SESSION_SHARE/hibernate_250_cycles.log @@ -127,6 +138,7 @@ _description: Attaches the log from the 250 cycle Hibernate/Resume test if it exists plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/suspend_250_cycles estimated_duration: 18750.0 depends: power-management/rtc @@ -145,6 +157,7 @@ _description: This is an automated stress test that will force the system to suspend/resume for 250 cycles. plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/suspend-250-cycle-log-attach estimated_duration: 1.0 command: [ -e $PLAINBOX_SESSION_SHARE/suspend_250_cycles.log ] && cat $PLAINBOX_SESSION_SHARE/suspend_250_cycles.log @@ -152,6 +165,7 @@ _description: Attaches the log from the 250 cycle Suspend/Resume test if it exists plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/suspend-250-cycles-time-check estimated_duration: 1.0 command: [ -e $PLAINBOX_SESSION_SHARE/suspend_250_cycles_times.log ] && sleep_time_check $PLAINBOX_SESSION_SHARE/suspend_250_cycles_times.log @@ -159,6 +173,7 @@ _description: Checks the sleep times to ensure that a machine suspends and resumes within a given threshold plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/reboot estimated_duration: 4500.0 requires: @@ -172,6 +187,7 @@ _description: Stress reboot system (100 cycles) plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/reboot_log estimated_duration: 1.0 depends: stress/reboot @@ -180,6 +196,7 @@ command: tar cvz $PLAINBOX_SESSION_SHARE/*reboot.100.log | base64 plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/reboot_30 requires: package.name == 'upstart' @@ -193,6 +210,7 @@ _description: Stress reboot system (30 cycles) plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/reboot_30_log depends: stress/reboot_30 command: @@ -200,6 +218,7 @@ command: tar cvz $PLAINBOX_SESSION_SHARE/*reboot.30.log | base64 plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/poweroff estimated_duration: 4500.0 requires: @@ -213,6 +232,7 @@ _description: Stress poweroff system (100 cycles) plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/poweroff_log estimated_duration: 1.0 depends: stress/poweroff @@ -221,6 +241,7 @@ command: tar cvz $PLAINBOX_SESSION_SHARE/*poweroff.100.log | base64 plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/poweroff_30 requires: package.name == 'upstart' @@ -234,6 +255,7 @@ _description: Stress poweroff system (30 cycles) plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/poweroff_30_log depends: stress/poweroff_30 command: @@ -241,6 +263,7 @@ command: tar cvz $PLAINBOX_SESSION_SHARE/*poweroff.30.log | base64 plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/reboot_30_check estimated_duration: 1.0 depends: stress/reboot_30 @@ -248,6 +271,7 @@ command: pm_log_check --log-level=notset $PLAINBOX_SESSION_SHARE/pm_test.reboot. _description: Check logs for the stress reboot (30 cycles) test case plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/reboot_30_check_log estimated_duration: 1.0 depends: stress/reboot_30_check @@ -256,6 +280,7 @@ command: tar cvz $PLAINBOX_SESSION_SHARE/pm_log_check_reboot.30.log | base64 plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/poweroff_30_check estimated_duration: 1.0 depends: stress/poweroff_30 @@ -263,6 +288,7 @@ command: pm_log_check --log-level=notset $PLAINBOX_SESSION_SHARE/pm_test.powerof _description: Check logs for the stress poweroff (30 cycles) test case plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/poweroff_30_check_log estimated_duration: 1.0 depends: stress/poweroff_30_check @@ -271,6 +297,7 @@ command: tar cvz $PLAINBOX_SESSION_SHARE/pm_log_check_poweroff.30.log | base64 plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/reboot_check estimated_duration: 1.0 depends: stress/reboot @@ -278,6 +305,7 @@ command: pm_log_check --log-level=notset $PLAINBOX_SESSION_SHARE/pm_test.reboot. _description: Check logs for the stress reboot (100 cycles) test case plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/reboot_check_log estimated_duration: 1.0 depends: stress/reboot_check @@ -286,6 +314,7 @@ command: tar cvz $PLAINBOX_SESSION_SHARE/pm_log_check_reboot.100.log | base64 plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/poweroff_check estimated_duration: 1.0 depends: stress/poweroff @@ -293,6 +322,7 @@ command: pm_log_check --log-level=notset $PLAINBOX_SESSION_SHARE/pm_test.powerof _description: Check logs for the stress poweroff (100 cycles) test case plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/poweroff_check_log estimated_duration: 1.0 depends: stress/poweroff_check @@ -301,6 +331,7 @@ command: tar cvz $PLAINBOX_SESSION_SHARE/pm_log_check_poweroff.100.log | base64 plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/graphics estimated_duration: 600.0 requires: @@ -312,6 +343,7 @@ _description: Run the graphics stress test. This test can take a few minutes. plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/graphics-tarball estimated_duration: 1.0 requires: @@ -320,6 +352,7 @@ command: [ -f $PLAINBOX_SESSION_SHARE/graphics-stress-results ] && tar cvz $PLAI _description: Attaches the graphics stress results to the submission. plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/usb estimated_duration: 710.0 user: root @@ -327,6 +360,7 @@ command: removable_storage_test -s 10240000 -c 100 -i 3 usb _description: Runs a test that transfers 100 10MB files 3 times to usb. plugin: user-interact +category_id: 2013.com.canonical.plainbox::stress id: stress/sdhc user: root _summary: Stress test for SDHC card @@ -346,6 +380,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::stress id: stress/network_restart estimated_duration: 1500.0 user: root @@ -354,12 +389,14 @@ command: network_restart -t 100 -o $PLAINBOX_SESSION_SHARE _description: Ping ubuntu.com and restart network interfaces 100 times plugin: attachment +category_id: 2013.com.canonical.plainbox::stress id: stress/network_restart_log estimated_duration: 1.0 depends: stress/network_restart command: file=$PLAINBOX_SESSION_SHARE/network_restart.log; if [ -e "$file" ]; then iconv -t 'ascii' -c "$file"; fi plugin: manual +category_id: 2013.com.canonical.plainbox::stress id: stress/wireless_hotkey estimated_duration: 60.0 requires: dmi.product in ['Notebook','Laptop','Portable'] diff --git a/jobs/suspend.txt.in b/jobs/suspend.txt.in index f60d938..e450b9f 100644 --- a/jobs/suspend.txt.in +++ b/jobs/suspend.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/network_before_suspend depends: ethernet/detect estimated_duration: 1.2 @@ -6,6 +7,7 @@ _description: Record the current network before suspending. command: set -o pipefail; gateway_ping_test | tee $PLAINBOX_SESSION_SHARE/network_before_suspend.txt plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/resolution_before_suspend estimated_duration: 1.2 _description: Record the current resolution before suspending. @@ -18,6 +20,7 @@ _summary: Record the current resolution before suspending. command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_resolution_before_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: graphics/`echo ${index}`_switch_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` estimated_duration: 1.2 @@ -26,6 +29,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/audio_before_suspend estimated_duration: 1.0 requires: @@ -35,12 +39,14 @@ _description: Record mixer settings before suspending. command: audio_settings store --file=$PLAINBOX_SESSION_SHARE/audio_settings_before_suspend plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/cpu_before_suspend estimated_duration: 1.2 _description: Verify that all the CPUs are online before suspending command: cpuinfo_resource > $PLAINBOX_SESSION_SHARE/cpuinfo_before_suspend plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/memory_before_suspend estimated_duration: 1.2 _description: @@ -48,6 +54,7 @@ _description: command: meminfo_resource > $PLAINBOX_SESSION_SHARE/meminfo_before_suspend plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_before_suspend depends: wireless/wireless_connection requires: device.category == 'WIRELESS' @@ -64,6 +71,7 @@ requires: command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: shell + category_id: 2013.com.canonical.plainbox::suspend id: suspend/iperf_before_suspend_ether_auto_$2 depends: ethernet/detect estimated_duration: 20.0 @@ -83,6 +91,7 @@ requires: command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=WIRELESS" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: shell + category_id: 2013.com.canonical.plainbox::suspend id: suspend/iperf_before_suspend_wifi_auto_$2 depends: wireless/wireless_connection estimated_duration: 20.0 @@ -96,6 +105,7 @@ _description: This is an automated test to gather some info on the current state of your network devices. If no devices are found, the test will exit with an error. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/iperf_before_suspend_mobilebroadband_gsm_auto depends: mobilebroadband/gsm_connection estimated_duration: 20.0 @@ -109,6 +119,7 @@ _description: This test executes iperf connection performance/stability against the broadband device found on the system before suspend. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/iperf_before_suspend_mobilebroadband_cdma_auto depends: mobilebroadband/cdma_connection estimated_duration: 20.0 @@ -123,6 +134,7 @@ _description: plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_before_suspend estimated_duration: 10.0 requires: @@ -150,6 +162,7 @@ _description: This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_send_before_suspend estimated_duration: 10.0 requires: @@ -177,6 +190,7 @@ _description: This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_browse_before_suspend estimated_duration: 10.0 requires: @@ -204,6 +218,7 @@ _description: This is an automated Bluetooth test. It emulates browsing on a remote device specified by the BTDEVADDR environment variable. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_get_before_suspend estimated_duration: 20.0 requires: @@ -231,6 +246,7 @@ _description: This is an automated Bluetooth test. It receives the given file from a remote host specified by the BTDEVADDR environment variable plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_before_suspend_manual estimated_duration: 120.0 requires: @@ -249,6 +265,7 @@ _description: Was the data correctly transferred? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/suspend_advanced requires: sleep.mem == 'supported' @@ -284,6 +301,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: sleep.mem == 'supported' @@ -316,6 +334,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/suspend_advanced_auto requires: sleep.mem == 'supported' @@ -334,6 +353,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_hybrid_sleep_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` user: root command: @@ -352,6 +372,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/suspend-single-log-check estimated_duration: 1.2 command: [ -e $PLAINBOX_SESSION_SHARE/suspend_single.log ] && sleep_test_log_check -v s3 $PLAINBOX_SESSION_SHARE/suspend_single.log @@ -359,6 +380,7 @@ _description: Automated check of the suspend log to look for errors reported by fwts plugin: attachment +category_id: 2013.com.canonical.plainbox::suspend id: suspend/suspend-single-log-attach command: [ -e $PLAINBOX_SESSION_SHARE/suspend_single.log ] && cat $PLAINBOX_SESSION_SHARE/suspend_single.log _description: @@ -371,6 +393,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: attachment + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_suspend-single-log-attach_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` command: [ -e $PLAINBOX_SESSION_SHARE/`echo ${index}`_suspend_single.log ] && cat $PLAINBOX_SESSION_SHARE/`echo ${index}`_suspend_single.log @@ -385,6 +408,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_hybrid-sleep-single-log-check depends: suspend/`echo ${index}`_hybrid_sleep_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` estimated_duration: 1.2 @@ -400,6 +424,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: attachment + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_hybrid-sleep-single-log-attach_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: suspend/`echo ${index}`_hybrid_sleep_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` command: [ -e $PLAINBOX_SESSION_SHARE/`echo ${index}`_hybrid_sleep_single.log ] && cat $PLAINBOX_SESSION_SHARE/`echo ${index}`_hybrid_sleep_single.log @@ -408,6 +433,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/suspend-time-check estimated_duration: 1.2 command: [ -e $PLAINBOX_SESSION_SHARE/suspend_single_times.log ] && sleep_time_check $PLAINBOX_SESSION_SHARE/suspend_single_times.log @@ -421,6 +447,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_suspend-time-check_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` estimated_duration: 1.2 @@ -429,6 +456,7 @@ command: EOF plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: power-management/usb_wakeup_keyboard user: root depends: suspend/suspend_advanced @@ -445,6 +473,7 @@ _description: Did the system wake up from suspend mode when you pressed a keyboard key? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: power-management/usb_wakeup_mouse user: root depends: suspend/suspend_advanced @@ -461,6 +490,7 @@ _description: Did the system wake up from suspend mode when you pressed the mouse button? plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/network_after_suspend estimated_duration: 20.0 depends: suspend/suspend_advanced suspend/network_before_suspend @@ -468,6 +498,7 @@ _description: Test the network after resuming. command: network_wait; gateway_ping_test | diff $PLAINBOX_SESSION_SHARE/network_before_suspend.txt - plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/resolution_after_suspend depends: suspend/suspend_advanced suspend/resolution_before_suspend estimated_duration: 1.2 @@ -481,6 +512,7 @@ _summary: Resolution consistency check after suspend with the valuse we got befo command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_resolution_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` estimated_duration: 1.2 depends: suspend/`echo ${index}`_resolution_before_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` @@ -489,6 +521,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/audio_after_suspend estimated_duration: 1.0 requires: @@ -501,6 +534,7 @@ command: diff $PLAINBOX_SESSION_SHARE/audio_settings_before_suspend $PLAINBOX_SESSION_SHARE/audio_settings_after_suspend plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/audio_after_suspend_auto estimated_duration: 1.2 requires: @@ -513,6 +547,7 @@ command: diff $PLAINBOX_SESSION_SHARE/audio_settings_before_suspend $PLAINBOX_SESSION_SHARE/audio_settings_after_suspend plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/speaker-headphone-plug-detection-after-suspend depends: suspend/suspend_advanced estimated_duration: 60.0 @@ -534,6 +569,7 @@ _description: The test times out after 30 seconds (and fails in that case). plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/microphone-plug-detection-after-suspend depends: suspend/suspend_advanced estimated_duration: 60.0 @@ -556,6 +592,7 @@ _description: The test times out after 30 seconds (and fails in that case). plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/playback_headphones-after-suspend estimated_duration: 20.0 depends: audio/list_devices suspend/suspend_advanced @@ -581,6 +618,7 @@ _description: Did you hear a sound through the headphones and did the sound play without any distortion, clicks or other strange noises from your headphones? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/alsa_record_playback_external-after-suspend estimated_duration: 20.0 depends: suspend/playback_headphones-after-suspend suspend/suspend_advanced @@ -607,6 +645,7 @@ _description: Did you hear your speech played back? plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/cpu_after_suspend estimated_duration: 1.2 depends: suspend/suspend_advanced suspend/cpu_before_suspend @@ -614,6 +653,7 @@ _description: Verify that all CPUs are online after resuming. command: cpuinfo_resource | diff $PLAINBOX_SESSION_SHARE/cpuinfo_before_suspend - plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/cpu_after_suspend_auto estimated_duration: 1.2 depends: suspend/suspend_advanced_auto suspend/cpu_before_suspend @@ -621,6 +661,7 @@ _description: Verify that all CPUs are online after resuming. command: cpuinfo_resource | diff $PLAINBOX_SESSION_SHARE/cpuinfo_before_suspend - plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/memory_after_suspend estimated_duration: 1.2 depends: suspend/suspend_advanced suspend/memory_before_suspend @@ -629,6 +670,7 @@ _description: command: meminfo_resource | diff $PLAINBOX_SESSION_SHARE/meminfo_before_suspend - plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/memory_after_suspend_auto estimated_duration: 1.2 depends: suspend/suspend_advanced_auto suspend/memory_before_suspend @@ -637,6 +679,7 @@ _description: command: meminfo_resource | diff $PLAINBOX_SESSION_SHARE/meminfo_before_suspend - plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/display_after_suspend estimated_duration: 120.0 depends: suspend/suspend_advanced @@ -656,6 +699,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: manual + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_display_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` _summary: Test display function after suspend for $product @@ -669,6 +713,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_after_suspend depends: suspend/suspend_advanced suspend/wireless_before_suspend requires: @@ -681,6 +726,7 @@ _description: connection is working as expected. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_wpa_bg depends: suspend/suspend_advanced estimated_duration: 20.0 @@ -703,6 +749,7 @@ _description: security and the 802.11b/g protocols after the system has been suspended. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_open_bg depends: suspend/suspend_advanced estimated_duration: 1.2 @@ -725,6 +772,7 @@ _description: security and the 802.11b/g protocols after the system has been suspended. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_wpa_n depends: suspend/suspend_advanced estimated_duration: 1.2 @@ -747,6 +795,7 @@ _description: security and the 802.11n protocol after the system has been suspended. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_open_n depends: suspend/suspend_advanced estimated_duration: 1.2 @@ -769,6 +818,7 @@ _description: security and the 802.11n protocol after the system has been suspended. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_wpa_ac depends: suspend/suspend_advanced estimated_duration: 1.2 @@ -792,6 +842,7 @@ _description: security and the 802.11ac protocol after the system has been suspended. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_open_ac depends: suspend/suspend_advanced estimated_duration: 1.2 @@ -815,6 +866,7 @@ _description: security and the 802.11ac protocol after the system has been suspended. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_wpa_bg_auto depends: suspend/suspend_advanced_auto estimated_duration: 1.2 @@ -837,6 +889,7 @@ _description: security and the 802.11b/g protocols after the system has been suspended. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_open_bg_auto depends: suspend/suspend_advanced_auto estimated_duration: 1.2 @@ -859,6 +912,7 @@ _description: security and the 802.11b/g protocols after the system has been suspended. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_wpa_n_auto depends: suspend/suspend_advanced_auto estimated_duration: 1.2 @@ -881,6 +935,7 @@ _description: security and the 802.11n protocol after the system has been suspended. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_open_n_auto depends: suspend/suspend_advanced_auto estimated_duration: 1.2 @@ -903,6 +958,7 @@ _description: security and the 802.11n protocol after the system has been suspended. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_wpa_ac_auto depends: suspend/suspend_advanced_auto estimated_duration: 1.2 @@ -926,6 +982,7 @@ _description: security and the 802.11ac protocol after the system has been suspended. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_open_ac_auto depends: suspend/suspend_advanced_auto estimated_duration: 1.2 @@ -956,6 +1013,7 @@ requires: command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: shell + category_id: 2013.com.canonical.plainbox::suspend id: suspend/iperf_after_suspend_ether_auto_$2 depends: suspend/suspend_advanced requires: device.path == "$1" @@ -976,6 +1034,7 @@ requires: command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=WIRELESS" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: shell + category_id: 2013.com.canonical.plainbox::suspend id: suspend/iperf_after_suspend_wifi_auto_$2 depends: suspend/suspend_advanced requires: device.path == "$1" @@ -989,6 +1048,7 @@ _description: This is an automated test to gather some info on the current state of your network devices. If no devices are found, the test will exit with an error. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/iperf_after_suspend_mobilebroadband_gsm_auto depends: suspend/suspend_advanced_auto estimated_duration: 20.0 @@ -1002,6 +1062,7 @@ _description: This test executes iperf connection performance/stability against the broadband device found on the system after suspend. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/iperf_after_suspend_mobilebroadband_cdma_auto depends: suspend/suspend_advanced_auto estimated_duration: 20.0 @@ -1015,6 +1076,7 @@ _description: This test executes iperf connection performance/stability against the broadband device found on the system after suspend. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_wpa_bg_manual depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1046,6 +1108,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_open_bg_manual depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1077,6 +1140,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_wpa_n_manual depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1108,6 +1172,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_open_n_manual depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1139,6 +1204,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_wpa_ac_manual depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1170,6 +1236,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wireless_connection_after_suspend_open_ac_manual depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1201,6 +1268,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_detect_after_suspend depends: suspend/suspend_advanced bluetooth/detect-output estimated_duration: 1.2 @@ -1226,6 +1294,7 @@ _description: This test grabs the hardware address of the bluetooth adapter after suspend and compares it to the address grabbed before suspend. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_after_suspend depends: suspend/suspend_advanced suspend/bluetooth_obex_before_suspend estimated_duration: 10.0 @@ -1254,6 +1323,7 @@ _description: This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_after_suspend_auto depends: suspend/suspend_advanced_auto suspend/bluetooth_obex_before_suspend estimated_duration: 10.0 @@ -1282,6 +1352,7 @@ _description: This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_send_after_suspend depends: suspend/suspend_advanced estimated_duration: 10.0 @@ -1310,6 +1381,7 @@ _description: This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_send_after_suspend_auto depends: suspend/suspend_advanced_auto estimated_duration: 10.0 @@ -1338,6 +1410,7 @@ _description: This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_browse_after_suspend depends: suspend/suspend_advanced estimated_duration: 10.0 @@ -1366,6 +1439,7 @@ _description: This is an automated Bluetooth test. It emulates browsing on a remote device specified by the BTDEVADDR environment variable. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_browse_after_suspend_auto depends: suspend/suspend_advanced_auto estimated_duration: 20.0 @@ -1394,6 +1468,7 @@ _description: This is an automated Bluetooth test. It emulates browsing on a remote device specified by the BTDEVADDR environment variable. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_get_after_suspend estimated_duration: 20.0 depends: suspend/suspend_advanced @@ -1422,6 +1497,7 @@ _description: This is an automated Bluetooth test. It receives the given file from a remote host specified by the BTDEVADDR environment variable plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_get_after_suspend_auto depends: suspend/suspend_advanced_auto estimated_duration: 20.0 @@ -1450,6 +1526,7 @@ _description: This is an automated Bluetooth test. It receives the given file from a remote host specified by the BTDEVADDR environment variable plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/bluetooth_obex_after_suspend_manual depends: suspend/suspend_advanced suspend/bluetooth_obex_before_suspend_manual estimated_duration: 120.0 @@ -1468,6 +1545,7 @@ _description: Was the data correctly transferred? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/cycle_resolutions_after_suspend estimated_duration: 120.0 requires: package.name == 'xorg' @@ -1489,6 +1567,7 @@ _summary: Cycle resolution after suspend tests command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_cycle_resolutions_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'xorg' depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` @@ -1504,6 +1583,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/cycle_resolutions_after_suspend_auto estimated_duration: 1.2 requires: package.name == 'xorg' @@ -1514,6 +1594,7 @@ _description: command: xrandr_cycle --keyword=after_suspend --screenshot-dir $PLAINBOX_SESSION_SHARE plugin: attachment +category_id: 2013.com.canonical.plainbox::suspend id: suspend/xrandr_screens_after_suspend.tar.gz depends: suspend/cycle_resolutions_after_suspend command: [ -f $PLAINBOX_SESSION_SHARE/xrandr_screens_after_suspend.tgz ] && base64 $PLAINBOX_SESSION_SHARE/xrandr_screens_after_suspend.tgz @@ -1526,6 +1607,7 @@ _summary: Attach screenshots from the suspend/cycle_resolution_after_suspend tes command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: attachment + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_xrandr_screens_after_suspend.tar.gz depends: suspend/`echo ${index}`_cycle_resolutions_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` command: [ -f $PLAINBOX_SESSION_SHARE/`echo ${index}`_xrandr_screens_after_suspend.tgz ] && base64 $PLAINBOX_SESSION_SHARE/`echo ${index}`_xrandr_screens_after_suspend.tgz @@ -1533,6 +1615,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/record_playback_after_suspend estimated_duration: 10.0 depends: suspend/suspend_advanced @@ -1549,6 +1632,7 @@ _description: This will check to make sure that your audio device works properly after a suspend and resume. This may work fine with speakers and onboard microphone, however, it works best if used with a cable connecting the audio-out jack to the audio-in jack. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/record_playback_after_suspend_auto estimated_duration: 10.0 depends: suspend/suspend_advanced_auto @@ -1565,12 +1649,14 @@ _description: This will check to make sure that your audio device works properly after a suspend and resume. This may work fine with speakers and onboard microphone, however, it works best if used with a cable connecting the audio-out jack to the audio-in jack. plugin: attachment +category_id: 2013.com.canonical.plainbox::suspend id: suspend/suspend-auto-single-log-attach command: [ -e $PLAINBOX_SESSION_SHARE/suspend_auto_single_log ] && cat $PLAINBOX_SESSION_SHARE/suspend_auto_single_log _description: Attaches the log from the single suspend/resume test to the results plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/screenshot_after_suspend estimated_duration: 10.0 depends: suspend/suspend_advanced_auto @@ -1585,12 +1671,14 @@ _description: Review attachment manually later plugin: attachment +category_id: 2013.com.canonical.plainbox::suspend id: screenshot_after_suspend.jpg depends: suspend/screenshot_after_suspend command: base64 ${PLAINBOX_SESSION_SHARE}/screenshot_after_suspend.jpg _description: Attaches the screenshot captured in graphics/screenshot. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/gpu_lockup_after_suspend estimated_duration: 10.0 requires: @@ -1610,6 +1698,7 @@ _description: After a 60s workload, check kern.log for reported GPU errors plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wifi_resume_time depends: suspend/suspend_advanced requires: @@ -1621,6 +1710,7 @@ _description: after a suspend/resume cycle. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/wifi_resume_time_auto estimated_duration: 1.2 depends: suspend/suspend_advanced_auto @@ -1632,6 +1722,7 @@ _description: after a suspend/resume cycle. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/network_resume_time depends: suspend/suspend_advanced estimated_duration: 1.2 @@ -1643,6 +1734,7 @@ _description: after a suspend/resume cycle. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/network_resume_time_auto depends: suspend/suspend_advanced_auto estimated_duration: 1.2 @@ -1654,6 +1746,7 @@ _description: after a suspend/resume cycle. plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/power depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1666,6 +1759,7 @@ _description: Does the power LED remain on after resuming from suspend? plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/battery-charging estimated_duration: 120.0 depends: suspend/suspend_advanced @@ -1679,6 +1773,7 @@ _description: Did the battery indicator LED still turn orange after resuming from suspend? plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/battery-charged depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1693,6 +1788,7 @@ _description: Does the orange battery LED still shut off when system is fully charged after resuming from suspend? plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/battery-low depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1706,6 +1802,7 @@ _description: Does the LED light orange when battery is low after resuming from suspend? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/hdd depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1720,6 +1817,7 @@ _description: Did the HDD LED still blink with HDD activity after resuming from suspend? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/numeric-keypad estimated_duration: 120.0 depends: suspend/suspend_advanced @@ -1736,6 +1834,7 @@ _description: 2. Numbers should only be entered in the keyboard verification window when the LED is on plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/caps-lock depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1749,6 +1848,7 @@ _description: Did the Cap Keys LED light as expected after resuming from suspend? plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/wlan depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1762,6 +1862,7 @@ _description: Did the WLAN LED light as expected after resuming from suspend? plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/wlan-disabled depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1777,6 +1878,7 @@ _description: Did the LED turn off then WLAN is disabled after resuming from suspend? plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/bluetooth depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1792,6 +1894,7 @@ _description: Did the bluetooth LED turn off and on twice after resuming from suspend? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/camera estimated_duration: 120.0 depends: @@ -1808,6 +1911,7 @@ _description: Did the camera LED still turn on and off after resuming from suspend? plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/touchpad depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1823,6 +1927,7 @@ _description: 3. When the LED is off, the mouse pointer should not move on touchpad usage plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/led_after_suspend/wireless depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1840,6 +1945,7 @@ _description: Did the WLAN/Bluetooth LED light as expected after resuming from suspend? plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/keys_after_suspend/brightness depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1853,6 +1959,7 @@ _description: Did the brightness change following to your key presses after resuming from suspend? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/keys_after_suspend/volume depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1871,6 +1978,7 @@ _description: Did the volume change following to your key presses? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/keys_after_suspend/mute depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1888,6 +1996,7 @@ _description: Did the volume mute following your key presses? plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/keys_after_suspend/sleep depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1903,6 +2012,7 @@ _description: Did the system go to sleep after pressing the sleep key after resuming from suspend? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/keys_after_suspend/battery-info depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1921,6 +2031,7 @@ _description: Did the Battery Info key work as expected after resuming from suspend? plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/keys_after_suspend/wireless depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1935,6 +2046,7 @@ _description: Did the wireless go off on the first press and on again on the second after resuming from suspend? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/keys_after_suspend/media-control estimated_duration: 120.0 depends: suspend/suspend_advanced @@ -1953,6 +2065,7 @@ _description: Do the keys work as expected after resuming from suspend? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/keys_after_suspend/super depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1970,6 +2083,7 @@ _description: Does the super key work as expected after resuming from suspend? plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/keys_after_suspend/video-out depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1984,6 +2098,7 @@ _description: Check that the video signal can be mirrored, extended, displayed on external or onboard only, after resuming from suspend. plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: suspend/keys_after_suspend/touchpad depends: suspend/suspend_advanced estimated_duration: 120.0 @@ -1999,6 +2114,7 @@ _description: Verify the touchpad has been disabled and re-enabled. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/usb_insert_after_suspend estimated_duration: 30.0 depends: @@ -2017,6 +2133,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/usb3_insert_after_suspend estimated_duration: 30.0 requires: @@ -2037,6 +2154,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/usb_remove_after_suspend estimated_duration: 30.0 depends: @@ -2054,6 +2172,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/usb3_remove_after_suspend estimated_duration: 30.0 depends: @@ -2073,6 +2192,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/usb_storage_automated_after_suspend estimated_duration: 1.2 depends: suspend/usb_insert_after_suspend @@ -2083,6 +2203,7 @@ _description: test is run. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/usb3_storage_automated_after_suspend estimated_duration: 1.2 requires: @@ -2095,6 +2216,7 @@ _description: test is run. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/usb_storage_preinserted_after_suspend estimated_duration: 1.2 user: root @@ -2106,6 +2228,7 @@ _description: is intended for servers and SRU automated testing. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/usb3_storage_preinserted_after_suspend estimated_duration: 1.2 user: root @@ -2119,6 +2242,7 @@ _description: is intended for servers and SRU automated testing. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/usb_performance_after_suspend depends: suspend/usb_insert_after_suspend user: root @@ -2129,6 +2253,7 @@ _description: minimum expected speed. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/usb3_superspeed_performance_after_suspend requires: usb.usb3 == 'supported' @@ -2141,6 +2266,7 @@ _description: as SuperSpeed USB device using xhci_hcd driver and transfers data correctly. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/mmc-insert-after-suspend estimated_duration: 30.0 depends: suspend/suspend_advanced @@ -2159,6 +2285,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/mmc-storage-after-suspend depends: suspend/mmc-insert-after-suspend estimated_duration: 10.0 @@ -2169,6 +2296,7 @@ _description: is run. It tests reading and writing to the MMC card after the system has been suspended. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/mmc-remove-after-suspend depends: suspend/mmc-insert-after-suspend estimated_duration: 30.0 @@ -2185,6 +2313,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/sd-insert-after-suspend estimated_duration: 30.0 depends: suspend/suspend_advanced @@ -2204,6 +2333,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/sd-storage-after-suspend estimated_duration: 10.0 depends: suspend/sd-insert-after-suspend @@ -2214,6 +2344,7 @@ _description: is run. It tests reading and writing to the SD card after the system has been suspended. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/sd-remove-after-suspend estimated_duration: 30.0 depends: suspend/sd-insert-after-suspend @@ -2231,6 +2362,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/sdhc-insert-after-suspend estimated_duration: 30.0 depends: suspend/suspend_advanced @@ -2250,6 +2382,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/sdhc-storage-after-suspend estimated_duration: 10.0 depends: suspend/sdhc-insert-after-suspend @@ -2260,6 +2393,7 @@ _description: is run. It tests reading and writing to the SDHC card after the system has been suspended. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: suspend/sdhc-remove-after-suspend estimated_duration: 30.0 depends: suspend/sdhc-insert-after-suspend @@ -2276,6 +2410,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/cf-insert-after-suspend estimated_duration: 30.0 depends: suspend/suspend_advanced @@ -2294,6 +2429,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/cf-storage-after-suspend estimated_duration: 10.0 depends: mediacard/cf-insert-after-suspend @@ -2304,6 +2440,7 @@ _description: is run. It tests reading and writing to the CF card after the system has been suspended. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/cf-remove-after-suspend estimated_duration: 30.0 depends: mediacard/cf-insert-after-suspend @@ -2320,6 +2457,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/sdxc-insert-after-suspend estimated_duration: 30.0 depends: suspend/suspend_advanced @@ -2338,6 +2476,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/sdxc-storage-after-suspend estimated_duration: 10.0 depends: mediacard/sdxc-insert-after-suspend @@ -2348,6 +2487,7 @@ _description: is run. It tests reading and writing to the SDXC card after the system has been suspended. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/sdxc-remove-after-suspend depends: mediacard/sdxc-insert-after-suspend estimated_duration: 30.0 @@ -2365,6 +2505,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/ms-insert-after-suspend estimated_duration: 30.0 depends: suspend/suspend_advanced @@ -2383,6 +2524,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/ms-storage-after-suspend estimated_duration: 10.0 depends: mediacard/ms-insert-after-suspend @@ -2393,6 +2535,7 @@ _description: is run. It tests reading and writing to the MS card after the system has been suspended. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/ms-remove-after-suspend estimated_duration: 30.0 depends: mediacard/ms-insert-after-suspend @@ -2409,6 +2552,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/msp-insert-after-suspend estimated_duration: 30.0 depends: suspend/suspend_advanced @@ -2427,6 +2571,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/msp-storage-after-suspend estimated_duration: 10.0 depends: mediacard/msp-insert-after-suspend @@ -2437,6 +2582,7 @@ _description: is run. It tests reading and writing to the MSP card after the system has been suspended. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/msp-remove-after-suspend estimated_duration: 30.0 depends: mediacard/msp-insert-after-suspend @@ -2453,6 +2599,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/xd-insert-after-suspend estimated_duration: 30.0 depends: suspend/suspend_advanced @@ -2471,6 +2618,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/xd-storage-after-suspend estimated_duration: 10.0 depends: mediacard/xd-insert-after-suspend @@ -2481,6 +2629,7 @@ _description: is run. It tests reading and writing to the xD card after the system has been suspended. plugin: user-interact +category_id: 2013.com.canonical.plainbox::suspend id: mediacard/xd-remove-after-suspend estimated_duration: 30.0 depends: mediacard/xd-insert-after-suspend @@ -2497,6 +2646,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: touchpad/touchpad_after_suspend depends: suspend/suspend_advanced_auto requires: @@ -2508,6 +2658,7 @@ _description: Determine if the touchpad is still functioning after suspend/resume. plugin: manual +category_id: 2013.com.canonical.plainbox::suspend id: touchscreen/touchscreen_after_suspend depends: suspend/suspend_advanced_auto estimated_duration: 30.0 @@ -2520,6 +2671,7 @@ _description: Does tap recognition work? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: camera/display_after_suspend estimated_duration: 120.0 depends: suspend/suspend_advanced @@ -2535,6 +2687,7 @@ _description: Did you see the video capture? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: camera/still_after_suspend estimated_duration: 120.0 depends: camera/display_after_suspend @@ -2553,6 +2706,7 @@ _description: Did you see the image? plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: camera/multiple_resolution_after_suspend estimated_duration: 1.2 depends: camera/still_after_suspend @@ -2573,6 +2727,7 @@ _description: Pointing device tests after suspend. command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=MOUSE" -w "category=TOUCHPAD" -w "category=TOUCHSCREEN"' plugin: manual + category_id: 2013.com.canonical.plainbox::suspend id: suspend/pointing-after-suspend_`echo "${product}_${category}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`_`basename $path` depends: suspend/suspend_advanced requires: device.path == "$path" @@ -2595,6 +2750,7 @@ _description: Click tests for pointing devices after suspend. command: cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=MOUSE" -w "category=TOUCHPAD"' plugin: manual + category_id: 2013.com.canonical.plainbox::suspend estimated_duration: 30.0 id: suspend/clicking-after-suspend_`echo "${product}_${category}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'`_`basename $path` depends: suspend/suspend_advanced @@ -2620,6 +2776,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_compiz_check_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'nux-tools' @@ -2630,6 +2787,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::suspend id: suspend/compiz_check_after_suspend depends: suspend/suspend_advanced requires: package.name == 'nux-tools' @@ -2646,6 +2804,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_glxgears_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` requires: package.name == 'mesa-utils' @@ -2663,6 +2822,7 @@ command: EOF plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend id: suspend/glxgears_after_suspend depends: suspend/suspend_advanced requires: @@ -2690,6 +2850,7 @@ command: id: suspend/`echo ${index}`_video_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::suspend requires: package.name == 'xorg' package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0' @@ -2706,6 +2867,7 @@ command: id: suspend/video_after_suspend plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::suspend depends: suspend/suspend_advanced requires: device.category == 'VIDEO' @@ -2729,6 +2891,7 @@ plugin: local command: cat <<'EOF' | run_templates -t -s 'graphics_card_resource' plugin: shell + category_id: 2013.com.canonical.plainbox::suspend id: suspend/`echo ${index}`_driver_version_after_suspend_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` depends: suspend/`echo ${index}`_suspend_after_switch_to_card_`echo "${product}" | sed 's/ /_/g;s/[^_a-zA-Z0-9-]//g'` command: graphics_driver @@ -2741,6 +2904,7 @@ id: suspend/driver_version_after_suspend requires: device.category == 'VIDEO' depends: suspend/suspend_advanced plugin: shell +category_id: 2013.com.canonical.plainbox::suspend command: graphics_driver estimated_duration: 0.500 _description: Parses Xorg.0.Log and discovers the running X driver and version after suspend @@ -2761,6 +2925,7 @@ command: id: suspend/oops_results_after_suspend.log plugin: attachment +category_id: 2013.com.canonical.plainbox::suspend estimated_duration: 0.5 command: [ -e ${PLAINBOX_SESSION_SHARE}/fwts_oops_results_after_s3_hwe.log ] && cat ${PLAINBOX_SESSION_SHARE}/fwts_oops_results_after_s3_hwe.log diff --git a/jobs/thunderbolt.pxu b/jobs/thunderbolt.pxu index 1268718..f4b4395 100644 --- a/jobs/thunderbolt.pxu +++ b/jobs/thunderbolt.pxu @@ -4,6 +4,7 @@ _name: Thunderbolt Support value-type: bool plugin: user-interact +category_id: 2013.com.canonical.plainbox::disk id: thunderbolt/insert imports: from 2013.com.canonical.plainbox import manifest requires: manifest.has_thunderbolt == 'True' @@ -23,6 +24,7 @@ _description: selected result plugin: shell +category_id: 2013.com.canonical.plainbox::disk id: thunderbolt/storage-test user: root imports: from 2013.com.canonical.plainbox import manifest @@ -36,6 +38,7 @@ _description: Thunderbolt HDD plugin: user-interact +category_id: 2013.com.canonical.plainbox::disk id: thunderbolt/remove imports: from 2013.com.canonical.plainbox import manifest requires: manifest.has_thunderbolt == 'True' @@ -55,6 +58,7 @@ _description: selected result plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::disk id: thunderbolt/daisy-chain imports: from 2013.com.canonical.plainbox import manifest requires: manifest.has_thunderbolt == 'True' diff --git a/jobs/touchpad.txt.in b/jobs/touchpad.txt.in index 80d8ef4..e24331f 100644 --- a/jobs/touchpad.txt.in +++ b/jobs/touchpad.txt.in @@ -1,4 +1,5 @@ plugin: manual +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/basic requires: dmi.product in ['Notebook','Laptop','Portable'] estimated_duration: 120.0 @@ -12,6 +13,7 @@ _description: Did the cursor move? plugin: user-interact +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/horizontal requires: dmi.product in ['Notebook','Laptop','Portable'] @@ -28,6 +30,7 @@ _description: Could you scroll right and left? plugin: user-interact +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/vertical requires: dmi.product in ['Notebook','Laptop','Portable'] @@ -44,6 +47,7 @@ _description: Could you scroll up and down? plugin: shell +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/singletouch-automated requires: dmi.product in ['Notebook','Laptop','Portable'] @@ -54,6 +58,7 @@ _description: Determine whether the touchpad is detected as a singletouch device automatically. plugin: shell +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/multitouch-automated estimated_duration: 1.2 requires: @@ -64,6 +69,7 @@ _description: Determine whether the touchpad is detected as a multitouch device automatically. plugin: manual +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/multitouch-manual requires: dmi.product in ['Notebook','Laptop','Portable'] depends: touchpad/singletouch-automated @@ -77,6 +83,7 @@ _description: Is the touchpad supposed to be multitouch? plugin: manual +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/singletouch-corner requires: dmi.product in ['Notebook','Laptop','Portable'] estimated_duration: 120.0 @@ -91,6 +98,7 @@ _description: Did the tap trigger a new tab to be opended? plugin: manual +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/singletouch-selection requires: dmi.product in ['Notebook','Laptop','Portable'] estimated_duration: 120.0 @@ -104,6 +112,7 @@ _description: Did a selection window open and were several files selected? plugin: manual +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/multitouch-rightclick requires: dmi.product in ['Notebook','Laptop','Portable'] depends: touchpad/multitouch-automated @@ -119,6 +128,7 @@ _description: Did the right click pop up menu appear? plugin: user-interact +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/multitouch-horizontal requires: dmi.product in ['Notebook','Laptop','Portable'] command: touchpad_test right left @@ -133,6 +143,7 @@ _description: Could you scroll right and left? plugin: user-interact +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/multitouch-vertical requires: dmi.product in ['Notebook','Laptop','Portable'] command: touchpad_test up down @@ -147,6 +158,7 @@ _description: Could you scroll up and down? plugin: manual +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/drag-and-drop requires: dmi.product in ['Notebook','Laptop','Portable'] estimated_duration: 120.0 @@ -161,6 +173,7 @@ _description: Did a selected folder move to the desktop? plugin: manual +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/multitouch-zoom requires: dmi.product in ['Notebook','Laptop','Portable'] depends: touchpad/multitouch-automated @@ -177,6 +190,7 @@ _description: Does the image zoom in and out? plugin: manual +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/multitouch-dash requires: dmi.product in ['Notebook','Laptop','Portable'] depends: touchpad/multitouch-automated @@ -191,6 +205,7 @@ _description: Did the tap open the Dash? plugin: shell +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/detected-as-mouse requires: dmi.product in ['Notebook','Laptop','Portable'] estimated_duration: 1.2 @@ -206,6 +221,7 @@ _description: This test will check if your touchpad was detected as a mouse. plugin: user-interact +category_id: 2013.com.canonical.plainbox::touchpad id: touchpad/continuous-move requires: dmi.product in ['Notebook','Laptop','Portable'] estimated_duration: 12 diff --git a/jobs/touchscreen.txt.in b/jobs/touchscreen.txt.in index 1069111..455f592 100644 --- a/jobs/touchscreen.txt.in +++ b/jobs/touchscreen.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::touchscreen id: touchscreen/nontouch-automated requires: xinput.device_class == 'XITouchClass' and xinput.touch_mode != 'direct' @@ -8,6 +9,7 @@ _description: Determine whether the screen is detected as a non-touch device automatically. plugin: shell +category_id: 2013.com.canonical.plainbox::touchscreen id: touchscreen/multitouch-automated requires: xinput.device_class == 'XITouchClass' and xinput.touch_mode == 'direct' @@ -17,6 +19,7 @@ _description: Determine whether the screen is detected as a multitouch device automatically. plugin: manual +category_id: 2013.com.canonical.plainbox::touchscreen id: touchscreen/multitouch-manual depends: touchscreen/nontouch-automated estimated_duration: 120.0 @@ -29,6 +32,7 @@ _description: Your screen was detected as a non touch screen. Select PASS if this is correct. plugin: manual +category_id: 2013.com.canonical.plainbox::touchscreen id: touchscreen/tap-detect depends: touchscreen/multitouch-automated estimated_duration: 120.0 @@ -41,6 +45,7 @@ _description: Does tap recognition work? plugin: manual +category_id: 2013.com.canonical.plainbox::touchscreen id: touchscreen/drag-n-drop depends: touchscreen/multitouch-automated estimated_duration: 120.0 @@ -54,6 +59,7 @@ _description: Does drag and drop work? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::touchscreen id: touchscreen/multitouch-zoom _summary: Check touchscreen pinch gesture for zoom depends: touchscreen/multitouch-automated @@ -69,6 +75,7 @@ _description: Did the blue square change size following the gesture? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::touchscreen id: touchscreen/multitouch-rotate _summary: Check touchscreen pinch gesture for rotate depends: touchscreen/multitouch-automated @@ -84,6 +91,7 @@ _description: Did the blue square rotate following the gesture? plugin: manual +category_id: 2013.com.canonical.plainbox::touchscreen id: touchscreen/multitouch-window-move depends: touchscreen/multitouch-automated estimated_duration: 120.0 @@ -97,6 +105,7 @@ _description: Did the window move along with the drag? plugin: manual +category_id: 2013.com.canonical.plainbox::touchscreen id: touchscreen/multitouch-dash depends: touchscreen/multitouch-automated estimated_duration: 120.0 @@ -109,6 +118,7 @@ _description: Did the tap open the Dash? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::touchscreen id: touchscreen/3-touch-tap depends: touchscreen/multitouch-automated estimated_duration: 15.0 @@ -128,6 +138,7 @@ command: exit $EXIT plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::touchscreen id: touchscreen/4-touch-tap depends: touchscreen/multitouch-automated estimated_duration: 15.0 diff --git a/jobs/usb.txt.in b/jobs/usb.txt.in index 2203275..68ac01d 100644 --- a/jobs/usb.txt.in +++ b/jobs/usb.txt.in @@ -1,10 +1,12 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::usb id: usb/detect estimated_duration: 1.0 command: set -o pipefail; lsusb 2>/dev/null | sed 's/.*\(ID .*\)/\1/' | head -n 4 || echo "No USB devices were detected" >&2 _description: Detects and shows USB devices attached to this system. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::usb id: usb/disk_detect depends: usb/detect estimated_duration: 1.0 @@ -21,6 +23,7 @@ _description: Were the drives detected? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::usb id: usb/HID depends: usb/detect estimated_duration: 1.0 @@ -36,6 +39,7 @@ _description: Did the device work as expected? plugin: user-interact +category_id: 2013.com.canonical.plainbox::usb id: usb/insert depends: usb/detect estimated_duration: 10.0 @@ -55,6 +59,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::usb id: usb3/insert requires: usb.usb3 == 'supported' @@ -75,6 +80,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::usb id: usb/remove depends: usb/insert estimated_duration: 10.0 @@ -91,6 +97,7 @@ _description: automatically selected result. plugin: user-interact +category_id: 2013.com.canonical.plainbox::usb id: usb3/remove depends: usb3/insert requires: @@ -109,6 +116,7 @@ _description: automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::usb id: usb/storage-transfer depends: usb/insert user: root @@ -126,6 +134,7 @@ _description: automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::usb id: usb3/storage-transfer requires: usb.usb3 == 'supported' @@ -145,6 +154,7 @@ _description: automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::usb id: usb/storage-automated depends: usb/insert user: root @@ -154,6 +164,7 @@ _description: This test is automated and executes after the usb/insert test is run. plugin: shell +category_id: 2013.com.canonical.plainbox::usb id: usb3/storage-automated requires: usb.usb3 == 'supported' @@ -165,6 +176,7 @@ _description: This test is automated and executes after the usb3/insert test is run. plugin: shell +category_id: 2013.com.canonical.plainbox::usb id: usb/storage-preinserted user: root estimated_duration: 45.0 @@ -175,6 +187,7 @@ _description: is intended for servers and SRU automated testing. plugin: shell +category_id: 2013.com.canonical.plainbox::usb id: usb3/storage-preinserted user: root requires: @@ -187,6 +200,7 @@ _description: is intended for servers and SRU automated testing. plugin: manual +category_id: 2013.com.canonical.plainbox::usb id: usb/panels _description: PURPOSE: @@ -201,6 +215,7 @@ _description: Do all USB slots work with the device? plugin: shell +category_id: 2013.com.canonical.plainbox::usb id: usb/performance depends: usb/insert user: root @@ -211,6 +226,7 @@ _description: minimum expected speed. plugin: shell +category_id: 2013.com.canonical.plainbox::usb id: usb3/superspeed_performance requires: usb.usb3 == 'supported' diff --git a/jobs/user_apps.txt.in b/jobs/user_apps.txt.in index db7a8f4..39edef4 100644 --- a/jobs/user_apps.txt.in +++ b/jobs/user_apps.txt.in @@ -1,4 +1,5 @@ plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/update_manager_finds_updates depends: ethernet/detect user: root @@ -16,6 +17,7 @@ _description: command: update-manager --check-dist-upgrades plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/nautilus_folder_create requires: package.name == 'nautilus' command: nautilus @@ -31,6 +33,7 @@ _description: Do you now have a new folder called Test Folder? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/nautilus_folder_copy depends: software/nautilus_folder_create command: nautilus @@ -48,6 +51,7 @@ _description: Do you now have a folder called Test Data? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/nautilus_folder_move depends: software/nautilus_folder_copy command: nautilus @@ -64,6 +68,7 @@ _description: Was the folder called Test Data successfully moved into the folder called Test Folder? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/nautilus_file_create depends: software/nautilus_folder_create command: nautilus $HOME/"Test Folder" @@ -79,6 +84,7 @@ _description: Do you now have a file called Test File 1? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/nautilus_file_copy depends: software/nautilus_file_create command: nautilus $HOME/"Test Folder" @@ -96,6 +102,7 @@ _description: Do you now have a file called Test File 2? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/nautilus_file_move depends: software/nautilus_file_copy command: nautilus $HOME/"Test Folder" @@ -112,6 +119,7 @@ _description: Was the file Test File 2 successfully moved into the Test Data folder? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/nautilus_file_delete depends: software/nautilus_file_create command: nautilus $HOME/"Test Folder" @@ -127,6 +135,7 @@ _description: Is Test File 1 now gone? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/nautilus_folder_delete depends: software/nautilus_folder_create command: nautilus @@ -150,6 +159,7 @@ command: cat <<'EOF' | run_templates -s "find $PLAINBOX_PROVIDER_DATA/documents -type f" id: `basename $0`_test plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::user_apps description: PURPOSE: This test will check that common office document types can be opened in the default application. @@ -171,6 +181,7 @@ command: cat <<'EOF' | run_templates -s "find $PLAINBOX_PROVIDER_DATA/audio -type f" id: `basename $0`_test plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::user_apps description: PURPOSE: This test will check that common audio files can be opened in the default player. @@ -192,6 +203,7 @@ command: cat <<'EOF' | run_templates -s "find $PLAINBOX_PROVIDER_DATA/video -type f" id: `basename $0`_test plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::user_apps description: PURPOSE: This test will check that common video files can be played in the default player. @@ -213,6 +225,7 @@ command: cat <<'EOF' | run_templates -s "find $PLAINBOX_PROVIDER_DATA/images -type f" id: `basename $0`_test plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::user_apps description: PURPOSE: This test will check that common image formats can be opened in the default viewer. @@ -226,6 +239,7 @@ command: EOF plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/firefox requires: package.name == 'firefox' command: firefox $PLAINBOX_PROVIDER_DATA/websites/testindex.html @@ -238,6 +252,7 @@ _description: Did the Ubuntu Test page load correctly? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/firefox-flash depends: software/firefox requires: package.name == 'firefox' @@ -252,6 +267,7 @@ _description: Did you see the text? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/firefox-flash-video depends: software/firefox-flash requires: package.name == 'firefox' @@ -266,6 +282,7 @@ _description: Did the video play correctly? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/firefox-totem depends: software/firefox requires: @@ -283,6 +300,7 @@ _description: Did the video play using a plugin? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/empathy-facebook_chat requires: package.name == "empathy" command: empathy @@ -297,6 +315,7 @@ _description: Were you able to connect correctly and send/receive messages? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/empathy-google_talk requires: package.name == "empathy" command: empathy @@ -311,6 +330,7 @@ _description: Were you able to connect correctly and send/receive messages? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/empathy-jabber requires: package.name == "empathy" command: empathy @@ -325,6 +345,7 @@ _description: Were you able to connect correctly and send/receive messages? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/empathy-aim requires: package.name == "empathy" command: empathy @@ -339,6 +360,7 @@ _description: Were you able to connect correctly and send/receive messages? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/empathy-msn requires: package.name == "empathy" command: empathy @@ -353,6 +375,7 @@ _description: Were you able to connect correctly and send/receive messages? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/evolution-pop3 requires: package.name == "evolution" command: evolution @@ -366,6 +389,7 @@ _description: Were you able to receive and read e-mail correctly? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/evolution-imap requires: package.name == "evolution" command: evolution @@ -379,6 +403,7 @@ _description: Were you able to receive and read e-mail correctly? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/evolution-smtp requires: package.name == "evolution" command: evolution @@ -392,6 +417,7 @@ _description: Were you able to send e-mail without errors? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/gcalctool requires: package.name == "gcalctool" command: gcalctool @@ -404,6 +430,7 @@ _description: Did it launch correctly? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/gcalctool-functions depends: software/gcalctool requires: package.name == "gcalctool" @@ -421,6 +448,7 @@ _description: Did the functions perform as expected? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/gcalctool-memory depends: software/gcalctool requires: package.name == "gcalctool" @@ -438,6 +466,7 @@ _description: Did the functions perform as expected? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/gcalctool-clipboard depends: software/gcalctool requires: package.name == "gcalctool" @@ -454,6 +483,7 @@ _description: Did the functions perform as expected? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/gedit requires: package.name == "gedit" command: gedit @@ -467,6 +497,7 @@ _description: Did this perform as expected? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/gedit-read depends: software/gedit requires: package.name == "gedit" @@ -481,6 +512,7 @@ _description: Did this perform as expected? plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::user_apps id: software/gnome-terminal requires: package.name == "gnome-terminal" command: gnome-terminal diff --git a/jobs/virtualization.txt.in b/jobs/virtualization.txt.in index 23b5a59..5a3ce76 100644 --- a/jobs/virtualization.txt.in +++ b/jobs/virtualization.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::virtualization id: virtualization/kvm_check_vm user: root environ: KVM_TIMEOUT KVM_IMAGE http_proxy https_proxy @@ -14,6 +15,7 @@ _summary: Verify KVM VM boots plugin: shell +category_id: 2013.com.canonical.plainbox::virtualization id: virtualization/xen_ok requires: package.name == 'libvirt-bin' user: root @@ -25,6 +27,7 @@ _summary: Verify Xen is running plugin: shell +category_id: 2013.com.canonical.plainbox::virtualization id: virtualization/xen_check_vm depends: virtualization/xen_ok user: root diff --git a/jobs/wireless.txt.in b/jobs/wireless.txt.in index 7c9b039..1a02303 100644 --- a/jobs/wireless.txt.in +++ b/jobs/wireless.txt.in @@ -1,4 +1,5 @@ plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_scanning requires: package.name == 'network-manager' @@ -21,6 +22,7 @@ estimated_duration: 0.645 _description: Wireless scanning test. It scans and reports on discovered APs. plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/info_automated requires: package.name == 'network-manager' @@ -31,6 +33,7 @@ _description: This is an automated test to gather some info on the current state of your wireless devices. If no devices are found, the test will exit with an error. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection command: network_check estimated_duration: 120.0 @@ -46,6 +49,7 @@ _description: Did a notification show and was the connection correctly established? plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_wpa_bg requires: device.category == 'WIRELESS' @@ -67,6 +71,7 @@ _description: security and the 802.11b/g protocols. plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_open_bg requires: device.category == 'WIRELESS' @@ -88,6 +93,7 @@ _description: security and the 802.11b/g protocols. plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_wpa_n requires: device.category == 'WIRELESS' @@ -109,6 +115,7 @@ _description: security and the 802.11n protocol. plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_open_n requires: device.category == 'WIRELESS' @@ -130,6 +137,7 @@ _description: security and the 802.11n protocol. plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_wpa_ac requires: device.category == 'WIRELESS' @@ -152,6 +160,7 @@ _description: security and the 802.11ac protocol. plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_open_ac requires: device.category == 'WIRELESS' @@ -174,6 +183,7 @@ _description: security and the 802.11ac protocol. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_wpa_bg_manual requires: device.category == 'WIRELESS' @@ -204,6 +214,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_open_bg_manual requires: device.category == 'WIRELESS' @@ -234,6 +245,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_wpa_n_manual requires: device.category == 'WIRELESS' @@ -265,6 +277,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_open_n_manual requires: device.category == 'WIRELESS' @@ -296,6 +309,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_wpa_ac_manual requires: device.category == 'WIRELESS' @@ -327,6 +341,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_open_ac_manual requires: device.category == 'WIRELESS' @@ -358,6 +373,7 @@ _description: Verification is automated, do not change the automatically selected result. plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/monitor_wireless_connection requires: package.name == 'iperf' @@ -378,6 +394,7 @@ _description: Tests the performance of a system's wireless connection through the iperf tool. plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/monitor_wireless_connection_udp requires: package.name == 'iperf' @@ -398,6 +415,7 @@ _description: Tests the performance of a system's wireless connection through the iperf tool, using UDP packets. plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_open_a requires: device.category == 'WIRELESS' @@ -420,6 +438,7 @@ _description: respond to requests on the 802.11a protocol. plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_open_b requires: device.category == 'WIRELESS' @@ -449,6 +468,7 @@ _description: Automated test that tests performance of each wireless network dev command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=WIRELESS" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: shell + category_id: 2013.com.canonical.plainbox::wireless id: wireless/stress_performance_$2 requires: device.path == "$1" user: root @@ -459,6 +479,7 @@ command: EOF plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_connection_open_g requires: device.category == 'WIRELESS' @@ -480,6 +501,7 @@ _description: respond to requests on the 802.11g protocol. plugin: shell +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_extension requires: device.category == 'WIRELESS' command: wireless_ext @@ -494,6 +516,7 @@ requires: command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=WIRELESS" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: shell + category_id: 2013.com.canonical.plainbox::wireless id: wireless/iwconfig_check_$2 requires: device.path == "$1" command: iwconfig $2 @@ -505,6 +528,7 @@ _description: This is an automated test to gather some info on the current state of your network devices. If no devices are found, the test will exit with an error. plugin: user-interact-verify +category_id: 2013.com.canonical.plainbox::wireless id: wireless/wireless_rfkill command: rfkill list | zenity --text-info --title rfkill-Info estimated_duration: 120.0 @@ -528,6 +552,7 @@ requires: command: cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=WIRELESS" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \`ls /sys\$0/net 2>/dev/null\`; do echo \$0 \$i; done"' plugin: user-interact-verify + category_id: 2013.com.canonical.plainbox::wireless user: root id: wireless/maximum_bandwidth_$2 requires: device.path == "$1" |