ubuntu-system-tests

Merge lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread into lp:ubuntu-system-tests

  1. upload_results_gspread
  2. Merge into trunk
Proposed by Brendan Donegan
Status: Work in progress
Proposed branch: lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread
Merge into: lp:ubuntu-system-tests
Diff against target: 128 lines (+114/-0)
2 files modified
README.rst (+23/-0)
ubuntu_sanity_tests/scripts/upload_to_gspread.py (+91/-0)
To merge this branch: bzr merge lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread
Reviewer Review Type Date Requested Status
Brendan Donegan (community) Disapprove
Leonardo Arias Fonseca (community) Needs Fixing
PS Jenkins bot continuous-integration Needs Fixing
Christopher Lee (community) Needs Information
Review via email: mp+258762@code.launchpad.net

Commit message

A simple script for parsing a subunit file and uploading the results to a Google spreadsheet

Description of the change

A simple script for parsing a subunit file and uploading the results to a Google spreadsheet

To post a comment you must log in.
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Reply
Revision history for this message
Federico Gimenez (fgimenez) wrote :

Looks very good in general, some comments inline.

Cheers!

Reply
lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread updated
113. By Brendan Donegan

Complete refactoring

114. By Brendan Donegan

Fix flake8 errors

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Reply
Revision history for this message
Leonardo Arias Fonseca (elopio) wrote :

Is there a way to test this?
We could put mocks in the gspread namespace, use a fake server, or maybe there's a staging google docs.
I wouldn't like to merge it before at least discussion testing.

Also, I think the readme should mention what this script is for, and the requirements to run it. Or maybe it could be in the help for the command.

I find it useful when the script ends in .py.

Reply
lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread updated
115. By Brendan Donegan

Move the script to under ubuntu_sanity_tests/scripts and add a section to the README

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Reply
lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread updated
116. By Brendan Donegan

Add a licence to upload_to_gspread.py

117. By Brendan Donegan

Add a simple unit test for the subunit parsing

Revision history for this message
Brendan Donegan (brendan-donegan) wrote :

I added a test for the most complicated part of the script that isn't mainly handled by something else - that is the subunit file parsing. I've not looked into using a staging server and I'm not sure mocking will be very useful here. I can still try something if you feel strongly about it.

Reply
Revision history for this message
Leonardo Arias Fonseca (elopio) wrote :

Hey, did you forget to upload the test? I can see only the subunit file.
As gspread takes care of all the remote processing, maybe you are right, we won't gain much testing that part. And if the spreadsheet is going away as Julien suggested in the meeting yesterday, we shouldn't worry that much about it.
About the subunit file, it would be nice to make it as small as possible, so I think you can remove all the debug finger messages.

Reply
lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread updated
118. By Brendan Donegan

Finish README update

119. By Brendan Donegan

Remove test and subunit file for upload_to_gspread script

120. By Brendan Donegan

add link on how to create credentials

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Reply
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Reply
Revision history for this message
Christopher Lee (veebers) wrote :

Looks good to me, one inline question which could be a bug. Otherwise this is an approve from me.

review: Needs Information
Reply
lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread updated
121. By Brendan Donegan

Mention the sheet parameter in the README

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Reply
Revision history for this message
Leonardo Arias Fonseca (elopio) wrote :

Things to add to the README:

As veebers mentioned, the sheet name argument.
I got stuck for a long time because I didn't got service API credentials, I first tried with the other two kinds of credentials. It would be good to mention this. And also that the drive api has to be enabled.
Then I got stuck because I didn't add the email address of the credentials to the spreadsheed with edit permissions.
The spreadsheet and the device sheet must exist.

When I finally got it working without errors like this:
$ ubuntu_sanity_tests/scripts/upload_to_gspread.py /tmp/test-002d7944d73f.json "Sanity Test Results" test 12 /tmp/output/artifacts/test-results.subunit

The test sheet is empty. Am I doing something wrong?

Some inline things to fix.

review: Needs Fixing
Reply
Revision history for this message
Brendan Donegan (brendan-donegan) wrote :

As discussed on IRC maybe we shouldn't merge this? We can always just use the script directly in jenkins if we need it temporarily. This is a short term thing and gspread is terribly akward to work with so let's not merge something that isn't going to be used for very long. We can always reconsider if things change.

review: Disapprove
Reply

Unmerged revisions

121. By Brendan Donegan

Mention the sheet parameter in the README

120. By Brendan Donegan

add link on how to create credentials

119. By Brendan Donegan

Remove test and subunit file for upload_to_gspread script

118. By Brendan Donegan

Finish README update

117. By Brendan Donegan

Add a simple unit test for the subunit parsing

116. By Brendan Donegan

Add a licence to upload_to_gspread.py

115. By Brendan Donegan

Move the script to under ubuntu_sanity_tests/scripts and add a section to the README

114. By Brendan Donegan

Fix flake8 errors

113. By Brendan Donegan

Complete refactoring

112. By Brendan Donegan

Add script to upload to gspread

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'README.rst'
2--- README.rst 2015-04-30 11:11:03 +0000
3+++ README.rst 2015-05-15 06:28:08 +0000
4@@ -235,3 +235,26 @@
5
6 If you don't pass any test as argument it will run all the tests in the sanity
7 suite.
8+
9+Upload the results to a Google sheet
10+------------------------------------
11+
12+Results from the sanity tests can be uploaded to a Google sheet using the
13+command:
14+
15+::
16+
17+ $ upload_to_gspread <secrets file> <sheet> <device> <build> <results subunit>
18+
19+The secrets file is a JSON file containing secrets necessary for authentication.
20+See the `Google OAuth2 documentation <https://developers.google.com/api-client-library/python/auth/installed-app#creatingcred>`_
21+for information on how to obtain it. The sheet is the name of the spreadsheet
22+to upload to, the device is used as the name of the tab
23+in the spreadsheet and the build is what the row is prefixed with.
24+The results subunit file contains the test results to be uploaded.
25+
26+It depends on the gspread and oauth2 modules which can be installed with:
27+
28+::
29+
30+ $ pip install gspread oauth2client
31
32=== added directory 'scripts'
33=== added directory 'ubuntu_sanity_tests/scripts'
34=== added file 'ubuntu_sanity_tests/scripts/upload_to_gspread.py'
35--- ubuntu_sanity_tests/scripts/upload_to_gspread.py 1970-01-01 00:00:00 +0000
36+++ ubuntu_sanity_tests/scripts/upload_to_gspread.py 2015-05-15 06:28:08 +0000
37@@ -0,0 +1,91 @@
38+#!/usr/bin/python3
39+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
40+#
41+# This file is part of the Ubuntu Sanity Tests project
42+# Copyright (C) 2015 Canonical
43+#
44+# This program is free software: you can redistribute it and/or
45+# modify it under the terms of the GNU General Public License version 3,
46+# as published by the Free Software Foundation.
47+#
48+# This program is distributed in the hope that it will be useful,
49+# but WITHOUT ANY WARRANTY; without even the implied warranty of
50+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
51+# GNU General Public License for more details.
52+#
53+# You should have received a copy of the GNU General Public License
54+# along with this program. If not, see <http://www.gnu.org/licenses/>.
55+#
56+
57+
58+import os
59+
60+import gspread
61+
62+from argparse import ArgumentParser
63+from oauth2client.client import GoogleCredentials
64+from subunit.v2 import ByteStreamToStreamResult
65+from testtools import StreamResult
66+
67+
68+class TestStreamResult(StreamResult):
69+
70+ results = {}
71+
72+ def status(self, test_id=None, test_status=None, test_tags=None,
73+ runnable=True, file_name=None, file_bytes=None, eof=False,
74+ mime_type=None, route_code=None, timestamp=None):
75+ if test_status and test_status != 'inprogress':
76+ self.results[test_id] = test_status
77+
78+
79+def parse_subunit(results):
80+ test_results = TestStreamResult()
81+ with open(results, 'rb') as subunit_file:
82+ converter = ByteStreamToStreamResult(source=subunit_file)
83+ test_results.startTestRun()
84+ converter.run(test_results)
85+ test_results.stopTestRun()
86+ return test_results.results
87+
88+
89+def authorize(secrets):
90+ credentials = get_credentials(secrets)
91+ return gspread.authorize(credentials)
92+
93+
94+def get_credentials(secrets):
95+ os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = os.path.expanduser(secrets)
96+ app_default = GoogleCredentials.get_application_default()
97+ return app_default.create_scoped(['https://spreadsheets.google.com/feeds'])
98+
99+
100+def upload_results(gc, sheet, device, build, results):
101+ results_doc = gc.open(sheet)
102+ device_sheet = results_doc.worksheet(device)
103+ results_row = [build]
104+ for result in sorted(results.keys()):
105+ results_row.append(results[result])
106+ device_sheet.append_row(results_row)
107+
108+
109+def _parse_arguments(argv=None):
110+ parser = ArgumentParser('Upload results to a Google Spreadsheet')
111+ parser.add_argument('secrets', help='Secrets json file')
112+ parser.add_argument('sheet', help='The name of the spreadsheet')
113+ parser.add_argument('device', help='The name of the device')
114+ parser.add_argument('build', help='The build number')
115+ parser.add_argument('subunit_file',
116+ help='The subunit file with the results')
117+ return parser.parse_args()
118+
119+
120+def main():
121+ args = _parse_arguments()
122+ results = parse_subunit(args.subunit_file)
123+ gc = authorize(args.secrets)
124+ upload_results(gc, args.sheet, args.device, args.build, results)
125+
126+
127+if __name__ == "__main__":
128+ main()

Subscribers

People subscribed via source and target branches

to all changes: