testtools

Merge lp:~jml/testtools/unexpected-success-2 into lp:~testtools-committers/testtools/trunk

  1. unexpected-success-2
  2. Merge into trunk
Proposed by Jonathan Lange
Status: Merged
Merged at revision: 142
Proposed branch: lp:~jml/testtools/unexpected-success-2
Merge into: lp:~testtools-committers/testtools/trunk
Diff against target: 570 lines (+233/-38)
5 files modified
NEWS (+6/-1)
testtools/testresult/doubles.py (+17/-1)
testtools/testresult/real.py (+38/-3)
testtools/tests/test_compat.py (+8/-7)
testtools/tests/test_testresult.py (+164/-26)
To merge this branch: bzr merge lp:~jml/testtools/unexpected-success-2
Reviewer Review Type Date Requested Status
Martin Packman Needs Fixing
Robert Collins Approve
Review via email: mp+42050@code.launchpad.net

Description of the change

This branch finished off the rest of bug 654474. It changes the contract of TestResult such that wasSuccessful returns False if there has ever been an unexpected success.

In addition, it updates the contract tests to test for the behaviour of wasSuccessful in many more cases, and adds contract tests for Python 2.6 and 2.7 which we run against our fake implementations.

To post a comment you must log in.
lp:~jml/testtools/unexpected-success-2 updated
145. By Jonathan Lange

Update NEWS

Revision history for this message
Robert Collins (lifeless) wrote :

Seems to me that wasSuccessful() should be reset by startTestRun - what do you think?

Rather than a mixin, this looks like a job for testscenarios. Its fine without but clearly not as nice.

review: Approve
Revision history for this message
Jonathan Lange (jml) wrote :

Agreed on both points. Will only update startTestRun for testtools
TestResult, since I'm not controlling Python.

Would be better with testscenarios, but not in this branch. Also, we need
to have a chat about dependencies.

Thanks,
jml

Revision history for this message
Martin Packman (gz) wrote :

On Python 3 (with lp:~gz/testtools/raises_regressions_675327 merged) I get three failures. All in TestMultiTestresultContract (nit, should that be TestMultiTest*R*esultContract?) the methods test_addError_is_failure, test_addFailure_is_failure, and test_addUnexpectedSuccess_was_successful all return True from `result.wasSuccessful()`.

There are actually four failures, but one is silent...

As noted by Martin Pool in bug 654474 comment 5 changing the outcome without also providing feedback is bad, so I think TextTestResult.stopTestRun also needs changing. Something along the lines of but prettier than:

=== modified file 'testtools/testresult/real.py'
--- testtools/testresult/real.py 2010-10-24 09:21:01 +0000
+++ testtools/testresult/real.py 2010-11-28 19:18:08 +0000
@@ -258,6 +261,8 @@
         stop = self._now()
         self._show_list('ERROR', self.errors)
         self._show_list('FAIL', self.failures)
+ for test in self.unexpectedSuccesses:
+ self.stream.write("%sUNEXPECTEDSUCCESS: %s\n%s" % (self.sep1, test.id(), self.sep2))
         self.stream.write("Ran %d test%s in %.3fs\n\n" %
             (self.testsRun, plural,
              self._delta_to_float(stop - self.__start)))
@@ -267,7 +272,7 @@
             self.stream.write("FAILED (")
             details = []
             details.append("failures=%d" % (
- len(self.failures) + len(self.errors)))
+ len(self.failures) + len(self.errors) + len(self.unexpectedSuccesses)))
             self.stream.write(", ".join(details))
             self.stream.write(")\n")
         super(TextTestResult, self).stopTestRun()

The silent failure that should then be revealed on Python 3 also needs fixing:

=== modified file 'testtools/tests/test_compat.py'
--- testtools/tests/test_compat.py 2010-11-11 09:46:18 +0000
+++ testtools/tests/test_compat.py 2010-11-28 19:18:08 +0000
@@ -19,6 +19,7 @@
     )
 from testtools.matchers import (
     MatchesException,
+ Not,
     Raises,
     )

@@ -246,7 +247,7 @@
         if newio:
             self.expectFailure("Python 3 StringIO expects text not bytes",
                 self.assertThat, lambda: soutwrapper.write(self.uni),
- Raises(MatchesException(TypeError)))
+ Not(Raises(MatchesException(TypeError))))
         soutwrapper.write(self.uni)
         self.assertEqual("pa???n", sout.getvalue())

Final nit:

+ def test_addUnexpectedSuccess_was_successful(self):
+ # addUnexpectedSuccess does not the test run in Python 2.7.

Verb?

review: Needs Fixing
Revision history for this message
Jonathan Lange (jml) wrote :

mgz, thanks very much for the review. I've fixed the nits, TextTestResult, the MultiTestResult failure (map(f, xs) returns an iterator in Python 3) and the hidden failure in test_compat. I also added tests for the TextTestResult changes.

lifeless, I've made the startTestRun change.

I'm going to merge it now, but I'd welcome any comments you have on the incremental diff.

lp:~jml/testtools/unexpected-success-2 updated
146. By Jonathan Lange

minor nits

147. By Jonathan Lange

Make MultiTestResult at all usable in Python 3

148. By Jonathan Lange

startTestRun resets unexpected successes and other errors

149. By Jonathan Lange

NEWS update

150. By Jonathan Lange

Actually report the error in TextTestResult (thanks mgz)

151. By Jonathan Lange

Fix a inaccurate unexpected success, hidden by our broken Python 3 support

Revision history for this message
Robert Collins (lifeless) wrote :

+ len(self.failures) + len(self.errors)))
+ len(self.failures) + len(self.errors)
+ + len(self.unexpectedSuccesses)))
sum(map(len, (self, self.failures, self.errors,
    self.unexpectedSuccesses)))

Would be cleaner, I think?

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'NEWS'
2--- NEWS 2010-11-29 00:11:19 +0000
3+++ NEWS 2010-11-29 00:28:14 +0000
4@@ -8,7 +8,12 @@
5 -------
6
7 * addUnexpectedSuccess is translated to addFailure for test results that don't
8- know about addUnexpectedSuccess. (Jonathan Lange, #654474)
9+ know about addUnexpectedSuccess. Further, it fails the entire result for
10+ all testtools TestResults (i.e. wasSuccessful() returns False after
11+ addUnexpectedSuccess has been called). (Jonathan Lange, #654474)
12+
13+* startTestRun will reset any errors on the result. That is, wasSuccessful()
14+ will always return True immediately after startTestRun() is called.
15
16 * Responsibility for running test cleanups has been moved to ``RunTest``.
17 This change does not affect public APIs and can be safely ignored by test
18
19=== modified file 'testtools/testresult/doubles.py'
20--- testtools/testresult/doubles.py 2009-12-31 03:15:19 +0000
21+++ testtools/testresult/doubles.py 2010-11-29 00:28:14 +0000
22@@ -1,4 +1,4 @@
23-# Copyright (c) 2009 Jonathan M. Lange. See LICENSE for details.
24+# Copyright (c) 2009-2010 Jonathan M. Lange. See LICENSE for details.
25
26 """Doubles of test result objects, useful for testing unittest code."""
27
28@@ -15,15 +15,18 @@
29 def __init__(self):
30 self._events = []
31 self.shouldStop = False
32+ self._was_successful = True
33
34
35 class Python26TestResult(LoggingBase):
36 """A precisely python 2.6 like test result, that logs."""
37
38 def addError(self, test, err):
39+ self._was_successful = False
40 self._events.append(('addError', test, err))
41
42 def addFailure(self, test, err):
43+ self._was_successful = False
44 self._events.append(('addFailure', test, err))
45
46 def addSuccess(self, test):
47@@ -38,6 +41,9 @@
48 def stopTest(self, test):
49 self._events.append(('stopTest', test))
50
51+ def wasSuccessful(self):
52+ return self._was_successful
53+
54
55 class Python27TestResult(Python26TestResult):
56 """A precisely python 2.7 like test result, that logs."""
57@@ -62,9 +68,11 @@
58 """A test result like the proposed extended unittest result API."""
59
60 def addError(self, test, err=None, details=None):
61+ self._was_successful = False
62 self._events.append(('addError', test, err or details))
63
64 def addFailure(self, test, err=None, details=None):
65+ self._was_successful = False
66 self._events.append(('addFailure', test, err or details))
67
68 def addExpectedFailure(self, test, err=None, details=None):
69@@ -80,6 +88,7 @@
70 self._events.append(('addSuccess', test))
71
72 def addUnexpectedSuccess(self, test, details=None):
73+ self._was_successful = False
74 if details is not None:
75 self._events.append(('addUnexpectedSuccess', test, details))
76 else:
77@@ -88,8 +97,15 @@
78 def progress(self, offset, whence):
79 self._events.append(('progress', offset, whence))
80
81+ def startTestRun(self):
82+ super(ExtendedTestResult, self).startTestRun()
83+ self._was_successful = True
84+
85 def tags(self, new_tags, gone_tags):
86 self._events.append(('tags', new_tags, gone_tags))
87
88 def time(self, time):
89 self._events.append(('time', time))
90+
91+ def wasSuccessful(self):
92+ return self._was_successful
93
94=== modified file 'testtools/testresult/real.py'
95--- testtools/testresult/real.py 2010-10-24 09:21:01 +0000
96+++ testtools/testresult/real.py 2010-11-29 00:28:14 +0000
97@@ -108,6 +108,18 @@
98 """Called when a test was expected to fail, but succeed."""
99 self.unexpectedSuccesses.append(test)
100
101+ def wasSuccessful(self):
102+ """Has this result been successful so far?
103+
104+ If there have been any errors, failures or unexpected successes,
105+ return False. Otherwise, return True.
106+
107+ Note: This differs from standard unittest in that we consider
108+ unexpected successes to be equivalent to failures, rather than
109+ successes.
110+ """
111+ return not (self.errors or self.failures or self.unexpectedSuccesses)
112+
113 if str_is_unicode:
114 # Python 3 and IronPython strings are unicode, use parent class method
115 _exc_info_to_unicode = unittest.TestResult._exc_info_to_string
116@@ -148,6 +160,9 @@
117
118 New in python 2.7
119 """
120+ self.unexpectedSuccesses = []
121+ self.errors = []
122+ self.failures = []
123
124 def stopTestRun(self):
125 """Called after a test run completes
126@@ -182,7 +197,7 @@
127
128 def __init__(self, *results):
129 TestResult.__init__(self)
130- self._results = map(ExtendedToOriginalDecorator, results)
131+ self._results = list(map(ExtendedToOriginalDecorator, results))
132
133 def _dispatch(self, message, *args, **kwargs):
134 return tuple(
135@@ -223,6 +238,13 @@
136 def done(self):
137 return self._dispatch('done')
138
139+ def wasSuccessful(self):
140+ """Was this result successful?
141+
142+ Only returns True if every constituent result was successful.
143+ """
144+ return all(self._dispatch('wasSuccessful'))
145+
146
147 class TextTestResult(TestResult):
148 """A TestResult which outputs activity to a text stream."""
149@@ -258,6 +280,10 @@
150 stop = self._now()
151 self._show_list('ERROR', self.errors)
152 self._show_list('FAIL', self.failures)
153+ for test in self.unexpectedSuccesses:
154+ self.stream.write(
155+ "%sUNEXPECTED SUCCESS: %s\n%s" % (
156+ self.sep1, test.id(), self.sep2))
157 self.stream.write("Ran %d test%s in %.3fs\n\n" %
158 (self.testsRun, plural,
159 self._delta_to_float(stop - self.__start)))
160@@ -267,7 +293,8 @@
161 self.stream.write("FAILED (")
162 details = []
163 details.append("failures=%d" % (
164- len(self.failures) + len(self.errors)))
165+ len(self.failures) + len(self.errors)
166+ + len(self.unexpectedSuccesses)))
167 self.stream.write(", ".join(details))
168 self.stream.write(")\n")
169 super(TextTestResult, self).stopTestRun()
170@@ -363,6 +390,9 @@
171 self._test_start = self._now()
172 super(ThreadsafeForwardingResult, self).startTest(test)
173
174+ def wasSuccessful(self):
175+ return self.result.wasSuccessful()
176+
177
178 class ExtendedToOriginalDecorator(object):
179 """Permit new TestResult API code to degrade gracefully with old results.
180@@ -376,11 +406,13 @@
181
182 def __init__(self, decorated):
183 self.decorated = decorated
184+ self._was_successful = True
185
186 def __getattr__(self, name):
187 return getattr(self.decorated, name)
188
189 def addError(self, test, err=None, details=None):
190+ self._was_successful = False
191 self._check_args(err, details)
192 if details is not None:
193 try:
194@@ -405,6 +437,7 @@
195 return addExpectedFailure(test, err)
196
197 def addFailure(self, test, err=None, details=None):
198+ self._was_successful = False
199 self._check_args(err, details)
200 if details is not None:
201 try:
202@@ -431,6 +464,7 @@
203 return addSkip(test, reason)
204
205 def addUnexpectedSuccess(self, test, details=None):
206+ self._was_successful = False
207 outcome = getattr(self.decorated, 'addUnexpectedSuccess', None)
208 if outcome is None:
209 try:
210@@ -487,6 +521,7 @@
211 return self.decorated.startTest(test)
212
213 def startTestRun(self):
214+ self._was_successful = True
215 try:
216 return self.decorated.startTestRun()
217 except AttributeError:
218@@ -517,7 +552,7 @@
219 return method(a_datetime)
220
221 def wasSuccessful(self):
222- return self.decorated.wasSuccessful()
223+ return self._was_successful
224
225
226 class _StringException(Exception):
227
228=== modified file 'testtools/tests/test_compat.py'
229--- testtools/tests/test_compat.py 2010-11-11 09:46:18 +0000
230+++ testtools/tests/test_compat.py 2010-11-29 00:28:14 +0000
231@@ -19,6 +19,7 @@
232 )
233 from testtools.matchers import (
234 MatchesException,
235+ Not,
236 Raises,
237 )
238
239@@ -196,34 +197,34 @@
240 super(TestUnicodeOutputStream, self).setUp()
241 if sys.platform == "cli":
242 self.skip("IronPython shouldn't wrap streams to do encoding")
243-
244+
245 def test_no_encoding_becomes_ascii(self):
246 """A stream with no encoding attribute gets ascii/replace strings"""
247 sout = _FakeOutputStream()
248 unicode_output_stream(sout).write(self.uni)
249 self.assertEqual([_b("pa???n")], sout.writelog)
250-
251+
252 def test_encoding_as_none_becomes_ascii(self):
253 """A stream with encoding value of None gets ascii/replace strings"""
254 sout = _FakeOutputStream()
255 sout.encoding = None
256 unicode_output_stream(sout).write(self.uni)
257 self.assertEqual([_b("pa???n")], sout.writelog)
258-
259+
260 def test_bogus_encoding_becomes_ascii(self):
261 """A stream with a bogus encoding gets ascii/replace strings"""
262 sout = _FakeOutputStream()
263 sout.encoding = "bogus"
264 unicode_output_stream(sout).write(self.uni)
265 self.assertEqual([_b("pa???n")], sout.writelog)
266-
267+
268 def test_partial_encoding_replace(self):
269 """A string which can be partly encoded correctly should be"""
270 sout = _FakeOutputStream()
271 sout.encoding = "iso-8859-7"
272 unicode_output_stream(sout).write(self.uni)
273 self.assertEqual([_b("pa?\xe8?n")], sout.writelog)
274-
275+
276 def test_unicode_encodings_not_wrapped(self):
277 """A unicode encoding is left unwrapped as needs no error handler"""
278 sout = _FakeOutputStream()
279@@ -232,7 +233,7 @@
280 sout = _FakeOutputStream()
281 sout.encoding = "utf-16-be"
282 self.assertIs(unicode_output_stream(sout), sout)
283-
284+
285 def test_stringio(self):
286 """A StringIO object should maybe get an ascii native str type"""
287 try:
288@@ -246,7 +247,7 @@
289 if newio:
290 self.expectFailure("Python 3 StringIO expects text not bytes",
291 self.assertThat, lambda: soutwrapper.write(self.uni),
292- Raises(MatchesException(TypeError)))
293+ Not(Raises(MatchesException(TypeError))))
294 soutwrapper.write(self.uni)
295 self.assertEqual("pa???n", sout.getvalue())
296
297
298=== modified file 'testtools/tests/test_testresult.py'
299--- testtools/tests/test_testresult.py 2010-11-28 12:15:49 +0000
300+++ testtools/tests/test_testresult.py 2010-11-29 00:28:14 +0000
301@@ -49,8 +49,39 @@
302 StringIO = try_imports(['StringIO.StringIO', 'io.StringIO'])
303
304
305-class TestTestResultContract(TestCase):
306- """Tests for the contract of TestResults."""
307+class Python26Contract(object):
308+
309+ def test_fresh_result_is_successful(self):
310+ # A result is considered successful before any tests are run.
311+ result = self.makeResult()
312+ self.assertTrue(result.wasSuccessful())
313+
314+ def test_addError_is_failure(self):
315+ # addError fails the test run.
316+ result = self.makeResult()
317+ result.startTest(self)
318+ result.addError(self, an_exc_info)
319+ result.stopTest(self)
320+ self.assertFalse(result.wasSuccessful())
321+
322+ def test_addFailure_is_failure(self):
323+ # addFailure fails the test run.
324+ result = self.makeResult()
325+ result.startTest(self)
326+ result.addFailure(self, an_exc_info)
327+ result.stopTest(self)
328+ self.assertFalse(result.wasSuccessful())
329+
330+ def test_addSuccess_is_success(self):
331+ # addSuccess does not fail the test run.
332+ result = self.makeResult()
333+ result.startTest(self)
334+ result.addSuccess(self)
335+ result.stopTest(self)
336+ self.assertTrue(result.wasSuccessful())
337+
338+
339+class Python27Contract(Python26Contract):
340
341 def test_addExpectedFailure(self):
342 # Calling addExpectedFailure(test, exc_info) completes ok.
343@@ -58,6 +89,52 @@
344 result.startTest(self)
345 result.addExpectedFailure(self, an_exc_info)
346
347+ def test_addExpectedFailure_is_success(self):
348+ # addExpectedFailure does not fail the test run.
349+ result = self.makeResult()
350+ result.startTest(self)
351+ result.addExpectedFailure(self, an_exc_info)
352+ result.stopTest(self)
353+ self.assertTrue(result.wasSuccessful())
354+
355+ def test_addSkipped(self):
356+ # Calling addSkip(test, reason) completes ok.
357+ result = self.makeResult()
358+ result.startTest(self)
359+ result.addSkip(self, _u("Skipped for some reason"))
360+
361+ def test_addSkip_is_success(self):
362+ # addSkip does not fail the test run.
363+ result = self.makeResult()
364+ result.startTest(self)
365+ result.addSkip(self, _u("Skipped for some reason"))
366+ result.stopTest(self)
367+ self.assertTrue(result.wasSuccessful())
368+
369+ def test_addUnexpectedSuccess(self):
370+ # Calling addUnexpectedSuccess(test) completes ok.
371+ result = self.makeResult()
372+ result.startTest(self)
373+ result.addUnexpectedSuccess(self)
374+
375+ def test_addUnexpectedSuccess_was_successful(self):
376+ # addUnexpectedSuccess does not fail the test run in Python 2.7.
377+ result = self.makeResult()
378+ result.startTest(self)
379+ result.addUnexpectedSuccess(self)
380+ result.stopTest(self)
381+ self.assertTrue(result.wasSuccessful())
382+
383+ def test_startStopTestRun(self):
384+ # Calling startTestRun completes ok.
385+ result = self.makeResult()
386+ result.startTestRun()
387+ result.stopTestRun()
388+
389+
390+class TestResultContract(Python27Contract):
391+ """Tests for the contract of TestResults."""
392+
393 def test_addExpectedFailure_details(self):
394 # Calling addExpectedFailure(test, details=xxx) completes ok.
395 result = self.makeResult()
396@@ -76,24 +153,12 @@
397 result.startTest(self)
398 result.addFailure(self, details={})
399
400- def test_addSkipped(self):
401- # Calling addSkip(test, reason) completes ok.
402- result = self.makeResult()
403- result.startTest(self)
404- result.addSkip(self, _u("Skipped for some reason"))
405-
406 def test_addSkipped_details(self):
407 # Calling addSkip(test, reason) completes ok.
408 result = self.makeResult()
409 result.startTest(self)
410 result.addSkip(self, details={})
411
412- def test_addUnexpectedSuccess(self):
413- # Calling addUnexpectedSuccess(test) completes ok.
414- result = self.makeResult()
415- result.startTest(self)
416- result.addUnexpectedSuccess(self)
417-
418 def test_addUnexpectedSuccess_details(self):
419 # Calling addUnexpectedSuccess(test) completes ok.
420 result = self.makeResult()
421@@ -106,32 +171,57 @@
422 result.startTest(self)
423 result.addSuccess(self, details={})
424
425- def test_startStopTestRun(self):
426- # Calling startTestRun completes ok.
427- result = self.makeResult()
428- result.startTestRun()
429- result.stopTestRun()
430-
431-
432-class TestTestResultContract(TestTestResultContract):
433+ def test_addUnexpectedSuccess_was_successful(self):
434+ # addUnexpectedSuccess fails test run in testtools.
435+ result = self.makeResult()
436+ result.startTest(self)
437+ result.addUnexpectedSuccess(self)
438+ result.stopTest(self)
439+ self.assertFalse(result.wasSuccessful())
440+
441+ def test_startTestRun_resets_unexpected_success(self):
442+ result = self.makeResult()
443+ result.startTest(self)
444+ result.addUnexpectedSuccess(self)
445+ result.stopTest(self)
446+ result.startTestRun()
447+ self.assertTrue(result.wasSuccessful())
448+
449+ def test_startTestRun_resets_failure(self):
450+ result = self.makeResult()
451+ result.startTest(self)
452+ result.addFailure(self, an_exc_info)
453+ result.stopTest(self)
454+ result.startTestRun()
455+ self.assertTrue(result.wasSuccessful())
456+
457+ def test_startTestRun_resets_errors(self):
458+ result = self.makeResult()
459+ result.startTest(self)
460+ result.addError(self, an_exc_info)
461+ result.stopTest(self)
462+ result.startTestRun()
463+ self.assertTrue(result.wasSuccessful())
464+
465+class TestTestResultContract(TestCase, TestResultContract):
466
467 def makeResult(self):
468 return TestResult()
469
470
471-class TestMultiTestresultContract(TestTestResultContract):
472+class TestMultiTestResultContract(TestCase, TestResultContract):
473
474 def makeResult(self):
475 return MultiTestResult(TestResult(), TestResult())
476
477
478-class TestTextTestResultContract(TestTestResultContract):
479+class TestTextTestResultContract(TestCase, TestResultContract):
480
481 def makeResult(self):
482 return TextTestResult(StringIO())
483
484
485-class TestThreadSafeForwardingResultContract(TestTestResultContract):
486+class TestThreadSafeForwardingResultContract(TestCase, TestResultContract):
487
488 def makeResult(self):
489 result_semaphore = threading.Semaphore(1)
490@@ -139,6 +229,36 @@
491 return ThreadsafeForwardingResult(target, result_semaphore)
492
493
494+class TestExtendedTestResultContract(TestCase, TestResultContract):
495+
496+ def makeResult(self):
497+ return ExtendedTestResult()
498+
499+
500+class TestPython26TestResultContract(TestCase, Python26Contract):
501+
502+ def makeResult(self):
503+ return Python26TestResult()
504+
505+
506+class TestAdaptedPython26TestResultContract(TestCase, TestResultContract):
507+
508+ def makeResult(self):
509+ return ExtendedToOriginalDecorator(Python26TestResult())
510+
511+
512+class TestPython27TestResultContract(TestCase, Python27Contract):
513+
514+ def makeResult(self):
515+ return Python27TestResult()
516+
517+
518+class TestAdaptedPython27TestResultContract(TestCase, TestResultContract):
519+
520+ def makeResult(self):
521+ return ExtendedToOriginalDecorator(Python27TestResult())
522+
523+
524 class TestTestResult(TestCase):
525 """Tests for `TestResult`."""
526
527@@ -308,6 +428,12 @@
528 self.fail("yo!")
529 return Test("failed")
530
531+ def make_unexpectedly_successful_test(self):
532+ class Test(TestCase):
533+ def succeeded(self):
534+ self.expectFailure("yo!", lambda: None)
535+ return Test("succeeded")
536+
537 def make_test(self):
538 class Test(TestCase):
539 def test(self):
540@@ -393,9 +519,18 @@
541 self.assertThat(self.getvalue(),
542 DocTestMatches("...\n\nFAILED (failures=1)\n", doctest.ELLIPSIS))
543
544+ def test_stopTestRun_not_successful_unexpected_success(self):
545+ test = self.make_unexpectedly_successful_test()
546+ self.result.startTestRun()
547+ test.run(self.result)
548+ self.result.stopTestRun()
549+ self.assertThat(self.getvalue(),
550+ DocTestMatches("...\n\nFAILED (failures=1)\n", doctest.ELLIPSIS))
551+
552 def test_stopTestRun_shows_details(self):
553 self.result.startTestRun()
554 self.make_erroring_test().run(self.result)
555+ self.make_unexpectedly_successful_test().run(self.result)
556 self.make_failing_test().run(self.result)
557 self.reset_output()
558 self.result.stopTestRun()
559@@ -428,7 +563,10 @@
560 self.fail("yo!")
561 AssertionError: yo!
562 ------------
563-...""", doctest.ELLIPSIS))
564+======================================================================
565+UNEXPECTED SUCCESS: testtools.tests.test_testresult.Test.succeeded
566+----------------------------------------------------------------------
567+...""", doctest.ELLIPSIS | doctest.REPORT_NDIFF))
568
569
570 class TestThreadSafeForwardingResult(TestWithFakeExceptions):

Subscribers

People subscribed via source and target branches