bpo-29941: Assert fixes

[Yhg1s]

There is a bit of confusion in the CPython source between Py_DEBUG and (C) asserts. By default Python builds without Py_DEBUG and without asserts (definining NDEBUG to disable them). Turning on Py_DEBUG also enables asserts. However, it is possible to turn on asserts without turning on Py_DEBUG, and at Google we routinely build CPython that way. (Doing this with the regular configure/make process can be done by setting CFLAGS=-UNDEBUG when running configure.) This happens to highlight two different problems:

• Code being defined in Py_DEBUG blocks but used in assertions: _PyDict_CheckConsistency() is defined in dictobject.c in an #ifdef Py_DEBUG, but then used in assert without a check for Py_DEBUG. This is a compile-time error.

• Assertions checking for things that are outside of CPython's control, like whether an exception is set before calling something that might clobber it. Generally speaking assertions should be for internal invariants; things that should be a specific way, and it's an error in CPython itself when it's not (I think Tim Peters originally expressed this view of C asserts). For example, PyObject_Call() (and various other flavours of it) does assert(!PyErr_Occurred()), which is easily triggered and the cause of which is not always apparent.

The second case is useful, mind you, as it exposes bugs in extension modules, but the way it does it is not very helpful (it displays no traceback), and if the intent is to only do this when Py_DEBUG is enabled it would be better to check for that. This PR fixes both issues.

I think what our codebase does (enable assertions by default, without enabling Py_DEBUG) is useful, even when applied to CPython, and I would like CPython to keep working that way. However, if it's deemed more appropriate to make assertions only work in Py_DEBUG mode, that's fine too -- but please make it explicit, by making non-Py_DEBUG builds require NDEBUG.

[mention-bot]

@Yhg1s, thanks for your PR! By analyzing the history of the files in this pull request, we identified @benjaminp, @serhiy-storchaka and @tim-one to be potential reviewers.

[benjaminp]

I support making assertions without Py_DEBUG work. We should add an easy way to do this with --configure and add a buildbot for it.

It seems like many of your changes would be simplified if there was a Py_DEBUG_ASSERT macro.

[benjaminp]

Probably this whole block should go under after line 1242 instead.

[Yhg1s]

Moved line 1242 up instead.

[Yhg1s]

Dropped the change to add Py_DEBUG guards around dubious asserts, as Tim doesn't think the're dubious enough :)

[vstinner]

I wrote most _PyXXX_CheckConsistency() methods. To be clear: these methods are not written for performances, but to catch bugs earlier. Be careful on slowdown if you enable assertions on production ;-)

[Yhg1s]

We enable them for tests (which we run a lot, all the time, but that's okay), not for deployed code.