Add docs validation to release process

[nezhar]

This is a way to automatically check if the documentation contains any broken links in order to avoid issues like #6928

The task is currently on allow_failures, but it could be added to the regular tasks in order to avoid linking to non existent pages in future.

The current documentation contains 17 broken links:

 not found (404): http://localhost:8000/tutorial/#api-guide from http://localhost:8000/tutorial/quickstart/ not found (404): http://localhost:8000/api-guide/pagination/github-link-pagination from http://localhost:8000/api-guide/pagination/ from http://localhost:8000/api-guide/pagination/#setting-the-pagination-style from http://localhost:8000/api-guide/pagination/#modifying-the-pagination-style not found (404): http://localhost:8000/topics/api-guide/schemas/#examples from http://localhost:8000/topics/documenting-your-api/ not found (404): http://localhost:8000/topics/api-guide/metadata/ from http://localhost:8000/topics/documenting-your-api/ not found (404): http://localhost:8000/community/topics/third-party-packages/#existing-third-party-packages from http://localhost:8000/community/third-party-packages/ not found (404): http://localhost:8000/community/3.10-announcement/community/funding.md from http://localhost:8000/community/3.10-announcement/ not found (404): http://localhost:8000/community/api-guide/schemas/#schemas-as-documentation from http://localhost:8000/community/3.5-announcement/ not found (404): http://localhost:8000/community/api-guide/schemas/#the-get_schema_view-shortcut from http://localhost:8000/community/3.5-announcement/ not found (404): http://localhost:8000/community/api-guide/schemas/#schemagenerator from http://localhost:8000/community/3.5-announcement/ not found (404): http://localhost:8000/community/3.4-announcement/api-clients#command-line-client from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/3.4-announcement/api-clients#python-client-library from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/tutorial/7-schemas-and-client-libraries/ from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/api-guide/schemas/ from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/api-guide/metadata/#custom-metadata-classes from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/3.4-announcement/release-notes#34 from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/api-guide/fields#jsonfield from http://localhost:8000/community/3.3-announcement/ not found (404): http://localhost:8000/community/api-guide/pagination/#custom-pagination-styles from http://localhost:8000/community/3.1-announcement/ 

[rpkilby]

Hi @nezhar. This is interesting. I'd recommend adding it to the tox.ini as a new test env. As an example, take a look at:

django-rest-framework/tox.ini

Lines 53 to 59 in 0fd72f1

	[testenv:docs]
	basepython = python3.7
	skip_install = true
	commands = mkdocs build
	deps =
	-rrequirements/requirements-testing.txt
	-rrequirements/requirements-documentation.txt

And then the travis job can be simplified to { python: "3.7", env: TOXENV=docs-links }

[lovelydinosaur]

I think it's probably not a good idea for our test cases to rely on making external requests.
If we do this, then if any interlinked site happens to temporarily go down or be unavailable for any reason then our tets cases will start failing, right?

[rpkilby]

@tomchristie I think this is okay:

• This is a separate job that the rest of the matrix doesn't depend upon
• It's marked as an allowable failure, which does not delay the overall build success/failure

[rpkilby]

@tomchristie per my above comment, is it worth reconsidering this? It would have caught #7226.

Well, it would require someone looking at the Travis build and noticing the failure, but it definitely improves visibility.

[lovelydinosaur]

is it worth reconsidering this?

My feeling is probably that it'd likely suck up more time from introducing flakiness into the build process than the value it'd add.

[rpkilby]

That's fair, maybe we could add it as a step in the release process? Currently, the docs have 20 broken links.

Should be as simple as

$ mkdocs serve & $ pylinkvalidate.py -P http://localhost:8000/

[lovelydinosaur]

That's fair, maybe we could add it as a step in the release process?

Ah, now something like that might be a good idea yup.

[rpkilby]

Okay, I've updated the PR and fixed most of the broken links. There are a few outstanding issues though:

• See 32636b5. Should we be linking to fund.django-rest-framework.org or should we be linking to the funding.md page? Both are used currently.
• /tutorial/2-requests-and-responses links to http://127.0.0.1:8000/snippets/ as a convenience, however this is always going to be an invalid link. Should we just leave it, or should we rewrite as a code block http://127.0.0.1:8000/snippets/?
• /topics/documenting-your-api has a broken schema link that should point to examples in the old coreapi docs. It's easy enough to fix the link, but should we instead embed those examples elsewhere in the docs? cc @carltongibson

[carltongibson]

/topics/documenting-your-api has a broken schema link that should point to examples in the old coreapi docs. It's easy enough to fix the link, but should we instead embed those examples elsewhere in the docs? cc @carltongibson

IIRC, the thought was we'd just link to the source for the old CoreAPI pages on GitHub, because at the time I couldn't find a way to put them in the docs without them appearing in the Nav bar. (We upgraded mkdocs since then, is that possible now?) I'll review when I go over the schema docs.

[rpkilby]

(We upgraded mkdocs since then, is that possible now?)

Yeah - it's possible to link to the core api docs without having them in the nav. I've corrected other references to the coreapi docs where appropriate (e.g., older release notes).

I guess the question for this specific case is whether whether the referenced examples are still relevant/correct and if they should be moved into the main docs or if they should remain in the outdated coreapi docs. Specifically, these examples demonstrate how to document the various handlers. e.g.,

class ListUsernames(APIView): def get(self, request): """  Return a list of all user names in the system.  """

vs

class UserList(generics.ListCreateAPIView): """  get: List all the users.  post: Create a new user.  """

They're linked to at the end of this section.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[nezhar]

I updated the PR so it can now work with GitHub Actions. The job is allowed to fail, but it will add an annotation (not sure if there is a way to signal a warning on the job as in GitLab CI):

Here is an output for the broken links: https://github.com/encode/django-rest-framework/actions/runs/5848946265/job/15856739665#step:8:151

ERROR Crawled 133 urls with 13 error(s) in 3.36 seconds Start URL(s): http://localhost:8000/ not found (404): http://localhost:8000/api-guide/schemas/openapi-operationid from http://localhost:8000/api-guide/schemas/ from http://localhost:8000/api-guide/schemas/#examples not found (404): http://localhost:8000/community/topics/third-party-packages/#existing-third-party-packages from http://localhost:8000/community/third-party-packages/ not found (404): http://localhost:8000/community/api-guide/schemas/#schemas-as-documentation from http://localhost:8000/community/3.5-announcement/ not found (404): http://localhost:8000/community/api-guide/schemas/#the-get_schema_view-shortcut from http://localhost:8000/community/3.5-announcement/ not found (404): http://localhost:8000/community/api-guide/schemas/#schemagenerator from http://localhost:8000/community/3.5-announcement/ not found (404): http://localhost:8000/community/3.4-announcement/api-clients#command-line-client from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/3.4-announcement/api-clients#python-client-library from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/tutorial/7-schemas-and-client-libraries/ from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/api-guide/schemas/ from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/api-guide/metadata/#custom-metadata-classes from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/3.4-announcement/release-notes#34 from http://localhost:8000/community/3.4-announcement/ not found (404): http://localhost:8000/community/api-guide/fields#jsonfield from http://localhost:8000/community/3.3-announcement/ not found (404): http://localhost:8000/community/api-guide/pagination/#custom-pagination-styles from http://localhost:8000/community/3.1-announcement/ 

[auvipy]

I think we can safely merge this now