Skip to content

Conversation

@alukach
Copy link
Member

@alukach alukach commented Nov 18, 2020

This PR does two things:

  1. Adds a script for documentation generation.
  2. Sets up a Jekyll site to host the documentation.

Script for Documentation Generation

Moves documentation generation from a lerna command that generates document on each package to a single script (scripts/docgen.js) that generates documentation for each package. This script works much like the jsii-docgen, but with a few exceptions:

  1. Any module not prefaced with @cdk-seed is ignored
  2. documentation is prepended with Jekyll Front Matter content
  3. links to AWS constructs are directed to the official AWS CDK documentation.

/**
* This script is an adaptation of the build tooling from the official jsii-docgen codebase:
* https://github.com/aws/jsii-docgen/blob/d69453e6191d86761359d8ab28c4fd5e3e5ac2ad/src/index.ts#L11-L37
* The primary differences:
* - only generates documentation if the module's `fqn` starts with `"@cdk-seed"`
* - renders link to non-local modules by pointing at the official CDK documentation
* - prepends each outputted markdown file with Jekyll front-matter to describe hierarchy of data
*/

Jekyll Site

A Jekyll site has been added in the site directory. This runs the Just The Docs theme which provides search out of the box. We utilize the front matter generated when we create our documentation markdown to create a sense of hierarchy to the modules (i.e. the README.md of each package renders as a link in the side-navbar, the classes and props of each package render as links under that README.md's documentation).

The process to generate documentation is a bit intricate, I attempted to capture the steps here:

cdk-seed/README.md

Lines 49 to 62 in b887955

## Generating documentation
The generation of our documentation website is a three part process:
1. `jsii` must be run within each package (this is done by running `npm run build` from the project base dir). This produces a `.jsii` in the root of each package.
2. `scripts/docgen.js` should be run to gather each package's `.jsii` file and to export markdown documentation for each package into the `site/docs` directory.
3. [Jekyll](https://jekyllrb.com/) should be run to generate HTML from the markdown documentation.
This process can be made easier by running two processes in separate terminals:
1. `npm start` which concurrently runs two operations:
* trigger `jsii` builds on changes to packages' `README.md` or `lib/*.ts` files.
* trigger `scripts/docgen.js` to run on changes to packages' `.jsii` files.
2. `npm run website` which starts the Jekyll server. It is assumed that Jekyll has been previously installed on the system.

The act of creating helper commands to watch for new builds and new documentation was a bit wild for the following reasons:

  1. jsii supports a -w flag for running in watch mode to generate .jsii files on changes to the *.ts files, however it does not trigger rebuilds on changes to README.md files (however, those README.md files are used to generate .jsii files). For this reason, I had to wrap the command in another watch command with nodemon.
  2. nodemon does not do well watching files without filenames (i.e. files that are only extensions, such as .env or .jsii). For this reason, we need to manually discover and add each .jsii filepath as a -w command to nodemon ($(for f in packages/*/.jsii; do echo \"-w $f\"; done)). Related issue.

cdk-seed/package.json

Lines 16 to 22 in b887955

"start": "concurrently \"npm run build:watch\" \"npm run docgen:watch\"",
"bootstrap": "lerna bootstrap",
"build": "lerna run build",
"build:watch": "nodemon --watch 'packages' --ext '.md' --exec 'lerna run --parallel build:watch'",
"package": "lerna run package",
"docgen": "node ./scripts/docgen.js --output site/docs 'packages/**/.jsii'",
"docgen:watch": "nodemon --exec 'npm run docgen' $(for f in packages/*/.jsii; do echo \"-w $f\"; done) --ext jsii --verbose",


Ultimately, I don't believe that jsii-docgen is the best tool for documentation generation. It gets us a lot for not much effort (docgen from both code and README.md), but it doesn't seem to display any JSDoc blocktags aside from @deprecated (which only renders with a warning triangle, but no message, issue). After doing some research, I think something like using TypeDoc to generate JSON which is then parsed and rendered by Jekyll would give much greater control about the UI elements but would require much more work (blogpost on topic).

Code documentation is generated automatically from the [JSDoc](https://jsdoc.app/) comment blocks provided within the TypeScript source code. At a minumum, every member of the `props` interface and the exported `Construct` should be annotated with JSDoc information. _NOTE: At time of writing, it appears that only the `@deprecated` tag makes its way from the JSDoc annotations to the rendered markdown. As such, content from other tags (e.g. `@default`) should be duplicated within the textual messaging of the JSDoc description._

@alukach
Copy link
Member Author

alukach commented Nov 18, 2020

Resolves #4, however it does not autodeploy to Github Pages (that should be covered in #3)

@alukach
Copy link
Member Author

alukach commented Nov 18, 2020

Thanks for the review @botanical, I'm in agreement on all suggestions and have made changes in a01501c.

Copy link
Member

@botanical botanical left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a few comments, but nothing major

@alukach
Copy link
Member Author

alukach commented Nov 19, 2020

@botanical Thanks Jennifer. Were you able to build the documentation locally?

@botanical
Copy link
Member

@botanical Thanks Jennifer. Were you able to build the documentation locally?

Yes 🎉

It makes me think of one thing - maybe in another PR we can add a troubleshooting page for things like if a user runs into:

[1] > root@ docgen /Users/jennifertran/Code/ds/cdk-seed [1] > node ./scripts/docgen.js --output site/docs 'packages/**/.jsii' [1] [1] (node:80263) ExperimentalWarning: The ESM module loader is experimental. [1] file:///Users/jennifertran/Code/ds/cdk-seed/scripts/docgen.js:65 [1] const output = args.output ?? "dist"; [1] ^ [1] [1] SyntaxError: Unexpected token '?' [1] at Loader.moduleStrategy (internal/modules/esm/translators.js:122:18) [1] at async link (internal/modules/esm/module_job.js:42:21) [1] npm ERR! code ELIFECYCLE [1] npm ERR! errno 1 [1] npm ERR! root@ docgen: `node ./scripts/docgen.js --output site/docs 'packages/**/.jsii'` [1] npm ERR! Exit status 1 [1] npm ERR! [1] npm ERR! Failed at the root@ docgen script. [1] npm ERR! This is probably not a problem with npm. There is likely additional logging output above. 

then we can suggest they remember to run nvm use.

That's the only gotcha I ran into because I am using different node versions for different projects. Just a thought 😄

Screen Shot 2020-11-19 at 8 01 39 AM

@alukach alukach merged commit dc60fd6 into main Nov 21, 2020
@alukach alukach deleted the documentation-generation branch November 21, 2020 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants