feat(sphinx): allow readme file to define sphinx header and reorganize topic (#229)

Elad Ben-Israel · web-flow · commit 405da9cae244 · 2018-09-17T12:36:30.000+03:00
* Fix #228: allow readme file to define sphinx topic header * Fix #185: move multi-language tabs after readme * Update README
diff --git a/README.md b/README.md
@@ -232,23 +232,73 @@ under `sphinx`, etc.
 
 That's it. You are ready to rock!
 
-## Configuration
+## Features
+
+### Language features
+
+ * Classes
+ * Inheritance
+ * Constructors
+ * Methods
+ * Properties
+ * Abstract Members
+ * Virtual Overrides
+ * Async Methods
+ * Variadic Arguments
+ * Static Methods and Properties
+ * Static Constants
+ * Abstract Classes
+ * Interfaces
+ * Enums
+ * Primitive Types: string, number, boolean, date, json, any
+ * Collection Types: arrays, maps
+ * Union Types (limited support)
+ * Module Dependencies
+ * Data Interfaces
+
+### Source Languages
 
-jsii configuration is read from the module's `package.json` and includes the following options:
+ * TypeScript
+
+### Target Languages
+
+ * __Java__ - generates a ready-to-publish Maven package.
+ * __.NET__ - generates a ready-to-publish NuGet package.
+ * __Sphinx__ - generates a Sphinx reStructuredText document for the module with README and reference docs.
+ * __Python__ (work in progress) - generates a ready-to-publish PyPI package.
+ * __Ruby__ (work in progress) - generates a ready-to-publish RubyGem.
+
+
+## Targets
+
+jsii configuration is read from the `jsii` section in the module's
+`package.json` and includes the following options:
 
  * `targets` - the list of target languages this module will be packaged for. For each
    target, you would need to specify some naming information such as namespaces, package manager
    coordinates, etc. See [supported targets](#targets) for details.
  * `outdir` - the default output directory (relative to package root) for
-   __jsii-pacmak__. This is where target artifacts are emitted during packaging. Each artifact
-   will be emitted under `<outdir>/<target>` (e.g. `dist/java`, `dist/js`, etc).
+   __jsii-pacmak__. This is where target artifacts are emitted during packaging.
+   Each artifact will be emitted under `<outdir>/<target>` (e.g. `dist/java`,
+   `dist/js`, etc). Conventionally we use `"dist"` for outdir.
 
-### Targets
+### Java
 
-The following targets are currently supported:
+The `java` target will produce a ready-to-deploy Maven package for your jsii module.
 
- * `js` - implicit - every module will always have a "js" target (dah!).
- * `java` - packages the module as in Java/Maven package. Requires the following config:
+The `$outdir/java` directory will include the contents of a staged offline Maven
+repository. javadocs and sources are included automatically in the Maven package
+
+This repository can be published to [Maven Central](https://search.maven.org/)
+via the `deploy-staged-repository` command of the
+[nexus-staging-maven-plugin](https://mvnrepository.com/artifact/org.sonatype.plugins/nexus-staging-maven-plugin).
+See [Sonatype
+documentation](https://mvnrepository.com/artifact/org.sonatype.plugins/nexus-staging-maven-plugin)
+and [this gist](https://gist.github.com/eladb/9caa04253b268e8a8f3d658184202806)
+as a reference.
+
+To package your jsii module for Java, add the following configuration to the `jsii`
+section in `package.json`:
 
 ```json
 {
@@ -262,77 +312,70 @@ The following targets are currently supported:
 }
 ```
 
-* `dotnet` - packages the module as a .NET/NuGet package. Requires the following config:
+### .NET
+
+The `dotnet` target will produce a ready-to-publish NuGet package for your module.
+
+The `$outdir/dotnet` directory will include `.nupkg` files, which can
+be [published to NuGet](https://docs.microsoft.com/en-us/nuget/create-packages/publish-a-package).
+
+To package your jsii module as for .NET, add this configuration to the `jsii`
+section in `package.json`:
 
 ```js
 {
   "dotnet": {
-    /* Required. */
-    "namespace": "Acme.HelloNamespace",
+    "namespace": "Acme.HelloNamespace", // required
+    "packageId": "Acme.HelloPackage",   // required
+    "title": "ACME Hello",              // optional (default: packageId)
+    "iconUrl": "path/to/icon.svg",      // optional (default: no icon)
+
+    // strong-name signing
+    "signAssembly": true,                          // optional (default: false)
+    "assemblyOriginatorKeyFile": "path/to/key.snk" // optional
+  }
+}
+```
 
-    /* Required. */
-    "packageId": "Acme.HelloPackage",
+### Sphinx
 
-    /* Optional. Default: Value of packageId. */
-    "title": "ACME Hello",
+The sphinx target emits a [Sphinx](http://www.sphinx-doc.org/en/master/)
+documentation topic for the module, that can be used to build a Sphinx
+documentation website. It's not a complete website.
 
-    /* Optional. Default: null (no icon). */
-    "iconUrl": "path/to/icon.svg",
 
-    /* Optional. Used in conjunction with assemblyOriginatorKeyFile. Default: false. */
-    "signAssembly": true,
+The `$outdir/sphinx` directory will include two files:
 
-    /* Optional. Used in conjunction with signAssembly. Default: null. */
-    "assemblyOriginatorKeyFile": "path/to/key.snk"
-  }
-}
-```
+ * `<module-name>.rst` - the Sphinx topic entry point
+ * `<module-name>.README.md` (optional) - the module's README.md file (if exists)
 
-* `sphinx` - produces sphinx documentation for the module. No config is required, but an empty
-  entry will be needed in order to package this target:
+The `.rst` file will use [m2r](https://github.com/miyakogi/m2r) to
+[`mdinclude`](https://miyakogi.github.io/m2r/example.html#include-markdown-file)
+the README.md file into the topic.
+
+NOTE: if the first line of your `README.md` file starts with `# ` (an H1
+header), the contents of this line will be used as the first header of the
+topic. Otherwise, the module's name will be used.
+
+You will need to build a Sphinx documentation website with this `.rst` included.
+
+To package your jsii module as a Sphinx topic, add an empty object to the
+`jsii` section in `package.json` under the `sphinx` key:
 
 ```json
 {
   "sphinx": { }
 }
 ```
 
+### JavaScript
 
-## Features
-
-### Language features
-
- * Classes
- * Inheritance
- * Constructors
- * Methods
- * Properties
- * Abstract Members
- * Virtual Overrides
- * Async Methods
- * Variadic Arguments
- * Static Methods and Properties
- * Static Constants
- * Abstract Classes
- * Interfaces
- * Enums
- * Primitive Types: string, number, boolean, date, json, any
- * Collection Types: arrays, maps
- * Union Types (limited support)
- * Module Dependencies
- * Data Interfaces
-
-### Source Languages
-
- * TypeScript
+An implicit JavaScript target will always be created. No configuration is needed.
 
-### Target Languages
+The `$outdir/js` directory will include that npm tarball of the module (created
+with [`npm pack`](https://docs.npmjs.com/cli/pack)).
 
- * __Java__ - generates a ready-to-publish Maven package.
- * __.NET__ - generates a ready-to-publish NuGet package.
- * __Sphinx__ - generates a Sphinx reStructuredText document for the module with README and reference docs.
- * __Python__ (work in progress) - generates a ready-to-publish PyPI package.
- * __Ruby__ (work in progress) - generates a ready-to-publish RubyGem.
+Tarballs can be published to npmjs.org using [`npm publish`](https://docs.npmjs.com/cli/publish)
 
 ## What kind of sorcery is this?
 
diff --git a/packages/jsii-calc/README.md b/packages/jsii-calc/README.md
@@ -1,3 +1,15 @@
-## JSII Calculator
+# jsii Calculator
 
 This library is used to demonstrate and test the features of JSII
+
+## Sphinx
+
+This file will be incorporated into the sphinx documentation.
+
+If this file starts with an "H1" line (in our case `# jsii Calculator`), this
+heading will be used as the Sphinx topic name. Otherwise, the name of the module
+(`jsii-calc`) will be used instead.
+
+
+
+
diff --git a/packages/jsii-calc/test/assembly.jsii b/packages/jsii-calc/test/assembly.jsii
@@ -141,7 +141,7 @@
   "license": "Apache-2.0",
   "name": "jsii-calc",
   "readme": {
-    "markdown": "## JSII Calculator\n\nThis library is used to demonstrate and test the features of JSII\n"
+    "markdown": "# jsii Calculator\n\nThis library is used to demonstrate and test the features of JSII\n\n## Sphinx\n\nThis file will be incorporated into the sphinx documentation.\n\nIf this file starts with an \"H1\" line (in our case `# jsii Calculator`), this\nheading will be used as the Sphinx topic name. Otherwise, the name of the module\n(`jsii-calc`) will be used instead.\n\n\n\n\n"
   },
   "repository": {
     "type": "git",
@@ -3070,5 +3070,5 @@
     }
   },
   "version": "0.7.5",
-  "fingerprint": "ySCSItPu9fqfNNEpdYkxXDfM/reHpBffHLSNsUPp2nI="
+  "fingerprint": "XrmsNUcNdYiHEC6BRVT5XoeVmYQZzjYgiu6MyibgOwk="
 }
diff --git a/packages/jsii-pacmak/lib/targets/sphinx.ts b/packages/jsii-pacmak/lib/targets/sphinx.ts
@@ -28,7 +28,6 @@ const HMARKS = ['=', '-', '^', '~', '"', '#'];
 
 class SphinxDocsGenerator extends Generator {
     private assemblyName?: string;
-    private readmeFile?: string;
     private namespaceStack = new Array<NamespaceStackEntry>();
     private tocPath = new Array<string>();
     private targets: { [name: string]: TargetConstructor } = {};
@@ -75,12 +74,8 @@ class SphinxDocsGenerator extends Generator {
 
     protected onBeginAssembly(assm: spec.Assembly, fingerprint: boolean) {
         this.tocPath = new Array<string>(); // As a safety measure, in case previous assembly somehow didn't get it back to 0.
-        if (assm.readme) {
-            this.readmeFile = `_${fsSafeName(assm.name)}.README.md`;
-            this.code.openFile(this.readmeFile);
-            this.code.line(assm.readme.markdown);
-            this.code.closeFile(this.readmeFile);
-        }
+
+        const { readmeFile, readmeHeader } = this.emitReadme(assm);
 
         this.code.openFile(`${fsSafeName(assm.name)}.rst`);
 
@@ -89,8 +84,17 @@ class SphinxDocsGenerator extends Generator {
             this.code.line();
         }
 
-        this.openSection(assm.name);
+        // use the readme header if defined or the assembly name otherwise
+        this.openSection(readmeHeader || assm.name);
+        this.code.line();
+
+        if (readmeFile) {
+            this.code.line(`.. mdinclude:: ./${readmeFile}`);
+            this.code.line();
+        }
+        this.openSection('Reference');
         this.code.line();
+
         if (assm.targets) {
             this.code.openBlock('.. tabs::');
             this.code.line();
@@ -136,7 +140,6 @@ class SphinxDocsGenerator extends Generator {
         this.closeSection();
         this.code.closeFile(`${fsSafeName(assm.name)}.rst`);
 
-        delete this.readmeFile;
         delete this.assemblyName;
     }
 
@@ -158,14 +161,8 @@ class SphinxDocsGenerator extends Generator {
                 this.code.line();
                 this.openSection(conciseName);
             }
-        } else {
-            if (this.readmeFile) {
-                this.code.line(`.. mdinclude:: ./${this.readmeFile}`);
-                this.code.line();
-            }
-            this.openSection('Reference');
-            this.code.line();
         }
+
         this.code.line(`.. py:module:: ${nativeName}`);
         this.code.line();
     }
@@ -597,6 +594,38 @@ class SphinxDocsGenerator extends Generator {
         }
         this.code.closeBlock();
     }
+
+    /**
+     * Emits the README markdown file, while stripping off the first H1 header (if exists).
+     * @returns readme: the name of the file (or undefined)
+     * @returns header: the contents of the header (or undefined)
+     */
+    private emitReadme(assm: spec.Assembly): { readmeFile?: string, readmeHeader?: string } {
+        if (!assm.readme) {
+            return {
+                readmeFile: undefined,
+                readmeHeader: undefined
+            };
+        }
+
+        let lines = assm.readme.markdown.split('\n');
+        let readmeHeader;
+
+        if (lines[0].startsWith('# ')) {
+            readmeHeader = lines[0].substr(2);
+            lines = lines.slice(1);
+        }
+
+        const readmeFile = `_${fsSafeName(assm.name)}.README.md`;
+        this.code.openFile(readmeFile);
+        this.code.line(lines.join('\n'));
+        this.code.closeFile(readmeFile);
+
+        return {
+            readmeFile,
+            readmeHeader
+        };
+    }
 }
 
 function dup(char: string, times: number) {
diff --git a/packages/jsii-pacmak/test/expected.jsii-calc-base/sphinx/_scope_jsii-calc-base.rst b/packages/jsii-pacmak/test/expected.jsii-calc-base/sphinx/_scope_jsii-calc-base.rst
@@ -1,6 +1,9 @@
 @scope/jsii-calc-base
 =====================
 
+Reference
+---------
+
 .. tabs::
 
    .. group-tab:: C#
@@ -118,9 +121,6 @@
 
 
 
-Reference
----------
-
 .. py:module:: @scope/jsii-calc-base
 
 Base
diff --git a/packages/jsii-pacmak/test/expected.jsii-calc-lib/sphinx/_scope_jsii-calc-lib.rst b/packages/jsii-pacmak/test/expected.jsii-calc-lib/sphinx/_scope_jsii-calc-lib.rst
@@ -1,6 +1,9 @@
 @scope/jsii-calc-lib
 ====================
 
+Reference
+---------
+
 .. tabs::
 
    .. group-tab:: C#
@@ -118,9 +121,6 @@
 
 
 
-Reference
----------
-
 .. py:module:: @scope/jsii-calc-lib
 
 EnumFromScopedModule (enum)
diff --git a/packages/jsii-pacmak/test/expected.jsii-calc/dotnet/Amazon.JSII.Tests.CalculatorPackageId/.jsii b/packages/jsii-pacmak/test/expected.jsii-calc/dotnet/Amazon.JSII.Tests.CalculatorPackageId/.jsii
@@ -141,7 +141,7 @@
   "license": "Apache-2.0",
   "name": "jsii-calc",
   "readme": {
-    "markdown": "## JSII Calculator\n\nThis library is used to demonstrate and test the features of JSII\n"
+    "markdown": "# jsii Calculator\n\nThis library is used to demonstrate and test the features of JSII\n\n## Sphinx\n\nThis file will be incorporated into the sphinx documentation.\n\nIf this file starts with an \"H1\" line (in our case `# jsii Calculator`), this\nheading will be used as the Sphinx topic name. Otherwise, the name of the module\n(`jsii-calc`) will be used instead.\n\n\n\n\n"
   },
   "repository": {
     "type": "git",
@@ -3070,5 +3070,5 @@
     }
   },
   "version": "0.7.5",
-  "fingerprint": "ySCSItPu9fqfNNEpdYkxXDfM/reHpBffHLSNsUPp2nI="
+  "fingerprint": "XrmsNUcNdYiHEC6BRVT5XoeVmYQZzjYgiu6MyibgOwk="
 }
diff --git a/packages/jsii-pacmak/test/expected.jsii-calc/sphinx/_jsii-calc.README.md b/packages/jsii-pacmak/test/expected.jsii-calc/sphinx/_jsii-calc.README.md
diff --git a/packages/jsii-pacmak/test/expected.jsii-calc/sphinx/jsii-calc.rst b/packages/jsii-pacmak/test/expected.jsii-calc/sphinx/jsii-calc.rst
