Integrating documentation markdown into a Zola (SSG) website (revised)

wold June 09, 2023 #website #Zola

Brief notes on integrating the AVATeR user manual into this website. The manual is maintained separately, allowing inclusion in a software package. While written for Zola (v1.7.2), most aspects apply to other SSGs such as Hugo. Discussing who reads manuals is left for another time ;)

Importing markdown files (one to a few pages)

For one or just a few pages, an easy solution is importing the raw content into an existing SSG page. This is what we currently use for the single page AVATeR manual.

To do so with Zola, use load_data() with format="plain" to import markdown text. Note:

using it within the markdown content, requires calling it indirectly from within a 'shortcode'
local source file changes may require restarting Zola (a known issue)

Generating 'frontmatter' metadata for multiple pages

For larger manuals, it is more practical to convert or export the source files to a format that a SSG accepts. Zola (Hugo and most SSGs) require each page to start with a metadata block ('frontmatter') like below:

+++
title = "Integrating documentation into a Zola (SSG) website"
draft = false
date = 2023-01-01
+++

Content here...

No frontmatter variables are actually mandatory, but the block itself is. Templates and certain section sorting modes (i.e. weight) may however complain if expected variables are missing.

Prepending such a block to each page can be achieved with scripting and/or a documentation generator that uses templates (i.e. Sphinx). This approach wasn't yet needed for syncoda.nl; we do touch on the topic of navigation menus, which can be generated using Zola as discussed next.

For Zola, navigation menus can be derived from the section structure, which is derived from the file layout.

Sections and templates

Directories containing an _index.md create a section, that registers any containing pages, content files and direct subsections (children). A templates that processes a section, can build this data into menus for example.

Its also possible to access arbitrary sections (and pages) if their path is provided, using get_section.

Themes

Themes are collections of templates. Most themes stick to the basics required for blogging. Some focus on documentation like book, adidoks or doks (Hugo). Any templates can be extended by overriding it or parts thereof, by creating a modified copy in the site root directory /templates/.

Page transparancy

Related is the section transparancy toggle: this will forward its pages to the parent section. For example we can amalgate yearly blogposts from /posts/2022/ and /posts/2023/ under /posts/. When building menus using section pages it's important to keep transparancy in mind: consider using subsections instead, as we'll discuss in a moment.

File layout considerations

For smaller sites, posts, pages and documentation can co-exist with some planning - assuming we use sections to build the navigation structure.

├──layout1
│       ├── avater
│       │   ├── _index.md
│       │   ├── info.md
│       │   ├── page.md
│       │   └── manual.md
│       ├── project #2
│       └── project #3

For 'layout1', section index /avater/_index.md can list its child pages (and direct subsections). Its pages in turn could access the /avater/ section using its path, and provide the same menu.

Creating the software projects list

Populating the project list for the parent /software/ section, could be done manually. Enabling transparancy for /avater/ forwards all its pages to the parent, including the manual (see layout1), which we may not want. Moving the manual into its own section (not transparant) solves this in part, and makes room for translations:

└── layout2
    └── software
        ├── avater
        │   ├── _index.md
        │   └── manual
        │       ├── _index.md

The manual section index may forward to a page, or be used to display any information.

As for the sofware listing, we could choose to keep just one information page for each project, and forward that to /software/... but what we actually want, is listing not the pages, but the subsections on /software/.

Getting subsection indexes

After checking the book theme, it turns out sections get an array with their subsections. We can iterate these on /software/ to generate a menu like:

List of subsections: <br />
{# where `section_item` is your current section: #}
{% for s in section_item.subsections %}
	{% set subsection = get_section(path=s, metadata_only=true) %}
	Subsection path: "{{ s }}" with title: "{{ subsection.title }}" and path: "{{ subsection.path }}"<br />
{% endfor %}

And possibly even their subpages/sections.

Initially this post didn't account for subsections - I even considered proposing an extension to transparancy, that would forward only the section indexes as a page. But it turns out it isn't necessary.

Project home URL and redirecting

A project's home URL would ideally be just /avater/ without the 'info/' part. The section index _index.md can perform a page forward, redirecting to info.md. This keeps a separation between section indexes and pages. It uses /templates/internal/alias.html, which sadly isn't setup to handle themes yet, causing a flicker with dark themes. I'll report this issue soon.

Another option is moving the info.md content into avater/_index.md, and treat it as a page. The section template must anticipate this, like including section variables for the content (section.content) and TOC (section.toc); often it won't, so expect customizing some parts.

Final layout

While straying a bit from the manual theme, recently the release pages were moved to /avater/releases/ (like /news/), and forwarded to the frontpage, making the final site layout:

├── posts/
│   ├── 2022/
│   ├── 2023/
│   │   └── _index.md
│   └── _index.md
├── software/
│   ├── avater/
│   │   ├── _index.md  # forwards to pages/info/
│   │   ├── manuals/
│   │   ├── pages/
│   │   └── releases/
│   ├── _index.md
│   └── project #2

Appendix: Modifying URLs

Before proceeding: be careful after testing with the sed -n and p (print) options. When not using git, a pre/post directory comparison can be made using diff -r or diff -ru0.

Image URLs

It's easiest to store images with the post content ('bundling'). If moved to /static/, the URLs may need to be modified. The regex below suffices, but may not cover all situations.

sed -E 's+\!\[(.*)\]\((.\+)\)+![\1](/files/avater/manual/\2)+' somewhere/manual_en.md > manual_en.md 
# Test using sed -En 's+regexhere+p' manual_en.md

With Powershell avoid the default unicode conversion (always something... ;)), if desired.

get-content somewhere/manual_en.md | 
 %{$_ -replace '!\[(.*)\]\((.+)\)' , '![$1](/files/avater/manual/$2)'} | 
 Out-File -Encoding ascii -FilePath manual_en.md

Internal URLs

File layout changes may necessitate changing internal URLs. A quick example using find -exec.

find . -iname "*.md" -exec sed -i -E 's+systemrequirements+requirements+' {} \;
# test using sed -n -E 's+systemrequirements+requirements+p' {} \;

Linux/GNU also offer the sed -s multi-file option:

sed -s -i 's+(@/software/avater/info.md#requirements)+(/software/avater/#requirements)+' *.md

Remarks

And to repeat: be careful after testing with the sed -n and p (print) options. When not using git, a pre/post directory comparison can be made using diff -r or diff -ru0.

Updated May 24th: some text changes
Updated June 10th: major rewrite, use subsections instead