A file format-agnostic static site generator

Overview

mksite

A file format-agnostic static site generator

Installation

If you already have Rust and Cargo installed:

cargo install mksite

Alternatively, you can install via git:

cargo install --git https://github.com/alterae/mksite

Usage

mksite <COMMAND>

Commands:
  build  Build the site according to `mksite.toml`
  clean  Delete all build outputs
  init   Initialize a `mksite.toml` file in the current directory
  new    Scaffold an empty site in a new directory
  help   Print this message or the help of the given subcommand(s)

Options:
  -q, --quiet                  Do not print log messages
      --log-level <LOG_LEVEL>  What level of logging to enable (error, warn, info, debug, or trace) [default: info]
  -h, --help     Print help information
  -V, --version  Print version information

mksite is a program for turning a tree of text files into a different tree of text files, usually a website. A typical mksite project has the following structure:

  • mksite.toml — The mksite.toml file is the core of a project, and defines the names of the other significant directories, the data available to templates, and the transforms to apply to files.
  • layout/ (optional) — The layout/ directory contains layouts, such as html boilerplate, to be applied to files after they are transformed. If you don't want any layouts, or don't want to use mksite's layout system, this folder can be safely omitted. The name of the layout/ directory can be customized in mksite.toml.
  • out/ (generated) — The out/ directory is generated by mksite when the site is built, and contains the transformed contents of the src/ directory, as well as the contents of the static/ directory, copied as-is. The name of the out/ directory can be customized in mksite.toml.
  • src/ — The src/ directory holds all the non-static source files for the website. Files in src must be valid UTF-8 and can contain template expressions using the Tera templating language. Other than that, they can be anything: LaTex, HTML, Markdown, your own custom markup language, or something else entirely. The name of the src/ directory can be customized in mksite.toml.
  • static/ (optional) — The static/ directory contains static files, such as stylesheets or images, which are copied as-is to the out/ directory. No templating or transformation occurs. The name of the static/ directory can be customized in mksite.toml.

Config

An example mksite.toml file looks like this:

dirs.src = "src"
dirs.out = "out"
dirs.static = "static"
dirs.layout = "layout"

[data]
author = { name = "Jane Doe", email = "[email protected]" }
copyright = 2022

[transforms]
md.html = "pandoc -f markdown -t html"
scd.html = ["scdoc", "pandoc -f man -t html"]

The first four lines tell mksite the names of the src/, out/, static/, and layout/ directories, respectively. Changing these will change where mksite reads and writes data. For example, dirs.out = "www" would cause mksite to write the build output in a folder called www/.

Next is the data section, which is where you can define arbitrary data that will be passed to the template rendering. In templates, this data will be made available under the data variable. For details on the template syntax, see the Tera documentation.

Finally, we have the transforms section. Transforms are commands or chains of commands that take a stream of bytes on standard input, and return a stream of bytes for standard output. Transforms can be used to trivially implement many features mksite does not natively support, such as markdown rendering and syntax highlighting. The basic syntax of a transform definition is in.out = "command" or in.out = ["command1", "command2", ...] where in is the file extension the transform operates on, out is the file extension the transform produces, and command is a command to pipe the page through. For more details on the finer points of transforms, see below.

All fields in this config file are optional.

Layouts

Layouts are simply Tera templates located in the layout/ directory that accept an additional page.content variable. Layouts must be valid UTF-8, but aside from that they can be any format.

An example layout file looks like this.

<!-- layout/_.html -->
<!DOCTYPE html>
<html>
  <head></head>
  <body>
    <!-- The "| safe" prevents Tera from html-escaping the content. -->
    {{ page.content | safe }}
  </body>
</html>

There are two kinds of layouts: default layouts and override layouts:

Override layouts have the same name, extension, and relative path as a specific file in the out/ directory, and only apply to that file. For example, the layout layout/blog/index.html will only apply to the page out/blog/index.html.

Default layouts have the name _.ext, where ext is some file extension. Default layouts apply to all files in the corresponding directory and in nested directories that don't have a default layout. For example:

layout/
    _.html
    blog/
        _.html
        index.html

In this example, layout/blog/_.html will apply to all html files in out/ except index.html, and layout/_.html will apply to every html file in out/ except the contents of the blog directory.

Note Layouts are applied after transforms, based on the file extension of the transform output. As a result, a layout like _.html will apply to all generated html files, regardless of whether these html files were hand-written or generated from markdown or something else via a transform.

If no applicable layouts are found for a file, or if there is no layout/ no layout will be applied.

If an applicable layout exists, but you would like to prevent it from being applied to a file or folder, you can define an "empty" layout like so:

{{ page.content | safe }}

Transforms

A transform has an input extension, an output extension, and a command or chain of commands.

Each page in src/ with a file extension matching a transform's input extension is piped into the transform as a stream of bytes, and the resulting stream of bytes is written to a file in the out/ directory with the same name and path, and a file extension matching the transform's output extension. Each page can be piped to multiple transforms, and multiple transforms can output the same format. The relationship between inputs and outputs is many-to-many.

Warning It is important to note that the inputs and outputs of transforms are streams of arbitrary bytes, not necessarily valid UTF-8 strings. This is important for interfacing with external non-rust tools, but there are some caveats:

Though this may change in the future, at present all templates and layouts must be valid UTF-8. This means that while transforms can both input and output arbitrary bytes, the original input to a transform (a file in the src/ directory) will be UTF-8. Additionally, layouts for non-UTF-8 files are not supported, and attempting to define a layout for, say, a .pdf file will result in an error.

If a transform has a single command, pages are piped to that command, and the output of the command is written to the output file. For example, these transforms:

[transforms]
md.html = "pandoc -f markdown"
md.pdf = "pandoc -f markdown -t pdf"

Will use pandoc to produce an html file and a pdf document in out/ for every markdown file in src/.

If a transform has a chain of commands, pages are piped to each command in the chain in order, and the ouput of the last command is written to the output file, like with shell pipelines. For example:

[transforms]
scd.html = ["scdoc", "pandoc -f man"]

Will use scdoc to generate a man page from each .scd file, and immediately pipe that man page to pandoc to convert it to html.

Contributing

Pull requests and issues are welcome, but please ensure you run cargo fmt before submitting a PR.

Example

See the docs/ folder for an example of a website built using mksite.

License

mksite is licensed under the MIT License.

Comments
  • Man page(s)

    Man page(s)

    Man pages are an important form of documentation. We should have one.

    Actually we should probably have at least two since we'll want to document the config format. We might even want three because Tera's official docs are lacking and it would be good to have some of our own for the templating syntax.

    documentation 
    opened by alterae 0
  • Transforms outputting to same file silently overwrite each other

    Transforms outputting to same file silently overwrite each other

    Minimal reproduction:

    .
    ├── layout
    ├── mksite.toml
    ├── src
    │  ├── foo.1
    │  └── foo.2
    └── static
    
    # mksite.toml
    [transforms]
    1.txt = "cat"
    2.txt = "cat"
    

    This will result in only one page called foo.txt in out/.

    This behavior should probably be at least a warning, probably an error.

    bug 
    opened by alterae 0
  • Website overhaul

    Website overhaul

    Something shiny, with good typography and nice gradients and shadows and the like. Should include an html version of the man pages(s) (see #8), as well as a tutorial and any other documentation necessary to explain how to use it. The main page shouldn't duplicate the readme, and the readme should be shortened to account for having proper docs.

    We should probably hold off on this until the software is more mature and less subject to change.

    documentation 
    opened by alterae 0
  • Explore other templating engines

    Explore other templating engines

    Tera gets the job done but there is a lot I just do not like about both the API and the templating syntax. At some point I should look into swapping it out for something I like more (or if I'm really masochistic I can do some heavy-duty yak shaving and write my own).

    Ideal characteristics (in no particular order):

    • Good API
    • Nice templating syntax (I really like Mustache)
    • Support for inputs that aren't valid UTF-8
    opened by alterae 0
  • Page frontmatter

    Page frontmatter

    I don't really want to do this but I do not see another way to allow users to define arbitrary per-page metadata, which we need in order to properly handle things like titles and descriptions in layouts.

    We could probably do TOML front matter like so:

    +++
    title = "Page Title"
    description = "A brief one-sentence description of the page"
    +++
    
    Page contents...
    

    And then make it accessible to templates like so:

    <head>
      <title>{{ page.data.title }}</title>
      <meta name="description" content="{{ page.data.description }}" />
    </head>
    

    This will require #3, and will also interact with #4.

    We might also want to support YAML front matter but I really do not like that idea.

    Alternatively, if we were using a templating engine that wasn't Tera, we could do something using the templating language (a {% export title "Page Title" %} type deal).

    enhancement 
    opened by alterae 0
  • Arbitrary globs instead of extensions

    Arbitrary globs instead of extensions

    Big maybe. This would be a bit more powerful but I'm not sure it's worth the massively increased complexity of transform processing.

    Basically we'd use file globs a-la-makefile to determine what files are piped to a given transform.

    enhancement 
    opened by alterae 0
Owner
Michelle S.
She/her or they/them. Fan of Rust, Typescript, game design, and good command-line tools.
Michelle S.
serve a static site, single page application or just a static file with Rust

cargo-server tl;dr: Does the same as "python -m http.server" or "npx serve" but for Rust ecosystem. cargo-server helps you serve a static site, single

Raphael Amorim 18 Oct 14, 2022
Serve a static site, single page application or just a static file with Rust

cargo-server tl;dr: Does the same as "python -m http.server" or "npx serve" but for Rust ecosystem. cargo-server helps you serve a static site, single

Raphael Amorim 18 Oct 14, 2022
A fast static site generator in a single binary with everything built-in. https://www.getzola.org

zola (né Gutenberg) A fast static site generator in a single binary with everything built-in. Documentation is available on its site or in the docs/co

Zola 9.8k Dec 3, 2022
static site generator from markdown files.

Mdblog Static site generator from markdown files with features: TeX style math support file path is the post url file name is the post title post can

Fu 49 Nov 11, 2022
the best static site generator

the best static site generator you've ever seen

Anirudh Balaji 14 Jan 24, 2022
Static site generator written in Rust

Sedum is a static site generator written in Rust. It can be used locally or with a service like Netlify to generate websites on the fly. Usage Local P

Elly 1 Jan 5, 2022
An ultra simple static site generator designed to appeal to technical as well as non technical users.

Hulk is an ultra simple static site generator designed to appeal to both technical and non technical users.

Bilal Tariq 1 Dec 27, 2021
Sūshì is a simple but customizable static site generator / blog generator written in Rust

sūshì Sūshì is a simple but customizable static site generator / blog generator written in Rust. Installation Install with Cargo (Recommended) cargo i

MrNothing233 2 Mar 20, 2022
serve a static site, single page application or just a static file with Rust

cargo-server tl;dr: Does the same as "python -m http.server" or "npx serve" but for Rust ecosystem. cargo-server helps you serve a static site, single

Raphael Amorim 18 Oct 14, 2022
Serve a static site, single page application or just a static file with Rust

cargo-server tl;dr: Does the same as "python -m http.server" or "npx serve" but for Rust ecosystem. cargo-server helps you serve a static site, single

Raphael Amorim 18 Oct 14, 2022
Hot reload static web server for deploying mutiple static web site with version control.

SPA-SERVER It is to provide a static web http server with cache and hot reload. 中文 README Feature Built with Hyper and Warp, fast and small! SSL with

null 6 Aug 17, 2022
A fast static site generator in a single binary with everything built-in. https://www.getzola.org

zola (né Gutenberg) A fast static site generator in a single binary with everything built-in. Documentation is available on its site or in the docs/co

Zola 9.8k Dec 3, 2022
static site generator from markdown files.

Mdblog Static site generator from markdown files with features: TeX style math support file path is the post url file name is the post title post can

Fu 49 Nov 11, 2022
A fast static site generator in a single binary with everything built-in. https://www.getzola.org

zola (né Gutenberg) A fast static site generator in a single binary with everything built-in. Documentation is available on its site or in the docs/co

Zola 9.8k Nov 26, 2022
the best static site generator

the best static site generator you've ever seen

Anirudh Balaji 14 Jan 24, 2022
Static site generator written in Rust

Sedum is a static site generator written in Rust. It can be used locally or with a service like Netlify to generate websites on the fly. Usage Local P

Elly 1 Jan 5, 2022
An ultra simple static site generator designed to appeal to technical as well as non technical users.

Hulk is an ultra simple static site generator designed to appeal to both technical and non technical users.

Bilal Tariq 1 Dec 27, 2021
Oinky - A blazing fast, data-oriented static site generator.

Oinky Oinky is a static site generator, written in Rust, that compiles Handlebars templates and Markdown files into a static site. Oinky also enables

Asko Nõmm 4 Jun 18, 2022
⚡ A Blazingly-Fast Static Site Generator, built with Rust.

Stuart A Blazingly-Fast Static Site Generator. Download Now » Stuart is a very fast and flexible static site generator, with build times as low as 0.1

William Henderson 5 Nov 25, 2022
Kurzlink is a simple static site generator built in rust

kurzlink What is kurzlink? Kurzlink is a simple static site generator built in rust.

Evy Garden 2 Nov 5, 2022