# Building the OpenXR Specification

<!--
Copyright 2014-2024, The Khronos Group Inc.

SPDX-License-Identifier: CC-BY-4.0
-->

Before building the specification, install the necessary prerequisite tools as described later in this document.

You may instead choose to use the `open-in-docker.sh` script, available in the
OpenXR-Docs (spec source) and OpenXR-SDK-Source (SDK, for loader docs build)
repo as well as the internal monorepo. This script which will mount the
repository in a container built from an image with all the necessary tools
installed: these are regularly pushed to Docker Hub. See that script for more
details. You can find the associated Dockerfile at
<https://github.com/KhronosGroup/DockerContainers/blob/main/openxr.Dockerfile>

## Conditional Inclusion of Extensions

By default, the specification is built without including the content belonging
to any extensions.

You can build the specification without any extensions by simply using:

```bash
make html
```

to build the HTML version of the specification.

To include extension content, set the `EXTENSIONS` Make variable when invoking `make`.
For example:

```bash
make EXTENSIONS="XR_KHR_performance_stats" APITITLE="(with XR_KHR_performance_stats)" html
```

* `EXTENSIONS` is a blank-separated list of extension name strings that you want included in the specification.
* `APITITLE` is a string used to annotate the specification title.  Here, it is used to indicate which specification is included in this specification build.
* `html` is a Makefile target specifying what type of document to generate.  See below for a list of targets.

### Helper Scripts

Since specifying lists of extensions is tedious, there are several "helper"
scripts that invoke `make` for you with a list of functions generated by
these scripts. The base script is `makeSpec` which accepts the following
options as well as the Makefile targets described further below:

* -genpath path - directory for generated files and outputs (default `generated`)
* -spec core - make a spec with no extensions (default)
* -spec khr - make a spec with all KHR extensions
* -spec all - make a spec with all registered extensions
* -ext name - add specified extension and its dependencies
* -clean - clean generated files before building
* -registry path - API XML to use instead of default (`registry/xr.xml`)
* -apiname name - API name to use instead of default (`openxr`)
* -v - verbose, print actions before executing  them
* -n - dry-run, print actions instead of executing them

Normally you will invoke the script to just generated an HTML output
containing all extensions:

```bash
./makeSpec -spec all html
```

For backward compatibility, these scripts exist as wrappers around
`makeSpec`:

* `makeAllExt` - build with all extensions defined in the registry
* `makeExt` - build with only the extension specified as a command-line argument
* `makeKHR` - build with all KHR extensions defined in the registry
* `makeKHRAndKHX` - build with all KHR and KHX extensions defined in the registry. *NOTE* there are no KHX extensions at present, so `makeKHRAndKHX` is identical to `makeKHR`.

These are invoked as follows:

```bash
./makeAllExt target
./makeExt extension_name target
./makeKHR target
./makeKHRAndKHX target
```

* `target` is one of the targets listed below
* `extension_name` is a string containing one or more space-separated extension name strings.

## Makefile Specification Targets

These targets generate a variety of output documents in the directory
specified by the Makefile variable `$(OUTDIR)` (by default,
`generated/out/1.1/`).

It is recommended to build these targets using a "helper" script from above,
unless you want to only build the core spec without any extensions.

Most of these are not supported if building in `OpenXR-SDK-Source`: this
repository is mostly software, and the only contained AsciiDoc content is the
loader design document source.

These targets are currently supported.

* API spec:
  * `html` - Single-file HTML5 in `$(OUTDIR)/openxr.html`
  * `pdf` - PDF in `$(OUTDIR)/openxr.pdf` (Letter [8x5 x 11] paper size)
  * `pdfA4` - PDF in `$(OUTDIR)/openxr.pdf` (A4 paper size)
* Reference pages:
  * `manhtmlpages` - Both combined (`$(OUTDIR)/man/html/openxr.html`) and per-entity
    (`$(OUTDIR)/man/html/*.html`) reference pages, extracted from the AsciiDoc spec chapters.
* OpenXR Header:
  * `header` - C language header files:
    * `$(OUTDIR)/openxr/openxr.h`
    * `$(OUTDIR)/openxr/openxr_platform.h`
    * `$(OUTDIR)/openxr/openxr_reflection.h`
  * `header-test` Compile-tests the header files
* Combined testing and output target (preview of some CI testing):
  * `all`

## Makefile Other Targets

These targets can be built by simply invoking `make` without using the "helper" scripts mentioned above.

For example:

```bash
make styleguide
```

Many of these are not supported if building in `OpenXR-SDK-Source`: this
repository is mostly software, and the only contained AsciiDoc content is the
loader design document source.

* "OpenXR Documentation and Extensions" guide (aka Style Guide):
  * `styleguide` - Single-file HTML5 in `$(OUTDIR)/styleguide.html`
* OpenXR Extensions Process guide:
  * `extprocess` - Single-file HTML5 in `$(OUTDIR)/extprocess.html`
* "OpenXR Loader - Design and Operation" guide (aka Loader Spec)
  * `loader`
* Extract example code from AsciiDoc and try to compile them.
  * `build-examples`
* Run a variety of self-tests and internal validations of the registry and spec
  * `check-spec-links`
* Remove targets and intermediate files
  * `clean`
* Remove intermediate files only
  * `clean_dirt`

## Generate standalone headers

When an extension is in progress it is often useful to have
a standalone header that can be used in combination with the
official OpenXR header files. The following command can be
used to generate these.

```bash
python3 scripts/genxr.py -registry registry/xr.xml -standaloneExtension XR_KHR_sample_extension standalone_header
```

## Installing the Prerequisite Tools

This section describes the software components used by the OpenXR spec
toolchain.

Before building the OpenXR spec, you must install the following tools:

* GNU make (make version: 4.0.8-1; older versions probably OK)
* Python 3 (python, version: 3.4.2 or newer, preferably 3.6 or newer)
  * These packages are recommended for check_spec_links on all platforms:
    `termcolor tabulate`
  * `networkx` is required for `scripts/xml_consistency.py`, which is run by the
    `all` target in the Makefile.
  * On Windows, `colorama` is also recommended.
  * On apt-based systems,
    `sudo apt install python3-termcolor python3-tabulate python3-networkx`
    will install the versions of those packages that are in your distribution,
    which are likely sufficient.
  * To install with pip, the typical `pip3 install -r requirements.txt`
    will work.
  * To be able to generate the Python docs, or perform spec diffs, you'll
    additionally need: `sudo apt install python3-pytest python3-pypdf2` and
    `python3 -m pip install pdoc3 pdf_diff`
* Ruby (`ruby`, version 2.3.3 tested, anything newer should work)
  * The Ruby development package (ruby-dev) may also be required in some
    environments.
  * See below for required gems.
* Git command-line client (`git`, tested with version: 2.1.4).
  The build can progress without a git client, but branch/commit
  information will be omitted from the build.
  Any version supporting the following operations should work:
  * `git symbolic-ref --short HEAD`
  * `git log -1 --format="%H"`

For (optional) verification of the registry XML against its schema in `make check-spec-links`
and `make all`, at least some of the following packages must be installed:

* `jing` for the validating XML against the schema, with the most usable error messages.
  (apt package `jing`)
* `trang` for RELAX-NG schema format verification and conversion. (apt package `trang`)
  Note: Related to `jing`, may be in the same package if you're using some other system.
* `xmllint` for validating XML against the schema. (apt package `libxml2-utils`)
  Also requires `trang`.
* `xmlstarlet` for validating XML against the schema. (apt package `xmlstarlet`)
  Also requires `trang`.

A recommended install of XML tooling for apt-based systems would be

```sh
sudo apt install jing trang
```

while a complete install of XML tooling (including redundant verifiers) would be

```sh
sudo apt install jing trang libxml2-utils xmlstarlet
```

The following Ruby Gems and platform package dependencies must also be
installed.
This process is described in more detail for individual platforms and
environment managers below.
Please read the remainder of this document (other than platform-specific
parts you don't use) completely before trying to install.

* Asciidoctor (`asciidoctor`, version: 2.0.10 or compatible, apt package
  `asciidoctor`)
* Rouge (`rouge`, apt package `ruby-rouge`)
* Asciidoctor PDF (`asciidoctor-pdf`, version: 1.5.0 through 1.6.2)

**Note:**
> Asciidoctor-pdf versions before `1.5.0.alpha15` have issues with multi-page
valid usage blocks, in that the background only renders for the first page.
`alpha.15` fixes this issue (as well as a few others); do not use prior
versions. On the other hand, versions later than 1.6.2 break our index
customization script, so 1.6.2 is ideal.

Only the `asciidoctor` and `rouge` gems (and their dependencies) are needed if
you do not intend to build PDF versions of the spec and supporting documents.

**Note:**
> While it's easier to install just the toolchain components for HTML builds,
people submitting MRs with substantial changes to the Specification are
responsible for verifying that their branches build *both* `html` and `pdf`
targets.

The easiest way to get the right version of these gems is to use "bundler":

```sh
bundle install
```

Then, pass `BUNDLER=1` to each `make` (or wrapper script) invocation.

### Platforms and System Packages

#### Windows

Most of the dependencies on Linux packages are light enough that it's
possible to build the spec natively in Windows, but it means bypassing the
makefile and calling functions directly.
This might be solved in future.
However, given how well Windows Subsystem for Linux works for most uses,
it is unlikely to be worth the effort to fully enable a native build.

When using Ubuntu 16.04 or 18.04 via the Windows 10 Subsystem for Linux, most
dependencies can be installed via apt-get, with the two ruby gems installed with
gem in the user's directory. The Ubuntu 16.04.6 default Ruby install (version
2.3.1) seems to be up-to-date enough to run all the required gems. Just follow
the Debian-derived Linux instructions below.

**Notes:**

* If you're already using [rvm](https://rvm.io) or
  [rbenv](https://github.com/rbenv/rbenv) to upgrade your version of Ruby,
  that's OK, but it's no longer necessary. Ubuntu 16.04 and newer (as well as
  probably most any distribution you'd find for WSL) contains a new enough Ruby,
  just not new enough AsciiDoctor and AsciiDoctor-PDF. Ubuntu 20.04 and newer
  contains a new-enough everything.
* Most of the tools on Bash for Windows are quite happy with Windows line
  endings (CR LF), but bash scripts expect Unix line endings (LF).
  The file `.gitattributes` at the top of the repo
  forces such scripts to be checked out with the proper line
  endings on non-Linux platforms.
  If you add new scripts whose names don't end in `.sh`, they should be
  included in `.gitattributes` as well.
* For building spec, you might have a better experience if you clone the
  repository using git installed inside WSL, rather than a native Windows build
  of Git.

#### Debian-derived Linux (including Ubuntu and Ubuntu on WSL)

Other distributions using different package managers, such as RPM (Fedora)
and Yum (SuSE) will have different requirements.

```sh
# Absolute bare minimum for only these makefile targets:
# header html manhtmlpages extprocess styleguide loader
sudo apt install make git ruby python3
gem install --user "asciidoctor:~>2.0.10" rouge

# More complete, for building these makefile targets:
# header html pdf pdfA4 manhtmlpages styleguide loader header-test: build-examples check-spec-links release
sudo apt -y install build-essential python3 git libxml2-dev ttf-lyx ghostscript ruby \
  python3-termcolor python3-tabulate python3-networkx
gem install --user "asciidoctor:~>2.0.10" rouge "asciidoctor-pdf:1.6.2"

# Full build: supports all makefile targets, including the "all" target
sudo apt -y install build-essential python3 git libxml2-dev ttf-lyx ghostscript ruby \
  trang jing python3-termcolor python3-tabulate python3-networkx
gem install --user "asciidoctor:~>2.0.10" rouge "asciidoctor-pdf:1.6.2"
```

Ubuntu 20.04+, Debian Bullseye, and Debian Buster Backports all have new enough
asciidoctor and asciidoctor-pdf in their repositories (Buster has asciidoctor-pdf
1.5.0 alpha17 which isn't ideal but should work) so the gem lines below can
be replaced with something like this (adding `-t buster-backports` if you're still
on Buster):

```
sudo apt install asciidoctor ruby-rouge

# or this, if you also want to build PDFs:
sudo apt install asciidoctor ruby-asciidoctor-pdf
```

#### macOS

It should be possible to install equivalent dependencies using Homebrew. If you
know how, please contribute instructions.

## Installing Gems

Since most Linux distributions do not have new enough AsciiDoctor/AsciiDoctor-PDF,
these can be installed by gem separately.

Something like this will work:

```sh
# Bare minimum install
gem install --user "asciidoctor:~>2.0.10"

# Adding PDF support
gem install --user "asciidoctor:~>2.0.10" "asciidoctor-pdf:1.6.2"
```

After the `gem install --user`, you may see a message that a directory isn't on
the path. You can run the following, modifying the version number as necessary
to match the error message.

```sh
echo 'export PATH=$HOME/.gem/ruby/2.3.0/bin:$PATH' >> ~/.bashrc
```

Restart your shell (e.g. open a new terminal window). Opening a new tab in your
existing terminal application is usually not enough. Note that you do not need
to use the `-l` option, since the modifications were made to `.bashrc` rather
than `.bash_profile`. If successful, `which asciidoctor` should print something
that starts with your home directory.

(If you run something other than bash, you'll want to do the equivalent in your
shell.)

## If Gems not working

If you've installed Ruby Gems before, sometimes things can get out of sync.
This is especially true after a system update.
In fact, you may have installed some under your personal account and some
under the system account (using sudo).

If you have used "sudo" to install any gems before, you should either remove
all your "sudo" gems, or all your personal gems.

I removed all my personal gems to fix collisions and reinstalled the system
ones using the steps below.

First, uninstall the personal gems:

```sh
gem uninstall asciidoctor-pdf \
              asciidoctor-mathematical \
              asciidoctor \
              rake \
              rouge \
              json-schema \
              mathematical \
              ruby-enum
```

Then, I uninstalled all the same packages for the system:

```sh
sudo gem uninstall asciidoctor-pdf \
                   asciidoctor-mathematical \
                   asciidoctor \
                   rake \
                   rouge \
                   json-schema \
                   mathematical \
                   ruby-enum
```

Then, update Gem in one location.  Again, I used "sudo", but you don't have to here.

```sh
sudo gem pristine --all
sudo gem update --system
```

This actually restored the packages I had installed before.
But, just in case, I asked to install them again.

```sh
sudo gem install asciidoctor \
                 rake \
                 rouge \
                 json-schema \
                 mathematical \
                 ruby-enum \
                 rubygems-update
MATHEMATICAL_SKIP_STRDUP=1 sudo gem install asciidoctor-mathematical
sudo gem install asciidoctor-pdf --pre
```

Notice, that I also installed rubygems-update, not sure if it helped, but I found this online.

This allowed my system to be properly setup again.
