GLEP 68: Package and category metadata

Metadata files

This specification provides two kinds of metadata files: category metadata files and package metadata files. Both kinds of files use the XML 1.0 file format [3]. They must not use external markup declarations, as defined in the XML specification. While they may reference or include a DTD, the parser must not fetch or process it.

The data structure of metadata files is defined in this GLEP. The elements and attributes do not use namespaces. Conforming files must not contain any elements or attributes that are not defined in this specification. However, parsers should ignore any unknown elements or attributes in order to permit future extension.

Category metadata files are named metadata.xml and located inside category directories in an ebuild repository. Their structure is described in Category metadata section.

Package metadata files are named metadata.xml and located inside package directories in an ebuild repository. Their structure is described in Package metadata section.

Text data

The following text data types are used:

• text data,
• multi-line text data.

In case of text data, all whitespace inside the element is normalized (consecutive whitespace sequences are replaced by a single SP). Trailing and leading whitespace is stripped.

In case of multi-line text data, all whitespace except for newline characters is normalized. Newlines are used to delimit lines of text. Leading and trailing lines of text that are either empty or consist purely of whitespace are stripped. Afterwards, the whitespace belonging to the indentation common to all non-empty lines of text is stripped.

Optionally, interspersing text with <cat/> and <pkg/> elements can be allowed. In this case, <cat/> element is used to reference a category inside the repository, and must contain a valid category name. <pkg/> is used to reference a package, and must contain a valid qualified package name.

Common attributes

The following common attributes are allowed on multiple elements:

• language specifiers,
• restriction specifiers.

Language specifiers are used whenever an element supports variants in different languages. In this case, each occurrence of the element may contain an optional lang="" attribute that contains an IETF language tag [4]. In case no lang="" attribute is provided, an implicit default of en is assumed.

Restriction specifiers are used whenever an element supports restricting to specific package versions. In this case, each occurence of the element may contain an optional restrict="" attribute that contains an EAPI 5 dependency specification that has to match one or more versions of the package. In this case, the metadata provided by the element applies only to the package versions matching the restriction.

Category metadata

The category metadata file uses <catmetadata/> top-level element. This element can contain, in any order:

• zero or more <longdescription/> elements containing category descriptions in different languages (at most one for each language). The category description is formed of multi-line text, optionally interspersed with <cat/> and <pkg/> elements.

Package metadata

Information sources

The basic source of information on current metadata.xml format was metadata.dtd as of 2016-03-02 [5]. Whenever the DTD was unclear, appropriate GLEPs were referenced in order to deduce the original intent. Whenever the GLEPs were unclear or the elements missed GLEPs, original mailing list discussions were referenced.

Removed elements

Compared to the original DTD, the following elements were removed (both in the spec and in the updated DTD file):

• package-scope <changelog/> element was removed. It dates back to the original metadata.xml proposal [5] but it was never implemented — instead, plain text ChangeLogs were used. Furthermore, GLEP 46 introduced <changelog/> inside <upstream/> with different type which collided with the global declaration due to DTD limitations.
• package-scope <natural-name/> element was removed. It was available for 1.5yr and after that time, it reached four packages providing it and no known tool supporting/using it. It was used only to provide a copy of package name with correct case (e.g. libressl -> LibreSSL), therefore the information provided by it was considered redundant.
• top-level <packages/> variant was removed. It was never used and it was really unclear what its use would be. In any case, this made the DTD simpler.

<pkg/> value format

A debate on valid format of <pkg/> element values preceded the writing of this GLEP. The DTD did not specify a value format restriction on this, only suggested that it is used for cross-linking. Further on, GLEP 56 redefined its value to a valid CP or CPV. The practical uses did not include the latter case; however, it was common to include EAPI 1 slot specifiers or even EAPI 5 slot operators following the qualified package names.

After finding the Doug Goldstein's blog post on introduction of <pkg/> elements [6], it turned out that the original intent was to allow cross-linking/referencing from packages.gentoo.org. Since the latter uses qualified package names as identifiers, it was decided to restrict <pkg/> elements to reference those. For entries that include slot specifiers, it is recommended to move the slot specifiers out of <pkg/> element.

Language identifiers

Originally, the DTD used implicit default value of C. However, this value was not in line with real language specifiers found in metadata.xml. The latter usually took form of ISO 639-1 language codes which do not form a valid (complete) locale identifiers, while the former is not a valid language identifier in any of the considered standards. Furthermore, since en was commonly used to identify English in metadata.xml files, and no tools relied on the implicit default defined in the DTD, it was decided to change the implicit default to en.

Language identifiers were later updated to allow full IETF language tags, so that codes like pt-BR or zh-Hant can be represented.

Package restrictions

Originally, the DTD described the restrict="" attribute as: the format of this attribute is equal to the format of DEPEND lines in ebuilds. This specification is based upon this definition. However, for practical reasons it added three clarifications to it:

• only package dependency specifications are allowed (i.e. no USE-conditionals or multiple dependency specifications),
• EAPI 5 dependency specifications are allowed. Although metadata.xml provides no EAPI identification mechanism, the top-level profile directory specifies EAPI 5, and Portage supports EAPI 5 since 2012.
• only dependencies referencing the same package are allowed.

Furthermore, DTD added a special case for * value that applies if there are no other tags that apply. This behavior was not used at all, and being at least a bit confusing (compared to the common use of * to imply matching everything), it was removed.

Upstream block

The upstream block was defined by GLEP 46. However, this GLEP is ambiguous at the best. Tiziano Müller (one of the original authors) has explained the intent behind most of the elements of the GLEP.

In particular, he confirmed that the GLEP lists all elements that are allowed explicitly, and no implicit inclusions were meant to be allowed. This means that the <maintainer/> element does not allow a <description/>.

He also confirmed that unless noted otherwise, elements were not allowed to be used more than once. This affects <bugs-to/> and <changelog/> elements. Repetitions of <doc/> were only allowed because DTD technically didn't permit restricting them while allowing uses of different languages.

At the time of writing this GLEP, only a single Gentoo package was using multiple <bugs-to/> elements, and no packages were using multiple <changelog/> or <doc/> elements (or non-English docs). For this reason, this GLEP enforces the original intent of at most one element.

Rationale for upstream maintainer descriptions

The proper contents of the <maintainer/> elements in <upstream/> blocks were unclear in the DTD since the technical file format limitation implied that all elements and attributes added for the Gentoo maintainers also applied to upstream maintainers, and vice versa.

The comments in the DTD clearly separated attributes between the two — i.e. stated that the type attribute is used only for Gentoo maintainers, while the status attribute is used only for upstream maintainers. However, package version restrictions and maintainer descriptions were also implicitly allowed on them. Since neither of the two was allowed by GLEP 46, this specification disallows them.

Parsing metadata.xml

Since the metadata.xml format provided by this specification is compatible with existing tool, no new implementation is required for reading those files.

Checking metadata.xml validity

To provide more strict checking of metadata.xml files, XML schema file is provided in the Gentoo xml-schema repository [7]. This schema provides:

• element structure checks,
• data duplication checks (e.g. multiple descriptions for the same flag but see below),
• partial value correctness checks.

The limitations of the schema are:

• values are verified using simple regular expressions, so not all format violations will be caught (e.g. the rule will consider app-foo/bar-1 a valid qualified package name when the version suffix is disallowed),
• cross-references can not be checked (package references, category references, URLs, project identifiers),
• <maintainer type=""/> correctness can not be checked,
• data duplication checks are done per restrict="" value rather than per every package version matched by the restriction. Therefore, multiple definitions that are applied to a single package by two different restrict="" rules will not be caught.

Example metadata.xml file

<?xml version='1.0' encoding='UTF-8'?>
<pkgmetadata>
  <maintainer type='person'>
    <email>developer@example.com</email>
    <name>Example Developer</name>
  </maintainer>
  <maintainer type='person' restrict='dev-libs/foo:11'>
    <email>anotherdev@example.com</email>
    <name>Another Developer</name>
    <description>CC only on bugs for libfoo.so.11</description>
  </maintainer>
  <maintainer type='project'>
    <email>project@example.com</email>
    <name>Example Project</name>
  </maintainer>
  <maintainer type='person'>
    <email>upstream@example.com</email>
    <name>Upstream Developer</name>
    <description>Upstream developer, wishing to be CC-ed on bugs</description>
  </maintainer>
  <longdescription>
    First paragraph of extensive description.

    Second paragraph.
  </longdescription>
  <longdescription lang='de'>
    Erster Absatz mit detaillierter Beschreibung.

    Zweiter Absatz.
  </longdescription>
  <slots>
    <slot name='11'>Compatibility slot providing libfoo.so.11 only.</slot>
    <subslots>
      Match SONAME of libfoo.so.
    </subslots>
  </slots>
  <slots lang='de'>
    <slot name='11'>Kompatibilitäts-Slot, installiert ausschließlich libfoo.so.11.</slot>
    <subslots>
      Subslot ist stets identisch mit dem SONAME von libfoo.so.
    </subslots>
  </slots>
  <use>
    <flag name='foo'>Enables foo feature</flag>
    <flag name='bar' restrict='&lt;dev-libs/foo-12'>Enables bar feature (requires <pkg>dev-libs/bar</pkg>)</flag>
    <flag name='bar' restrict='&gt;=dev-libs/foo-12'>Enables bar feature</flag>
  </use>
  <use lang='de'>
    <flag name='foo'>Konfiguriert das Paket mit Unterstützung für foo</flag>
    <flag name='bar' restrict='&lt;dev-libs/foo-12'>Konfiguriert das Paket mit Unterstützung für bar (benötigt <pkg>dev-libs/bar</pkg>)</flag>
    <flag name='bar' restrict='&gt;=dev-libs/foo-12'>Konfiguriert das Paket mit Unterstützung für bar</flag>
  </use>
  <upstream>
    <maintainer status='active'>
      <email>upstream@example.com</email>
      <name>Upstream Developer</name>
    </maintainer>
    <maintainer status='inactive'>
      <!-- e-mail unknown -->
      <name>John Smith</name>
    </maintainer>
    <changelog>http://www.example.com/releases.html</changelog>
    <doc>http://www.example.com/doc.html</doc>
    <doc lang='de'>http://www.example.com/doc.de.html</doc>
    <bugs-to>http://www.example.com/issues.html</bugs-to>
    <remote-id type='foohub'>example/foo</remote-id>
  </upstream>
</pkgmetadata>

German translations provided by tamiko.