MP: A compiler for formal metadata

Title: MP: A compiler for formal metadata

Abstract:

MP is a program for validating the syntactical structure of formal metadata, testing the structure against the Content Standards for Digital Geospatial Metadata devised by the Federal Geographic Data Committee (FGDC). MP is described as a compiler because it contains not only a lexical parser but also code to analyze the tree that the parser generates, and code to output the metadata in several different formats. It is written in Standard C (i.e. ANSI C) and runs on Linux, UNIX, and all versions of Microsoft Windows (95 and later, including XP).

MP generates a textual report indicating errors in the metadata, primarily in the structure but also in the values of some of the scalar elements (i.e. those whose values are restricted by the standard). Output formats include text (the same as the input format), Hypertext Markup Language (HTML), Standard Generalized Markup Language (SGML), Extensible Markup Language (XML) and Directory Interchange Format (DIF).

MP has the ability to recognize and process elements that are not part of the FGDC standard, provided these elements are properly described in a local file.

Supplemental_Information:

1. Command line options

MP is started by issuing a command to the operating system. Its behavior is controlled both by switches on the command line and through a configuration file. Command-line switches are as follows:

 option      resulting action
 --------------------------------------------------------
 -l code     use language indicated by code (en,es,id,fr)
 -c cfile    obtain configuration information from cfile
 -e efile    direct syntax errors to efile
 -t tfile    create text output in tfile
 -h hfile    create html output in hfile
 -f ffile    create FAQ-style html in ffile
 -s sfile    create sgml output in sfile
 -d dfile    create DIF  output in dfile

The name of the input file that contains formal metadata is given on the command line but is not preceded by a switch. The input file name must not begin with a hyphen.

Example:

 mp -c config.dat foo -e foo.err -t foo.txt -h foo.html -f foo.faq.html -s foo.sgml -d foo.dif

This command causes mp to read options from the file config.dat and metadata from the file foo. Errors are directed to the file foo.err, and output is directed to the following files:

 foo.txt      ASCII, using the input format
 foo.html     Hypertext Markup Language
 foo.faq.html FAQ-style Hypertext Markup Language
 foo.sgml     Standard Generalized Markup Language
 foo.xml      Extensible Markup Language
 foo.dif      Directory Interchange Format

Names of output files may be omitted if the corresponding information is included in the configuration file, which is described in section 3 below.

Errors are printed to the error file or, if none was specified, to the standard error device (stderr), which is usually the console or the terminal from which the program was launched. Error messages and warnings refer to the line in the input metadata file by number (i.e. "Error (line 1):" refers to the first line in the input file). Warnings are issued when conditions that are considered unusual are detected; errors indicate a condition that runs counter to the dictates of the FGDC standard.

2. Input file format

Since the FGDC Content Standards for Digital Geospatial Metadata, as the name implies, specifies only the contents of metadata files and not their encoding, it is necessary to choose or devise a specification for metadata encoding in order to create formal metadata.

The encoding format interpreted by this compiler is purely textual. It describes the hierarchical structure of the metadata using indentation, in which the members of compound elements are indented more than their parent element and all of the elements at the same level in the hierarchy are indented alike. The full hierarchy is specified, even elements like Contact_Information that exist only to contain other elements and would not normally be needed by a human reader. The format is designed to be both parseable and human- readable, with a minimum of unnecessary jargon and code, but the requirement to be parseable makes it a lot more structured than ordinary text.

Precise rules governing the format of the input file are as follows:

Terms:

tab := ASCII 9

space := ASCII 32

element name := A sequence of bytes consisting of alphanumeric characters, the underscore, hyphen, and forward slash. This sequence is one of the formal names given to metadata elements in the standard.

Examples:

Citation
Identification_Information
Data_Set_G-Polygon_Outer_G-Ring
Range_of_Dates/Times

value := A text string associated with an element by the originator of the metadata.

Arrangement:

a. Metadata files are plain text without markup. Non-ASCII character may be used and will be passed through to the output files but in the HTML output these may be interpreted as having ISO-8859-1 encoding by subsequent software.

b. The number of characters per line is not limited.

c. Indentation is accomplished using tabs, spaces, or a combination of the two, but for purposes of determining indentation level, one tab equals one space.

d. Blank lines may occur anywhere in the file.

e. Element names are spelled out in the metadata file exactly as in the syntax rules of the metadata content standard. Where the descriptive portion of the standard differs from the syntactical rules, the syntactical rules are regarded as authoritative, e.g. Attribute_Units_of_Measure is correct.

f. A single colon or equal sign may follow each element name but is not required.

g. Whitespace may occur between element name and colon or equal sign, and may occur after the colon or equal sign.

h. Values are associated with an element in one of three ways:

(1) The value begins at the first nonblank following the element name (or following colon or equal sign) and extends to the end of the line.

(2) The value begins on the line following the element name. It is indented more than the element name, i.e. there are more spaces or tabs preceding the value than precede the element name.

(3) The value begins on the line containing the element name. It extends onto subsequent lines, where it is indented more than the element name, i.e. there are more spaces or tabs preceding the value on lines following the element name than precede the element name.

i. Values of compound types occur on successive lines using the same degree of indentation.

Example:

  Citation_Information:
    Originator:
    Publication_Date:
    Publication_Time:
    Title:
    Type_of_Map:
    Serial_Information:
      Serial_Name:
      Issue_Identification:
    Publication_Information:
      Publication_Place:
      Publisher:
    Other_Citation_Details:
    Online_Linkage:
    Larger_Work_Citation:

3. Configuration file format

Described in <http://geology.usgs.gov/tools/metadata/tools/doc/config.html>

4. Local extension file format

The local extension file is encoded like the metadata; hierarchical structure is indicated using indentation.

local:
  name <element name>
  sgml <tag name>
  parent <element name>
  child <element name>

Compound types:

 local = name + (sgml) + 1{parent}n + 1{child}n

Scalar types:

name

The name of the element as it will appear in the metadata. This must consist only of upper- and lower-case letters, numbers, and the underscore character.

sgml

The element tag to be used for output in SGML. If omitted, the full name is used.

parent

The name of the element under which this element may appear. These names may be standard elements or local extensions.

child

The name of an element which may appear under this element.

ALBERSCEA AZIMUTHAL EQUIDISTC EQUIRECT GENERALVNP GNOMONIC LAMBERTAZ LAMBERTCC MERCATOR MILLER MODSALASKA OBMERCATOR ORTHO POLYCONIC PSTEREO ROBINSON SINUSOIDAL SPOBLMERC STEREO TM VANDERG

Data format:	Executable and source code Size: 3 megabytes
Network links:	<http://geology.usgs.gov/tools/metadata/mp-2.9.*.zip>

Data format:

Executable and source code Size: 3 megabytes

Network links:

<http://geology.usgs.gov/tools/metadata/mp-2.9.*.zip>

Data format:	Source and executable code with documentation in HTML in format C (version ANSI (1987)) Size: 800 kilobytes
Network links:	<http://geology.usgs.gov/tools/metadata/src.tar.gz>

Data format:

Source and executable code with documentation in HTML in format C (version ANSI (1987)) Size: 800 kilobytes

Network links:

<http://geology.usgs.gov/tools/metadata/src.tar.gz>

MP: A compiler for formal metadata

Frequently anticipated questions:

What does this data set describe?

Who produced the data set?

Why was the data set created?

How was the data set created?

How reliable are the data; what problems remain in the data set?

How can someone get a copy of the data set?

Who wrote the metadata?