XProc 3.1: dynamic pipeline execution

Draft Community Group Report

Editor's Draft at (build 140)
Latest editor’s draft:
http://spec.xproc.org/master/head/run/
Editors:
Norman Walsh
Achim Berndzen
Gerrit Imsieke
Erik Siegel
Participate:
GitHub xproc/3.0-steps
Report an issue
Changes:
Diff against current “status quo” draft
Commits for this specification

This document is also available in these non-normative formats: XML and HTML with automatic change markup courtesy of DeltaXML.


Abstract

This specification describes the p:run step for XProc 3.1: An XML Pipeline Language.

Status of this Document

This document is an editor's draft that has no official standing.

This specification was published by the XProc Next Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Contributor License Agreement (CLA) there is a limited opt-out and other conditions apply. Learn more about W3C Community and Business Groups.

If you wish to make comments regarding this document, please send them to xproc-dev@w3.org. (subscribe, archives).

1. Introduction

This specification describes the p:run XProc step. A machine-readable description of these steps may be found in steps.xpl.

Note

It will most probably end up in an XProc document of its own.

Familarity with the general nature of [XProc 3.0] steps is assumed; for background details, see [XProc 3.0 Steps].

2. p:run

The p:run step runs a dynamically loaded pipeline.

<p:run
  name? = NCName>
    p:with-input*
</p:run>

The p:run step functions mostly like an atomic step in that you can define inputs connections and option values for it. However, unlike atomic steps, it has no fixed signature. Any inputs are allowed and any outputs may be connected.

One input port is specifically designated the pipeline input port. By default, that port is named pipeline, but its name can be changed by specifying a different name in the p:pipeline option.

The p:run step expects a single pipeline document on the pipeline port. It is a dynamic error (err:XC0080) if the pipeline input to the p:run step is not a valid pipeline.

The pipeline that appears on the pipeline port is evaluated using the inputs and options specified on the p:run step. The default readable port of the p:run step becomes the default readable port of the pipeline. It is a dynamic error (err:XC0081) if the pipeline has declared inputs that are not specified on the p:run step. It is a dynamic error (err:XC0082) if the pipeline has required options that are not specified on the p:run invocation.

Note

Do we really need to have C0082? Isn’t it sufficient that the dynamically executed pipeline raises an error if a required option is missing?

If we raise this error on p:run, we might as well require that the types of the options given on p:run be compatible with what the pipeline’s option expects. Instead of raising such an error on p:run, we can wait until the pipeline is executed and any dynamic errors occur in the pipeline.

Each output port of the pipeline can appear as a same-named output port of the p:run step. In order for this to happen, the port needs to be explicitly declared in the p:run step. In contrast to output declarations of compound steps or of p:declare-pipeline with a subpipeline, such an output declaration may not establish a connection to any port of another step or of the pipeline to be run.

If the pipeline has an output that is not declared on the p:run step, that output is discarded, and the corresponding port on the p:run step does not exist. If the p:run step declares an output port that is not provided by the pipeline, an empty sequence appears on that port.

Note

Should we do a similar thing for inputs? That is, if the pipeline declares an input but the p:run invocation doesn’t provide a connection, it is treated as if the connection were declared explicitly empty. If we do so, C0081 is no error any more.

This might be useful in that it extends the range of pipelines that a given p:run invocation may run. Suppose that you modified an EPUB building step by adding an input port that accepts a text document with extra CSS to be used by every HTML page within the EPUB. So this extended EPUB step has an additional input port extra-css in addition to its (hypothetic) normal ports source and conf. It would be perfectly useful to run this extended EPUB building pipeline as if it were the standard EPUB building pipeline. It will see zero documents on its extra-css port, which is fine.

In that sense, a p:run invocation declares an interface that the dynamically executed pipelines must implement but may extend.

We don’t do this default-to-empty for unsaturated input connections of other step invocations, do we?

Likewise, what happens if the p:run invocation has a p:with-option element (or an option attribute) that does not exist in the pipeline? I suggest that this option be simply ignored, no attempt be made by the processor to pass this option to the pipeline, no error raised.

Note

This was Norm’s original draft:

The p:run step is assumed to have a primary output port. If the pipeline evaluated does not have a primary output port, an empty sequence appears on that port. Alternatively, that could be a dynamic error.

I think Norm has proposed both alternatives before he agreed that a p:run step needs to declare its outputs. With these output declarations mandatory, can we safely omit the primary port requirement?

2.1. Example

Example 1. Dynamic Execution of a Transformation/Validation Pipeline
<p:run name="runme" xslt-parameters="{map{{'foo':'bar'}}}">
  <p:with-input port="pipeline">
    <p:inline expand-text="true">
      <p:declare-step name="transform-n-validate">
        <p:input name="source" primary="true" sequence="true"/>
        <p:input name="stylesheet"/>
        <p:input name="xsd"/>
        <p:option name="assert-valid" as="xs:boolean" select="false()"/>
        <p:option name="xslt-parameters" as="map(xs:QName, item()*)?"/>
        <p:output port="result" primary="true"/>
        <p:output port="report" pipe="report@xsdval"/>
        <p:xslt parameters="{$xslt-parameters}">
          <p:with-input port="stylesheet" pipe="stylesheet"/>
        </p:xslt>
        <p:validate-with-xml-schema assert-valid="{$assert-valid}" name="xsdval">
          <p:with-input port="schema" pipe="xsd"/>
        </p:validate-with-xml-schema>
      </p:declare-step>
    </p:inline>
  </p:with-input>
  <p:with-input name="source" href="my.xml"/>
  <p:with-input name="stylesheet" href="my.xsl"/>
  <p:with-input name="xsd" href="my.xsd"/>
  <p:output port="result" primary="true"/>
  <p:output port="report"/>
</p:run>

2.2. Document properties

No document properties are preserved.

Note

Shouldn’t we say, the extent to which document properties are preserved depend on the steps in the dynamically executed pipeline?

3. Step Errors

This step can raise dynamic errors.

[Definition: A dynamic error is one which occurs while a pipeline is being evaluated.] Examples of dynamic errors include references to URIs that cannot be resolved, steps which fail, and pipelines that exhaust the capacity of an implementation (such as memory or disk space). For a more complete discussion of dynamic errors, see Dynamic Errors in XProc 3.0: An XML Pipeline Language.

If a step fails due to a dynamic error, failure propagates upwards until either a p:try is encountered or the entire pipeline fails. In other words, outside of a p:try, step failure causes the entire pipeline to fail.

The following errors can be raised by this step:

err:XC0080

It is a dynamic error if the pipeline input to the p:run step is not a valid pipeline.

See: p:run

err:XC0081

It is a dynamic error if the pipeline has declared inputs that are not specified on the p:run step.

See: p:run

err:XC0082

It is a dynamic error if the pipeline has required options that are not specified on the p:run invocation.

See: p:run

A. Conformance

Conformant processors must implement all of the features described in this specification except those that are explicitly identified as optional.

Some aspects of processor behavior are not completely specified; those features are either implementation-dependent or implementation-defined.

[Definition: An implementation-dependent feature is one where the implementation has discretion in how it is performed. Implementations are not required to document or explain how implementation-dependent features are performed.]

[Definition: An implementation-defined feature is one where the implementation has discretion in how it is performed. Conformant implementations must document how implementation-defined features are performed.]

A.1. Implementation-defined features

The following features are implementation-defined:

    A.2. Implementation-dependent features

    The following features are implementation-dependent:

      B. References

      [XProc 3.0] XProc 3.0: An XML Pipeline Language. Norman Walsh, Achim Berndzen, Gerrit Imsieke and Erik Siegel, editors.

      [XProc 3.0 Steps] XProc 3.0 Steps: An Introduction. Norman Walsh, Achim Berndzen, Gerrit Imsieke and Erik Siegel, editors.

      C. Glossary

      dynamic error

      A dynamic error is one which occurs while a pipeline is being evaluated.

      implementation-defined

      An implementation-defined feature is one where the implementation has discretion in how it is performed. Conformant implementations must document how implementation-defined features are performed.

      implementation-dependent

      An implementation-dependent feature is one where the implementation has discretion in how it is performed. Implementations are not required to document or explain how implementation-dependent features are performed.

      D. Ancillary files

      This specification includes by reference a number of ancillary files.

      steps.xpl

      An XProc step library for the declared steps.