Patrick Schmitz
Microsoft Research
Growing interest in SMIL underscores the importance of a common language to describe timing and synchronization in documents. SMIL 1 defined syntax and semantics for a basic multimedia language. This was an important first step, but does not allow authors to use the timing semantics with other languages, like HTML and CSS. Authors need to be able to describe timing for wide range of documents, and to combine the semantics of timing and synchronization with other existing tools and semantics. In addition, they need the flexibility to accommodate different editing/authoring scenarios and constraints.
We need to provide a common model for timing so that authors are not forced to learn and remember different models for different document types or different authoring scenarios. In addition, where different syntactic approaches are available (e.g. to address different authoring scenarios), it is important that the model allow authors to combine the different approaches in a straightforward manner.
SMIL Boston has extended the SMIL timing support, and added support for integration of timing and synchronization markup with other languages. Working Drafts describe three approaches to this integration:
SMIL Boston only specifies the inline syntax variant, mostly due to time constraints of the Working Group, but also due to the range of open issues with styled timing and timesheets. Looking to the future, each of these three options for integration is needed for a number of applications, and with this comes the need to combine the approaches.
A unified model must be developed that allows authors and language designers to choose the most appropriate syntax, and to combine the different methods for a given document. This timing integration model must preserve common timing semantics, and should work well with other new document tools such as XSLT.
Inline syntax provides a straightforward and easy-to-understand means of adding timing to a language. This approach is fairly well understood, and is documented in some detail in the SMIL Boston working drafts. Even in documents that use styled timing and/or timesheets, the inline syntax can be useful to augment or override more general rules defined with the other methods. This presumes that there is a clear hierarchy among inline timing, styled timing and timesheets.
CSS provides a good precedent for such a hierarchy, defining ordering and priorities in the stylesheet cascade. While the specifics of the CSS model may not apply, the general approach is a good one. Given the many authors familiar with CSS, a unified timing model should align to the general model and principles of the cascade model. In particular, inline syntax should override styled timing and timesheets.
Styled timing is well suited to documents (or document sections) in which the
document structure and the timing structure are fairly closely aligned. An
example of this is list structures that are presented as a sequence. For example
the HTML <ol> construct may be presented with a
highlight style applied to each list item for a given duration, in sequence.
In order to layer styled timing with inline syntax, the semantics must be consistent between the two approaches. CSS again provides a good precedent for a unified model, supporting both stylesheets and inline style specification in the cascade. Based upon the experience with CSS, a unified model for timing integration should base styled timing upon the manipulation of the same set of properties that the inline syntax describes.
Styled timing has some inherent issues, especially related to distinguishing presentation
style and timing. Two of the useful actions to perform over time
are application of an inline style rule (i.e. as described by a "style"
attribute), and application of a class to an element (effectively adding a token
to the "class" attribute for an element). Both of these time
actions can result in the modification of style properties on the element. In a
styled timing model, this includes the possibility of modifying the timing
properties. If timing invokes styling that in turn modifies the timing,
the complexity of the time model increases considerably. It is then too easy for authors to
create documents that have very non-obvious or undefined behavior.
A unified model should preclude this feedback complexity. One possible approach is to "lock" the timing properties while applying the time actions for an element so that the styling side-effects of the time action cannot feed back into the timing. Other approaches may also be explored.
Timesheets allows authors to specify timing and synchronization in a separate
document, or in a separate section of a document. The name borrows from CSS
stylesheets, and the idea applies some of the principles of stylesheets to the
description of timing and synchronization. The timesheet looks much like the timing section
of a SMIL language document, except that referenced content can be an
individual element in an XML file (e.g. a <span> in XHTML). In addition,
integration semantics such as "timeAction" are supported.
Timesheets address those authoring scenarios in which it is easier or simpler to separate the timing and synchronization description from the content and presentation controls. The specific requirements that timesheets address are that it must be possible to:
The first three requirements are addressed simply by virtue of separating
the timing and synchronization markup into the timesheet. The fourth requirement is
related to the overriding need for a unified model for timing integration. Multiple
time action support and a unified model are easily achieved if we model
timesheets as animating properties. That is, all time actions are
simply manipulations of some property over time. The property can be an abstract
like the intrinsic action of an HTML <b> (bold) element, or a
specific property like CSS visibility. Multiple references to
the same property on a given element are resolved as multiple animations of the
property, according to the rules already detailed in SMIL Animation.
Mixing timesheets, styled timing and inline timing can be thought of as a variant on the mix of XSL FO and CSS stylesheets that can be used to present XML documents. In effect, each stylesheet is a filter on the document. A key issue to work out is whether there is a fixed priority between timesheets and styled timing (i.e. a defined ordering of the filters), or whether they can be arbitrarily ordered. A fixed ordering may be easier to explain to authors, but may be too constraining.
A unified model for timing and synchronization can built upon a property based model for timing manipulation. This extends the SMIL Boston (inline) timing and synchronization syntax, integration controls and animation model to support styled timing and timesheet support. The model aligns with the widely understood model for cascading syntax, and should work well with a range of XML tools.