I have been working in multimedia platforms, authoring, annotation and semantic technologies for a long time. I started working on video games and game authoring systems, did some work on embedded systems, and helped build a real-time currency trading system. For some time I built professional video editing and effects gear, animation authoring systems and the integration of timing and animation models with web languages. I managed to return to my interest in linguistics as I began to work on semantic technologies for search and media annotation.
Much of this happened at a bunch of little companies you have probably never heard of: Imagic, Geoworks, Olsen & Assoc.(Zurich), Ampex, Accom, DimensionX, Nervana, Microsoft and Yahoo! Research (okay, you've heard of those last ones).
A significant interest has been in user models, runtimes and tools for managing, editing and presentation of multimedia. A few years back, I began looking at how multimedia and the Semantic Web will be combined, including community annotation models and the automatic generation of narrative. My current work applies semantic and social media technologies to the problems of academic and museums collections management.
In my copious spare time, I enjoy reading, cooking, hiking, woodworking and digital photography. I began blogging but got distracted by too much other writing. I may get back to this...
Current Work
After many years building products and pursuing research, I decided it was time to go back to school to deepen my skills in some key areas and to enjoy the collaboration with interesting students and faculty. I recently completed a Masters of Information Management and Science at UC Berkeley's School of Information. I had so much fun being back at Berkeley and working with museums, that I took a job as a Semantic Services Architect for the IST/Data Services group.
I have thought about tools to support community moderation and filtering of public annotation, with the hope of building a rich and semantically aware body of annotation to physical artifacts. This in turn would better enable semantic queries within a museum and across related museums, to support educational programs and more directed or focused museum visits (e.g., for students and researchers). Ultimately, it could also provide a means for museums to leverage the assets (media, annotations, research notes and even physical assets) of related museums, creating a semantic museum web. These ideas found expression in my Masters Final Project, and in my current research to bring semantic services to the campus community.
Some other recent projects include:
- An art installation ("The Bridge") with TouchableStories that will link the UC Berkeley campus to the nearby community of Richmond through plantscapes, media and oral history.
- A fun mashup using Flickr notes and regions to adapt photos to small format presentation. (see pubs below for the ACM short paper).
- A great project with the San Francisco International Film Festival, in which we got a few minutes of footage from about 18 directors, built an online tool to let the public cut up this video and create their own 1-minute films, and then screened the best 50 films at a special festival event. It was great fun, and we also got some good research out of it (see pubs below for the ACM paper).
- A project for Dan Klein's Statistical Natural Language Processing class to induce ontology from Flickr tags (see pubs below for the workshop paper).
- A family photobase project based using MySQL/PHP/Ajax that explored several areas of personal
photo management and sharing (roughly 10K photos):
- Ontology translation to map keywords into a faceted ontology
- Automatic generation of a web-based faceted browser UI (from the ontology) allowing novices (e.g., my kids and non-technical relatives) to explore the collection.
- Logical (functional) categories to represent concepts like cyclical time (e.g., "Spring", "weekends", "evening").
- Adaptive denormalization of the category association table to improve query performance and reduce table size.
I have been collaborating with Lynda Hardman's group at CWI Amsterdam, looking at the intelligent, automatic generation of multimedia and hypermedia presentations. We are exploring how we can use semantics of multimedia query results (either directly as meta-data and associated ontologies, or discovered patterns and connections) to more appropriately choose and populate discourse models, presentation patterns and then hypermedia formatting objects.
I work when I can with Peter King at University of Manitoba and Simon Thompson at University of Kent, on extensions to SMIL declarative languages for functional content description.
I continue to participate as an invited expert to the Synchronized Multimedia (SYMM), Timed Text and Scalable Vector Graphics (SVG) working groups at the World Wide Web Consortium (W3C). On a related theme, I wrote a detailed design document for Timing and Animation Support in Batik - while still a work in progress it may be of interest to some folks. Brian Birtles at the University of Technology, Sydney is using this as a basis for his work on SVG animation in Mozilla.
I serve on program and review committees for multimedia and web technology conferences and journals.
Earlier Work
At Nervana, I developed the client architecture for a semantic discovery and awareness application. I spent a lot of time optimizing incremental XSLT processing and managing async/streaming XML. I also built some tools for analysis and maintenance of categorization ontologies.
While I was at Microsoft, I developed models for representing time in XML documents, and worked to implement these in various media standards. I am particularly interested in the integration of SMIL and timing into HTML (see the original HTML+TIME submission, the SMIL 2.0 work and the XHTML+SMIL Note). This functionality is supported in IE5 and later (play with it!). To find out more, see the related MSDN page, and the W3C SYMM public and (member only) Working Group pages.
I spent some time pestering people on the synchronization issues related to the integration of HTML and television, radio and DVD, and what happens when you can record and review HTML-enhanced media. I still think this makes more sense than the hacks described in ATVEF and DVB 1.
Check out the demos page for examples of my SMIL work in action.
At DimensionX, a little startup that was digested by Microsoft, I led the development of Liquid Motion, a Java-based authoring tool and runtime for 2-D animation. Many of the ideas about animation and timing that ended up in my contributions to SMIL germinated in this tool. Microsoft let this die on the vine, which is too bad - I miss playing with it.
At AMPEX, Axial and Accom, I worked on video editing and video special effects gear. This included some very cool dedicated hardware used in Hollywood, New York, London and elsewhere to edit and manipulate video. I do not miss the vagaries of discontinuous (drop-frame) time, but I do miss working directly with the creative folks that were our customers.
Before that, I was a researcher for the US Forest Service, worked as a naturalist on whale-watching boat tours, wrote video games and game development tools, wrote the front-end for a currency trading system in Zurich, and played at odds and ends too numerous to remember.
Publications:
- The CONCUR Framework for Community Maintenance of Curated Resources
- Patrick Schmitz, ACM Document Engineering 2008, Sao Paolo, Brazil(to appear).
- The Delphi toolkit: Enabling Semantic Search for Museum Collections
- Patrick Schmitz, Michael Black, Museums and the Web 2008, Apr. 9-12 2008, Montréal, Québec, Canada.
- Leveraging Community Annotations for Image Adaptation to Small Presentation Formats
- Patrick Schmitz, ACM Multimedia 2006, Oct. 23-27, Santa Barbara, CA.
- Community Annotation and Remix: a Research Platform and Pilot Deployment
- Ryan Shaw, P. Schmitz, ACM Multimedia 2006, Oct. 23-27, Santa Barbara, CA. [Note: PDF is 5MB huge]
- Inducing Ontology from Flickr Tags
- Patrick Schmitz, Collaborative Web Tagging workshop, 15th International World Wide Web Conference 23-26 May 2006, Edinburgh, Scotland.
- Behavioral Reactivity and Real Time Programming in XML
- Peter King, P. Schmitz, S. Thompson, ACM Document Engineering 2004 Proceedings pp. 57-66.
- The Convergence of Applications and Documents - Requirements and Directions
- Patrick Schmitz, Position Paper, W3C Workshop on Web Applications and Compound Documents, 2004.
- Presentation Dynamism in XML: Functional Programming meets SMIL Animation
- Patrick L. Schmitz, Simon Thompson, Peter King, Poster Session, Twelfth International World Wide Web Conference 20-24 May 2003, Budapest, HUNGARY
- Presentation Dynamism in XML: Functional Programming meets SMIL Animation
- Patrick L. Schmitz, Simon Thompson, Peter King, Technical Note Nov 2002
- Multimedia Goes Corporate
- Patrick L. Schmitz, IEEE Multimedia, July 2002 (also available here)
- Multimedia Meets Computer Graphics in SMIL2.0: A Time Model for the Web
- Patrick L. Schmitz, Eleventh International World Wide Web Conference, May 2002.
- Improving Media Fragment Integration in Emerging Web Formats
- Lloyd Rutledge, P. Schmitz, 8th International Conference on Multimedia Modeling, Nov 2001.
- SMIL Timing for the Web
- Patrick Schmitz. Introduction to the SMIL 2.0 Timing model (view in HTML format, in IE 5.5 or later to see the demos inline with the doc). You can also download a ZIP file with demo media. The associated demos are also available separately.
- The Link vs. the Event: Activating and Deactivating Elements in Time-Based Hypermedia
- Lynda Hardman, P. Schmitz, J. van Ossenbruggen, W. ten Kate, L. Rutledge, New Review of Hypermedia and Multimedia, Volume 6, 2000.
- A Unified Model for Representing Timing in XML Documents
- Patrick Schmitz. WWW9 Multimedia Workshop Position Paper.
- Unifying Scheduled Time models with Interactive Event-based Timing
- Patrick Schmitz (W3C Internal note). This is an early note describing how to
merge two timing models into one.
Available at: UnifyingNote.html, UnifyingNote.doc (MSWord) and UnifyingNote.ps (PostScript) - XHTML+SMIL Language Profile
- Debbie Newman, A. Patterson, P. Schmitz, (eds). W3C Note, 31 Jan. 2002.
This describes a language that adds SMIL multimedia extensions to HTML/CSS. - SMIL Animation
- Patrick L. Schmitz and Aaron Cohen. W3C Recommendation 4 Sep. 2001.
This is a standalone recommendation that describes a simplified timing model and basic animation support. - SMIL 2 Animation Module
- Patrick Schmitz, A. Cohen, K. Day, editors. SMIL 2.0 W3C Recommendation 7 Aug 2001. This is mostly the same animation support as in SMIL Animation, but integrated with the complete SMIL 2 Timing and other functionality.
- SMIL 2 Timing and Synchronization Module
- Patrick L. Schmitz, B. Saccocio, J. Ayers, editors. SMIL 2.0 W3C Recommendation 7 Aug 2001.
- SMIL 2 Time Manipulations Module
- Patrick L. Schmitz, editor. SMIL 2.0 W3C Recommendation 7 Aug 2001.
- SMIL 2 Document Object Model
- Philippe Le Hégaret, P. Schmitz, editors. W3C working draft (not included as part of SMIL 2 Recommendation).
- Synchronized Multimedia Integration Language (SMIL) Modules
- Warner ten Kate, Ted Wugofski, Patrick Schmitz, editors. W3C Note 23 Feb 1999.
- XHTML Events Module
- An updated events syntax for XML-based markup languages, Ted Wugofski, Patrick
Schmitz, Shane P. McCarron.
This early draft has been superceded by XML Events that reduces the scope to event binding. - HTML+TIME submission to W3C
- Patrick Schmitz, Jin Yu, Peter Santangeli. Submission to W3C that proposed integration with HTML and CSS.
- Delivering on the Promise of Java-based Animation
- Patrick Schmitz and A. Pineda, in "Mastering Web Design", John McCoy Ed., Sybex 1996.
- Reprioritizable Asynchronous Media Server Request Mechanism and Method
- Patrick L. Schmitz and S. Jahaul (U.S. Patent #5,544,318)
- Energy Effects of Forest Management Alternatives
- Gideon Schwarzbart and P. Schmitz, U.S.D.A. Forest Service Technical Publication, 1982.
Talks and Presentations:
- Synchronization of XHTML Documents: The XHTML+SMIL Language
- SMIL Europe 2003 Conference, Paris France, Feb. 2003 (IE6 only - uses XSLT to generate XHTML+SMIL)
- XHTML+SMIL, SVG and SVG-Animation Generation with XSLT
- SMIL Europe 2003 Conference, Paris France, Feb. 2003 (IE6 only - uses XSLT to generate XHTML+SMIL)
- Advancing Multimedia on the Web
- SBMidia 2001 Conference, Florianoplis Brazil, Oct. 2001 (IE6 only - uses XSLT to generate XHTML+SMIL)
- Integrating Timing into XML Documents
- INRIA PDMS 'summer school', Autrans, France, Aug 2001
- Integrating Timing with XML
- Phillips Research Center, Eindhoven, Netherlands, May 2000
- Integrating Timing with XML
- Slide deck to the WWW9 Multimedia Workshop Position Paper.
- Integrating Timing with XML Documents
- Guest lecture for CS 298, U.C. Berkeley, Apr 2000
- Using RealAudio/RealVideo Plug-ins, ActiveX with JavaScript/Java
- RealMedia Dev. Conf. 1997
- Multimedia Animation: Tips and Techniques
- MS SiteBuilder Conf., San Jose, Oct. 1996
- Multimedia Design with Java
- Seybold Seminars, San Francisco, Sep. 1996 and Tokyo, Dec. 1996
- Designing Class Libraries
- JavaOne, May 1996
- Multimedia Animation in Java
- Netscape Developers Forum, Mar. 1996 and Internet Developers Forum, May 1996
- Principles of Animation Runtimes
- Guest lecture for CS 298, U.C. Berkeley, Spring 1996