XQuery Design Patterns Reusable Solutions for Large-Scale XQuery Applications William Candillon {william.candillon@28msec.com} Balisage 2010 Monday, August 9, 2010
Design Patterns • Very mature in the object community • Reusable Software and Design • Documentation • Communication and Teaching Monday, August 9, 2010
Design Patterns Because Design Patterns bills itself as being concerned with OOP software alone, I fear that software developers outside the object community may ignore it. Tom DeMarco, IEEE Software, 1996 Monday, August 9, 2010
XQuery has grown... • Update Facility • Scripting Extension • Full-Text Support • Data Definition Facility (Zorba) • Rich variety of module Libraries (EXPath / EXQuery) Monday, August 9, 2010
XQuery has grown... ...so have its applications An XQuery Application (~15k LOC) Monday, August 9, 2010
Existing XQuery Application • Enterprise Resource Planning application entirely written in XQuery • Featuring • Web Front-end and APIs • Workflow Engine • Wiki Engine • 28 000 lines of code • 135 XQuery modules Monday, August 9, 2010
Symptoms • Strong coupling between modules • Low extensibility • Heterogeneous vocabulary Monday, August 9, 2010
Strong Coupling Between Modules module namespace oauth = "http://wwww.example.com/modules/oauth/client"; import module namespace io = "http://www.zorba-xquery.com/modules/readline"; import module namespace fs = "http://www.zorba-xquery.com/modules/file"; import module namespace xqddf = "http://www.zorba-xquery.com/modules/xqddf"; import module namespace utils = "http://www.28msec.com/modules/utils"; import module namespace random = "http://www.28msec.com/modules/random"; import module namespace cookies = "http://www.28msec.com/modules/http/cookies"; import module namespace http = "http://www.28msec.com/modules/http"; import module namespace rest = "http://expath.org/ns/http-client"; Monday, August 9, 2010
Low Extensibility module namespace oauth = "http://wwww.example.com/modules/oauth/client"; module namespace atom = "http://www.example.com/modules/atom/client"; declare function atom:get($feed-uri) { (: ... :) }; declare function atom:post($feed-uri, $entry) { (: ... :) }; declare function atom:put($feed-uri, $entry-uri, $entry) { (: ... :) }; declare function atom:delete($entry-uri) { (: ... :) }; Monday, August 9, 2010
Heterogeneous Vocabulary Monday, August 9, 2010
Use-Cases AtomPub Client / Server Description Pattern Language 1 Store an Atom entry on the server 2 Transform an Atom entry into XHTML 3 Notify Twitter for each new Atom entry Monday, August 9, 2010
Use-Case 1 Store an Atom entry on the server Monday, August 9, 2010
Use-Case 1 Store an Atom entry on the server Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) AtomPub post(http:request) Monday, August 9, 2010
Use-Case 1 Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) AtomPub post(http:request) ... create(atom:entry) Monday, August 9, 2010
Use-Case 1 Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) if(...) then coll:create($entry) AtomPub else if(...) post(http:request) file:create($entry) ... ... create(atom:entry) Monday, August 9, 2010
Use-Case 1 Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) if(...) then coll:create($entry) AtomPub else if(...) post(http:request) file:create($entry) ... Monday, August 9, 2010
Use-Case 1 Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) AtomPub post(http:request, store) Monday, August 9, 2010
Strategy Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) AtomPub Client post(http:request, store) post($request, file:create#1) Monday, August 9, 2010
Strategy declare sequential function atompub:post( $feed-uri as xs:string, $entry as schema-element(atom:entry), $store as (function(xs:string, atom:entry) as item()*)) ) { (: Processing ...:) (: Storing the entry :) $store($feed-uri, $entry-uri) }; Monday, August 9, 2010
Strategy Benefits • Reduced Coupling • Reusability • Flexibility Use it when • Multiple implementation share the same interface • Hide specific implementation details from a module • Large amount of conditional statements Monday, August 9, 2010
Use-Cases Description Pattern Language 1 Store an Atom entry on the server Strategy XQuery 1.1 2 Transform an Atom entry into XHTML 3 Notify Twitter for each new Atom entry Monday, August 9, 2010
Use-Case 2 • Transform an Atom entry into XHTML let $title := $feed/atom:title/text() return <html xmlns=”http://www.w3.org/1999/xhtml”> <head> <title>{$title}</title> </head> <body> <h1>{$title}</h1> { for $entry in $feed/atom:entry return <div id="{$entry/atom:id/text()}"> <h2>{$entry/atom:title/text()}</h2> {$entry/atom:content/*} </div> } </body> </html> Monday, August 9, 2010
Use-Case 2 • Providing transformations for a particular data type <?xml version="1.0" encoding="utf-8"?> <feed xmlns="http://www.w3.org/2005/Atom" xmlns:georss="http://www.georss.org/georss"> <title>Where is Waldo?</title> <link href="http://example.org/"/> <entry> <title>Cacilienstrasse 5, 8032 Zurich, Switzerland</title> <link href="http://example.org/2009/09/09/atom01"/> <updated>2009-08-17T07:02:32Z</updated> <georss:point>45.256 -71.92</georss:point> </entry> </feed> Monday, August 9, 2010
Use-Case 2 • Providing transformations for a particular data type Atom Syndication Format Atom http://www.w3.org/2005/Atom toHTML() GeoRSS GeoRSS http://www.georss.org/georss toHTML() Monday, August 9, 2010
Use-Case 2 • Providing transformations for a particular data type • How to combine them? Atom Syndication Format Atom http://www.w3.org/2005/Atom toHTML() Convertor toHTML() GeoRSS GeoRSS http://www.georss.org/georss toHTML() Monday, August 9, 2010
Use-Case 2 • Providing transformations for a particular data type • How to combine them? ... toHTML() Atom Syndication Format Atom http://www.w3.org/2005/Atom toHTML() Convertor toHTML() GeoRSS GeoRSS http://www.georss.org/georss toHTML() ... • Doesn’t scale toHTML() Monday, August 9, 2010
Translator • Separate interpretations of heterogeneous elements. • Bringing the XSLT programming paradigm into XQuery. Templates for namespace A Atom Syndication Format match-feed($node) http://www.w3.org/2005/Atom apply-feed($node, $templates) ... if($match($node)) then Client $apply($node, $templates) transform($node, $templates) else () Templates for namespace B match-email($node) Google Data Protocol apply-email($node, $templates) http://schemas.google.com/g/2005 match-...($node) apply-...($node, $templates) ... Monday, August 9, 2010
Template - Match Function declare function atom:feed-template( $transform, $node, $templates) as function(){ ((: match function :) function($node) as xs:boolean { typeswitch ($node) case $n as schema-element(atom:feed) return true() default return false() }, (: apply :) function($transform, $feed, $templates) as element(html:html) { <html xmlns="http://www.w3.org/1999/xhtml"> <h1>Feed</h1> <div id="entries"> {$transform($feed/atom:entry, $templates)} </div> </html> }) }; Monday, August 9, 2010
Template - Apply Function declare function atom:feed-template( $transform, $node, $templates) as function(){ ((: match :) function($node) as xs:boolean { typeswitch ($node) case $n as schema-element(atom:feed) return true() default return false() }, (: apply function :) function($transform, $feed, $templates) as element(html:html) { <html xmlns="http://www.w3.org/1999/xhtml"> <h1>Feed</h1> <div id="entries"> {$transform($feed/atom:entry, $templates)} </div> </html> }) }; Monday, August 9, 2010
Translator - Transform Function declare function local:transform($node, $templates) as item()* { for $tpl in $templates let $template := $tpl() let $match := $template[1] let $apply := $template[2] return if($match($node)) then $apply(local:transform#2, $node, $templates) else () }; Monday, August 9, 2010
Translator - Conclusion Benefits • Independent modules can easily collaborate on the same XDM instance • Extending the XDM translation is easy Use it when: • Transform/Interpret an heterogeneous XML document • Use the power of XSLT paradigm in XQuery Monday, August 9, 2010
Use-Cases Description Pattern Language 1 Store an Atom entry on the server Strategy XQuery 1.1 2 Transform an Atom entry into XHTML Translator XQuery 1.1 3 Notify Twitter for each new Atom entry Monday, August 9, 2010
Use-Case 3 • Publish/Subscribe mechanism for the Atom server • Enable new kind of collaboration with an arbitrary number of modules • Example: Sending a message on Twitter for each new Atom entry Monday, August 9, 2010
Observer Monday, August 9, 2010
Observer (: Hold the observer references :) declare variable $atompub:on-post := (); (: Add an observer to the post request :) declare sequential function atompub:on-post($o) as item()*)){ set $atompub:on-post := ($atompub:on-post, $o) }; declare sequential function atompub:post() { (: Processing :) (: Notify observers :) for $o in $atompub:on-post return $o($entry); }; Monday, August 9, 2010
Observer - Conclusion Benefits • Extensible behavior • Collaboration with an arbitrary number of modules • Low coupling Applicability • Publish/Subscribe paradigm within XQuery • Keep consistency between module states Monday, August 9, 2010
Use-Cases Description Pattern Language 1 Store an Atom entry on the server Strategy XQuery 1.1 2 Transform an Atom entry into XHTML Translator XQuery 1.1 3 Notify Twitter for each new Atom entry Observer Scripting Monday, August 9, 2010
XQuery Design Pattern Catalog Monday, August 9, 2010
XQuery Design Pattern Catalog Join the community: http://patterns.28msec.com Monday, August 9, 2010
Conclusion • Design Patterns are going beyond the object community • Companies are building large-scale applications in XQuery • Promoting better designs and low coupling • XQuery Design Patterns... • ...are pragmatic solutions • ...they belong to the community Monday, August 9, 2010
Thank You! Questions? More info http://patterns.28msec.com Monday, August 9, 2010

XQuery Design Patterns

  • 1.
    XQuery Design Patterns Reusable Solutions for Large-Scale XQuery Applications William Candillon {william.candillon@28msec.com} Balisage 2010 Monday, August 9, 2010
  • 2.
    Design Patterns • Very mature in the object community • Reusable Software and Design • Documentation • Communication and Teaching Monday, August 9, 2010
  • 3.
    Design Patterns Because Design Patterns bills itself as being concerned with OOP software alone, I fear that software developers outside the object community may ignore it. Tom DeMarco, IEEE Software, 1996 Monday, August 9, 2010
  • 4.
    XQuery has grown... • Update Facility • Scripting Extension • Full-Text Support • Data Definition Facility (Zorba) • Rich variety of module Libraries (EXPath / EXQuery) Monday, August 9, 2010
  • 5.
    XQuery has grown... ...so have its applications An XQuery Application (~15k LOC) Monday, August 9, 2010
  • 6.
    Existing XQuery Application • Enterprise Resource Planning application entirely written in XQuery • Featuring • Web Front-end and APIs • Workflow Engine • Wiki Engine • 28 000 lines of code • 135 XQuery modules Monday, August 9, 2010
  • 7.
    Symptoms • Strong coupling between modules • Low extensibility • Heterogeneous vocabulary Monday, August 9, 2010
  • 8.
    Strong Coupling BetweenModules module namespace oauth = "http://wwww.example.com/modules/oauth/client"; import module namespace io = "http://www.zorba-xquery.com/modules/readline"; import module namespace fs = "http://www.zorba-xquery.com/modules/file"; import module namespace xqddf = "http://www.zorba-xquery.com/modules/xqddf"; import module namespace utils = "http://www.28msec.com/modules/utils"; import module namespace random = "http://www.28msec.com/modules/random"; import module namespace cookies = "http://www.28msec.com/modules/http/cookies"; import module namespace http = "http://www.28msec.com/modules/http"; import module namespace rest = "http://expath.org/ns/http-client"; Monday, August 9, 2010
  • 9.
    Low Extensibility module namespace oauth = "http://wwww.example.com/modules/oauth/client"; module namespace atom = "http://www.example.com/modules/atom/client"; declare function atom:get($feed-uri) { (: ... :) }; declare function atom:post($feed-uri, $entry) { (: ... :) }; declare function atom:put($feed-uri, $entry-uri, $entry) { (: ... :) }; declare function atom:delete($entry-uri) { (: ... :) }; Monday, August 9, 2010
  • 10.
  • 11.
    Use-Cases AtomPub Client / Server Description Pattern Language 1 Store an Atom entry on the server 2 Transform an Atom entry into XHTML 3 Notify Twitter for each new Atom entry Monday, August 9, 2010
  • 12.
    Use-Case 1 Store an Atom entry on the server Monday, August 9, 2010
  • 13.
    Use-Case 1 Store an Atom entry on the server Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) AtomPub post(http:request) Monday, August 9, 2010
  • 14.
    Use-Case 1 Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) AtomPub post(http:request) ... create(atom:entry) Monday, August 9, 2010
  • 15.
    Use-Case 1 Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) if(...) then coll:create($entry) AtomPub else if(...) post(http:request) file:create($entry) ... ... create(atom:entry) Monday, August 9, 2010
  • 16.
    Use-Case 1 Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) if(...) then coll:create($entry) AtomPub else if(...) post(http:request) file:create($entry) ... Monday, August 9, 2010
  • 17.
    Use-Case 1 Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) AtomPub post(http:request, store) Monday, August 9, 2010
  • 18.
    Strategy Collection Store File System Store REST Store create(atom:entry) create(atom:entry) create(atom:entry) AtomPub Client post(http:request, store) post($request, file:create#1) Monday, August 9, 2010
  • 19.
    Strategy declare sequential function atompub:post( $feed-uri as xs:string, $entry as schema-element(atom:entry), $store as (function(xs:string, atom:entry) as item()*)) ) { (: Processing ...:) (: Storing the entry :) $store($feed-uri, $entry-uri) }; Monday, August 9, 2010
  • 20.
    Strategy Benefits • Reduced Coupling • Reusability • Flexibility Use it when • Multiple implementation share the same interface • Hide specific implementation details from a module • Large amount of conditional statements Monday, August 9, 2010
  • 21.
    Use-Cases Description Pattern Language 1 Store an Atom entry on the server Strategy XQuery 1.1 2 Transform an Atom entry into XHTML 3 Notify Twitter for each new Atom entry Monday, August 9, 2010
  • 22.
    Use-Case 2 • Transform an Atom entry into XHTML let $title := $feed/atom:title/text() return <html xmlns=”http://www.w3.org/1999/xhtml”> <head> <title>{$title}</title> </head> <body> <h1>{$title}</h1> { for $entry in $feed/atom:entry return <div id="{$entry/atom:id/text()}"> <h2>{$entry/atom:title/text()}</h2> {$entry/atom:content/*} </div> } </body> </html> Monday, August 9, 2010
  • 23.
    Use-Case 2 • Providing transformations for a particular data type <?xml version="1.0" encoding="utf-8"?> <feed xmlns="http://www.w3.org/2005/Atom" xmlns:georss="http://www.georss.org/georss"> <title>Where is Waldo?</title> <link href="http://example.org/"/> <entry> <title>Cacilienstrasse 5, 8032 Zurich, Switzerland</title> <link href="http://example.org/2009/09/09/atom01"/> <updated>2009-08-17T07:02:32Z</updated> <georss:point>45.256 -71.92</georss:point> </entry> </feed> Monday, August 9, 2010
  • 24.
    Use-Case 2 • Providing transformations for a particular data type Atom Syndication Format Atom http://www.w3.org/2005/Atom toHTML() GeoRSS GeoRSS http://www.georss.org/georss toHTML() Monday, August 9, 2010
  • 25.
    Use-Case 2 • Providing transformations for a particular data type • How to combine them? Atom Syndication Format Atom http://www.w3.org/2005/Atom toHTML() Convertor toHTML() GeoRSS GeoRSS http://www.georss.org/georss toHTML() Monday, August 9, 2010
  • 26.
    Use-Case 2 • Providing transformations for a particular data type • How to combine them? ... toHTML() Atom Syndication Format Atom http://www.w3.org/2005/Atom toHTML() Convertor toHTML() GeoRSS GeoRSS http://www.georss.org/georss toHTML() ... • Doesn’t scale toHTML() Monday, August 9, 2010
  • 27.
    Translator • Separate interpretations of heterogeneous elements. • Bringing the XSLT programming paradigm into XQuery. Templates for namespace A Atom Syndication Format match-feed($node) http://www.w3.org/2005/Atom apply-feed($node, $templates) ... if($match($node)) then Client $apply($node, $templates) transform($node, $templates) else () Templates for namespace B match-email($node) Google Data Protocol apply-email($node, $templates) http://schemas.google.com/g/2005 match-...($node) apply-...($node, $templates) ... Monday, August 9, 2010
  • 28.
    Template - MatchFunction declare function atom:feed-template( $transform, $node, $templates) as function(){ ((: match function :) function($node) as xs:boolean { typeswitch ($node) case $n as schema-element(atom:feed) return true() default return false() }, (: apply :) function($transform, $feed, $templates) as element(html:html) { <html xmlns="http://www.w3.org/1999/xhtml"> <h1>Feed</h1> <div id="entries"> {$transform($feed/atom:entry, $templates)} </div> </html> }) }; Monday, August 9, 2010
  • 29.
    Template - ApplyFunction declare function atom:feed-template( $transform, $node, $templates) as function(){ ((: match :) function($node) as xs:boolean { typeswitch ($node) case $n as schema-element(atom:feed) return true() default return false() }, (: apply function :) function($transform, $feed, $templates) as element(html:html) { <html xmlns="http://www.w3.org/1999/xhtml"> <h1>Feed</h1> <div id="entries"> {$transform($feed/atom:entry, $templates)} </div> </html> }) }; Monday, August 9, 2010
  • 30.
    Translator - TransformFunction declare function local:transform($node, $templates) as item()* { for $tpl in $templates let $template := $tpl() let $match := $template[1] let $apply := $template[2] return if($match($node)) then $apply(local:transform#2, $node, $templates) else () }; Monday, August 9, 2010
  • 31.
    Translator - Conclusion Benefits • Independent modules can easily collaborate on the same XDM instance • Extending the XDM translation is easy Use it when: • Transform/Interpret an heterogeneous XML document • Use the power of XSLT paradigm in XQuery Monday, August 9, 2010
  • 32.
    Use-Cases Description Pattern Language 1 Store an Atom entry on the server Strategy XQuery 1.1 2 Transform an Atom entry into XHTML Translator XQuery 1.1 3 Notify Twitter for each new Atom entry Monday, August 9, 2010
  • 33.
    Use-Case 3 • Publish/Subscribe mechanism for the Atom server • Enable new kind of collaboration with an arbitrary number of modules • Example: Sending a message on Twitter for each new Atom entry Monday, August 9, 2010
  • 34.
  • 35.
    Observer (: Hold the observer references :) declare variable $atompub:on-post := (); (: Add an observer to the post request :) declare sequential function atompub:on-post($o) as item()*)){ set $atompub:on-post := ($atompub:on-post, $o) }; declare sequential function atompub:post() { (: Processing :) (: Notify observers :) for $o in $atompub:on-post return $o($entry); }; Monday, August 9, 2010
  • 36.
    Observer - Conclusion Benefits • Extensible behavior • Collaboration with an arbitrary number of modules • Low coupling Applicability • Publish/Subscribe paradigm within XQuery • Keep consistency between module states Monday, August 9, 2010
  • 37.
    Use-Cases Description Pattern Language 1 Store an Atom entry on the server Strategy XQuery 1.1 2 Transform an Atom entry into XHTML Translator XQuery 1.1 3 Notify Twitter for each new Atom entry Observer Scripting Monday, August 9, 2010
  • 38.
    XQuery Design PatternCatalog Monday, August 9, 2010
  • 39.
    XQuery Design PatternCatalog Join the community: http://patterns.28msec.com Monday, August 9, 2010
  • 40.
    Conclusion • Design Patterns are going beyond the object community • Companies are building large-scale applications in XQuery • Promoting better designs and low coupling • XQuery Design Patterns... • ...are pragmatic solutions • ...they belong to the community Monday, August 9, 2010
  • 41.
    Thank You! Questions? More info http://patterns.28msec.com Monday, August 9, 2010