Development and support

Related links

Home

DERI Pipes: Open Source, Extendable, Embeddable Web Data Mashups

 Inspired by Yahoo's Pipes, DERI Pipes is an engine and graphical environment for general Web Data transformations and Mashup.

  • Supports RDF, XML, Microformats, JSON and binary streams.
  • Use it as a "Web Pipe" or embedded in your applications
  • Works as a mashup command Line tool
  • Supports SPARQL, XQUERY, Several scripting languages.
  • Extend it as needed

DERI Pipes, in general, produce as an output streams of data (e.g. XML, RDF,JSON) that can be used by applications. However, when invoked by a normal browser, they provide a end user GUI for the user to enter parameter values and browse the results (see the examples below) .

 

Tim Berners-Lee on the Semantic Web  pipe  Search city's facts from Sindice.com and Geonames

Pipe examples in XML (see specification )

Pipes are written in a simple XML language to specify the operators in the pipe and the connections between them. E.G. a mixing data from 2 sources is as simple as this:

<pipe>
<simplemix>
<source><fetch><location>http://www.w3.org/People/Berners-Lee/card</source></fetch></location>
<source><fetch><location>http://g1o.net/foaf.rdf</source></fetch></location>
</simplemix>
</pipe>

The Construct block allows RDF Transformation by the means of SPARQL Construct queries. Moreover, Pipes operators can be then nested as needed. In the following example Tim Berners-Lee's DBLP record is fetched and transformed to use his real URI:


<pipe>
<
construct>
<source><fetch><location>http://dblp.l3s.de/d2r/resource/authors/Tim_Berners-Lee</source></fetch></location>
<query>
<![CDATA[
CONSTRUCT {<http://www.w3.org/People/Berners-Lee/card#i> ?p ?o.
?s2 ?p2 <http://www.w3.org/People/Berners-Lee/card#i>} where
{{<http://dblp.l3s.de/d2r/resource/authors/Tim_Berners-Lee> ?p ?o} UNION
{?s2 ?p2 <http://dblp.l3s.de/d2r/resource/authors/Tim_Berners-Lee>}}
]]>
</query>
</construct>
</pipe>

Pipes are easily defined using a graphical DERI Pipes Editor. In this environment pipes are created by drag-and-drop operators onto a panel then connecting operators to create pipes. The following figure is a screen shot of DERI Pipes Editor of the pipe mentioned above (click on the picture to see the screencast).

 

 

A list of pipes is available in the Pipe editor, each with its own stable invocation URL. To prevent unwanted deletion or editing, pipes can be password protected.

 

People:

  • Danh Le Phuoc (DERI Galway)
  • Christian Morbidoni (Semedia, Universita' Politecnica delle Marche, Ancona)
  • Axel Polleres (DERI Galway)
  • Matthias Samwald
  • Robert Fuller (DERI Galway)
  • Giovanni Tummarello (DERI Galway)