A similar metaphor has been implemented in Yahoo Web Pipes3, which allows to implement customized services and information streams by processing and combiningWeb sources (usually RSS feeds) using a cascade of simple operators.

Since Web pipes are themselves HTTP retrievable data sources, they can be reused and combined to form other pipes. Also, Web pipes are “live’: they are computed on demand at each HTTP invocation, thus reflect the current status of the original data sources.

Unfortunately Yahoo Web Pipes are engineer to operate using fundamentally the RSS paradigm (item list) which does not map well at all with the graph based data model of RDF. For this purpose SemanticWeb Pipes have been written from the start to operate also on Semantic Web data offering specialized operators to perform the most important data aggregation and transformation tasks.

When a pipe is invoked, simply fetching the pipe URL, the external sources are fetched and transformed transparently and thus the SemanticWeb pipe will reflect the most up to date data available online.

Using Semantic Web Pipes you can fetch, mix and process RDF files published on the Web. As the output of a Pipe is an HTTP retrievable RDF model, simple pipes can also work as inputs to more complex Pipes.

Simple examples (see specification http://pipes.deri.org/documentation.html)

Pipes are written in a simple XML language. E.G. a mixing 2 sources is as simple as this:

   <mix>
     <source><fetch><location>http://www.w3.org/People/Berners-Lee/card</source></fetch></location>
     <source><fetch><location>http://g1o.net/foaf.rdf</source></fetch></location>
   </mix>

The output of this pipe is calculated live when you fetch http://pipes.deri.org:8080/pipes/Pipes/?id=simplemix.

The Construct block allows RDF Transformation by the means of SPARQL Construct queries. In the following example Tim Berners-Lee's DBLP record is fetched and transformed to use his real URI:

<construct>
  <source><fetch><location>http://dblp.l3s.de/d2r/resource/authors/Tim_Berners-Lee</source></fetch></location> 
  <query>
       <![CDATA[
          CONSTRUCT {<http://www.w3.org/People/Berners-Lee/card#i> ?p ?o. 
                     ?s2 ?p2 <http://www.w3.org/People/Berners-Lee/card#i>} where 
         {{<http://dblp.l3s.de/d2r/resource/authors/Tim_Berners-Lee> ?p ?o} UNION  
          {?s2 ?p2 <http://dblp.l3s.de/d2r/resource/authors/Tim_Berners-Lee>}}
	 ]]>
   </query>
</construct>

The output of this pipe is available at http://pipes.deri.org:8080/pipes/Pipes/?id=TBLsPublications.
Pipes operators can be then nested as needed.

As proper documentation is still to be written, the suggestion as the moment is to look at the example pipes in the Pipe Editor. Also, the AJAX enabled pipe editor will provide inline operator documentation when inserting a block.

A list of pipes is available in the Pipe editor, each with its own stable invocation URL. To prevent unwanted deletion or editing, pipes can be password protected.

Related Publications