Semantic Web Pipes

SW Pipes Project

Home
Pipe Editor
Documentation
FAQs
Publications
Installation Instructions
Pipes Ontology

Pipe Modules

Development and Support

Mailing list
Source code
Issue tracker

Pipe Operator

Each Pipe operator has one or more inputs and only one output. The inputs can be various things, e.g. URIs, text or output from another operator. The output of an operator can be:

a single RDF graph
an RDF datasets (consisting of a default graph an zero or more named graphs)
an SPARQL result set.

General Pipe syntax

Syntax template:

<pipe>
<parameters>
    <parameter>
      <id>parameter's name (must be unique)</id>
      <label>description of the paramenter</label>
    </parameter>
....
</parameters>

Example (See also http://pipes.deri.org:8080/pipes/Pipes/?id=custompipe, http://pipes.deri.org:8080/pipes/Pipes/?id=mixderifoaf ? )

<pipe>

<parameters>
    <parameter>
      <id>id1</id>
      <label>First Deri team member</label>
    </parameter>
    <parameter>
      <id>id2</id>
      <label>Second Deri team member</label>
     </parameter>
</parameters>

<code>
    <simplemix>
       <source>
         <fetch>
           <location>http://www.deri.ie/fileadmin/scripts/foaf.php?id=$id1$</location>
         </fetch>
       </source>
       <source>
         <fetch>
           <location>http://www.deri.ie/fileadmin/scripts/foaf.php?id=$id2$</location>
         </fetch>
       </source>
    </simplemix>
</code>
</pipe>

FETCH

The Fetch operator is used to fetch data from an URI in RDF/XML or SPARQL-RESULT/XML . There is an optional attribute "accept" which determines the HTTP accept header for the request. Allowed value are :"rdfxml" and "sparqlxml".

Syntax template:

<fetch accept="rdfxml/sparqlxml">
   <location>
   URL OF THE WEB FILE TO FETCH
   </location>
</fetch>

<fetch accept="rdfxml/sparqlxml">
   <sparqlendpoint>
      <defaultgraph>URI of the default graph</defaultgraph>
      <endpoint>URL to the Sparql endpoint</endpoint>
      <query>the remote SPARQL query</query>
   </sparqlendpoint>
</fetch>

Example:

<fetch accept="rdfxml">
<location>http://www.w3.org/People/Berners-Lee/card?</location>
</fetch>

MIX

The MIX operator is used to merge triples from multiple RDF sources. Each RDF Source can be either a constant (directly input as RDF/XML) or another Pipe operator which can output RDF/XML data. There is an optional attribute "uri". If it is speficfied, then the sourcedata will be placed into an named graph with attribute's value as graph name.

Note: Constant RDF/XML text has to be wrapped into a CDATA section

Syntax template:

<simplemix>
<source>Enter one or more sources</source>
....
</simplemix>

Example (see http://pipes.deri.org:8080/pipes/Pipes/?id=simplemix )

   <simplemix>
     <source>
     <fetch>
     <location>http://www.w3.org/People/Berners-Lee/card?</location>
     </fetch>
     </source>
     <source>
     <![CDATA[
    <rdf:RDF
   xmlns:j.0="http://xmlns.com/foaf/0.1/">http://xmlns.com/foaf/0.1/"
    xmlns:wot="http://xmlns.com/wot/0.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<j.0:PersonalProfileDocument>
    <j.0:primaryTopic>
      <j.0:Person rdf:about="http://g1o.net/foaf.rdf#me"
         j.0:givenname="Giovanni"
         j.0:family_name="Tummarello"
         j.0:nick="Jccq"
         j.0:title="Ph.D"
         j.0:icqChatID="68832951">
        <j.0:knows>
         ...
</rdf:RDF>
     ]]>
     </source>
   </simplemix>

CONSTRUCT

The Construct query is used to create expected triples by using SPARQL-CONSTRUCT query on specified RDF sources. The output of this operator is the result of a SPARQL-CONSTRUCT query executed on the one or more sources. Each RDF Source can be either a constant (directly input as RDF/XML) or another Pipe operator which can output RDF/XML data. There is an optional attribute "uri". If it is speficfied, then the source data will be placed into an named graph with attribute's value as graph name.

Note: Constant RDF/XML text has to be wrapped into a CDATA section.

Syntax template:

<source uri=uri>Enter one source syntax here!</source>

...
<query>
Enter SPARQL construct query here!
</query>
</construct>

SELECT

Similar to CONSTRUCT operator, the SELECT operator is used performs a SELECT query and outputs the result in the SPARQL-Result XML format. This operator is only used as an input for FOR operator. Each RDF Source can be either a constant (directly input as RDF/XML) or another Pipe operator which can output RDF/XML data. There is an optional attribute "uri". If it is speficfied, then the sourcedata will be placed into an named graph with attribute's value as graph name.

Note: Constant RDF/XML text has to be wrapped into a CDATA section.

Syntax template:

<select>
<source uri="uri">Enter one source syntax here!</source>
<source >Enter one source syntax here!</source>
<query>
Enter SPARQL SELECT query here!
</query>
</select>

Example:

<select>
     <source><fetch><location>http://www.w3.org/People/Berners-Lee/card?</location></fetch></source>
     <source><fetch><location>http://g1o.net/foaf.rdf?</location></fetch></source>
<query>
       <![CDATA[
         select ?uri where {?s <http://xmlns.com/foaf/0.1/knows?> ?uri} ORDER by desc(?uri) LIMIT 2 offset 10
     ]]>
   </query>
</select>

RDFS

Similar to MIX operator,this RDFS operator will merge specified RDF sources and then infer implicit triples from the merged triples.The output of this operator is also RDFXML format. Each RDF Source can be either a constant (directly input as RDF/XML) or another Pipe operator which can output RDF/XML data. There is an optional attribute "uri". If it is speficfied, then the sourcedata will be placed into an named graph with attribute's value as graph name.

Note: Constant RDF/XML text has to be wrapped into a CDATA section.

Syntax template:

<rdfs>
<source>Enter one or more sources</source>

</rdfs>

<construct>
<source>
<rdfs>
     <source><fetch><location>http://www.w3.org/People/Berners-Lee/card?</location></fetch></source>
     <source><fetch><location>http://xmlns.com/foaf/spec/index.rdf?</location></fetch></source>
</rdfs>
</source>
<query>
<![CDATA[
PREFIX foaf:   <http://xmlns.com/foaf/0.1/?>
PREFIX rdf:   <http://www.w3.org/1999/02/22-rdf-syntax-ns#?>
CONSTRUCT

{ ?x foaf:name ?name }
WHERE

{ ?x foaf:name ?name . ?x rdf:type foaf:Agent}
]]>
</query>
</construct>

FOR

The FOR operator will invoke a parametrized pipe multiple times and merge the resulting outputs of each invocation. The <sourcelist> specifies an operator that ouputs a SPARQL result set. For each result in the set, the variable values will be subtituted into the parametrized pipe specified in <forloop> and the pipe will be invoked.

Syntax template:

<for>
<sourcelist>
requires SPARQL XML FORMAT
</sourcelist>
<forloop>
<simplemix><fetch><location>$uri$</location></fetch><simplemix>
</forloop>
</for>

Example (See also http://pipes.deri.org:8080/pipes/Pipes/?id=forloop ): This pipe fech from the URIs of all five people known by the author of a FOAF file.

<for>
   <sourcelist>
   <select>
     <source><fetch><location>http://www.w3.org/People/Berners-Lee/card?</location></fetch></source>
<query>
       <![CDATA[
         select ?uri where {?s <http://xmlns.com/foaf/0.1/knows?> ?uri} ORDER by desc(?uri) LIMIT 5
     ]]>
   </query>
</select>
   </sourcelist>
<forloop>
<simplemix><source><fetch><location>$uri$</location></fetch></source></simplemix>
</forloop>
</for>

Patching Operators

The purpose of these operators is to provide means for producing and automatically applying patches to RDF data sources.
Let us explain their usage with a use case (see also http://pipes.deri.org:8080/pipes/Pipes/?id=patch ?).

Suppose I want to build a pipe that outputs the merge of my own and Bob's FOAF files, but I also want to automatically remove from the output some statements that are included in Bob's file, which I think are wrong (e.g. Bob is declaring to be able to program in Java, but I'm sure he is cheating ). Of course Bob has the right to state such a fact in his own RDF file, and obviously I don't have the rights (and the technical means) to hack the file. So I need a mechanism to automatically exclude the statement in question from the "view" I want to produce and perhaps expose to the world, while being able to reflect the future updates Bob will apply to his FAOF description. Let's see how to do it with Semantic Web pipes.

I first have to publish on the web (say to http://mywebsite/patch.rdf ?) an RDF file including, in this case, a single triple:

ex:Bob ex:skill ex:JavaProgramming

Then I use the patch-generator block to produce the patch:

        <patch-generator>
           <source>
                <fetch><location>http://mywebsite/patch.rdf?</location></fetch>
          </source>
        </patch-generator>

The output of this simple operator will be an RDF which express the "negation" of all the triples in the source. In this case something that means: NOT(ex:Bob ex:skill ex:JavaProgramming).

The next step is to create a pipe that applies the patch to Bob's FOAF file. To do this I'll use the patch-executor operator, with the previous block nested:

    <patch-executor>
          <source>
                <fetch><location>http://bobswebsite.org/foaf.rdf?</location></fetch>
          </source>
        <source>
                <fetch><location>http://mywebsite.org/foaf.rdf?</location></fetch>
          </source>
          <source>
                <patch-generator>
                   <source>
                        <fetch><location>http://mywebsite/patch.rdf?</location></fetch>
                  </source>
                </patch-generator>
          </source>
    </patch-executor>

This composite pipe will take as input the Bob's FOAF, my FOAF file and my patch and will produce as output an RDF which contains all the statements form the two FOAF minus the one I distrusted.

SMOOSHER

This operator produces a merge of all the sources smooshing URIs based on the owl:sameAs statement included in the sources themselves.

For example (see also http://pipes.deri.org:8080/pipes/Pipes/?id=smoosher?), let us consider two RDF files as input. The first one (published at http://bobwebsite.org/foaf.rdf ?) says:

http://bobwebsite.org/Bob/ ? ex:skill ex:JavaProgramming

and the second (published at http://charleswebsite.org/info.rdf ?) says:

http://charleswebsite.org/Bob/? foaf:knows http://charleswebsite.org/Charles/?
http://bobwebsite.org/Bob/? owl:sameAs http://charleswebsite.org/Charles/ ?

By creating a pipe like the following:

    <smoosher>
          <source>
            <fetch><location>http://bobwebsite.org/foaf.rdf?</location></fetch>
          </source>
          <source>
            <fetch><location>http://charleswebsite.org/info.rdf?</location></fetch>
          </source>
    </smoosher>

I would obtain, as output, an RDF including the following triples:

http://bobwebsite.org/Bob/? ex:skill ex:JavaProgramming
http://bobwebsite.org/Bob/? foaf:knows http://charleswebsite.org/Charles/ ?

where only one URI is used to address Bob as an entity (the shortest one).

Dynamic Operators

This operator invokes stored pipes that have been defined the Pipe editor.

This will be transformed into the execution of a local pipe with the specified parameter =

http://pipes.deri.org:8008/pipes?pipename?parametername=value ?

<pipe>
<code>
<construct>
   <source>
   <mixderifoaf>
     <id1>12</id1>
     <id2>70</id2>
   </mixderifoaf>
   </source>
   <query>
       <![CDATA[
          CONSTRUCT {?book <http://purl.oclc.org/NET/nknouf/ns/bibtex#hasTitle?> ?title. ?book <http://purl.oclc.org/NET/nknouf/ns/bibtex#hasAuthor?> ?author} where {?book <http://purl.oclc.org/NET/nknouf/ns/bibtex#hasTitle?> ?title . ?book <http://purl.oclc.org/NET/nknouf/ns/bibtex#hasAuthor?> ?author}
     ]]>
   </query>
</construct>
</code>
</pipe>

DTD

<?xml version="1.0"?>
<!DOCTYPE pipe [
<!ELEMENT pipe (parameters,code)>
<!ELEMENT parameters(id,label)>
<!ELEMENT id (#PCDATA)>
<!ELEMENT label (#PCDATA)>
<!ELEMENT code (simplemix|rdfs|fetch|for|construct)>
<!ELEMENT simplemix (source+)>
<!ELEMENT rdfs (source+)>
<!ELEMENT fetch (location|sparqlendpoint)>
<!ELEMENT location (#PCDATA)>
<!ELEMENT sparqlendpoint (defaultgraph,endpoint,query)>
<!ELEMENT defaultgraph (#PCDATA)>
<!ELEMENT endpoint (#PCDATA)>
<!ATTLIST fetch accept (rdfxml|sparqlxml) "rdfxml">
<!ELEMENT construct (source+,query)>
<!ELEMENT query (#PCDATA)>
<!ELEMENT for (sourcelist,forloop)>
<!ELEMENT sourcelist (#PCDATA|fetch|select)>
<!ELEMENT select (source+,query)>
<!ELEMENT forloop (simplemix|rdfs|fetch|for|construct)>
<!ELEMENT source (#PCDATA|simplemix|rdfs|fetch|for|construct)>
<!ATTLIST source uri CDATA #IMPLIED>
]>

News

1 November 2007

SW Pipes first public release.

9 November 2007

Technical report published

Last modified 1 November 2007.

SW Pipes Project

Pipe Modules

Development and Support

Related Links

Pipe Operator

General Pipe syntax

FETCH

MIX

CONSTRUCT

SELECT

RDFS

FOR

Patching Operators

SMOOSHER

Dynamic Operators

DTD

News

1 November 2007

9 November 2007