Vision

REST: The SharePoint RESTful interface

Written by: Margriet Bruggeman, Nikander Bruggeman.

April 30, 2010

REST: The SharePoint RESTful interface

SOAP may be the de facto standard when it comes to solving problems surrounding remote communication and interoperability, but it's also true that SOAP can be quite complex and for many types of applications complete overkill. REST on the other hand, which stands for Representational State Transfer, is rapidly gaining ground as a lightweight Web service protocol that allows you to remotely manipulate resources over the Internet. Although not strictly necessary, REST is usually used in conjunction with the HTTP protocol. The exciting news is that SharePoint 2010 supports a RESTful interface, and it does this by leveraging ADO.NET Data Services.

Note. By the time you are reading this blog post, this note has hopefully become superfluous. The SharePoint 2010 product team promises that REST is enabled by default in the RTM version of SharePoint 2010. At the time of writing however, you still need to enable it by installing ADO.NET Data Services v1.5 ctp and resetting IIS. There's one strange quirk though, if you install the full version (called ADONETDataServices_v15_CTP2.exe) it always result in an access denied error and a failed installation. Instead, you need to install the run time only version called ADONETDataServices_v15_CTP2_RuntimeOnly.exe.

All REST calls to SharePoint are nothing more than HTTP methods. A typical REST interface supports the following HTTP methods:

  • OPTIONS, this method returns a list of all HTTP methods that are supported. The SharePoint RESTful interface doesn't support this method.
  • GET, this method retrieves a resource, but doesn't alter it in any way. Or to define it real precisely: the method returns a representation of the server resource and doesn't alter the server resource in any way.
  • HEAD, this method returns only the HTTP headers (the metadata), but not the actual resource data. The SharePoint RESTful interface doesn't support this method.
  • POST, this method creates a new resource on the server. You can POST to anything that represents a set. At a later time, we will create a custom list called Products. A Products list is a collection, so you would be able to POST to that.
  • PUT, this method creates or replaces a resource on the server.

The SharePoint RESTful interface supports another, custom, HTTP method called MERGE. You can use it if you want to avoid replacing an entire resource using PUT. For example, you may want to do this if all you want to do is replace a couple of properties of a resource. The MERGE method ensures your changes will be merged with the resource properties found on the server. The MERGE method is a custom construct by Microsoft, and it's identical to the PATCH method that hasn't been standardized yet. Once the PATCH method is standardized, Microsoft will support that too.

The examples in this section use the HTTP GET method exclusively. If you want to experiment with other HTTP methods as well, you may consider downloading the HTTP client tool cURL, which makes this easy to do. On a side note, although using cURL is very easy, installing it is not. There may be easier alternatives out there.

REST Responses can be represented via JSON or Atom (more on these response types later). The HTTP Accept header determines which content type is returned. The Accept header value for Atom is application/atom+xml, the Accept header value for JSON is application/json.

Note. In some ways, REST reminds us of WebDAV, only it's much more simple to use. This is just an observation that attempts of manipulating resources have been around for quite some time now. REST itself saw the first light of day in 2000.

Atom

Atom applies to two different standards, the Atom Syndication Format and the Atom Publishing Protocol (AtomPub). The Atom Syndication Format is an XML language used for web feeds, but this is not the standard we're interested in as it's not supported by SharePoint 2010. SharePoint 2010 does offer full AtomPub support, which is a simple HTTP-based protocol for creating and updating web resources. Sounds like an excellent match with REST, doesn't it? If you want to see examples of what Atom looks like, you will just need to follow the examples later on and you will see plenty of it.

Compared to JSON, most data services client libraries use Atom because it has richer type expression possibilities, although JSON is more compact and therefore more suitable for AJAX applications.

JSON

JSON (pronounced Jason) is a serialization format created in 2002 as a cleaner and lighter alternative to XML. It is possible to express simple as well as complex types in JSON. JSON is based on a subset of the JavaScript programming language and is very easy to parse and generate in a JavaScript programming environment. This makes JSON a well-suited, lightweight data-interchange format for browser-server communication. If you want to learn more about JSON, visit http://www.json.org/.

A JSON representation of a string is a collection of zero or more characters wrapped in double quotes. This means the JSON representation of the name William Bender looks like this: "William Bender". A JSON representation can be converted to a JavaScript (simple or complex) type using the JavaScript eval() function. You could also dynamically generate a new function that handles the conversion using the Function() JavaScript function. Both approaches are shown here:

var strJSON = "\"William Bender\"";
var strResult = eval(strJSON);
strResult = new Function("return " + strJSON);

The JSON representation of a number is identical to the number itself. For example, the JSON representation of the number 7 is 7:

var strJSON = "7";
var intResult = eval(strJSON);

The JSON representation of a Boolean is either true or false. Here’s an example:

var strJSON = "true";
var blnResult = eval(strJSON);
if ( blnResult )
{
alert(blnResult);
}

An array is an ordered collection of values. The JSON representation of an array begins with a left bracket ([) and ends with a right bracket (]). Values are separated by a comma (,). The JSON representation of a number array looks like this: [1, 2, 3]. Here’s an example:

var strJSON = "[1, 2, 3]";
var arrResult = eval(strJSON);
alert(arrResult[0]);

The JSON representation of a string array looks like this: ["Bob", "Sue"]. Here's an example:

var strJSON = "[\"Bob\", \"Sue\"]";
var arrResult = eval(strJSON);
alert(arrResult[1]);

Objects are unordered sets of name-value pairs. JSON representations of objects are enclosed by curly braces ({}). Each name is followed by a colon (:). Name-value pairs are separated by commas (,). Look at the following definition of a JavaScript object called Person containing Name and Age properties:

function Person(strName, strAge)
{
this.Name = strName;
this.Age = strAge;
}

The JSON representation of the Person object looks like this: {"Name":"Jack","Age":47}. Although you are able to use the eval() function to convert this JSON string representation to a JavaScript object containing Name and Age properties, this is more complex than converting JSON representations to simple types. Instead, we recommend you use a JSON parser to do the conversion for you. Nowadays, you can download several JavaScript frameworks that contain JSON parsers (among other functionalities). A good example of a very popular JavaScript framework that contains a JSON parser is jQuery. We definitely recommend using it.

Another argument for using a JSON parser is that the eval() function compiles and executes any JavaScript code. This makes you vulnerable ftor script injection attacks. A good JSON parser will offer safeguards against this sort of attack and thus will be safer.

Note It is possible to create complex object hierarchies via JSON because nested types are allowed.

Compared to Atom, JSON is more lightweight and very suitable for AJAX applications, but due it's lack of expressiveness it's type fidelity is less.

Working with REST

If you start working with REST in a SharePoint 2010 environment, you will directly find out that the ListData WCF services is the pinnacle of SharePoint's RESTful interface. The ListData service is virtualized, which means you can access it within every SharePoint context. To make this possible, the ListData service is stored in a virtual folder called _vti_bin. The _vti_bin folder is a virtual directory in IIS that is mapped to the [drive letter]:\Program Files\Common Files\Microsoft Shared\web server extensions\14\ISAPI folder. The name of the _vti_bin folder originates from a company called Vermeer Technologies Incorporated, which was acquired by Microsoft in the beginning of 1996. Vermeer Technologies was responsible for creating the original version of FrontPage. The _vti_bin folder is a special Common Gateway Interface (CGI )–like folder that contains server-side functionality that needs to be available within the entire virtual server. On a SharePoint server, the _vti_bin folder is the place where the default SharePoint web service files are stored.

To demonstrate the use of the ListData service, we have created a separate site collection called REST which can be reached at the following URL: http://neptune/sites/REST. If you call the ListData service within the context of the site collection, you will get a list of all lists within the root web site. The call to the ListData service looks like this:

http://neptune/sites/REST/_vti_bin/listdata.svc

The result is an XML response in Atom format that looks like this:

Now, if you would have been interested in a list of all lists in a subsite called TestSite, all you have to do is pass that context to the ListData service, like so:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc

Suppose we have created a custom list in TestSite called Products that contains the columns ID, Title, Description, and Available. We have included two test entries in the Products list called My Product and Another Product. If we want to learn more about the Products list via REST, we need to pass the list context to the ListData service, like so:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Products

Sometimes MSIE is too smart for it's own good. You already know that the previous call leads to an Atom response. We're using a default installation of MSIE 8 and this is what we get:

When working with REST, it makes perfect sense to turn of the Feed Reader view so you can inspect the actual Atom response. The next procedure explains how to do this for MSIE 8.

  1. Open MSIE 8.
  2. Click Tools > Internet Options. This opens the Internet Options dialog window.
  3. Click the Content tab.
  4. In the Feeds and Web Slices section, click Settings.
  5. Uncheck the Turn on feed reading view checkbox.
  6. Click OK twice.

Now, if you refresh the page, the full Atom response will be displayed. In addition, we also like to use a tree web debugging tool called Fiddler (http://www.fiddler2.com). With it, you can easily collect and inspect the HTTP traffic generated by your browser, as shown in the next Figure. This comes in handy if you temporarily want to keep track of the REST calls you're making.

Note. Another interesting feature of Fiddler is that is allows you to modify and re-issue HTTP requests again. If you replace the HEADER "Accept: */*" with "Accept application/json" and execute it, you will see a REST response in JSON format.

Making a ListData call to the Products list results in an Atom response containing basic information about the Products list and the items within this list. To get an idea what a response looks like, we have included the XML bit describing the list item entry for My Product:

One of the critical pieces of information about a list entry is the element that contains a URL that you can use to further inspect this particular list entry. You can do this by issuing the following command:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Products(1)

Note. Please note that the number listed in the previous command is the SharePoint 2010 list item id.

If you want to learn more about a specific column, you can add it's name to the URL:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Products(1)/Title

If you want to loose the XML that surrounds the response, you can also access the text value of the property itself, like so:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Products(1)/Title/$value

REST supports multiple options for presenting data. The first one that we're showing is the possibility to sort the result set based on a column name.

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Products?$orderby=Title

Note. REST options are prefixed by a dollar sign ("$"). This makes it easy to distinguish them from normal querystring parameters.

The previous command also could have been rewritten as:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Products?$orderby=Title asc

As you would expect, the command can be reversed like this:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Products?$orderby=Title desc

You can also specify orderings for multiple columns:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Products?$orderby=Title desc, ID asc

The $skip and $top parameters are ideal for implementing a paging mechanism. The $skip=[n] parameter allows you to skip the first n options, the $top=[n] parameter allows you to return the next n items. Typically, you will use this in conjunction with the $orderby parameter (appended using the & (ampersand) symbol). The following command skips the first 5 product list items (ordered by title) and displays the next one:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Products?$skip=5&$top=1&$orderby=Title

Suppose we're adding a new list called Purchases that has a lookup column called PurchasedProduct that refers to the Title column of the Products list. Using REST, you can do a lookup traversal too:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Purchases(1)/PurchasedProduct/Description

If you want to learn more about other properties of a related Product item, you can do that by appending the name of the Product list Column you are interested in:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Purchases(1)/PurchasedProduct/Description

When doing remote calls, it is usually a good idea too get big chunks of data in one go instead of using a "chatty" interface consisting of multiple calls. The $expand parameter can assist here because it allows you to embed one or more sets of related entities in the overall result set (much similar to a SQL JOIN). The next example shows how to use it:

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Purchases?$expand=Products

You can use the $metadata parameter if you want to learn what properties are available for a certain resource. If it makes it easier to understand, you can also look ath the $metadata parameter as providing WSDL for REST.

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Purchases?$metadata

The final parameter is the $filter parameter. It allows you to define filters that decrease the size of your result set. You can do all sorts of handy stuff such as comparisons (eq, ne, gt, lt, and, or, not), simple arithmetic operations (such as add, sub, mod), more advanced string operations (such as substring, startswith, trim, toupper), and data functions (such as day, month, and year). For a full list and description of all REST operations, please refer to the MSDN article "Using Microsoft ADO.NET Data Services". The next example shows how to filter the Title column, where the column value needs to be equal to the value "Product 1".

http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Purchases?$filter=Title eq 'Product 1'

Issuing HTTP GET requests using C# isn't hard to do. The next piece of code shows how to do it:

string url = " http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc/Purchases(1)";
WebRequest request = WebRequest.Create(url);
request.Credentials = CredentialCache.DefaultCredentials;

WebResponse response = request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string responseContent = reader.ReadToEnd().Trim();

As the default returns XML (Atom), the string variable responseContent contains it's response in XML. You can take it, and further process it from there. Typically, you will also use the XMLHttpRequest object, jQuery framework, ASP.NET AJAX, Silverlight etc. to communicate with the SharePoint RESTful interface. Unfortunately, a discussion of these techniques fall outside the scope of this section. However, there is still one way left to create REST clients that we need to discuss…

Building a REST client using Visual Studio.NET 2010

Creating a REST client is made really easy in Visual Studio.NET 2010. Using Visual Studio.NET 2010, you can add a service reference to a RESTful end point. In the next procedure, you will learn how to iterate through all purchases on our test site.

  1. Create a new C# console application via Visual Studio.NET 2010.
  2. In the Solution Explorer window, right-click the Service References node and select Add Service Reference. This opens the Add Service Reference wizard.
  3. Open the REST List Service in the context of your test web site. Our URL looks like this: http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc.
  4. In the Namespace textbox, type: TestSvc.
  5. Click OK. This adds a new service reference.

Finally, you need to add the following code to the Main() method of your console application:

string url = "http://neptune/sites/REST/TestSite/_vti_bin/listdata.svc";
TestSvc.TestSiteDataContext context = new TestSvc.TestSiteDataContext(new Uri(url));
context.Credentials = CredentialCache.DefaultCredentials;
foreach (TestSvc.PurchasesItem in context.Purchases)
{
Console.WriteLine(item.Title);
}

Alternatively, you could have opted to use Linq to interact with the Purchases result set, like so:

var purchases = from pur in context.Purchases select pur;

As you can see, interacting with a RESTful service is a breeze in Visual Studio.NET 2010.

« back to overview page