Dimuthu’s Blog

Few weeks back I wrote a blog post about Writing RESTful Services in C which explain the use of Axis2/C REST API. Basically when you provide a HTTP Method (GET, POST, PUT or DELETE) and a HTTP URL, it is matched with a given HTTP method and a URL pattern in order to identify the operation and extract out the request parameters. For the example mentioned in the above blog, we can summarize the URL mapping like this.

You can watch an application with this URL mapping in live, written using WSF/PHP which in fact run Axis2/C algorithms underneath.

Last week I updated this REST mapping algorithm and started a discussion about the changes on Axis2/C Dev list. I thought it would be better explain the idea on by blog too.

What the early algorithm (before my changes) did was, it search each pattern in the order it was declared, and returns when a match is found. Sequential searching for a matching pattern can reduce the performance as the number of operations grows. So my solutions was to keep the url pattern in a multi level (recursive) structure and match the url from one level to another.

Here is the structure of the ‘c struct’. (defined in src/core/util/core_utils.c)

/* internal structure to keep the rest map in a multi level hash */
typedef struct {
    /* the structure will keep as many as following fields */

    /* if the mapped value is directly the operation */
    axis2_op_t *op_desc;

    /* if the mapped value is a constant, this keeps a hash map of
    possible constants => corrosponding map_internal structure */
    axutil_hash_t *consts_map;

    /* if the mapped value is a param, this keeps a hash map of
    possible param_values => corrosponding_map_internal structre */
    axutil_hash_t *params_map;

} axutil_core_utils_map_internal_t;

Here is how it will looks like when the above URL pattern set (shown in the above table) is kept inside this multi-level (recursive) structure.

svc->op_rest_map  (hash)
                |
            "GET:students" --------- axutil_core_utils_map_internal_t (instance)
                |                                            |
                |                                        op_desc (axis2_op_t* for "GET students" op)
                |                                            |
                |                                        consts_map (empty hash)
                |                                            |
                |                                        params_map (hash)
                |                                                         |
                |                                                      "{student_id}" ------------- axutil_core_utils_map_internal_t (instance)
                |                                                                                            |
                |                                                                                op_desc (axis2_op_t* for "GET students/{student_id}" op)
                |                                                                                            |
                |                                                                                parms_map (empty hash)
                |                                                                                            |
                |                                                                                 const_map (hash)
                |                                                                                            |
                |                                                                                        "marks" ------------------- axutil_core_utils_map_internal_t (instance)
                |                                                                                                                            |
                |                                                                                                                    op_desc (NULL)
                |                                                                                                                            |
                |                                                                                                                   consts_map (empty hash)
                |                                                                                                                            |
                |                                                                                                                   params_map (hash)
                |                                                                                                                            |
                |                                                                                                                      "{subject_id}" ----------- axutil_core_utils_map_internal_t (instance)
                |                                                                                                                                                                               |
                |                                                                                                                                       op_desc (axis2_op_t* for "GET students/{student_id}/marks/{subject_id}" op)
                |                                                                                                                                                                               |
                |                                                                                                                                                                 consts_map / params_map (Both NULL)
                |
            "GET:students" --------- axutil_core_utils_map_internal_t (instance)
                                                            |
                                                        op_desc (axis2_op_t* for "GET students" op)
                                                            |
                                                        consts_map (empty hash)
                                                            |
                                                        params_map (hash)
                                                            |
                                                      "{student_id}" ------------- axutil_core_utils_map_internal_t (instance)
                                                                                                          |
                                                                                  op_desc (axis2_op_t* for "GET students/{student_id}" op)
                                                                                                          |
                                                                                             consts_map / params_map (Both NULL)

This structure is build at the time the server initialize the services. (from the “axis2_svc_get_rest_op_list_with_method_and_location” function in src/core/description/svc.c)

As each request hit the service, the request HTTP method and the URL is matched (which we call ‘rest dispatching’) with the above structure using the following algorithm. (defined in the “axis2_rest_disp_find_op” function in src/core/engine/rest_disp.c). Note that here we are extracting out the user REST parameters as well, but it is not shown in here.

1. The request URL is spitted in to URL components from ‘/’ character. Retrive the instance of axutil_core_utils_map_internal_t  from the svc->rest_map to the varaible ‘mapping_struct’.
2. Check the existance of URL components, count(URL components) > 0.
3. If it doesn’t exist any URL components, get the value of mapping_struct->op_desc where the mapping_struct is the current mapping instance of axutil_core_utils_map_internal_t. if the mapping_struct->op_desc is not NULL, we found the operation. If it is NULL just exit returning NULL.
4. Else If some URL component(s) exist, check the most former URL component in the mapping_struct->const_map hash. If mapping_struct->const_map['former_url_component'] is not NULL, assign the mapping struct->const_map['former_url_component'] value to mapping_struct and follow the step 2 with the remaining URL components. (note that here hash['key'] syntax is used to take the value for the key from the hash ‘hash’. If that returns TRUE, we found the opeartion, if not countine to step5.
5. if mapping_struct->const_map['former_url_component'] is NULL, match the former url component with each key (which is a URL component pattern) in mapping_struct->param_map hash. (We use the function  “axis2_core_utils_match_url_component_with_pattern” in src/core/util/core_utils.c to map URL component with the URL component pattern). If matching pattern found assign the mapping_struct->param_map['key'] to mapping struct and follow the step 2 with the remaining URL components. If that returns TRUE for some key it will be the matching operation.

Where as the earlier algorithm can be simplified to,

1. Match the request URL with URL patterns of each operation. This will be like calling the function “axis2_core_utils_match_url_component_with_pattern” (mentioned in step5 of the above algorithm) for the complete URL rather than for a URL component
   
2. If the pattern is matched, matching operation is the selected operation for the request.

I approximately calculated the time complexity of both of these algorithm.

Here is the time complexity of the later described algorithm.

Time complexity of the formerly described algorithm. (which is currently in the SVN).

Considering the facts, O(logn) < O(n),d < p and k < p/d we can safely conclude

O(long(n)*d^2*k) < O(n*p^2)  => The newer algorithm has better (low) time complexity.

However the time complexity is valid only it takes high values for the parameters. For low values  the actual time taken by the newer algorithm can have high values, considering the constant overhead of the recursions and the hash search. So in order to judge the performance of the algorithm, we have to run some test cases and measure the actual times. Possibly a task for the weekend

Here is a simple categorization of the WSF/PHP samples. You can access all the wsf/php samples from http://labs.wso2.org/wsf/php/solutions/samples/index.html.

If you have downloaded the WSF/PHP binaries or souce code package you can find all these samples, inside the ’samples’ directory

You can write REST as well as SOAP web services using Apache Axis2/C web services framework. There you can make existing Axis2/C web services RESTful just by providing the URL patterns and the HTTP methods to each operation in  the services.xml which act as a simple descriptor for an Axis2/C service.

So if we rewrite the RESTful Demo (Written in PHP) using Axis2/C, the services.xml would be something like following.

<service name="RESTfulSchool">
    <!-- mentioning the service library-->
    <parameter name="ServiceClass" locked="xsd:false">RESTfulSchool</parameter>

    <!-- some description -->
    <description>
        The RESTful School demo
    </description>

    <!-- list of operations -->
    <operation name="getSubjects">
            <parameter name="RESTMethod">GET</parameter>
            <parameter name="RESTLocation">subjects</parameter>
    </operation>
    <operation name="getSubjectInfoPerName">
            <parameter name="RESTMethod">GET</parameter>
            <parameter name="RESTLocation">subjects/{name}</parameter>
    </operation>
    <operation name="getStudents">
            <parameter name="RESTMethod">GET</parameter>
            <parameter name="RESTLocation">students</parameter>
    </operation>
    <operation name="getStudentInfoPerName">
            <parameter name="RESTMethod">GET</parameter>
            <parameter name="RESTLocation">students/{name}</parameter>
    </operation>
    <operation name="getMarksPerSubjectPerStudent">
            <parameter name="RESTMethod">GET</parameter>
            <parameter name="RESTLocation">students/{student}/marks/{subject}</parameter>
    </operation>
</service>

We will check how to write the service logic for a operation like “getMarksPerSubjectPerStudent”.

axiom_node_t *
RESTfulSchool_getMarksPerSubjectPerStudent(
    const axutil_env_t * env,
    axiom_node_t * request_payload)
{
    axiom_node_t *student_node = NULL;
    axiom_node_t *subject_node = NULL;

    /* Extracting out the child nodes from the request */
    student_node = axiom_node_get_first_child(request_payload, env);
    subject_node = axiom_node_get_next_sibling(student_node, env);

    /* now we can write the logic to retrieve the marks
       for the given student and subject and build and
       return the response payload */

    return response_payload;
}

As you can see the variables {student} and {subject} given in the services.xml can be easily accessed from your business logic, so we can build the response accordingly.

This way you can build a RESTful web services easily using C language.

Last week I wrote a how to on Writing a SOAP and REST Service with PHP. It shows how to write a single script that enables the SOAP and REST web services interfaces using WSF/PHP. Today I’m going to show you how you can use WSF/PHP to write clients in both SOAP and REST form.

For that I use the Amazon Client Demo hosted in the PHP Web Services Demo Site. With this demo you can do shopping with Amazon using their web services API. It uses a php library (AmazonClient.php) that allows us to connect Amazon using SOAP or REST libraries.

You can see in the library code It is using different values for “useSOAP” argument depending on what form of messaging we like to use. Amazon SOAP service uses the SOAP 1.1 and the REST service uses the “GET” HTTP method for messaging. We can configure these details as mentioned in the following table.

array(
      "to"=>self::AMAZON_REST_ENDPOINT,
      "HTTPMethod"=>"GET",
      "useSOAP" => FALSE)

array(
      "to"=>self::AMAZON_SOAP_ENDPOINT,
      "useSOAP" => "1.1",
      "action" => "http://soap.amazon.com")

Since Amazon requires different request messages for SOAP and REST services, we have to create them separately depending on the request type. But the response returns by the service is same in both cases so we can handle it using the same code.

Here is how ItemLookup operation of the Amazon service is written within the above mentioned constrains.

    /**
     * ItemLookup
     * @param $ASIN Amaxon Item Id
     * @return associate array consist of the response parameters
     */
    public function ItemLookup($ASIN)
    {

        if($this->is_soap)
        {
             $req_payload = <<<XML
                <ItemLookup xmlns="http://webservices.amazon.com/AWSECommerceService/2007-10-29">
                    <SubscriptionId>{$this->amazon_key}</SubscriptionId>
                    <ResponseGroup>Medium</ResponseGroup>
                    <Request>
                        <ItemId>{$ASIN}</ItemId>
                        <ReviewPage>1</ReviewPage>
                    </Request>
                </ItemLookup>
XML;
        }
        else
        {
             $req_payload = <<<XML
                <ItemLookup xmlns="http://webservices.amazon.com/AWSECommerceService/2007-10-29">
                    <SubscriptionId>{$this->amazon_key}</SubscriptionId>
                    <Service>AWSECommerceService</Service>
                    <ResponseGroup>Large</ResponseGroup>
                    <Operation>ItemLookup</Operation>
                    <ItemId>{$ASIN}</ItemId>
                    <ReviewPage>1</ReviewPage>
                </ItemLookup>
XML;
        }

        $ret_message = $this->request($req_payload);

        $simplexml = new SimpleXMLElement($ret_message->str);

        $res = $simplexml;

        return $res;
    }

WSF/PHP enables you to write both REST and SOAP services in PHP from a single script. I have written about how you can expose your Database as a REST and SOAP services in few of my previous posts using the Data Service capability of WSF/PHP. But there can be situations where your service is not based on a Database. For an example it can use results of some calculations, or a mashup calling other services. In that case you will prefer to write the service logic yourself. Here is how you can do it.

Lets think we have weather forecast data (may be from another service) and I want to make a web service using it and make it accessible via both REST and SOAP protocols.

In our demo service we give forecasts of temperature, humidity and some other parameters for a given date. So I expect

SOAP request payload as following.

<weatherReport>
  <date>{date}</date>
  <parameter>{parameter}</parameter>
</weatherReport>

And REST Request will be like

weatherReport/{date}/forecast/{parameter}

Note that here parameter can hold values like temperature, humidity or sunset-time.

First we declare our operation and the REST Request Mapping like this,

$operations = array("weatherReport" => "weather_report");
$restmap = array("weatherReport" =>
				array("HTTPMethod" =>"GET",
				      "RESTLocation" => "weatherReport/{date}/forecast/{parameter}"));

When you declare your rest mapping like above , in the service operation you will have the same request XML for both SOAP and REST form like this,

<weatherReport>
  <date>{date}</date>
  <parameter>{parameter}</parameter>
</weatherReport>

So in your service logic you just handling the request in only above format. You can easily extract out the request parameters using SimpleXML functions and return the corresponding result. So you service operation would be something like this,

function weather_report($in_message) {

	// create the simple xml element for the request xml
	$request_xml = new SimpleXMLElement($in_message->str);

	// extract out the parameter and the date
	$date = $request_xml->date;
	$parameter = $request_xml->parameter;

	// It is up to you to retrun the weather data ($result) for the requested date and parameter

	return "<response>$result</response>";
}

Finally you create the WSService object with the “operations” and “RESTMapping” and call its reply method which actually response to the requests.

$service = new WSService(array("operations" => $operations, "RESTMapping" => $restmap));
$service->reply();

You just created a web service which will handle both SOAP and REST requests.

The REST API for Twitter is very simple to learn and implement. And it has a comprehensive documentation.

Here is some selected operations to just to show its design. Note that here userid should be replaced with a valid twitter user id or user name and the format should be changed to the required output format (.xml, json, rss, atom are possible output formats)

After having look at this API, the first question I had was whether this API is actually RESTful. In RESTful design we expect to map a resource to a URL and do CRUD (Create, Read, Update and Delete) operations using request with different Http Verbs (POST, GET, PUT, DELETE) with that same URL. Look at my blog on RESTful CRUD Data Services Demo for more clarification.

So if ever the API is designed following the above theory it would have been like this.

So I think although Twitter API is really nice and easy, it is not really a RESTful API. If it was really RESTful, URLs might have been more organized so more easier to remember or predict. But still this API allows thousands of third party application to talk to the twitter, demonstrating the value of  providing web services over just providing some web pages in a website.

When you are developing Web Service for CRUD (Create, Read, Update, Delete) operations you may find it is easy to implement it as RESTful service. In this Demo on RESTful CRUD Service You can have an idea how you develop such a service with WSO2 WSF/PHP.

Here we take a scenario of submitting applications (say for a school).

In RESTful world we map a resource to a unique URL. In this demo, application is a resource. We use the URL “application/{id}” to represent a particular application with the id {id}.

You can  use  HTTP verb + Resource URL touples to manipulate the resource with CRUD operations.  Here is how it is done in this particular demonstration.

Go for the wsf/php demo sitefor the live demo of this service. Visit the demo service source code to see how easy to implement it with WSF/PHP Data Services library.

I wrote a similar blog on Data Services last week to demonstrate how you design the mapping of url to different resources in a RESTful Service.

In RESTful paradigm we give a piece of data ( or in other word ‘Resource’) a unique URL. And in order to manipulate data we use HTTP verbs POST/PUT (create, update), GET (read), DELETE (delete). For an example
take the scenario of manipulating Students data in a high school. Here is how each operation is mapped to a http request (URL + HTTP verb)

With the addition of all these HTTP verbs WSO2 WSF/PHP 2.0.0 become a great tool for RESTful developers. Specially with the introducing Data Services library it was so easy to make your database a REST service. I m thinking of preparing a series of application to demonstrate the power of WSF/PHP with all these new features.

This demo -RESTful School- shows how you map a URL to a peice of data. Here we use only the http “GET” method (which is the most to used in practicle data service).

Here is some description of the operations you find in there. Just check the source code for RESTful School demo to see how this is done in code level.

subjects

SELECT subjectName, subjectTeacher FROM Subjects

subjects/{name}

SELECT subjectName, subjectTeacher FROM Subjects where SubjectName = ?

students

SELECT * FROM Students

students/{name}

SELECT subjectName, marks FROM Marks m, Subjects s ".
        " where m.studentId = ? and m.subjectID = s.subjectId

SELECT * FROM Students where StudentName = ?

students/{student}/marks/{subject}

SELECT marks FROM Marks, Subjects, Students where StudentName = ?".
        " and SubjectName = ? and Marks.subjectId = Subjects.subjectId".
        " and Marks.studentID = Students.StudentId;