WSO2 -An open source middleware company- announced the release of bunch of their SOA enabling products along with Carbon 2.0.0 which is an OSGI based unified platform that all the WSO2 java products are built on.

  • WSAS (Web Service Application Server) – Provide and consume web services, data services with full WS-*, REST support, GUI tools, Integration with Eclipse, etc..
  • ESB (Enterprise Service Bus) – Message routing, mediation, transformation, logging, task scheduling, load balancing, failover routing, event brokering, etc. with number of transports support
  • Governance Registry – Govern you SOA platform, introducing number of new features including Dashboard with Gadgets, lifecycle management with checklists, eventing and notifications, remote/symbolic links, checkin/checkout functionality and new meta data management UI.

You will be able to get the advantage of the carbon platform by extending the functionality of any of these products, just by mixing components from other products (Just like running an ESB within WSAS).

Download, Play around with them and Enjoy:)!!!

The release of WSO2 SOA platform – Carbon has unified the process of development to deployment of SOA in several aspects. Here is a list of 5 aspects unified across all the components of the SOA platform namely WSAS (The App Server), ESB (Enterprise Service Bus) and BPS (Business Process Server)

1. Unified QoS configurations – Allows you to add/drop modules for security, reliability, etc of your services + edit their policies in a unified view.
2. Unified Registry Storage – Provides a unified view over the governance of the SOA platform.
3. Unified Trackers – Availability of logs, statistics, graphs and  message tracers making it easy to debug and test your system.
4. Unified User Experience – Well designed unified User interfaces allowing admins/users to get familiar with each of the components very quickly and easily.
5. Unified Extensibility – The underline OSGI environment and the design of the carbon platform itself make it possible to add new OSGI bundles as extensions to fit equally across all the products.

WSO2 has announced the release of WSO2 Mashup Server 1.5.2. This release introduced the support for Data Services (Exposing Database, Excel sheet, CSV as a service), Open ID Logins and personalized dashboards to manage mashups and some security improvements. Just have a look at the release note for all the details about the new features, fixes of the release.

WSO2 Mashup Server enables you to develop your enterprise mashups very easily. You will write your code for the mashups in javascript with the full help of integrated javascript objects to consume Web services or Atom feeds, manage sessions, scrape web pages and manipulate storage. Once your put your .js file in to the mashup server repository, your services automatically will have the WSDLs, ‘Try It’ functionality and SOAP, REST interfaces.

If you are new to Mashup Server or to develop Mashups, just have a look at the series of screenshots released by WSO2 sometime ago. That will no doubt take you from zero knowledge to a comfortable state to write full-fledged mashups.

February 10th, 2009WSO2 Carbon Products Released

WSO2 announced the release of the revolutionary series of products introducing a componentized design to implement SOA in cost effective and simple manner.

  1. WSO2 Web Services Application Server (WSAS) – WSO2 WSAS is an enterprise ready web services engine based on Apache Axis2. It is incorporated with many features including,
    • Data services support – Expose you enterprise data as a services in a jiffy
    • WSAS IDE – Eclipse IDE integration
    • Clustering support for High Availability & High Scalability
    • Full support for WS-Security, WS-Trust, WS-Policy and WS-Secure Conversation and XKMS
    • EJB service provider support – Expose your EJBs as services
    • Axis1 backward compatibility – Deploy Axis1 services on WSAS & Engage advanced WS-* protocols in front of legacy services
    • JMX & Web interface based monitoring and management
    • WS-* & REST support
    • GUI, command line & IDE based tools for Web service development
  2. WSO2 Enterprise Service Bus (ESB)- WSO2 ESB is a lightweight and easy-to-use Open Source Enterprise Service Bus based on Apache Synapse. Here is the listing of some of key features,
    • Proxy services – facilitating transport, interface (WSDL/Schema/Policy), message format (SOAP 1.1/1.2, POX/REST, Text, Binary), QoS (WS-Addressing/WS-Security/WS-RM) and optimization switching (MTOM/SwA).
    • Non-blocking HTTP/S transports based on Apache HttpCore for ultrafast execution and support for thousands of connections at high concurreny with constant memory usage.
    • Built in Registry/Repository, facilitating dynamic updating and reloading of the configuration and associated resources (e.g. XSLTs, XSD, JS, ..)
    • Easily extended via custom Java class (mediator and command)/Spring mediators, or BSF Scripting languages (Javascript, Ruby, Groovy, etc.)
    • Built in support for scheduling tasks using the Quartz scheduler.
    • Load-balancing (with or without sticky sessions) /Fail-over, and clustered Throttling and Caching support
    • WS-Security, WS-Reliable Messaging, Caching and Throttling configurable via (message/operation/service level) WS-Policies
    • Lightweight, XML and Web services centric messaging model
    • Support for industrial standards (Hessian binary web service protocol/Financial information exchange protocol)
    • Enhanced support for the VFS/JMS/Mail transports
    • Support for message splitting and aggregation using the EIP
    • Database lookup and store support with DBMediators with reusable database connection pools
    • JMX monitoring support
  3. WSO2 Registry – WSO2 Registry is a user-friendly, but comprehensive enterprise resource / metadata management solution. It is capable of
    • Storing and managing arbitrary resources and collections
    • Tagging, commenting and rating
    • Managing users and roles
    • Authentication and authorization on all resources and actions
    • Resource / collection versioning and rollback
    • Advanced search capabilities – tags, users, etc.
    • Built in media type support for common types (WSDL, XSD)
    • Dependency management – maintain relationships between dependent resources
    • Pluggable media type handlers for handling custom media types
    • Support for processing custom URL patterns via pluggable URL handlers
    • Support for custom query languages via pluggable query processors
    • Activity log and filtering support for the activity logs
    • Atom Publishing Protocol (APP) support for reading/writing the data store remotely
    • Subscribe to directories, comments, tags, etc. with any standard feed reader (Bloglines, Google Reader, etc)
    • Java client for remote access via APP
    • Embedded and WAR deployments
    • Web based user interface with Web 2.0 look and feel
  4. WSO2 Business Process Server (BPS) – WSO2 Business Process Server (BPS) is an easy-to-use open source business process server that executes business processes written using the WS-BPEL standard and it’s powered by Apache ODE (Orchestration Director Engine). It provides you the following features,
    • Deploying Business Processes written in compliance with WS-BPEL 2.0 Standard and BPEL4WS 1.1 standard.
    • Managing BPEL packages, processes and process instances.
    • Data Sources support.
    • External Database support for BPEL engine.
    • WS-Security support for business processes.
    • WS-RM support for business processes.
    • Caching support for business processes.
    • Throttling support for business processes.
    • Transport management.
    • Internationalized web based management console.
    • System monitoring.
    • Try-it for business processes.
    • SOAP Message Tracing.
    • Web Services tooling support such as WSDL2Java, Java2WSDL and WSDL Converter.
    • Customizable server – You can customize the BPS to fit into your exact requirements, by removing certain features or by adding new optional features.

All these products are pre-bundled components built on top of WSO2 Carbon framework which is based on the OSGI technology. In fact you can bundle the components to suite your enterprise architecture as all the major features have been developed as pluggable Carbon components.

January 31st, 2009Making Good SOA Great

WSO2 is preparing for the first major release of their enterprise java product series after adapting the OSGI technology. You can already try out the betas from the wso2.org site.

  1. WSO2 Web Services Application Server (WSAS)
  2. WSO2 Enterprise Service Bus (ESB)
  3. WSO2 Registry
  4. WSO2 Business Process Server (BPS)

With the power of OSGI you will be able to customize these products for your need just by mixing and matching the components within these products. If you like to learn more about this, just have a loot at the following ebook released by WSO2.

Making Good SOA Great

Making Good SOA Great

Now you can view the article I wrote titling “Introduction to PHP Data Services“. There I explain how you can design and implement Data Services in PHP using WSF/PHP Data Services Library.

This article covers,

  1. Designing your Data Service API.
  2. Writing the Data Service.
  3. Deploying and Testing Data Service.
  4. Make the Data Service available in both SOAP and RESTful form.
  5. Use of WS-* features in your Data Service.

If you are thinking of adapting SOA in to your database backed PHP applications, this article will be a good starting point.

“WSO2 carbon is a componentized, customizable SOA Platform, You can adapt the middleware to your enterprise architecture, rather than adapt your architecture to the middleware”.

December 18th, 2008Data Services Best Practices

In simple term, data services are exposing data as web services. Anyway it is not a complete definition. Actually there are situations where we use data services not only to read data, but also to create, update or delete data. So it is better say data services are doing CRUD (Create, Read, Update, Delete) operations for data through web services. Simply it is like providing a web service interface for the database.

Anyway exposing  a database directly as a web service is like violating the first principles of software engineering. It will tightly couple the database structure with the interface, so whenever you do a simple change to the database schema, you will have to change the web service interface which will no doubt break all the clients depending on it.

So first of all, you have to design the service interface independent of the database schema you have. Most of the time you will able to find some query that would map the service interface to the database schema.

For an example think of publishing data in database table (say for table name “Games”) like this.

Teams
GameID Venue Date Team1 Team2 Team1Score Team2Score
1 xxx stadium 2008-12-18 Italy Sweden 34 33
2 yyy stadium 2008-12-19 France Spain 51 50

You will directly able to map these data to data service. So the response payload for a “getGames” operation would be something like,

<getGamesResponse>
   <Game>
      <Venue>
         xxx stadium
      </Venue>
      <Date>
         2008-12-18
      </date>
      <Team1>
          Italy
      </Team1>
      <Team2>
          Sweden
      </Team2>
      <Team1Score>
          34
      </Team1Score>
      <Team2Score>
          33
      </Team2Score>
   </Game>
   <Game>
      <Venue>
         yyy stadium
      </Venue>
      <Date>
         2008-12-19
      </date>
      <Team1>
          France
      </Team1>
      <Team2>
          Spain
      </Team2>
      <Team1Score>
          51
      </Team1Score>
      <Team2Score>
          50
      </Team2Score>
   </Game>
</getGamesResponse>

You can get this done with a SQL query simply as this,

SELECT * FROM `Games`

Say later if you decided to restructure the database table so the new database schema would be like this,

Games
GameId Venue Date
1 xxx stadium 2008-12-18
1 yyy stadium 2008-12-19
GamesTeams
GameId TeamId score
1 1 70
1 2 33
2 1 51
2 3 50
Teams
TeamId Name Coach
1 Italy Mr. ABC
1 Canada Mr. PQR
2 Spain Mr. XYZ

(Note here the Games and Teams are associated in the GamesTeams table.)

You can still use a query like the one in following to provide the same service interface, because it returns the same result set as the earlier one.

   SELECT Games.Venue,
              Games.Date,
              Team1.Name AS Team1,
              Team2.Name AS Team2,
              GameTeam1.Score AS Score1,
              GameTeam2.Score AS Score2
         FROM Teams Team1,
              Teams Team2,
              GamesTeams GameTeam1,
              GamesTeams GameTeam2,
              Games
        WHERE GameTeam1.gameId = Games.gameId  AND
              GameTeam2.gameId = Games.gameId AND
              GameTeam1.teamId = Team1.teamId AND
              GameTeam2.teamId = Team2.teamId AND
              Team1.teamId <> Team2.teamId AND
              Team1.name=?

This allows you to keep the service interface unchanged, regardless of the changes you done to the database schema.

In addition to that, we can uses the features of the data service libraries to give meaningful names for the response elements. If we take above example itself, say you want to rename ‘Score1′ to ‘Team1-Score’ and ‘Score2′ to ‘Team2-Score’. But the dash character (‘-’) cannot be used as a variable in a database query. But you can provide that in the map of sql name to element name when you are writing the data service.

If you are using WSF/PHP php data services Here is how you provide that mapping,

$outputFormat = array("resultElement" => "getGamesResponse",
                      "rowElement" => "game",  // this is the repeating wrapper element for each game
                      "elements" => array( "Venue" => "Venue", // this is the mapping of xml name => sql name
                                           "Date" => "Date",
                                           "Team1" => "Team1",
                                           "Team2" => "Team2",
                                           "Team1-score" => "Score1", // we are using different names for sql and xml
                                           "Team2-score" => "Score2"));

If you write it using Java Data Services, you can use do this with the following configuration xml,

        <result element="getGamesResponse" rowName="game">
            <element name="Venue" column="Venu" />
            <element name="Date" column="Date" />
            <element name="Team1" column="Team1" />
            <element name="Team2" column="Team2" />
            <element name="Team1-Score" column="Score1" />
            <element name="Team2-Score" column="Score2" />
        </result>

Another consideration, when talking about the best practices of designing web services or data services is the granularity of the service interface. We say a web service is fine grained if the service contains tons of small operations. The opposite of that is the coarse grained services, which contains large but few operations. And it is recommended to use the later approach, you can see why from the following example.

Lets say you are developing a web service to upload some information. In order to upload the information first the user have to be authenticated and then the content should be validated. Then only he can submit the actual information. Say you design a fine grained web service for that. So it has three operations.

  1. authenticateUser
  2. validateContent
  3. submitInformation

If you design a coarse grained service interface it will be just the ‘uploadInformation’ operation. And all the three operation defined earlier will be called within the service logic and the clients will not be aware of that.

So here are some disadvantages that I see in the fine grained interface design compared with coarse grained design.

  1. The coupling is too high. Since the client is linked with the service in three adapters. Say you decided to change the service so that you first validate the content and depending on the content it sometime bypasses the authentication. You can’t do this with the former approach (fine grained approach) without changing the clients. But if it were designed as a coarse grained service (just one uploadInformation operation), the client need not to be changed.
  2. Two much time consumed for the transmission. Since we used three web service calls to do a single task the latency of the operation will largely depend on the network latency which will no doubt is comparatively very low. So the performance of the operation is degraded.
  3. The clients can bypass some steps!, Say in your fine grained service, some client bypass the authentication and validateContent steps and jump directly to the sumbitContent operation. In fact you have to write special code to make sure the clients call the service in the correct sequence, otherwise it will be a big security hole.

So I think these three points will be enough to explain why you should try to design a coarse grained interface for your web service.

You can apply these principle when you write data services as well. Lets take the same example explained above. Say there is another table that keep the scoring shots of each game like the one in following.

ScoringShots
GameID Time ScoringTeam ScoringPlayer

Here also you can keep the operation like ‘getGames’ which we defined in the above section. That operation only provide the basic information like the winner and the scores. So if the clients want to know about scoring shots as well, he have to call another operation, say getScoringShots(game) that will return the scoring shots results for a given game. If in practice the clients only need to know about scoring shots of few selected games, then this approach is ok.
But say normally clients need to know about the scoring shots of each and every games. Then they have to call the operation ‘getScoringShots’ multiple times. That’s when the lesson we just learn about the granularity can be applied. We can actually provide another operation, say ‘getGamesDetailed’ that actually bundle the details of scoring shots for all games with other information about the game. Here is snip of the response XML, I’m talking about.

<getGamesDetailedResponse>
   <Game>
      <Venue>
         xxx stadium
      </Venue>
      <Date>
         2008-12-18
      </date>
      <Team1>
          Italy
      </Team1>
      <Team2>
          Sweden
      </Team2>
      <Team1Score>
          34
      </Team1Score>
      <Team2Score>
          33
      </Team2Score>
      <!-- additionally we have ScoringShots element -->
      <ScoringShots>
          <Shot>
              <Time> xxx </Time>
              <ScoringPlayer> xxx </ScoringPlayer>
              <ScoringTeam> xxx </ScoringTeam>
              <Score> xxx </Score>
          </Shot>
          <Shot>
              <Time> yyy </Time>
              <ScoringPlayer> yyy </ScoringPlayer>
              <ScoringTeam> yyy </ScoringTeam>
              <Score> yyy </Score>
          </Shot>
      </ScoringShots>
   </Game>

   <Game>
    <!-- Another game details are mentioned here -->
   </Game>

   <!-- More Games -->
</getGamesDetailedResponse>

You can generate this kind of response using the nested query support of the data services libraries.
You can checkout more details about nested queries in php data services from my old post about php data service API.

LAMP (Linux + Apache + Mysql + PHP) stack powers many servers in the Internet today. For a LAMP  server, PostgreSQL could be the first alternative to Mysql. Similar to PHP + MySQL,  PHP + PostgreSQL too can be easily used in to host data services. Here are the steps to do it.

  1. If you already don’t have Apache + PHP + PostgreSQL download them from the following locations. Apache – http://httpd.apache.org/download.cgi, PHP – http://php.net and PostgreSQL – http://www.postgresql.org/download/
  2. You have to enable the PHP pdo_pgsql, pdo and pgsql plugins. Read here for the instructions to setup these libraries. (For an example: if you are windows you have to set the system ‘PATH’ variable to the <postgresql_installed_dir>/bin directory.
  3. If you already don’t have WSF/PHP, download and install it according to the guidelines provided in wsf/php installation guide.NOTE: You can check pdo_pgsql and wsf/php has properly installed with the help of phpinfo() function.
  4. Now lets start with creating a sample Database table. For this example I created a database called ‘workshop’, schema called ‘workshop’ and inside there the table ‘Employee’ with the following schema.
    Column Name Column Type
    employId integer
    name character varying
    email character varying
    jobTitle character varying
    project character varying

    Note: You can use phpPgAdmin (web based) or pgAdmin III to create tables from GUI

  5. Then you can write a small php script to expose the data in the above table as a web service.
    <?php
    
    //Including the Data Services library
    require_once("wso2/DataServices/DataService.php");
    
    // Including the connection information (i.e. PGSQUL USERNAME
    // and PGSQL_PASSWORD) for my PGSQL Connection
    require_once("constants.php");
    
    // database configurations
    $config = array(
    		"db" => "pgsql",
    		"username" => PGSQL_USERNAME,
    		"password" => PGSQL_PASSWORD,
    		"dbname" => "workshop",
    		"dbhost" => "localhost"
    		);
    
    $output_format = array(
                        "resultElement" => "employees",
                        "rowElement" => "employee",
                        "elements" => array(
    			    "id" => "employeeId",
                                "name" => "name",
                                "email" => "email",
                                "jobTitle" => "jobTitle",
                                "project" => "project"));
    
    $sql = "SELECT * FROM workshop.Employees";
    
    $get_employees_op = array("outputFormat" => $output_format, "sql" => $sql);
    
    $get_employees_url = array("HTTPMethod" => "GET", "RESTLocation" => "employees");
    
    // list of operations
    $operations = array(
                    "getEmployees" => $get_employees_op,
                    );
    
    // list of rest url mappping (operation => url)
    $restmap = array(
                    "getEmployees" => $get_employees_url,
                    );
    
    // creating DSService and reply
    $service = new DataService(array(
             "config" => $config,
             "operations" => $operations, "RESTMapping"=>$restmap));
    $service->reply();
    ?>
  6. We just wrote a PostgreSQL Data Services that provides its service as both REST and SOAP form. To deploy this service, We just need to copy this in to the web root directory. And the web URL for the script will be the endpoint to the web service.
  7. We can test the service either by calling its SOAP interface, which we may need to write a small SOAP client or by calling its REST interface, which only need a GET request from the browser. Say my script name is “my_dataservice.php” and I’ve put it in the web root directory, then the URL to call the REST interface of the service is
    http://localhost/my_dataservice.php/employees

WSDL Generation for PostgreSQL Data Service
You can get the WSDL for the service from the URL formed adding the suffix “?wsdl” (or “?wsdl2″ to wsdl v2.0) to the service URL,

http://localhost/my_dataservice.php?wsdl

Here all the schema data types are shown as xsd:anyType, which may not be the behavior that you want. In fact you can provide the schema data types to the fields from the code itself. Lets change the $outputFormat variable to provide the schema information as well using the following code snip.

$output_format = array(
                    "resultElement" => "employees",
                    "rowElement" => "employee",
                    "elements" => array(
			    "id" => array("column" => "employeeId",
		    			  "xsdType" => "xsd:int"),
			    "name" => array("column" => "name",
		    			  "xsdType" => "xsd:string"),
			    "email" => array("column" => "email",
		    			  "xsdType" => "xsd:string"),
			    "jobTitle" => array("column" => "jobTitle",
		    			  "xsdType" => "xsd:string"),
			    "project" => array("column" => "project",
		    			  "xsdType" => "xsd:string")));

Note that you provide the xsd type for each field explicitly. In fact this change is not needed for mysql pdo extension since it allows identifying field types programatically. Since this feature is not available in all the other pdo drivers, we have to explicitly give xsd type information for them.

If you wan to provide data services with SQLite or MSSQL, You can check my other posts on MSSQL(Microsoft SQL) Data Services In PHP and Data Services with SQLite in PHP.

In Apache Axis2/C AXIOM is used as the basic object model to represent XML. AXIOM provide a DOM like API that allows to traverse and build the XML very easily.

Anyway in underneath, AXIOM is different from DOM, as it has used some techniques to optimize the parsing of the XML as suited specially for SOAP message processing in web services. For an example the SOAP processor can validate a SOAP message by reading only some parts of the SOAP header fields, and if it is not valid, they can completely skip processing the body part. And since AXIOM is designed to built from a stream of data retrieved from a transport, sometimes SOAP processors can validate the message without the need of reading the full stream.

Anyway there should be lot of application that needs this optimization in parsing XMLs. They can easily adapt AXIOM/C to their application. Here is an AXIOM/C tutorial that covers both parsing and building XMLs from AXIOM. In this post I’d like to mention a code that can be used to retrieve an AXIOM from a String (char buffer) which we call as deserialization.

    axiom_node_t* AXIS2_CALL
    deserialize_my_buffer (
        const axutil_env_t * env,
        char *buffer)
    {
        axiom_xml_reader_t *reader = NULL;
        axiom_stax_builder_t *builder = NULL;
        axiom_document_t *document = NULL;
        axiom_node_t *payload = NULL;

        reader = axiom_xml_reader_create_for_memory (env,
            buffer, axutil_strlen (buffer),
            AXIS2_UTF_8, AXIS2_XML_PARSER_TYPE_BUFFER);

        if (!reader)
        {
            return NULL;
        }

        builder = axiom_stax_builder_create (env, reader);

        if (!builder)
        {
            return NULL;
        }
        document = axiom_stax_builder_get_document (builder, env);
        if (!document)
        {
            AXIS2_LOG_ERROR (env->log, AXIS2_LOG_SI,
                    "Document is null for deserialization");
            return NULL;
        }

        payload = axiom_document_get_root_element (document, env);

        if (!payload)
        {
            AXIS2_LOG_ERROR (env->log, AXIS2_LOG_SI,
                    "Root element of the document is not found");
            return NULL;
        }
        axiom_document_build_all (document, env);

        axiom_stax_builder_free_self (builder, env);

        return payload;
    }

Regardless of the fact this piece of code is been used many time by Axis2 and application that uses Axis2, it has never been identified as a core AXIOM function. I think it is better we have this function as an alternative method to create an axiom.

axiom_node_t *AXIS2_CALL
axiom_node_create_from_buffer(const axutil_env_t *env, axis2_char_t *buffer);

I already suggested this in Axis2/C mailing list and hopefully it will be included from the next release.

Here when we create the axiom tree function from the character buffer, we used “axiom_xml_reader_create_for_memory” function. Anyway whenever transport read data stream from wire it always uses the “axiom_xml_reader_create_for_io” function.

    /**
     * This create an instance of axiom_xml_reader to
     * parse a xml document in a buffer. It takes a callback
     * function that takes a buffer and the size of the buffer
     * The user must implement a function that takes in buffer
     * and size and fill the buffer with specified size
     * with xml stream, parser will call this function to fill the
     * buffer on the fly while parsing.
     * @param env environment MUST NOT be NULL.
     * @param read_input_callback() callback function that fills
     * a char buffer.
     * @param close_input_callback() callback function that closes
     * the input stream.
     * @param ctx, context can be any data that needs to be passed
     * to the callback method.
     * @param encoding encoding scheme of the xml stream
     */
    AXIS2_EXTERN axiom_xml_reader_t *AXIS2_CALL
    axiom_xml_reader_create_for_io(
        const axutil_env_t * env,
        AXIS2_READ_INPUT_CALLBACK read_callback,
        AXIS2_CLOSE_INPUT_CALLBACK close_callback,
        void *ctx,
        const axis2_char_t * encoding);

As you may have noticed it requires us to implement a “read_callback” function. Here is an example function prototype to implement this callback.

    int AXIS2_CALL
    some_function(
            char *buffer,
            int size,
            void *ctx);

This function will be called by the parser as required to parse the XML read from some stream.

So if your application involves reading data from a stream you are always recommended to use this function (i.e. “axiom_xml_reader_create_for_io”) instead of “axiom_xml_read_create_for_buffer” to create the AXIOM model more effectively.


© 2007 Dimuthu’s Blog | iKon Wordpress Theme by Windows Vista Administration | Powered by Wordpress