Data Services Best Practices

In simple term, data services are exposing data as web services. Anyway it is not a complete definition. Actually there are situations where we use data services not only to read data, but also to create, update or delete data. So it is better say data services are doing CRUD (Create, Read, Update, Delete) operations for data through web services. Simply it is like providing a web service interface for the database.

Anyway exposing  a database directly as a web service is like violating the first principles of software engineering. It will tightly couple the database structure with the interface, so whenever you do a simple change to the database schema, you will have to change the web service interface which will no doubt break all the clients depending on it.

So first of all, you have to design the service interface independent of the database schema you have. Most of the time you will able to find some query that would map the service interface to the database schema.

For an example think of publishing data in database table (say for table name “Games”) like this.

Teams
GameID Venue Date Team1 Team2 Team1Score Team2Score
1 xxx stadium 2008-12-18 Italy Sweden 34 33
2 yyy stadium 2008-12-19 France Spain 51 50

You will directly able to map these data to data service. So the response payload for a “getGames” operation would be something like,

<getGamesResponse>
   <Game>
      <Venue>
         xxx stadium
      </Venue>
      <Date>
         2008-12-18
      </date>
      <Team1>
          Italy
      </Team1>
      <Team2>
          Sweden
      </Team2>
      <Team1Score>
          34
      </Team1Score>
      <Team2Score>
          33
      </Team2Score>
   </Game>
   <Game>
      <Venue>
         yyy stadium
      </Venue>
      <Date>
         2008-12-19
      </date>
      <Team1>
          France
      </Team1>
      <Team2>
          Spain
      </Team2>
      <Team1Score>
          51
      </Team1Score>
      <Team2Score>
          50
      </Team2Score>
   </Game>
</getGamesResponse>

You can get this done with a SQL query simply as this,

SELECT * FROM `Games`

Say later if you decided to restructure the database table so the new database schema would be like this,

Games
GameId Venue Date
1 xxx stadium 2008-12-18
1 yyy stadium 2008-12-19
GamesTeams
GameId TeamId score
1 1 70
1 2 33
2 1 51
2 3 50
Teams
TeamId Name Coach
1 Italy Mr. ABC
1 Canada Mr. PQR
2 Spain Mr. XYZ

(Note here the Games and Teams are associated in the GamesTeams table.)

You can still use a query like the one in following to provide the same service interface, because it returns the same result set as the earlier one.

   SELECT Games.Venue,
              Games.Date,
              Team1.Name AS Team1,
              Team2.Name AS Team2,
              GameTeam1.Score AS Score1,
              GameTeam2.Score AS Score2
         FROM Teams Team1,
              Teams Team2,
              GamesTeams GameTeam1,
              GamesTeams GameTeam2,
              Games
        WHERE GameTeam1.gameId = Games.gameId  AND
              GameTeam2.gameId = Games.gameId AND
              GameTeam1.teamId = Team1.teamId AND
              GameTeam2.teamId = Team2.teamId AND
              Team1.teamId <> Team2.teamId AND
              Team1.name=?

This allows you to keep the service interface unchanged, regardless of the changes you done to the database schema.

In addition to that, we can uses the features of the data service libraries to give meaningful names for the response elements. If we take above example itself, say you want to rename ‘Score1’ to ‘Team1-Score’ and ‘Score2’ to ‘Team2-Score’. But the dash character (‘-‘) cannot be used as a variable in a database query. But you can provide that in the map of sql name to element name when you are writing the data service.

If you are using WSF/PHP php data services Here is how you provide that mapping,

$outputFormat = array("resultElement" => "getGamesResponse",
                      "rowElement" => "game",  // this is the repeating wrapper element for each game
                      "elements" => array( "Venue" => "Venue", // this is the mapping of xml name => sql name
                                           "Date" => "Date",
                                           "Team1" => "Team1",
                                           "Team2" => "Team2",
                                           "Team1-score" => "Score1", // we are using different names for sql and xml
                                           "Team2-score" => "Score2"));

If you write it using Java Data Services, you can use do this with the following configuration xml,

        <result element="getGamesResponse" rowName="game">
            <element name="Venue" column="Venu" />
            <element name="Date" column="Date" />
            <element name="Team1" column="Team1" />
            <element name="Team2" column="Team2" />
            <element name="Team1-Score" column="Score1" />
            <element name="Team2-Score" column="Score2" />
        </result>

Another consideration, when talking about the best practices of designing web services or data services is the granularity of the service interface. We say a web service is fine grained if the service contains tons of small operations. The opposite of that is the coarse grained services, which contains large but few operations. And it is recommended to use the later approach, you can see why from the following example.

Lets say you are developing a web service to upload some information. In order to upload the information first the user have to be authenticated and then the content should be validated. Then only he can submit the actual information. Say you design a fine grained web service for that. So it has three operations.

  1. authenticateUser
  2. validateContent
  3. submitInformation

If you design a coarse grained service interface it will be just the ‘uploadInformation’ operation. And all the three operation defined earlier will be called within the service logic and the clients will not be aware of that.

So here are some disadvantages that I see in the fine grained interface design compared with coarse grained design.

  1. The coupling is too high. Since the client is linked with the service in three adapters. Say you decided to change the service so that you first validate the content and depending on the content it sometime bypasses the authentication. You can’t do this with the former approach (fine grained approach) without changing the clients. But if it were designed as a coarse grained service (just one uploadInformation operation), the client need not to be changed.
  2. Two much time consumed for the transmission. Since we used three web service calls to do a single task the latency of the operation will largely depend on the network latency which will no doubt is comparatively very low. So the performance of the operation is degraded.
  3. The clients can bypass some steps!, Say in your fine grained service, some client bypass the authentication and validateContent steps and jump directly to the sumbitContent operation. In fact you have to write special code to make sure the clients call the service in the correct sequence, otherwise it will be a big security hole.

So I think these three points will be enough to explain why you should try to design a coarse grained interface for your web service.

You can apply these principle when you write data services as well. Lets take the same example explained above. Say there is another table that keep the scoring shots of each game like the one in following.

ScoringShots
GameID Time ScoringTeam ScoringPlayer

Here also you can keep the operation like ‘getGames’ which we defined in the above section. That operation only provide the basic information like the winner and the scores. So if the clients want to know about scoring shots as well, he have to call another operation, say getScoringShots(game) that will return the scoring shots results for a given game. If in practice the clients only need to know about scoring shots of few selected games, then this approach is ok.
But say normally clients need to know about the scoring shots of each and every games. Then they have to call the operation ‘getScoringShots’ multiple times. That’s when the lesson we just learn about the granularity can be applied. We can actually provide another operation, say ‘getGamesDetailed’ that actually bundle the details of scoring shots for all games with other information about the game. Here is snip of the response XML, I’m talking about.

<getGamesDetailedResponse>
   <Game>
      <Venue>
         xxx stadium
      </Venue>
      <Date>
         2008-12-18
      </date>
      <Team1>
          Italy
      </Team1>
      <Team2>
          Sweden
      </Team2>
      <Team1Score>
          34
      </Team1Score>
      <Team2Score>
          33
      </Team2Score>
      <!-- additionally we have ScoringShots element -->
      <ScoringShots>
          <Shot>
              <Time> xxx </Time>
              <ScoringPlayer> xxx </ScoringPlayer>
              <ScoringTeam> xxx </ScoringTeam>
              <Score> xxx </Score>
          </Shot>
          <Shot>
              <Time> yyy </Time>
              <ScoringPlayer> yyy </ScoringPlayer>
              <ScoringTeam> yyy </ScoringTeam>
              <Score> yyy </Score>
          </Shot>
      </ScoringShots>
   </Game>

   <Game>
    <!-- Another game details are mentioned here -->
   </Game>

   <!-- More Games -->
</getGamesDetailedResponse>

You can generate this kind of response using the nested query support of the data services libraries.
You can checkout more details about nested queries in php data services from my old post about php data service API.

This entry was posted in data services, DataServices, design, php, SOA, SQL, Tutorial/Guide, web services, wso2 and tagged , , , , , . Bookmark the permalink.

8 Responses to Data Services Best Practices

  1. Eric says:

    Hi there,

    If im going to consume a REST web service created using WSF/PHP, is there anyway to modify the xml output format ($outputFormat) to include additional element?

    say my output xml format should be like this:

    ….

    ….

    notice the data element? How would i go about this in WSF/PHP?

    thanks,

    Eric

  2. Eric says:

    opppss, the sample xml format was not displayed, should something like this:

    response
    data
    record

    /record

    /data
    /response

  3. dimuthu says:

    Hi Eric,
    Looks like you want to add an additional wrapper element to the generated response. That mean,
    <response>
    <data>
    <record>
    <field1>
    </field1>
    <field2>
    </field2>
    </record>
    </data>
    </response>

    If it is so, you can’t do that. 🙁
    But if you want is something like this, (see custom fields)
    <response>
    <record>
    <field1>
    </field1>
    <field2>
    </field2>
    <customfield1>
    </customfield1>
    <customfield2>
    </customfield2>
    </record>
    </response>
    then you can do it. Please read this, http://wso2.org/wiki/display/wsfphp/API+for+Data+Services+Revised

    Thanks
    Dimuthu

  4. _revo says:

    Dear Dimuthu!! Many thanks for your great tutorial. Very clear and helpful.

    Please advice how to manage more than 1 operation in one project file? For example is there a way to pack operations in functions or class? Please write example how to make 2 operations in 1 file, both with sql queries and different output model. Please..

    Regards,
    Alexey

  5. _revo says:

    Sure, I did that way, but how you can work with $inMessage – check variables, threw custom WSFaults, WSMessages, addressing etc – according to requested operation. That is why I try to find solution to incapsulate operation body in function or class.

  6. dimuthu says:

    If you are going to have a logic in the service method, you should use WSService object and proceed in normal way.
    Data Services (Using DataService object) we just wrap database query from a data service, So no space to write additional code. 🙁

    Thanks
    Dimuthu

  7. _revo says:

    oh 🙁 unexpected turn.. this killed a lot from positive of DataService.. all simplicity came to zero if we need to handle input manually, then build sql, parse rowset and output handmade replyMessage :(( Comparing with regular php coding – only autoWSDL generation seems to be as plus from using wso2/php framework. Of course for simple oneway methods it is still good but could (will?) be more. Please keep developing!!

Leave a Reply

Your email address will not be published. Required fields are marked *