Achieving the Integrity quality attribute in WCF SOA Services

Non-Functional Requirements - Integrity

This article follows on from my previous one where I introduced a pattern for creating service-orientated applications.

The set of components introduced in the previous article in themselves offer no help in meeting the technical requirements of; Integrity, Flexibility, Maintainability, Performance, Re-usability, Scalability and Security.

It is the interaction of the model and the technology used to construct it that will allow us to achieve the required non-functional aspects of our system. It can be said that architecture and the process used to build it are really just different representations of the same thing, and when the two are aligned one is able to build powerful systems very productively.

So how then do we leverage the technology to help us achieve our required technical requirements with minimal effort? Our chosen technology in this case will be Windows Communication Foundation (WCF), but it is equally achievable in Java with EJB 3 or the equivalent in other languages, all that will differ is productivity.

WCF can be said to be three minutes and three weeks, three weeks to think about how to do it, three minutes to do it, the beauty of WCF is that you almost don’t use it. WCF moves the focus of development from implementation to design, as gives developers a fighting chance to deliver professional software in productive time-frames.

Using the above mentioned model and patterns I will demonstrate the architecture and code implementation to achieve the required technical requirement, which is Integrity in this case. The level of adherence will depend on your systems requirements. The configuration below or a slight variation will suit 90% of most business applications.

Integrity

The first non-functional requirement I introduce is Integrity.

Integrity is the ability to detect and manage invalid data coming in to system as well as the imposition of complete transactions or rollbacks.

Transactions

I choose to adopt what is called a client/service transactional model in most cases; this is that if the consumer of the service presents our service a transaction we will join that one and allow the consumer to commit or rollback with our service as an atomic unit, possibly across a distributed system, otherwise we will make the Channel service the root of all transactions.

If no Isolation level is provided by the client we will assume the strictest Isolation level of Serializable, Ihave found this to be suitable in most systems except read intensive ones or in places where batch like functionality is required.

In a complex system, transactions are essential for ensuring the integrity and consistency of the data modified by the call chains. Directly managing the transactions is error prone and extremely labor intensive. The transaction support in the .NET Framework and WCF make this considerably easier.

Decisions still need to be made on where to apply transactional boundaries and when to span those transactions to other services. The decision includes not only what call chains to include in a transactional scope, but also what level of isolation to use for that transaction. The default in .NET is a Serializable transaction, which provides the maximum protection for consistent data resulting from a transaction.

However, Serializable transactions also use the most locking at the database level to ensure that integrity, so it will have throughput impacts. Those impacts are generally best addressed by scaling out the application as needed for throughput rather than relaxing the isolation unless detailed analysis of each call chain is done by data access experts.

Certain calling patterns, such as a read-only SELECT style call chain do not need to participate in a transaction. But if there is any potential for database modifications to be made as part of the overall business process of a call chain, then starting with a serializable transaction encapsulating all the activity of the call chain is the safest starting point.

Through analysis, the decision to back off on the span of the transaction scope or the isolation level may be made if performance challenges dictate and the query pattern is permissive. These have to be treated on a case by case basis.

Many raise a question of performance and locking issues due to this strict isolation level, research has found that in typical enterprise systems between 1-3% of users are concurrent at any one instance in time, 1% is normal 3% in a highly transactional system, for example, a system with 1500 users could expect 15 – 45 users to be concurrently accessing the system. This number is further reduced by assessing if they are all contending over the same resource at that instance in time.

In instances where intensive reads are done on a resource before an edit is performed one may relax the Isolation level to ReadCommited, but this should be an architectural decision and not taken lightly.

I recommend starting with Serializable, with the exception of select type call chains where a transaction is not required, but I would still recommend flowing the transaction at the interface level as in a SOA one can never be certain of the order and actual call chain that your service may participate in, so to be safe we flow the transaction trough to the next call to use if it needs too.

The diagram below depicts an iDesign type model of the default model; the red line in this case represents the transaction boundary

Transaction Models

Client/Server
Client/Server
Server
Server
Transactions with Publish/Subscribe
Transactions with Publish/Subscribe


With these settings we scope the entire logical unit of work into a single transaction that will enlist any transactional resource that it touches and if any exception occurs will trigger a rollback across the entire call chain even if the pieces are physically deployed on separate machines.

This is distributed transaction is achieved by using the Distributed Transaction Controller packaged into the windows operating system. If publish/subscribe mechanisms are used the service calls will at least one transaction removed from the original causing a rollback to the senders queue.

The settings for the Channel services are as follows

Turn on transaction flow in the bindings

<netTcpBinding>

  <binding

name="ReliableTcp" maxBufferSize="100000"

maxReceivedMessageSize="100000" transactionFlow="true">

          <readerQuotas maxArrayLength="100000"

maxStringContentLength="100000"/>

          <reliableSession enabled="true"/>

        </binding>

</netTcpBinding>

Set the transaction flow on the interface methods to Allowed

 [OperationContract]

        [TransactionFlow(TransactionFlowOption.Allowed)]

        void CaptureClientDetails(Client client);

Set the transaction scope required on the service implementation to true

[OperationBehavior(TransactionScopeRequired = true)]

        void IClientManager.RequestEmployer(Employer employer)

        {

                           //some code

        }


The TransactionScopeRequired Attribute is not required for SELECT type call chains where no data is modified.

If you want the transactions to flow between machines will need to turn on DTC on those machines

Flowing transactions in a load balanced environment

In a load balanced environment, it is advisable to remain as stateless as possible, if you can even avoid sticky sessions do it. Under these circumstances I would set the transaction flow option on the service contract to NotAllowed. This way I make certain that no transport layer sessions are created (you will need to set up you security to be sessionless, but more on that later)

I would still set the TransactionScopeRequired to True on the service implementation, so that my Channel service becomes the root of all transactions. Even if I do not want my client to be able to participate, I would still like to scope all my work into a transaction.

All the services in the call chain below the Channel will be set to flow transactions, or to start a new transaction. As the Channel is the hosted element of my service stack, it start transactions and all participating services join to this one. I am then able to scope the work of many services into one unit of work.

Data Validation

In order to validate invalid data we rely can on an interface called IDataErrorInfo, we implement this interface on our data contracts, allowing them to become self validating data contracts.

The usual use for this interface is to allow the user interface components to validate the data that needs to bind to a certain contract, thus the contract is able to encapsulate all the information it needs about what is valid data in its context. This method prevents the scattering of validation logic between the client and service.

We extend this behavior in two ways;

Firstly we extend this validation to the channel service by implementing a WCF behavior called ServiceDataContractValidation to validate input that arrives at the Channel service; or OperationDataContractValidation to do the same on a per method basis, IDataErrorInfo will contain any error messages prompted by the binding of incorrect data.

To validate if valid data is being bound to our contract we will use our second extension, the DataMemberValidator will inspect the data bound to the data contract and compare it to the rules that where set up for valid data for that contract at design time by calling the Validate method. If any invalid data is bound to the contract a run-time the validator will flag it as invalid data.

By using these two attributes together we thus create a mechanism we can use in presentation layer controls to validate input against a particular contract at run-time and provide visual prompts, and reject any contracts that have invalid data bound to them at the service boundary rather that executing all the layers to have an exception thrown from the gateway because a sting was too long.

Notice that IExtensibleDataObject is also implemented to allow for round trip versioning.

[DataContract(Namespace = "http://www.metallemon.co.za/WCF/2010/01")]

public class Client : IExtensibleDataObject, IDataErrorInfo

[DataMember]

[DataMemberValidator("Application Date", ValidatorConstraint.IsRequiredProperty)]

public DateTime ApplicationDate { get; set; }

[DataMember]

[DataMemberValidator("Account Number", ValidatorConstraint.IsRequiredProperty, MaximumLength = 50)]

public string AccountNumber { get; set; }

protected string Validate(string columnName)

{

                   m_Error = String.Empty;

 

MemberValidator<Client> validator = new MemberValidator<Client>(this);

       m_Error = validator.ValidateColumn(columnName);

           

       return m_Error;

}

On the Channel Service

void IClientManager.ManageClient(OperationMode operationMode, Client client)

{

   CheckArgument(client, "Client");

 

   using (var manager = new ProxyHelper<ClientManager, IClientManager>())

   {

      manager.Execute(m => m.ManageClient(operationMode, client));

   }

}

Enterprise Library 6.0

The Microsoft Enterprise Library is a collection of reusable software components (application blocks) designed to assist software developers with common enterprise development cross-cutting concerns (such as logging, validation, data access, exception handling, and many others).

The ideas presented above have also been implemented using the validation block of the Enterprise Library, remember its the pattern that is important, the implementation will and must evolve as your understanding of the problem improves.

This concept is at the heart of SOA, stable interfaces and ever evolving implementations.

Build Integrity into the framework

So by establishing architectural patterns around how your services manage transactions and data validation, you build integrity into your framework in such a way that few mistakes can be made by the implementers of your designs.

More by this Author


No comments yet.

    Sign in or sign up and post using a HubPages Network account.

    0 of 8192 characters used
    Post Comment

    No HTML is allowed in comments, but URLs will be hyperlinked. Comments are not for promoting your articles or other sites.


    Click to Rate This Article
    working