Drools!

Drools, or JBoss Rules is an open source framework that facilitates an easy maintenance of rules. And it comes with a feature called decision matrices which I quite perticularly like. Makes it easy. The best thing is that a decision matrix could be a MS Excel sheet, which means I can have a 2D expression of conditions. It also means that this matrix can be maintained by business analysts and can be created at the time of requirements gathering, instead of being developed by coders at the time of implementation. Gives you a bit of a head start!

Do I use Drools everywhere I have rules? No way! I use it at places that have a high density of if-else kind of statements, otherwise I prefer to use plain old if-elses. The reason is simple, drools does add an overhead, although minimal, of its own. Plus, the logic is decentralised and developers typically have an apprehension dealing with a "different" technology. Anyways, the simplicity achieved via decision matrices scores much higher.

I used Drools recently for a e-commerce function, we had to provide a list of products that a customer could buy based on a number of factors. Had we implemented in the classical if-else manner, it would have been a nightmare to find issues and fix, but decision matrices ensured that we had minimal bugs and any found were easily fixed.

Its super cool, if you haven't tried it yet, I strongly recommend you work out some time soon :-)

Need for Stub!

Do you have many external system interactions? Too many third parties? Do they provide easy interfaces for testing different scenarios in your application? Do I hear a resounding no? :-)

We have an application that depends on 30+ external systems that either do not provide test interfaces, or the ones that are provided, are quite primitive. That's quite bad because we can not effectively test our application and most of the logic goes untested to production. Take a guess... yes we have a large number of issues on production :-(

When we started writing the application, we decided we needed stubs, i.e. programs that will block our actual interaction with the third party and give us some response back. The problem with this approach is that you never look at the request! So we scratched our heads and we thought the following were the options:

1. Dumb stubs: Any request, you get a fixed response. No good unless you don't have any logic around the response that comes back.

2. Wise stubs: Based on the request coming in, validate it, point out errors, and prepare a response. The response could be different for different requests so can enable a wider scale of testing.

3. Simulators: Enhance the wise stubs so that they maintain a state of requests as well (may be across stubs if required). So if you have a stub for withdraw money and another to deposit, both should update a "balance" which should be visible to both.

In all of these cases, another dimension is that you could have stubs that sit in the same layer of application as the end point connectors, or you could have them available as services over protocols similar to how it will be on live.

I suggest building wise stubs that are available as services. Simulators are too much of a pain to write so unless you need them desperately, avoid them.

This is what we did recently: we used Mule. How does Mule help me ? Greatly!

1. Mule provides me with a central way of interacting with all different stubs.

2. We now have a common framework for designing and implementing stubs.

3. Most of the protocols are supported out of the box (we had to write a custom handler for Telnet though) so its kind of plug-and-play.

4. It can be easily run on a seperate JVM and can be deployed anywhere in the system.

5. I can easily build delays in the responses so that performance testing can see real life type responses.

6. And its easy!

Unit Testing with HSQL

Ever wondered how you could increase the speed of execution of test cases that run against a database? Or how you could unit test without impacting the data in your database? The obvious answer to the latter is to use a seperate database schema, but speed? How about an in-memory database? See, now we are talking!

HSQL provides a means of running an in-memory all Java RDBMS, so everything runs in the heap. All you need to do is to include the JARs from HSQL website (no I wont give a link here, please use internet search :-) ) into the JUnit execution classpath, configure the datasource as jdbc:hsqldb:mem:mydatabase?shutdown=true. If you use Hibernate in your application, you can use the SchemaExport feature (with Spring over Hibernate, use session factory bean to create database schema) to create the tables et al in the database as well. Bingo, you have a new and shinny blank database schema in the Java heap to run tests against.

I had to write my own Oracle specific functions and register them with HSQL because people had used them in HQLs :-(. Anyways, its not difficult and all you need to do is to have a class with static methods that implement the required feature and run "CREATE ALIAS" SQL queries against the database. That's it, easy peasy!

Using HSQL, I was able to bring down test execution time from 1.5 hrs to 20 mins.

WSRP with Weblogic Portal

Introduction
We have recently implemented WSRP using Oracle’s Weblogic Portal technology for our selfcare customer experiene for a client. This application had been in production for over 4 years and we had started to face severe challenges around its scalability and maintainability. The application had grown huge, was packaged in a single enterprise archive (EAR) and testing the smallest changes required a full regression test to be conducted. This meant that the system could not scale well in future, and also that we could not propagate changes to production swiftly and without impacting the whole website.
Once we read about WSRP, we knew it was the way to go!

Enter WSRP!
Webservices for Remote Portlets (WSRP) specification provides means for integrating remote applications and content into a website (portal) via pluggable and interactive webservices. These webservices have a well defined interface and protocol for processing user interactions and fragments that can be brought together to render web pages. This also means that all of your remote applications now integrate with each other using one single mechanism and one single adaptor, removing the need for having different adaptors for different applications.
An example of where WSRP shows its true might is where you have to show device information, price plan information, billing information, stock feeds and weather feeds on the same web page – say a customer’s home page. You may have device information coming from a different data source, plans and bills from yet another, stock and weather feeds may be third party. In a classical web application, you would have different parts of the application working sequentially sourcing all the necessary data and once done, would dump it all on the screen. If any one breaks, the page may entirely break. This is not very different to the way a regular portlet based application would behave either. However, in a WSRP based application, you have the option of executing these in parallel through minor configuration, and also, the impact is limited to a part of the page rather than the entire webpage. So you may have weather and stock information on the page even if your database is down! This also means you can pull information from completely independent sources without any code changes to your application.

The integrating part of the application (that puts everything together) is called a consumer, and all applications supplying the fragments are called producers. There can obviously be many consumers (different web sites) using a number of producers. So you could have different websites, say a public one, a customer service one, an intranet, an administration application etc all using different fragments coming from a set of producers. These websites may have a different look and feel and an entirely different purpose.

Typically, the interaction between producers and consumers happens in form of SOAP (over HTTP) and hence these producers could well be located in separate geographies. This enables horizontal scalability in the same application – we can now have parts of our application running on one server and other parts on other servers (of course, managing response times could be a challenge!).

Advantages of Using WSRP
We see the following benefits with the use of WSRP:

• There are smaller and more manageable applications now which can be developed and deployed in isolation.
• The development and testing effort now goes down as the impacts of changes are contained to smaller applications.
• A smaller regression test effort is required which implies a shorter test phase.
• Project lifecycle is shortened with the impact being contained and hence there is more agility in delivery.
• There is a larger possibility of code reuse in the application with individual deployable components.
• Applications can be deployed onto production without a full outage on the currently running website.
• With the use of shared libraries/ optional packages, it is possible to share code between different applications as well.

Performance Considerations

While WSRP provides many advantages in terms of maintainability, there is some obvious performance overhead associated with the same:

• The number of applications now increase which means there will be a default static load on the system (each application will come with its own overhead). In our case, there was a 60M overhead with each EAR.
• There is a CPU overhead because of the increase in number of interactions (each producer-consumer interaction is a new request).
• The memory utilization goes up as well as there are more sessions created (one per producer) and more caches created.
• Weblogic recommends using local proxy portlets rather than using remote proxy.
  We saw the response timings jump up as well. For static pages, the jump was significant (almost double).
• We saw the heap memory jump up from 2 GB to 2.5 GB and the CPU jump up from 20% utilization to 30%-35%.
• The startup memory jumped up from 200 MB to 600 MB.
• The size of LDAP files grows rapidly with WSRP as it is used for securing all communication between producers and consumers. The impact of LDAP is significant as Weblogic caches the entire LDAP in heap and having a large LDAP means high memory utilization, leaving small space for application to run. There are ways to limit the size of LDAP so they must be leveraged.

Unit Testing Using EasyMock

One of the most frustrating problems I have faced over years is around how to test just the unit of code I am interested in. The problem is aggravated by the fact that many a times my unit tests actually depend on the environment (data/ system settings). So my unit tests dont actually test units, do they?

I came across something called MockObjects way back in 2004, but even though we used it extensively then, I always found it a bit troublesome to use, one of the reasons why I did not use it much in the following years. I had heard about EasyMock over the years, but did not quite use it until recently. And hey, its the most amazing thing i have used in years! After Hibernate ofcourse :-)

So whats so cool about EasyMock? One, you dont have to write a seperate class called MockBlah to mock up Blah. EasyMock uses CGLIB proxies to create mocks on the fly! By default, you can only do it for interfaces, but with its class extensions, it can be done for classes as well.

Secondly, you dont need to tell EasyMock exactly what parameter value to expect going into a method, just give it the class type (EasyMock.isA(...) or EasyMock.anyInt(...)) and it will work for any object of the given type.

Thirdly, you can simulate exceptions by expecting method calls to throw exceptions. So you dont need a database dead lock or insertion failure to see how your code will behave in a case like that, just throw a SQLException from the interacting method's expectation.

Plus, you can stub method behaviours using the Answer functionality. Grab a hold of the parameters passed into a method and play with them - return whatever you expect, or set values in the passed objects. Brilliant, isn't it ?

What this means for me is that the JUnits are no longer data/ environment dependent, so there's no real set up and tear down needed. The test cases are now fantastically fast as well -I dont interact with the system and worry only about my Java code now. I also dont need to test on large and useless data sets, I create different types of data in my test cases so as to cover my code execution paths. Data independence, fast execution times and an ability to achieve a higher code coverage.

Read more about EasyMock at http://www.easymock.org/