Shahzad Bhatti Welcome to my ramblings and rants!

June 16, 2006

Integrating ActiveMQ with JBoss

Filed under: Computing — admin @ 8:49 am

Integrating ActiveMQ with JBoss
Recently, I had to integrate ActiveMQ 4 with JBoss 4
Here is how I did it:

Download and install JBoss 4.0.4

set JBOSS_HOME environment variable to point to installation directory of JBoss.

Download and install ActiveMQ 4.0

set ACTIVEMQ_HOME environment variable to point to installation directory of ActiveMQ.

Copy ActiveMQ JCA (rar) file to JBoss’s deploy directory, e.g.

cp $ACTIVEMQ_HOME/lib/optional/optional/activemq-ra-4.0-RC2.rar $JBOSS_HOME/server/all/deploy

Start Active MQ Server

cd $ACTIVEMQ_HOME/bin
sh activemq

Start JBoss Server

cd $JBOSS_HOME
sh bin/run.sh -c all
You need to keep the window open as you will see some debug messages on the console.

Create a folder for all source code, e.g.

mkdir mdbtest
cd mdbtest

Create a source, build and META-INF folder, e.g.

mkdir src
mkdir build
mkdir build/META-INF

Create a source package directory for Message-Driven Bean and Client

mkdir src/mdb
mkdir src/publisher

Define Message Driven Bean, in src/mdb/TestMessageBean.java e.g.

 1  package mdb;
 2  import java.io.PrintStream;
 3  import java.rmi.RemoteException;
 4  import java.util.Hashtable;
 5  import java.util.Properties;
 6  import javax.ejb.CreateException;
 7  import javax.ejb.MessageDrivenBean;
 8  import javax.ejb.MessageDrivenContext;
 9  import javax.jms.JMSException;
10  import javax.jms.Message;
11  import javax.jms.MessageListener;
12  import javax.jms.TextMessage;
13  import javax.naming.InitialContext;
14  import javax.naming.NamingException;
15  import javax.rmi.PortableRemoteObject;
16  import org.apache.commons.logging.Log;
17  import org.apache.commons.logging.LogFactory;
18 
19  /**
20   * @ejb.bean name="TestMessageEJB"
21   */
22  public class TestMessageBean implements MessageDrivenBean, MessageListener {
23    public TestMessageBean() {
24    }
25    public void onMessage(Message message)  {
26      try {
27        TextMessage textMessage = (TextMessage) message;
28        System.out.println(">>>Received " + textMessage.getText());
29      } catch (Exception e) {
30        e.printStackTrace();
31      }
32    }
33 
34    public void ejbRemove() {
35    }
36 
37    public void setMessageDrivenContext(MessageDrivenContext messageDrivenContext) {
38    }
39 
40    public void ejbCreate() {
41    }
42  }

Define Client Publisher in src/client/Publisher.java, e.g.

 1  package client;
 2  import java.io.PrintStream;
 3  import java.util.Properties;
 4  import javax.jms.ConnectionFactory;
 5  import javax.jms.JMSException;
 6  import javax.jms.Message;
 7  import javax.jms.Session;
 8  import javax.jms.TextMessage;
 9  import javax.naming.InitialContext;
10  import javax.rmi.PortableRemoteObject;
11  import org.apache.activemq.ActiveMQConnectionFactory;
12  import org.springframework.jms.core.JmsTemplate;
13  import org.springframework.jms.core.MessageCreator;
14 
15  public class Publisher {
16    private static final String URL = "tcp://localhost:61616";
17    private static final String DEST = "destQ";
18    private static final boolean topic = false;
19    public static void main(String[] args) throws Exception {
20      ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
21      connectionFactory.setBrokerURL(URL);
22      JmsTemplate jt = new JmsTemplate(connectionFactory);
23      if (topic) jt.setPubSubDomain(true);
24      jt.send( DEST, new MessageCreator() {
25        public Message createMessage(Session session) throws JMSException {
26          Message m = session.createTextMessage("hello world");
27            m.setIntProperty("add_id", id);
28           return m;
29          }
30        }
31       );
32    }
33  }

Define META-INF/ejb-jar.xml as follows:

 <?xml version="1.0" encoding="US-ASCII"?>

 <ejb-jar xmlns="http://java.sun.com/xml/ns/j2ee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/ejb-jar_2_1.xsd"
    version="2.1">

    <description> Test ActiveMQ </description>

    <enterprise-beans>
       <message-driven>
          <description>TestMDB</description>
          <ejb-name>TestMessageEJB</ejb-name>
          <ejb-class>mdb.TestMessageBean</ejb-class>
          <messaging-type>javax.jms.MessageListener</messaging-type>
          <transaction-type>Container</transaction-type>
          <activation-config>
             <activation-config-property>
                <activation-config-property-name>destination</activation-config-property-name>
                <activation-config-property-value>destQ</activation-config-property-value>
             </activation-config-property>
             <activation-config-property>
                <activation-config-property-name>destinationType</activation-config-property-name>
                <activation-config-property-value>javax.jms.Queue</activation-config-property-value>
             </activation-config-property>
             <activation-config-property>
                <activation-config-property-name>acknowledgeMode</activation-config-property-name>
                <activation-config-property-value>Auto-acknowledge</activation-config-property-value>
             </activation-config-property>
             <activation-config-property>
                <activation-config-property-name>messageSelector</activation-config-property-name>
                <activation-config-property-value/>
             </activation-config-property>
          </activation-config>
       </message-driven>
    </enterprise-beans>
    <assembly-descriptor>
      <security-role>
         <description>Open</description>
         <role-name>everyone</role-name>
      </security-role>
      <method-permission>
         <role-name>everyone</role-name>
         <method>
           <ejb-name>TestMessageEJB</ejb-name>
           <method-name>*</method-name>
         </method>
      </method-permission>
      <container-transaction>
        <method>
          <ejb-name>TestMessageEJB</ejb-name>
          <method-name>*</method-name>
        </method>
        <trans-attribute>Required</trans-attribute>
      </container-transaction>

       <message-destination>
          <message-destination-name>jbossQ</message-destination-name>
       </message-destination>
    </assembly-descriptor>
 </ejb-jar>
Define META-INF/jboss.xml as follows:

 <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE jboss PUBLIC
       "-//JBoss//DTD JBOSS 4.0//EN"
       "http://www.jboss.org/j2ee/dtd/jboss_4_0.dtd">
   <jboss>
      <jmx-name>activemq.rar:name=iTests</jmx-name>
      <enterprise-beans>
        <message-driven>
          <ejb-name>TestMessageEJB</ejb-name>
          <resource-adapter-name>activemq-ra-4.0.rar</resource-adapter-name>
          <configuration-name>ActiveMQ Message Driven Bean</configuration-name>
        </message-driven>
      </enterprise-beans>
      <assembly-descriptor>
         <message-destination>
          <message-destination-name>jbossQ</message-destination-name>
          <jndi-name>activemq/queue/outbound</jndi-name>
         </message-destination>
      </assembly-descriptor>
      <resource-managers>
         <resource-manager>
            <res-name>queuefactoryref</res-name>
            <res-jndi-name>java:/activemq/QueueConnectionFactory</res-jndi-name>
         </resource-manager>
         <resource-manager>
            <res-name>topicfactoryref</res-name>
            <res-jndi-name>java:/activemq/TopicConnectionFactory</res-jndi-name>
         </resource-manager>
      </resource-managers>

      <invoker-proxy-bindings>
         <invoker-proxy-binding>
            <name>activemq-message-driven-bean</name>
            <invoker-mbean>default</invoker-mbean>
            <proxy-factory>org.jboss.ejb.plugins.inflow.JBossMessageEndpointFactory</proxy-factory>
            <proxy-factory-config>
              <endpoint-interceptors>
                <interceptor>org.jboss.proxy.ClientMethodInterceptor</interceptor>
                <interceptor>org.jboss.ejb.plugins.inflow.MessageEndpointInterceptor</interceptor>
                <interceptor>org.jboss.proxy.TransactionInterceptor</interceptor>
                <interceptor>org.jboss.invocation.InvokerInterceptor</interceptor>
             </endpoint-interceptors>
           </proxy-factory-config>
         </invoker-proxy-binding>
      </invoker-proxy-bindings>

      <container-configurations>
         <container-configuration>
            <container-name>ActiveMQ Message Driven Bean</container-name>
            <call-logging>false</call-logging>
            <invoker-proxy-binding-name>activemq-message-driven-bean</invoker-proxy-binding-name>
            <container-interceptors>
               <interceptor>org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor</interceptor>
               <interceptor>org.jboss.ejb.plugins.LogInterceptor</interceptor>
               <interceptor>org.jboss.ejb.plugins.RunAsSecurityInterceptor</interceptor>
               <!-- CMT -->
               <interceptor transaction="Container">org.jboss.ejb.plugins.TxInterceptorCMT</interceptor>
               <interceptor transaction="Container">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
               <interceptor transaction="Container" metricsEnabled="true">org.jboss.ejb.plugins.MetricsInterceptor</interceptor>
               <interceptor transaction="Container">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
               <!-- BMT -->
               <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
               <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenTxInterceptorBMT</interceptor>
               <interceptor transaction="Bean">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
               <interceptor transaction="Bean" metricsEnabled="true">org.jboss.ejb.plugins.MetricsInterceptor</interceptor>
               <interceptor>org.jboss.resource.connectionmanager.CachedConnectionInterceptor</interceptor>
            </container-interceptors>
            <instance-pool>org.jboss.ejb.plugins.MessageDrivenInstancePool</instance-pool>
            <instance-cache></instance-cache>
            <persistence-manager></persistence-manager>
            <container-pool-conf>
             <MaximumSize>100</MaximumSize>
            </container-pool-conf>
         </container-configuration>
      </container-configurations>
 </jboss>
Define activemq-jms-ds.xml in JBoss’s deployment directory as follows:


 <?xml version="1.0" encoding="UTF-8"?>

 <!DOCTYPE connection-factories
     PUBLIC "-//JBoss//DTD JBOSS JCA Config 1.5//EN"
     "http://www.jboss.org/j2ee/dtd/jboss-ds_1_5.dtd">

 <connection-factories>

    <tx-connection-factory>
       <jndi-name>activemq/QueueConnectionFactory</jndi-name>
       <xa-transaction/>
       <track-connection-by-tx/> <!-- Thanks to Adrian Brock for pointing this one out! -->
       <rar-name>activemq-ra-4.0.rar</rar-name>
       <connection-definition>javax.jms.QueueConnectionFactory</connection-definition>
       <security-domain-and-application>JmsXARealm</security-domain-and-application>
    </tx-connection-factory>

    <tx-connection-factory>
       <jndi-name>activemq/TopicConnectionFactory</jndi-name>
       <xa-transaction/>
       <track-connection-by-tx/> <!-- Thanks to Adrian Brock for pointing this one out too! -->
       <rar-name>activemq-ra-4.0.rar</rar-name>
       <connection-definition>javax.jms.TopicConnectionFactory</connection-definition>
       <security-domain-and-application>JmsXARealm</security-domain-and-application>
    </tx-connection-factory>

    <!--
    <mbean code="org.jboss.resource.deployment.AdminObject" name="activemq.topic:name=topicSearchRequest">
       <attribute name="JNDIName">topic/search.request</attribute>
       <depends optional-attribute-name="RARName">jboss.jca:service=RARDeployment,name='activemq-ra-4.0.rar'</depends>
       <attribute name="Type">javax.jms.Topic</attribute>
       <attribute name="Properties">PhysicalName=topic.search.request</attribute>
    </mbean>

    <mbean code="org.jboss.resource.deployment.AdminObject" name="activemq.topic:name=topicSearchResponse">
       <attribute name="JNDIName">topic/search.response</attribute>
       <depends optional-attribute-name="RARName">jboss.jca:service=RARDeployment,name='activemq-ra-4.0.rar'</depends>
       <attribute name="Type">javax.jms.Topic</attribute>
       <attribute name="Properties">PhysicalName=topic.search.response</attribute>
    </mbean>
    -->

 </connection-factories>
Compile your code as follows:

javac -d build -classpath $JBOSS_HOME/server/all/lib/jboss-j2ee.jar:$ACTIVEMQ_HOME/lib/optional/spring-1.2.4.jar:\
$ACTIVEMQ_HOME/lib/optional/activemq-optional-4.0.jar:$ACTIVEMQ_HOME/lib/activeio-core-3.0.jar:\
$ACTIVEMQ_HOME/lib/activemq-core-4.0.jar:$ACTIVEMQ_HOME/lib/activemq-console-4.0.jar src/*/*java

Package EJB as follows:

jar -C build -cf mdbtest.jar *

Deploy EJB as follows:

cp mdbtest.jar $JBOSS_HOME/server/all/deploy

Verify that EJB was deployed successfully in JBoss console logs
Run publisher as follows:

java -classpath build:$JBOSS_HOME/server/all/lib/jboss-j2ee.jar:$ACTIVEMQ_HOME/lib/optional/spring-1.2.4.jar:\
$ACTIVEMQ_HOME/lib/optional/activemq-optional-4.0.jar:$ACTIVEMQ_HOME/lib/activeio-core-3.0.jar:\
$ACTIVEMQ_HOME/lib/activemq-core-4.0.jar:$ACTIVEMQ_HOME/lib/activemq-console-4.0.jar client.Publisher

Verify debug message on the JBoss Console
Voila

(Note that you could use xdoclet and ant script, but I tried to focus on integration here)

See inbound-communication.html

See outbound-communication

June 13, 2006

Agile Purists

Filed under: Computing — admin @ 9:26 am

Agile Purists
I find a number of software concepts have religious following where a passionate leader can find a lot of followers even when the ideas are not very sound or have limited application. There are number of examples like OO, agile methodologies, dynamic languages, Microsoft, etc. In a number of cases
purists ignore real world or refuse to have a dialog with opposite side.

I read post of Cedric Beust, in which he raised interesting points about :

  • Tests are (executable) specs.
  • If it’s not testable, it’s useless.

I have been thinking about these as well and I agree that these purists have not seen most of the code in real world. Working for a large eCommerce site with millions of lines of code, I understand that you cannot expect 100% coverage. Though, personally I like writing tests and have even wrote a framework TestPlayer that sniffs legacy applications and automatically generate:

  • Unit Tests with Mocks using jMock, JUnit 3.8.1, JUnit 4.0 and TestNG 4.0
  • Integration Tests using JUnit 3.8.1, JUnit 4.0 and TestNG 4.0
  • Acceptance Tests using FitNesse
  • Stress/Load Tests

The fact that leader of TDD, Kent Beck works for a company that writes tools for exploratory testing (though I am skeptical about it) shows that having 100% coverage is unrealistic. In my experience I have found that 100% coverage takes about 2-3 times the time that it takes to write the software and in some cases up to 4 times. This is where Cedric recommends risk based approach. I also read reply of Uncle Bob where he suggests “Either make sure it works, or don’t ship it. Period.” Again such hard nose
attitude shows little experience with real life code. This tells me that unless you don’t have 100% coverage, dont’ ship your software. Yet, with over ten years of industry experience I have never seen this actually worked. I see buggy code released all the time, it’s just matter of risk factor, i.e., you ship the software as long as there aren’t any serious bugs. Also, when agilists talk about TDD they mean unit tests with mocks, but in real world you also need integrations tests to test against real services and load tests to make sure that you don’t have any unexpected performance problems when the software is shipped. Also, for any online software, it is also important to do security testing or penetration testing.
However, later three get very little attention. In fact, in order to cover all facets, you need to start
with acceptance test (using Fit/FitNesse), then unit tests and followed by integration/performance/security tests. Clearly, you need to balance all this and focus on high risk components of your software or your software may never ship if you stick to 100% coverage. Despite what you think 100% coverage does not mean elimination of your QA department, it’s just that they will be doing a lot more exploratory
testing.

The second point about using tests as specs is equally unrealistic. For any one who has worked with over 500 developers and millions of lines of code, you can’t expect anyone to understand software by reading tests. In fact, I had to learn a number of commercial and open source software where developers had same attitude that just read the code or tests (if you are lucky) and though with enough time you can understand the software by reading code and tests, but with good brief documentation it can be done much faster. Also, in many cases code or tests don’t tell show why that code is there. In many cases, there could be more than one way to do things, without documentation it is hard to see which way is preferred under what conditions.

Though, I like TDD and Agile methodologies specially their emphasis on people and collaboration over processes and technology, but each team and each project is different. It is time that agilists stop proposing prescriptive methodologies. In the end, agility means how fast you respond to change and reduce time to market.

May 28, 2006

Implementing Const in Java

Filed under: Computing — admin @ 12:33 pm

Implementing Const in Java
For those who migrate to Java from C++ (as I did back in ’95), they miss
const keyword, which provides you a nice way to prevent called functions
from mutating objects.

In C++, const keyword like static keyword is a bit overloaded (Note that
static in C++ could mean locally global variable, local variable in method,
shared field/method in class). Similarly, const in C++ can be used for

  • defining Constants, e.g. const int MAX = 100;
  • Protecting pointee in pointer, e.g. int const * ptr
  • Protecting pointer, e.g. int * const ptr
  • Protecting both pointee and pointer, e.g. int const * const ptr
  • Protecting methods from modifying any internal data, e.g.
    int get() const {return x;}
  • Protecting reference, e.g. const AClass& instance

When working on a complicated software with dozens of layers, you need to
define security policies as to what layers can change what information.
Though, C++’s security policies are not adequate, but const gives you some
basic capability.

Though, const is a keyword in Java, but it is not implemented and available
to Java programmers. The closest thing it provides is final, but it is used for :

Here are a couple of ways to add this capability to Java
Suppose you have a Widget class, and accessor and setter methos for
following four attributes:

 1  package widget;
 2
 3  public class Widget {
 4    public Widget() {
 5    }
 6    public Widget(String name, String description, int quantity, double price) {
 7      setName(name);
 8      setDescription(description);
 9      setQuantity(quantity);
10      setPrice(price);
11    }
12    public String getName() {
13      return this.name;
14    }
15
16    public void setName(String name) {
17      this.name = name;
18    }
19
20    public String getDescription() {
21      return this.description;
22    }
23
24    public void setDescription(String description) {
25      this.description = description;
26    }
27
28    public int getQuantity() {
29      return this.quantity;
30    }
31
32    public void setQuantity(int quantity) {
33      this.quantity = quantity;
34    }
35
36    public double getPrice() {
37      return this.price;
38    }
39
40    public void setPrice(double price) {
41      this.price = price;
42    }
43
44    private String name;
45    private String description;
46    private int quantity;
47    private double price;
48  }

and following are major components in the application:

  • web presentation
  • shopping cart application
  • supply chain
  • cms (content management system)
  • persistence

However, the widget name and description can only be changed by CMS component
and price and quantity can only be changed by supply chain system.

1. Adapter Pattern

If the class implements an interface then you can define another class
like WidgetAdapter, e.g.

1         public class WidgetAdapter implements IWidget {
2            public WidgetAdapter(Widget delegate) {
3              this.delegate = delegate;
4            }
5            public void setName(String aName) {
6             throw new IllegalAccessException("name cannot be changed");
7            }
8

If the class does not an interface then you can derive your class from the
Widget class.
Note that both of above approaches require that application layer be aware
of security policies and know how to instantiate these objects. Though, you
can use Factory pattern, but it is still somewhat limited solution.

2. Value pattern

This is similar to above approach except, you define a completely new class
with only the information that you need to pass to another layer. All of the
attributes in value class are final, e.g.

1     public class WidgetValue {
2       public WidgetValue(String name, String description, int quantity, double price) {
3       ...
4       public final String name;
5     }
6 

This approach clearly defines contract between layers. Often objects passed to other layers have very tight data coupling and unnecessary information is passed. With this approach, services clearly identifies what information is needed. The disadvantage of this approach are a number of additional types that look very similar.

3. AOP

This is probably most flexible way to implement security policies as it
allows you to define security policies in one place and does not require
changes to domain object classes. For example,

 1  public aspect WidgetAccessAspect {
 2      declare error
 3          : (call(* widget.Widget.set*(..)))
 4          && !within(CMSManager)
 5          : "Changes to widget name/description are restricted to the CMSManager class";
 6      declare error
 7          : (call(* widget.Widget.set*(..)))
 8          && !within(SupplyChainManager)
 9          : "Changes to widget quantity/price are restricted to the SupplyChainManager class";
10  }

Above aspect will throw compile time errors when any code other than CMSManager tries to modify name and description of the Widget or any code other than SupplyChainManager that tries to modify quantity and price of the widget.Also, above approach can also be used to another nice feature of C++, which is friend keyword. It allows private access to another class. This can be quite handy because often I have to leak encapsulation to unit tests and it would be nice to do this easily.

On the downside, most people are not familiar with AOP and since AOP works like
a magic, it could be intimidating to most people.

4. AccessController/Permission

Another way is to use Java’s security policy to enforce access policies.
However, this requires that each layer be defined in a separate jar file, .e.g

  • application.jar — defines domain classes and core application logic
  • web.jar – presentation logic
  • supply.jar — supply chain
  • cms.jar — CMS

WidgetPermission looks like this:

 1  package widget;
 2 
 3  import java.security.*;
 4 
 5  public class WidgetPermission extends BasicPermission {
 6      public WidgetPermission(String name) {
 7          super(name);
 8      }
 9 
10      public WidgetPermission(String name, String actions) {
11          super(name, actions);
12      }
13  }

Then you can define a java policy file as:

 // or grant signedBy "party" codebase codebase "file://opt/application/lib/cms.jar" {
 grant codebase "file://opt/application/lib/cms.jar" {
     permission widget.WidgetPermission "setName";
     permission widget.WidgetPermission "setDescription";
 };

 grant codebase "file://opt/application/lib/supply.jar" {
     permission widget.WidgetPermission "setQuantity";
     permission widget.WidgetPermission "setPrice";
 };

Then inside Widget class, change setName, setDescription, setQuantity, setPrice as follows:

 1  import java.security.AccessController;
 2 
 3  public class Widget {
 4      private String name;
 5      private String description;
 6      private int quantity;
 7      private double price;
 8 
 9      public void setName(String aName) {
10          AccessController.checkPermission(
11                            new WidgetPermission("setName"));
12          this.name = aName;
13      }
14 
15      public void setDescription(String aDesc) {
16          AccessController.checkPermission(
17                            new WidgetPermission("setName"));
18          this.desc = aDesc;
19      }
20 

5. Combination of AOP and AccessController/Permission

As you can see above code of adding checkPermission can also be done with AOP, e.g.

1  public aspect WidgetAccessAspect {
2      public pointcut setOperations()
3          : execution(* widget.Widget.set*(..));
4 
5      before() : setOperations() {
6          AccessController.checkPermission(
7                           getPermission(thisJoinPointStaticPart));
8      }
9 

May 15, 2006

Is Google Evil?

Filed under: Computing — admin @ 7:44 am

Is Google Evil?
Since its start back in 1998, Google tried to be anti Microsoft and claimed to follow “Don’t be evil”. But as in Orson Welles’s character in Citizen Kane, it’s looking like Google is following Microsoft pursuit after all. Over the years, Google has used PageRank algorithm to censor some sites especially in China, where Chinese government controls access to pages about human rights.
Another source of controversy has been 2038 expiration date that Google uses in its cookie, which could provide a security risk to millions of users. Another risk with giant companies like Google is killing innovation especially in search and ad market where the hegemony of a giant company practically turns into monoply.

Though, most of Google’s profit comes from ads, but it has its eye on the desktop for a while. With its Google Toolbar and Desktop Search, it has complete access on what users are searching and on information stored on the computer. In fact, new version of tool bar uses AutoLink feature to add links to other sites similar to SmartTags features of Microsoft. With GMail and calendar it can read your emails and your schedule. Who knows how Google plans to use all this information. Though, Google has been resisting providing access to the users data to the Government, but it can certainly change in future.

Updated Don’t be fucking evil: Google hands over anonymous Blogger user’s IP address to Israeli court

April 20, 2006

Brainstorm Chicago April 19-20

Filed under: Computing — admin @ 9:35 pm

Brainstorm Chicago April 19-20
I spent yesterday at BPM/SOA conference sponsored by soainstitute.org and bpminstitute.org. There were five different tracks going on: business rules, bpm, process improvements, soa and enterprise architecture. The target audience for
the conference were primarily business side, but there was interesting exchange
of ideas on collaboration between IT and business. A number of people narrated
stories about failed projects due to lack of collaboration. In many instances,
projects were run by technical side without full involvement of business side.
I heard excuses from business side about lack of time, it reminded some of
my own experience and some ongoing work.

There was interesting concepts introduced on Rummler’s methodology on value
chain that tries to integrate vertical division of most organizations with
processes. A lof of information seemed repeat or spinning of old ideas, e.g. BPM echoed BPR initiative of early 90s. Ken Orr contented that SOA could have
been possible in 70s. There were also some good case studies of EA, e.g.
Bill Roth from state of Kansas showed a lot of interesting work. Apparently,
government is promoting a lot of EA work.

I found most interesting session was panel discussion “SOA vs EA”. A lot of
orgnizations are vying for agility and trying to find balance between IT
governance and faster ROI delivery cycle. EA, which is more mature than SOA
can add a lot of processes that can impendiment SOA initiatives. EA folks
considered SOA ugly hack, in fact Ken Orr called it lipstick on a pig.

Though, there was not much technical stuff and being hands on guy it was a
bit disappointment. However, I got free admission due to my professional
membership with Architects Association and day off from work,
so can’t complain.

April 13, 2006

Keeping the truck number low

Filed under: Computing — admin @ 4:02 pm

Keeping the truck number low
A general rule of thumb for most project management and methodologies is to
keep the truck or bus number low for your employees. Often, I have seen
misproportion allocation of skills among tiny number of employees. In a number
of places I worked, where people tried to hide their knowledge and created
cryptic and convoluated processes. They directly tied their knowledge and
skills with their job security and power. Often those people were gate keepers
and became Czars of entire delivery process. The problem becomes worse when
a company uses some proprietary technology and there aren’t enough people
outside to help. Often, this problem is caused by gap between
architects and developers, where software process is
ruled by a small number of architects. They create obscure frameworks that
works like magic, but no one else understands them. In such organizations,
architects treat developers like dumb workers, where developers follow
architects’s processes like McDonald employees follow recipe for big mac.

How can a company get out of this situation: by sharing knowledge across
multiple people and keeping the truck number low. So that there is minimal
impact to the project or organization when that person hit by a truck or
leave the company. Another tip is to use standard and widely used technologies
instead of nitch technology so that there is plenty of help outside. Last,
but not least by empowering developers to make decisions. This is especially
true if a team is following agile methodologies, where the role of
architect is largely diminished and developers use incremental and
evolutionary architecture.

April 2, 2006

Sudoku Solver in Ruby

Filed under: Computing — admin @ 10:37 pm

Sudoku Solver in Ruby
This weekend I spent a couple of hours to write a simple solver for Sudoku
puzzles in Ruby. Despite Here is how the algorithm works:

  • Create a matrix of 9×9
  • Initialize each cell in 9×9 matrix with all possible values, which will
    be 1..9.
  • Load partial solution from a data file specified by the user. The data
    file stores number in simple lines, where each number is separated by
    space and empty cells are represented by 0. For example:

     0 6 0 1 0 4 0 5 0
     0 0 8 3 0 5 6 0 0
     2 0 0 0 0 0 0 0 1
     8 0 0 4 0 7 0 0 6
     0 0 6 0 0 0 3 0 0
     7 0 0 9 0 1 0 0 4
     5 0 0 0 0 0 0 0 2
     0 0 7 2 0 6 9 0 0
     0 4 0 5 0 8 0 7 0

  • Loop – while sudoku puzzle is not solved
    • Normalize grid by comparing the set of possible values for each
      unsolved cell against values in its row, column and minigrid
      and remove those values from the set of possible values.
    • If there are no remaining unsolved cells remaining then quit.
    • Sort grid cells by least number of possible values. This means that
      solved cells will come up first, followed by cells with only two,three
      or four possible values.
    • Pick the cell that is not yet solved and select a number from its
      set of possible values.
    • Go back to loop

Here is complete source code or download it directly from http://bhatti.plexobject.com/sudoku_solver.rb.

  1 #
  2 # == Synopsis
  3 # sudoku_solver.rb solves Sudoku Puzzles.
  4 #
  5 # == Usage
  6 #    ruby sudoku_solver.rb  [ -h | --help ] [ -v | -- verbose ] [ -f | --file inputfile ]
  7 #    The input file stores Sudoku puzzle. Separate each number by a space
  8 #    in the file and use 0 for empty cells.
  9 #
 10 # == Author
 11 #  Shahzad Bhatti
 12 #
 13 # == Copyright
 14 #   None, whatsover.
 15 #
 16 require 'optparse'
 17 require 'rdoc/usage'
 18 require "benchmark"
 19 include Benchmark
 20 
 21 #
 22 ### Cell represents a single cell in the in the Soduku matrix, i.e., there
 23 ### will be 9 cells in a row of matrix and 9 cells in column.
 24 #
 25 class Cell
 26   include Comparable
 27   attr_reader :id
 28 
 29   public
 30   def initialize(id, min, max)
 31     @id = id
 32     @possible_numbers = []
 33     fill(min, max)
 34   end
 35 
 36   #
 37   ### fills internal array values with all possible values
 38   #
 39   def fill(min, max)
 40     for i in 0 ... max
 41       @possible_numbers[i] = i+min
 42     end
 43   end
 44 
 45   #
 46   ### allows sorting based on number of possibilities
 47   ### this will be used so that we sort cells by reducing number of
 48   ### possibilities.
 49   #
 50   def &lt;=&gt;(other)
 51     size &lt;=&gt; other.size
 52   end
 53 
 54   #
 55   ### accessor array
 56   #
 57   def [](indice)
 58     @possible_numbers[indice]
 59   end
 60 
 61   #
 62   ### return number of possible values
 63   #
 64   def size
 65     @possible_numbers.length
 66   end
 67 
 68   #
 69   ### pick any number randomly from the set of possible values
 70   #
 71   def pick_any
 72     num = random_number
 73     if (num &lt; 0)
 74       num = 0
 75     else
 76       num = num % @possible_numbers.length
 77     end
 78     @possible_numbers = [@possible_numbers[num]] if (!solved?)
 79   end
 80 
 81   #
 82   ### settor array operator
 83   #
 84   def set(value)
 85     raise ArgumentError if (value == nil || value.length == 0)
 86     @possible_numbers = value if (value[0] &gt; 0)
 87   end
 88 
 89   #
 90   ### return array of possible values for this cell
 91   #
 92   def get
 93     @possible_numbers
 94   end
 95 
 96   #
 97   ### remove solved solutions from possible values
 98   #
 99   def remove(already_solved)
100     return if (already_solved == nil || already_solved.length == 0)
101     new_values = []
102     for i in 0 ... @possible_numbers.length
103       @possible_numbers.delete_at(i) if (already_solved.include?(@possible_numbers[i]))
104     end
105   end
106 
107   #
108   ### returns true if solved
109   #
110   def solved?()
111     return @possible_numbers.length == 1
112   end
113 
114   #
115   ### stringified array
116   #
117   def to_s
118     buffer = ""
119     for i in 0 ... @possible_numbers.length
120       buffer += @possible_numbers[i].to_s
121     end
122     buffer
123   end
124   #
125   ### generates a random number
126   #
127 
128   private
129   def random_number
130     t = Time.now.to_f / (Time.now.to_f % Time.now.to_i)
131     random_seed = t * 1103515245 + 12345;
132     num = Integer((random_seed / 65536) % 32768)
133   end
134 end
135 
136 #
137 ### A general purpose class to represent multi-dimensional arrays in Ruby
138 #
139 class MultiDimensionalArray
140   #
141   ### multi-dimensional array object
142   #
143   def initialize(*dimensions)
144     @dimensions = Array.new(dimensions.length)
145     @factors = Array.new(dimensions.length)
146     product = 1
147     i = dimensions.length - 1
148     while i &gt;= 0
149       @dimensions[i] = dimensions[i]
150       @factors[i] = product
151       product *= @dimensions[i]
152       i -= 1
153     end
154     @data = Array.new(product)
155   end
156   #
157   ### find offset from Array
158   #
159   def getOffset(indices)
160     raise IndexError if indices.length != @dimensions.length
161     offset = 0
162     for i in 0 ... @dimensions.length
163       if indices[i] &lt; 0 or indices[i] &gt;= @dimensions[i]
164         raise IndexError
165       end
166       offset += @factors[i] * indices[i]
167     end
168     return offset
169   end
170 
171   #
172   ### accessor array
173   #
174   def [](*indices)
175     @data[self.getOffset(indices)]
176   end
177 
178   #
179   ### setter array
180   #
181   def []=(*indicesAndValue)
182     value = indicesAndValue.pop
183     @data[self.getOffset(indicesAndValue)] = value
184   end
185 
186   #
187   ### stringified object
188   #
189   def to_s
190     @data
191   end
192 end
193 
194 #
195 ### This class stores entire Sudoku matrix or matrix, where each cell
196 ### is initialized with partial solution specified by the user (from file).
197 #
198 class Matrix
199   MIN_NUMBER = 1
200   MAX_NUMBER = 9
201 
202   include Comparable
203   attr_reader :cells
204   attr_reader :dims
205   attr_reader :mini_dims
206 
207   public
208   #
209   ### constructor that initializes Sudoku matrix 9x9
210   #
211   def initialize(dims=MAX_NUMBER)
212     @dims = dims
213     @mini_dims = Math.sqrt(dims)
214     @cells = MultiDimensionalArray.new(dims, dims)
215     for i in 0 ... dims
216       for j in 0 ... dims
217         @cells[i, j] = Cell.new(i*dims+j, MIN_NUMBER, MAX_NUMBER)
218       end
219     end
220   end
221 
222   #
223   ### accessor array operator
224   #
225   def [](i, j)
226     @cells[i, j]
227   end
228 
229   #
230   ### return array of solved numbers for given row and column
231   #
232   def get_solved_numbers(row, col)
233     solved = []
234     for i in 0 ... dims
235       solved.push(@cells[row, i][0]) if (@cells[row, i].solved?)
236       solved.push(@cells[i, col][0]) if (@cells[i, col].solved?)
237     end
238     mini = mini_matrix_for_row(row, col)
239     for i in 0 ... mini.dims
240       for j in 0 ... mini.dims
241         solved.push(mini.cells[i, j][0]) if (mini.cells[i, j].solved?)
242       end
243     end
244     solved
245   end
246 
247   #
248   ### settor array operator
249   #
250   def []=(i, j, value)
251     @cells[i, j] = value
252   end
253 
254   #
255   ### return sorted array of cells by reducing possibilities
256   #
257   def as_sorted_array
258     arr = []
259     for i in 0 ... dims
260       for j in 0 ... dims
261         arr.push(@cells[i, j])
262       end
263     end
264     arr.sort
265   end
266 
267   #
268   ### return all cells in a given row
269   #
270   def get_solved_cells_for_row(row)
271     arr = []
272     for i in 0 ... dims
273       arr.push(@cells[row, i]) if (@cells[row, i].solved?)
274     end
275     arr.sort
276   end
277 
278   #
279   ### return all cells in a given row
280   #
281   def get_solved_cells_for_column(col)
282     arr = []
283     for i in 0 ... dims
284       arr.push(@cells[i, col]) if (@cells[i, col].solved?)
285     end
286     arr.sort
287   end
288 
289   #
290   ### This method checks whether the matrix follows Sudoku's rules
291   #
292   def valid?()
293     for i in 0 ... dims
294       arr = get_solved_cells_for_row(i)
295       return false if (duplicates_cells?(arr))
296       arr = get_solved_cells_for_column(i)
297       return false if (duplicates_cells?(arr))
298       arr = get_solved_mini_matrix(i)
299       return false if (duplicates_cells?(arr))
300     end
301     true
302   end
303 
304   #
305   ### This method normalizes the matrix by checking value in each cell that is
306   ### not yet solved and comparing it against all possible values for its
307   ### row and column. Basically, it means that if any of possible numbers
308   ### for that cell exist in its row, column or minimatrix, then that number
309   ### will be removed from possible values.
310   #
311   def normalize()
312     is_solved = true
313     for i in 0 ... @dims
314       for j in 0 ... @dims
315         if (!@cells[i, j].solved?)
316           already_solved = get_solved_numbers(i, j)
317           @cells[i, j].remove(already_solved)
318           is_solved = false
319         end
320       end
321     end
322     is_solved
323   end
324 
325   #
326   ### returns mini matrix
327   #
328   #     [0,0][0,1][0,2]         [0,3][0,4][0,5]         [0,6][0,7][0,8]
329   #     [1,0][1,1][1,2]         [1,3][1,4][1,5]         [1,6][1,7][1,8]
330   #     [2,0][2,1][2,2]         [2,3][2,4][2,5]         [2,6][2,7][2,8]
331   #
332   #     [3,0][3,1][3,2]         [3,3][3,4][3,5]         [3,6][3,7][3,8]
333   #     [4,0][4,1][4,2]         [4,3][4,4][4,5]         [4,6][4,7][4,8]
334   #     [5,0][5,1][5,2]         [5,3][5,4][5,5]         [5,6][5,7][5,8]
335   #
336   #     [6,0][6,1][6,2]         [6,3][6,4][6,5]         [6,6][6,7][6,8]
337   #     [7,0][7,1][7,2]         [7,3][7,4][7,5]         [7,6][7,7][7,8]
338   #     [8,0][8,1][8,2]         [8,3][8,4][8,5]         [8,6][8,7][8,8]
339   #
340   # matrixs are number left to right and top to bottom
341   #
342   def mini_matrix_for_row(row, col)
343     row = Integer(row / @mini_dims)
344     col = Integer(col / @mini_dims)
345     mini_matrix(Integer(row*@mini_dims+col))
346   end
347   #
348   ### returns mini matrix by number
349   #
350   def mini_matrix(num)
351     start_row = Integer(num/@mini_dims) * @mini_dims
352     start_col = Integer(num % @mini_dims * @mini_dims)
353     mini_cells = Matrix.new(@mini_dims)
354     for i in 0 ... @mini_dims
355       for j in 0 ... @mini_dims
356         row = Integer(start_row+i)
357         col = Integer(start_col+j)
358         mini_cells[i, j] =  @cells[row, col]
359       end
360     end
361     mini_cells
362   end
363 
364   #
365   ### returns solved cell values for given mini matrix
366   #
367   def get_solved_mini_matrix(num)
368     arr = []
369     mini = mini_matrix(num)
370     for i in 0 ... mini.dims
371       for j in 0 ... mini.dims
372         arr.push(mini.cells[i, j]) if (@cells[i, j].solved?)
373       end
374     end
375     arr.sort
376   end
377 
378   #
379   ### stringified object
380   #
381   def to_s
382     buffer = ""
383     for i in 0 ... @dims
384       for j in 0 ... @dims
385         buffer += @cells[i, j].to_s + " "
386       end
387       buffer += "rn"
388     end
389     buffer
390   end
391 
392   private
393   def duplicates_cells?(arr)
394     count = Hash.new(0)
395     arr.each { |box|
396       count[box[0]] += 1        # this is only possible with Fixnum
397     }
398     duplicates = false
399     count.each { |key, value|
400       duplicates = true if (value &gt; 1)
401     }
402     duplicates
403   end
404 end
405 
406 #
407 ### This is the real workhorse class that solves Sudoku puzzle.
408 #
409 class Solver
410   attr_reader :matrix
411   attr_reader :data_file
412   attr_reader :verbose
413 
414   public
415   #
416   ### default constructor
417   #
418   def initialize
419     @verbose = false
420     @opts = OptionParser.new
421     @opts.on("-h", "--help") { RDoc::usage }
422     @opts.on("-v", "--verbose") { @verbose = true}
423     @opts.on("-f", "--file FILENAME") {|arg| @data_file = arg}
424     @opts.parse(ARGV) rescue RDoc::usage('usage')
425     die("!!!!!!!!!!! No file specified !!!!!!!!") if @data_file == nil
426     die("!!!!!!!!!!! No such file #{@data_file} !!!!!!!!") if !File.exist?(@data_file)
427     load
428   end
429 
430   #
431   ### Loads partial solution from the file specified by the user.
432   #
433   def load
434     row = 0
435     @matrix = Matrix.new
436     File.open(@data_file, "r") do |file|
437       while line = file.gets
438         next if line =~ /^s*#/
439         nums = line.chomp.split(/[ trn,;:]/)
440         for col in 0 ... nums.length
441           @matrix[row, col].set([Integer(nums[col])])
442         end
443         row = row+1
444       end
445     end
446   end
447 
448   #
449   ### This is the main method for solving Sudoku puzzle. Here is the algorithm:
450   ### - initialize matrix where each cell has 1..9 possible values
451   ### - load partial solution from user's file
452   ### - after loading some of cells will be initialized with user's
453   ###           specified solution value and other will be remain untouched.
454   ### - while not solved
455   ###   - normalize matrix by comparing set of possible values for each
456   ###           unsolved cell against values in its row, column and minimatrix
457   ###           and remove those values from the set of possible values.
458   ###   - if there are no remaining unsolved cells remaining then quit.
459   ###   - sort matrix by least number of possible values. This means that solved
460   ###           cells will come up first.
461   ###   - pick any cell that is not yet solved and select a number from its
462   ###           set of possible values.
463   ###   - go back to loop
464   ###
465   #
466   def solve
467     moves = 0
468     started = Time.now
469     bm(1) do |x|
470       x.report("BM:") {
471         while (!@matrix.normalize())
472           moves += try_solve
473           puts "After try #{moves} moves:n#{@matrix}" if (@verbose)
474         end
475       }
476     end
477     elapsed = (Time.now - started).to_f
478     puts "Solution: after #{moves} moves in #{elapsed}:n#{@matrix}"
479   end
480 
481   private
482   #
483   ### Tries to solve puzzle by first sorting the matrix by least number of
484   ### possible values and then picking any number randomly from the cell
485   ### that is not yet solved.
486   #
487   def try_solve
488     moves = 0
489     sorted = @matrix.as_sorted_array
490     sorted.each do |cell|
491       next if cell.solved?
492       puts "before picking cell #{cell.id} with possible values #{cell}" if (@verbose)
493       prev = cell.get
494       cell.pick_any
495       if (@matrix.valid?)
496         puts "after picking random cell #{cell.id}: with value #{cell}" if (@verbose)
497         return moves
498       end
499 
500       #
501       prev.each do |num|
502         cell.set([num])
503         moves += 1
504         if (@matrix.valid?)
505           puts "after (#{moves}) picking cell #{cell.id}: with value #{cell}" if (@verbose)
506           return moves
507         end
508       end
509       puts "failed to validate after (#{moves}) picking cell #{cell.id}: with value #{cell}" if (@verbose)
510       cell.set(prev)
511     end
512     moves
513   end
514   #
515   ### class method to exit if input arguments are incorrect.
516   #
517   def die(msg = "")
518     puts msg
519     RDoc::usage
520     exit
521   end
522 end
523 
524 Solver.new.solve

Usage:

ruby -w sudoku_solver.rb -v –file your-file

Benchmark: Here is benchmark results for above puzzle

        user     system      total        real
 BM:  8.762000   0.000000   8.762000 (  8.903000)
 Solution: after 771 moves in 8.903:
 9 6 3 1 2 4 7 5 8
 1 7 8 3 7 5 6 2 9
 2 5 5 6 8 9 4 3 1
 8 2 1 4 3 7 5 9 6
 4 9 6 8 5 2 3 1 7
 7 3 5 9 6 1 2 8 4
 5 1 9 7 4 3 8 6 2
 3 8 7 2 1 6 9 4 5
 6 4 2 5 9 8 1 7 3

Future: Note this solution provides one solution, however it can be
easily extended using some backtracking techniques (similar to
some Chess solvers) to provide multiple solutions. By the way, if you
rerun the program with same data, you will see different solutions to
the same problem.

March 30, 2006

Articles from Login Magazine

Filed under: Computing — admin @ 9:23 am

Articles from Login Magazine

March 27, 2006

Nexus for next generation languages

Filed under: Computing — admin @ 7:39 am

Nexus for next generation languages
It’s said every ten to fifteen years you need a new programming language.
Today, Java is over ten years old and one of the most popular language.
Though, C# is another important language, but its design for most part is
similar to Java. A lot of people lately have been trying to find next
generation language. I bought Bruce Tate’s book “Beyond Java” hoping to
get some insight, but was somewhat disappointed as he only covered Ruby and
despite the fact he mentioned other languages like Lisp, Python, Smalltalk,
he didn’t really elaborated. Also, he ignored many other neat languages like
Haskell, Scala, PHP, Groovy. To some extent, a lot of these languages don’t
present fundamentally new idea, but extend a lot of ideas from 70s that were
first used in Lisp and later Smalltalk such as data-driven paradigm using
closures, functional programming, meta programming, and generics programming.
One of the battle going on between widely popular languages like Java/C# and
scripting languages is acceptance of scripting languages in large organizations
or in enterprise environment. Both James Gosling and James McGovern have been
trying to rip scripting languages for their lack of speed and lack of support
for enterprise environment. To some extent, there is truth to that, but as
most of the evangelists for the new scripting languages have been saying is
that not every organization needs sophisticated application servers, messaging
middlewares, EIS or ESB. One of interesting revelation came out in recent
symposium of theserverside, where Dian Almaer took survey of Rails in
production and only found one. Though, according to Mike Clark who was also
there said there were about four. It clearly shows that most people are
looking for bridge to connect with scripting languages.
I think one of the biggest effect of scripting
languages is emphasis on simplicity. It doesn’t matter if Fortune 500 can’t
use these scripting languages due to their complexed environment, but rest of
tens of thousand organizations from small and medium size can greatly get
benefit from them.

One of biggest barrier in acceptance of these scripting languages is lack
of Nexus. If we look at last twenty years, the new languages become widely
popular only when they created powerful nexus between existing language and
the new language. It’s sort of like hand-off from old-generation Star Trek
to new-generation Star Trek. For example, despite the fact C++ was object-oriented, but it accepted C syntax and allowed programmers to slowly adapt to it.
Similarly, Java’s syntax was very similar to C++ and despite the fact that
it was complete object-oriented, it used primitive types to give some compfort
to C++ programmers. This is why next generation language needs a nexus. I
thought Groovy might create that nexus for Java, but was very disappointed.
I like Ruby a lot, but I wish JRuby could be fully compatible and would be
able to run all libraries and frameworks like Rails. This is one of the reason,
I like Scala a lot, because it offers a lot of benefits of data-flow based
programming similar to Lisp, Ruby or Smalltalk, functional programming
similar to Haskell and full Generics support. On top of it, it can run both
on .NET’s CLR and Java’s VM. Unfortunately, it hasn’t got much attention as
other scripting languages like Ruby or Python.

March 24, 2006

Scripting Flame war

Filed under: Computing — admin @ 4:12 pm

There is a heating debates going on between scripting languages such as Python and Ruby against corporate friendly languages like Java and C#.
First, Gostling Claimed on JDJ that scripting languages are very simple and limited their use for simple
web pages. Then he derided scripting languages for lack of performance on Artima’s weblog.
Another flaming post by James McGovern suggested that Ruby is train reck waiting for happen.
Gosling tried to clarify in his blog, but it does not help much. He still tries to corner
scripting languages into domain specific. Despite the performance issues with the scripting languages like Ruby, Python, PHP, and Scheme, they are gaining popularity due to simplification and programmer productivity.
As hardware has been becoming more and more powerful, the performance issues would become less important. There are also plans for Java like VMs in Ruby, which can make it more acceptable to corporate world.
There has been growing trend in solutions that are open source and easier to develop, and many corporations resist for the fear of loosing market. The time is right for scripting languages and I hope languages like
Ruby, Python, Haskell, Scala, Smalltalk or Lisp variants become more acceptable in most places.

« Newer PostsOlder Posts »

Powered by WordPress