Shahzad Bhatti Welcome to my ramblings and rants!

RailsConf 2006 Day 3
I started today with Beyond DHTML: Introducing Laszlo on Rails by Mike Pence. It was interesting talk, but speaker could not give proper demo due to Internet connection. Overall, I am not convinced why I would need Lazlo, if I can use Ajax especially it is more resource intensive. I wish, I had attended Selinium track because I caught last few minutes and it seemed pretty useful.

Second session was Just the Facts (and Dimensions) — using Rails with your OLAP data model by Ken Kunz. It was slighly useful talk, but I didn’t learn a whole a lot.

Third session was Using Ruby on Rails to Succeed in Selling Music in the 21st Century by Benjamin Curtis. It was by far the best session of the conference and I got a lot of practical information on eCommerce. Though, I have been working for a very large eCommerce, but you don’t always get to see all areas. He also has a book called “Money Train”, though I saw it only had 18 pages. This is one area that I have not found a lot of information in books and tutorials so it might be worth buying it.

Then I attended Deploying Rails Applications by  James Duncan Davidson. Clearly, deployment has been more complicated piece of Rails development and there aren’t any clean solutions available. So, there was good discussion of using Apache/FastCGI, Lighttpd+fcgi. He recommended Capistrano and start small. He also suggested RSS/Campfire to keep track of deployment notifications. He also cautioned with log files, database sockets (firewall), file permissions (public directory). The other solution as few other people mentioned is Mongrel. It also reminded me of proxy servers and reverse proxy servers I used in a number of projects and helped setup in some consulting gigs (and a number of ugly issues along with it [keep-alive] ). I was surprised he ended the talk with using “Container” to tackle these deployment issues. Does this mean that Rails will turn into J2EE?

Final thoughts

This conference was not very well organized and simple things like lack of slides or Internet were bothersome. Also, there were no birds of a feather sessions. I found most of the speakers were not experienced and many of top level leaders of Rails such as DHH, Dave Thomas, Chad Fowler, David Black didn’t have any sessions. The only exceptions were Mike Clark, James Duncan Davidson and Justin Getland and except Justin, the talks of Mike Clark and JDD were very high level (sort of like keynotes). I found many of those sessions were light on content as a number of them had much higher agendas in their talk summaries, but speakers could not finish the APIs, demo, research or work needed. So, overall I found the conference useful, but it didn’t meet expectations. It was sort of brown bags, but didn’t have high caliber speakers like other conference. Also, I am not sure why they didn’t bother inviting Bruce Tate or Jim Weirich.

Since boom of Ruby/Rails, I find that biggest beneficiaries of all this revolution have been either 37signals, Dave Thomas (et al), Thoughtworks and a number of other startups, book publishers and trainers. I have bought almost all of Ruby/Rails books of Pragmatic series (along with other) and in this conference I heard a number of upcoming books from their series (and yes I will buy their books as well). Also, Dave Thomas, Mike Clark and Chad Fowler have pretty good training program going where they charge about $1800 for 3-day class (but they didn’t offer any short sessions from their training program). So, clearly there is a lot of money in it for all these people, but, I hope some of this windfall goes to Yukihiro Matz Matsumoto, the guy behind all this.

Another observation, I found that 90% of people at conference used Mac and I was one of very few people who didn’t have Mac and received a non-conformity certificate:

You can visit flicker for this picture as well and other pictures are available at pictures from Rails conference.

RailsConf 2006 Day 2
I started today with Ajax on Rails by Justin Gehtland. It was excellent talk, though most of it was nothing new. But Justin
gave demo of his upcoming project Streamlined that has scaffold to create views and has builtin
filter and allows managing relationships. It’s going to be very exciting
framework.

I then sat on Lessons from Blinksale and IconBuffet by Scott Raymond. Scott has worked as Rails contractor and he started
with a bit history of Vitruvius and described three tenants of architecture, i.e., firmitas (firmness), utilitas (usefulness) and venustas (beauty/delight). He presented a
number of interesting lessons such as starting project with templates,
mocking templates and then incrementally adding functionality with
very closed loop. He also emphasized testing markups before adding any
code.

Third session, I saw was Putting the BBC’s Programme Catalogue on Rails by Matt Biddulph. That talk was not as good
as I expected, I was hoping to get more practical information on
managing large database, but most of the talk covered history of BBC.
Also, I was shocked that he used ActiveRecord to migrate 2Gig data from
old database from legacy database to mysql. Clearly, an SQL script is
more suitable for such (I guess I would have used stored procedure,
but I am not sure which version of MySQL he used). He admitted, 99%
of database load was spent in creating ActiveRecords.

Then I attended an excellent talk on Rails Deployment by Ezra Zygmuntowicz. He showed a lot of practical tips on deploying
applications such as Apache/FastCGI, Lighttpd, Mongrel. A lot of tips
were from his article Rails Stack. He also plugged his upcoming Pragmatic book on Rails deployment.
Though, his talk summary included BackgrounDRb, but he didn’t talk
about it. Anyway, I look forward to his book.

I then attended talk on Testing Migrations by Glenn Vanderburg. There was a lot of good information on testing
migrations, though a lot of his approaches are still half-baked.

My last talk was by Mike Clark on
Testing Rails Apps
. I didn’t get much new from his talk, it pretty much boiled to:
Rails/Rake has excellent support for testing such as unit, integration,
functional, features and mocks. I don’t understand, why people still
inviting people to use TDD in conferences. By now, most develoeprs
understand benefits of TDD. I would be more interested in new support of
TDD in Rails or tackling difficults issues of testing. For example, I found a couple of articles of
Bruce Tate on Functiona/Integration testing much more useful, but Mike Clark’s
session was very high level sort of like keynote.

Finally, I could not connect to Internet for whole day. I don’t understand,
why RailsConf/Wyndham hotel has such poor Wifi support. My laptop connected to Wifi,
but never could get IP address from DHCP server. And one more thing,
I was forced to pay $50.0 for slides/audio sessions. I wish that was
available freely as we already paid for the conference.

RailsConf 2006 Day 1
I finally was able to find tickets for RailsConf 2006 in Chicago. Though,
I could not find tickets back in Feb., but I managed to find someone who
could not make it. Anyway, I missed most of sessions due to work, but I
was able to checkout “Rails Application Optimization – Techniques and Tools” by Stefan Kaes.
It was excellent talk on profiling Rails applications and improving
performance. He also posted most of tips at A Look at Common Performance Problems in Rails about 1-2 weeks ago. Though, I am disappointed most of tips encourage
bypassing rails framework. I think we need to tackle these problems
at language and framework level and not go around and write ugly hacks.
This also shows that Rails is not really ready for real businesses despite
such claims by rails community.

Second session I attended was Rails Deployment on Shared Hosts by Geoffrey Grosenbach. It was big disappointed, if he only had said “Don’t use shared hosts”, that would have saved me an hour.

I also enjoyed Martin Fowler’s keynote. He admitted, he has never
used Rails, but he is an avid proponent of Ruby. He basically ravished
design principles of Rails, i.e., opinionated software, simplicity,
quick and clean setup (which encourages agile dev. — tracer bullet — fire/aim/fire/aim). He talked about postmodern programming which was interesting. It just means that world isn’t perfect so languages or frameworks that
try to think nothing exist outside their world fails. Clearly, Ruby has
clean design and has a lot of influence from Lisp/Smalltalk, but it also
takes tips from Perl and Python by integrating with other languages and
external environment.

Finally, I found this conference not very well organized. For example,
for popular sessions, there weren’t enough seats and people had to stand
in the doorways. Also, there are no slides handout or conference proceeding.
Worst, organizers are going to charge $50.0 for session slides and audio.
At very least, slides should have made freely available.

Integrating ActiveMQ with JBoss
Recently, I had to integrate ActiveMQ 4 with JBoss 4
Here is how I did it:

Download and install JBoss 4.0.4

set JBOSS_HOME environment variable to point to installation directory of JBoss.

Download and install ActiveMQ 4.0

set ACTIVEMQ_HOME environment variable to point to installation directory of ActiveMQ.

Copy ActiveMQ JCA (rar) file to JBoss’s deploy directory, e.g.

cp $ACTIVEMQ_HOME/lib/optional/optional/activemq-ra-4.0-RC2.rar $JBOSS_HOME/server/all/deploy

Start Active MQ Server

cd $ACTIVEMQ_HOME/bin
sh activemq

Start JBoss Server

cd $JBOSS_HOME
sh bin/run.sh -c all
You need to keep the window open as you will see some debug messages on the console.

Create a folder for all source code, e.g.

mkdir mdbtest
cd mdbtest

Create a source, build and META-INF folder, e.g.

mkdir src
mkdir build
mkdir build/META-INF

Create a source package directory for Message-Driven Bean and Client

mkdir src/mdb
mkdir src/publisher

Define Message Driven Bean, in src/mdb/TestMessageBean.java e.g.

 1  package mdb;
 2  import java.io.PrintStream;
 3  import java.rmi.RemoteException;
 4  import java.util.Hashtable;
 5  import java.util.Properties;
 6  import javax.ejb.CreateException;
 7  import javax.ejb.MessageDrivenBean;
 8  import javax.ejb.MessageDrivenContext;
 9  import javax.jms.JMSException;
10  import javax.jms.Message;
11  import javax.jms.MessageListener;
12  import javax.jms.TextMessage;
13  import javax.naming.InitialContext;
14  import javax.naming.NamingException;
15  import javax.rmi.PortableRemoteObject;
16  import org.apache.commons.logging.Log;
17  import org.apache.commons.logging.LogFactory;
18 
19  /**
20   * @ejb.bean name="TestMessageEJB"
21   */
22  public class TestMessageBean implements MessageDrivenBean, MessageListener {
23    public TestMessageBean() {
24    }
25    public void onMessage(Message message)  {
26      try {
27        TextMessage textMessage = (TextMessage) message;
28        System.out.println(">>>Received " + textMessage.getText());
29      } catch (Exception e) {
30        e.printStackTrace();
31      }
32    }
33 
34    public void ejbRemove() {
35    }
36 
37    public void setMessageDrivenContext(MessageDrivenContext messageDrivenContext) {
38    }
39 
40    public void ejbCreate() {
41    }
42  }

Define Client Publisher in src/client/Publisher.java, e.g.

 1  package client;
 2  import java.io.PrintStream;
 3  import java.util.Properties;
 4  import javax.jms.ConnectionFactory;
 5  import javax.jms.JMSException;
 6  import javax.jms.Message;
 7  import javax.jms.Session;
 8  import javax.jms.TextMessage;
 9  import javax.naming.InitialContext;
10  import javax.rmi.PortableRemoteObject;
11  import org.apache.activemq.ActiveMQConnectionFactory;
12  import org.springframework.jms.core.JmsTemplate;
13  import org.springframework.jms.core.MessageCreator;
14 
15  public class Publisher {
16    private static final String URL = "tcp://localhost:61616";
17    private static final String DEST = "destQ";
18    private static final boolean topic = false;
19    public static void main(String[] args) throws Exception {
20      ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
21      connectionFactory.setBrokerURL(URL);
22      JmsTemplate jt = new JmsTemplate(connectionFactory);
23      if (topic) jt.setPubSubDomain(true);
24      jt.send( DEST, new MessageCreator() {
25        public Message createMessage(Session session) throws JMSException {
26          Message m = session.createTextMessage("hello world");
27            m.setIntProperty("add_id", id);
28           return m;
29          }
30        }
31       );
32    }
33  }

Define META-INF/ejb-jar.xml as follows:

 <?xml version="1.0" encoding="US-ASCII"?>

 <ejb-jar xmlns="http://java.sun.com/xml/ns/j2ee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/ejb-jar_2_1.xsd"
    version="2.1">

    <description> Test ActiveMQ </description>

    <enterprise-beans>
       <message-driven>
          <description>TestMDB</description>
          <ejb-name>TestMessageEJB</ejb-name>
          <ejb-class>mdb.TestMessageBean</ejb-class>
          <messaging-type>javax.jms.MessageListener</messaging-type>
          <transaction-type>Container</transaction-type>
          <activation-config>
             <activation-config-property>
                <activation-config-property-name>destination</activation-config-property-name>
                <activation-config-property-value>destQ</activation-config-property-value>
             </activation-config-property>
             <activation-config-property>
                <activation-config-property-name>destinationType</activation-config-property-name>
                <activation-config-property-value>javax.jms.Queue</activation-config-property-value>
             </activation-config-property>
             <activation-config-property>
                <activation-config-property-name>acknowledgeMode</activation-config-property-name>
                <activation-config-property-value>Auto-acknowledge</activation-config-property-value>
             </activation-config-property>
             <activation-config-property>
                <activation-config-property-name>messageSelector</activation-config-property-name>
                <activation-config-property-value/>
             </activation-config-property>
          </activation-config>
       </message-driven>
    </enterprise-beans>
    <assembly-descriptor>
      <security-role>
         <description>Open</description>
         <role-name>everyone</role-name>
      </security-role>
      <method-permission>
         <role-name>everyone</role-name>
         <method>
           <ejb-name>TestMessageEJB</ejb-name>
           <method-name>*</method-name>
         </method>
      </method-permission>
      <container-transaction>
        <method>
          <ejb-name>TestMessageEJB</ejb-name>
          <method-name>*</method-name>
        </method>
        <trans-attribute>Required</trans-attribute>
      </container-transaction>

       <message-destination>
          <message-destination-name>jbossQ</message-destination-name>
       </message-destination>
    </assembly-descriptor>
 </ejb-jar>

Define META-INF/jboss.xml as follows:

 <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE jboss PUBLIC
       "-//JBoss//DTD JBOSS 4.0//EN"
       "http://www.jboss.org/j2ee/dtd/jboss_4_0.dtd">
   <jboss>
      <jmx-name>activemq.rar:name=iTests</jmx-name>
      <enterprise-beans>
        <message-driven>
          <ejb-name>TestMessageEJB</ejb-name>
          <resource-adapter-name>activemq-ra-4.0.rar</resource-adapter-name>
          <configuration-name>ActiveMQ Message Driven Bean</configuration-name>
        </message-driven>
      </enterprise-beans>
      <assembly-descriptor>
         <message-destination>
          <message-destination-name>jbossQ</message-destination-name>
          <jndi-name>activemq/queue/outbound</jndi-name>
         </message-destination>
      </assembly-descriptor>
      <resource-managers>
         <resource-manager>
            <res-name>queuefactoryref</res-name>
            <res-jndi-name>java:/activemq/QueueConnectionFactory</res-jndi-name>
         </resource-manager>
         <resource-manager>
            <res-name>topicfactoryref</res-name>
            <res-jndi-name>java:/activemq/TopicConnectionFactory</res-jndi-name>
         </resource-manager>
      </resource-managers>

      <invoker-proxy-bindings>
         <invoker-proxy-binding>
            <name>activemq-message-driven-bean</name>
            <invoker-mbean>default</invoker-mbean>
            <proxy-factory>org.jboss.ejb.plugins.inflow.JBossMessageEndpointFactory</proxy-factory>
            <proxy-factory-config>
              <endpoint-interceptors>
                <interceptor>org.jboss.proxy.ClientMethodInterceptor</interceptor>
                <interceptor>org.jboss.ejb.plugins.inflow.MessageEndpointInterceptor</interceptor>
                <interceptor>org.jboss.proxy.TransactionInterceptor</interceptor>
                <interceptor>org.jboss.invocation.InvokerInterceptor</interceptor>
             </endpoint-interceptors>
           </proxy-factory-config>
         </invoker-proxy-binding>
      </invoker-proxy-bindings>

      <container-configurations>
         <container-configuration>
            <container-name>ActiveMQ Message Driven Bean</container-name>
            <call-logging>false</call-logging>
            <invoker-proxy-binding-name>activemq-message-driven-bean</invoker-proxy-binding-name>
            <container-interceptors>
               <interceptor>org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor</interceptor>
               <interceptor>org.jboss.ejb.plugins.LogInterceptor</interceptor>
               <interceptor>org.jboss.ejb.plugins.RunAsSecurityInterceptor</interceptor>
               <!-- CMT -->
               <interceptor transaction="Container">org.jboss.ejb.plugins.TxInterceptorCMT</interceptor>
               <interceptor transaction="Container">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
               <interceptor transaction="Container" metricsEnabled="true">org.jboss.ejb.plugins.MetricsInterceptor</interceptor>
               <interceptor transaction="Container">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
               <!-- BMT -->
               <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
               <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenTxInterceptorBMT</interceptor>
               <interceptor transaction="Bean">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
               <interceptor transaction="Bean" metricsEnabled="true">org.jboss.ejb.plugins.MetricsInterceptor</interceptor>
               <interceptor>org.jboss.resource.connectionmanager.CachedConnectionInterceptor</interceptor>
            </container-interceptors>
            <instance-pool>org.jboss.ejb.plugins.MessageDrivenInstancePool</instance-pool>
            <instance-cache></instance-cache>
            <persistence-manager></persistence-manager>
            <container-pool-conf>
             <MaximumSize>100</MaximumSize>
            </container-pool-conf>
         </container-configuration>
      </container-configurations>
 </jboss>

Define activemq-jms-ds.xml in JBoss’s deployment directory as follows:

 <?xml version="1.0" encoding="UTF-8"?>

 <!DOCTYPE connection-factories
     PUBLIC "-//JBoss//DTD JBOSS JCA Config 1.5//EN"
     "http://www.jboss.org/j2ee/dtd/jboss-ds_1_5.dtd">

 <connection-factories>

    <tx-connection-factory>
       <jndi-name>activemq/QueueConnectionFactory</jndi-name>
       <xa-transaction/>
       <track-connection-by-tx/> <!-- Thanks to Adrian Brock for pointing this one out! -->
       <rar-name>activemq-ra-4.0.rar</rar-name>
       <connection-definition>javax.jms.QueueConnectionFactory</connection-definition>
       <security-domain-and-application>JmsXARealm</security-domain-and-application>
    </tx-connection-factory>

    <tx-connection-factory>
       <jndi-name>activemq/TopicConnectionFactory</jndi-name>
       <xa-transaction/>
       <track-connection-by-tx/> <!-- Thanks to Adrian Brock for pointing this one out too! -->
       <rar-name>activemq-ra-4.0.rar</rar-name>
       <connection-definition>javax.jms.TopicConnectionFactory</connection-definition>
       <security-domain-and-application>JmsXARealm</security-domain-and-application>
    </tx-connection-factory>

    <!--
    <mbean code="org.jboss.resource.deployment.AdminObject" name="activemq.topic:name=topicSearchRequest">
       <attribute name="JNDIName">topic/search.request</attribute>
       <depends optional-attribute-name="RARName">jboss.jca:service=RARDeployment,name='activemq-ra-4.0.rar'</depends>
       <attribute name="Type">javax.jms.Topic</attribute>
       <attribute name="Properties">PhysicalName=topic.search.request</attribute>
    </mbean>

    <mbean code="org.jboss.resource.deployment.AdminObject" name="activemq.topic:name=topicSearchResponse">
       <attribute name="JNDIName">topic/search.response</attribute>
       <depends optional-attribute-name="RARName">jboss.jca:service=RARDeployment,name='activemq-ra-4.0.rar'</depends>
       <attribute name="Type">javax.jms.Topic</attribute>
       <attribute name="Properties">PhysicalName=topic.search.response</attribute>
    </mbean>
    -->

 </connection-factories>

Compile your code as follows:

javac -d build -classpath $JBOSS_HOME/server/all/lib/jboss-j2ee.jar:$ACTIVEMQ_HOME/lib/optional/spring-1.2.4.jar:\
$ACTIVEMQ_HOME/lib/optional/activemq-optional-4.0.jar:$ACTIVEMQ_HOME/lib/activeio-core-3.0.jar:\
$ACTIVEMQ_HOME/lib/activemq-core-4.0.jar:$ACTIVEMQ_HOME/lib/activemq-console-4.0.jar src/*/*java

Package EJB as follows:

jar -C build -cf mdbtest.jar *

Deploy EJB as follows:

cp mdbtest.jar $JBOSS_HOME/server/all/deploy

Verify that EJB was deployed successfully in JBoss console logs

Run publisher as follows:

java -classpath build:$JBOSS_HOME/server/all/lib/jboss-j2ee.jar:$ACTIVEMQ_HOME/lib/optional/spring-1.2.4.jar:\
$ACTIVEMQ_HOME/lib/optional/activemq-optional-4.0.jar:$ACTIVEMQ_HOME/lib/activeio-core-3.0.jar:\
$ACTIVEMQ_HOME/lib/activemq-core-4.0.jar:$ACTIVEMQ_HOME/lib/activemq-console-4.0.jar client.Publisher

Verify debug message on the JBoss Console

Voila

(Note that you could use xdoclet and ant script, but I tried to focus on integration here)

See inbound-communication.html

See outbound-communication

Agile Purists
I find a number of software concepts have religious following where a passionate leader can find a lot of followers even when the ideas are not very sound or have limited application. There are number of examples like OO, agile methodologies, dynamic languages, Microsoft, etc. In a number of cases
purists ignore real world or refuse to have a dialog with opposite side.

I read post of Cedric Beust, in which he raised interesting points about :

• Tests are (executable) specs.
• If it’s not testable, it’s useless.

I have been thinking about these as well and I agree that these purists have not seen most of the code in real world. Working for a large eCommerce site with millions of lines of code, I understand that you cannot expect 100% coverage. Though, personally I like writing tests and have even wrote a framework TestPlayer that sniffs legacy applications and automatically generate:

• Unit Tests with Mocks using jMock, JUnit 3.8.1, JUnit 4.0 and TestNG 4.0
• Integration Tests using JUnit 3.8.1, JUnit 4.0 and TestNG 4.0
• Acceptance Tests using FitNesse
• Stress/Load Tests

The fact that leader of TDD, Kent Beck works for a company that writes tools for exploratory testing (though I am skeptical about it) shows that having 100% coverage is unrealistic. In my experience I have found that 100% coverage takes about 2-3 times the time that it takes to write the software and in some cases up to 4 times. This is where Cedric recommends risk based approach. I also read reply of Uncle Bob where he suggests “Either make sure it works, or don’t ship it. Period.” Again such hard nose
attitude shows little experience with real life code. This tells me that unless you don’t have 100% coverage, dont’ ship your software. Yet, with over ten years of industry experience I have never seen this actually worked. I see buggy code released all the time, it’s just matter of risk factor, i.e., you ship the software as long as there aren’t any serious bugs. Also, when agilists talk about TDD they mean unit tests with mocks, but in real world you also need integrations tests to test against real services and load tests to make sure that you don’t have any unexpected performance problems when the software is shipped. Also, for any online software, it is also important to do security testing or penetration testing.
However, later three get very little attention. In fact, in order to cover all facets, you need to start
with acceptance test (using Fit/FitNesse), then unit tests and followed by integration/performance/security tests. Clearly, you need to balance all this and focus on high risk components of your software or your software may never ship if you stick to 100% coverage. Despite what you think 100% coverage does not mean elimination of your QA department, it’s just that they will be doing a lot more exploratory
testing.

The second point about using tests as specs is equally unrealistic. For any one who has worked with over 500 developers and millions of lines of code, you can’t expect anyone to understand software by reading tests. In fact, I had to learn a number of commercial and open source software where developers had same attitude that just read the code or tests (if you are lucky) and though with enough time you can understand the software by reading code and tests, but with good brief documentation it can be done much faster. Also, in many cases code or tests don’t tell show why that code is there. In many cases, there could be more than one way to do things, without documentation it is hard to see which way is preferred under what conditions.

Though, I like TDD and Agile methodologies specially their emphasis on people and collaboration over processes and technology, but each team and each project is different. It is time that agilists stop proposing prescriptive methodologies. In the end, agility means how fast you respond to change and reduce time to market.