Articles from Login Magazine
March 30, 2006
March 27, 2006
Nexus for next generation languages
Nexus for next generation languages
It’s said every ten to fifteen years you need a new programming language.
Today, Java is over ten years old and one of the most popular language.
Though, C# is another important language, but its design for most part is
similar to Java. A lot of people lately have been trying to find next
generation language. I bought Bruce Tate’s book “Beyond Java” hoping to
get some insight, but was somewhat disappointed as he only covered Ruby and
despite the fact he mentioned other languages like Lisp, Python, Smalltalk,
he didn’t really elaborated. Also, he ignored many other neat languages like
Haskell, Scala, PHP, Groovy. To some extent, a lot of these languages don’t
present fundamentally new idea, but extend a lot of ideas from 70s that were
first used in Lisp and later Smalltalk such as data-driven paradigm using
closures, functional programming, meta programming, and generics programming.
One of the battle going on between widely popular languages like Java/C# and
scripting languages is acceptance of scripting languages in large organizations
or in enterprise environment. Both James Gosling and James McGovern have been
trying to rip scripting languages for their lack of speed and lack of support
for enterprise environment. To some extent, there is truth to that, but as
most of the evangelists for the new scripting languages have been saying is
that not every organization needs sophisticated application servers, messaging
middlewares, EIS or ESB. One of interesting revelation came out in recent
symposium of theserverside, where Dian Almaer took survey of Rails in
production and only found one. Though, according to Mike Clark who was also
there said there were about four. It clearly shows that most people are
looking for bridge to connect with scripting languages.
I think one of the biggest effect of scripting
languages is emphasis on simplicity. It doesn’t matter if Fortune 500 can’t
use these scripting languages due to their complexed environment, but rest of
tens of thousand organizations from small and medium size can greatly get
benefit from them.
One of biggest barrier in acceptance of these scripting languages is lack
of Nexus. If we look at last twenty years, the new languages become widely
popular only when they created powerful nexus between existing language and
the new language. It’s sort of like hand-off from old-generation Star Trek
to new-generation Star Trek. For example, despite the fact C++ was object-oriented, but it accepted C syntax and allowed programmers to slowly adapt to it.
Similarly, Java’s syntax was very similar to C++ and despite the fact that
it was complete object-oriented, it used primitive types to give some compfort
to C++ programmers. This is why next generation language needs a nexus. I
thought Groovy might create that nexus for Java, but was very disappointed.
I like Ruby a lot, but I wish JRuby could be fully compatible and would be
able to run all libraries and frameworks like Rails. This is one of the reason,
I like Scala a lot, because it offers a lot of benefits of data-flow based
programming similar to Lisp, Ruby or Smalltalk, functional programming
similar to Haskell and full Generics support. On top of it, it can run both
on .NET’s CLR and Java’s VM. Unfortunately, it hasn’t got much attention as
other scripting languages like Ruby or Python.
March 24, 2006
Scripting Flame war
There is a heating debates going on between scripting languages such as Python and Ruby against corporate friendly languages like Java and C#.
First, Gostling Claimed on JDJ that scripting languages are very simple and limited their use for simple
web pages. Then he derided scripting languages for lack of performance on Artima’s weblog.
Another flaming post by James McGovern suggested that Ruby is train reck waiting for happen.
Gosling tried to clarify in his blog, but it does not help much. He still tries to corner
scripting languages into domain specific. Despite the performance issues with the scripting languages like Ruby, Python, PHP, and Scheme, they are gaining popularity due to simplification and programmer productivity.
As hardware has been becoming more and more powerful, the performance issues would become less important. There are also plans for Java like VMs in Ruby, which can make it more acceptable to corporate world.
There has been growing trend in solutions that are open source and easier to develop, and many corporations resist for the fear of loosing market. The time is right for scripting languages and I hope languages like
Ruby, Python, Haskell, Scala, Smalltalk or Lisp variants become more acceptable in most places.
March 21, 2006
Design by Committee
Design by Committee
Most large organizations have an architecture committee, platform team,
center of excellence or other forms of review committees that determine
vision and direction of software projects along with technology. It is
generally good idea to have a uniform vision and a fewer set of technologies
used across projects in order to minimize maintenance and learning curve.
One of the things that these committees are responsible is the enterprise
architecture. The enterprise architecture includes hardware platform,
software platform, tools, etc. In addition, it also includes application
architecture that determines how the system will be broken into subsystems.
In some respect these committees work similar to industry standard bodies or
consortium of organizations. Depending on size and diversity of these
committees, often the design process becomes somewhat bureaucratic. The
situation can be worse if you follow strict IT governance practices or ITIL
process.
The disagreement often results in lowest common denominator or low
quality solution.
Another frequest observation is that solution is often overly complexed
and over-engineered for the problem domain. Worse, due to diversity, the
architecture is not uniform and consistent. One of the reason for
such inconsistent or complexed architure is pissing contest. In general,
software architects don’t work well with other architects and the design
meetings turn into show-offs for coming up with most clever solution.
On the other hand, an architecture by a single experienced person is often
simple and consistent. The open source community offers many examples
of creating a simple solutions that works bettern than design
by committee’s. The open source community often uses the term “benevolent
dictator” who is incharge of unified vision and architecture. So,
the software project should probably be only designed by a single architect.
Software architecture is still an art that is learned through apprenticeship.
So, it helps if there is an apprentice who can do the grunt work such as
documenting designs and taking care of details. However, I don’t mean an
architect who does not do hands on work, but rather similar to
James O’ Coplien’s pattern “Architects-Also-Implements”.
March 20, 2006
How overwork leads to dumb workers
One of best ideas from agile methodologies is the balance of work and life. In the original XP book, Kent Beck laid down 40-hour week as one of the core practices of XP. However, in the second edition he started the term “Energized Work”, which is a bit vague. I guess if you want to be less controversial then choose more vague terms. Anyway, the idea is pretty much same that knowledge work such as developing software needs a relaxed mind. I have seen everywhere that when you are tired you make stupid mistakes and spend next day or days making up for it.
I read another interesting article Be smarter at work, slack off that reinforces same idea. It quotes Peter Capelli, a professor of management at Wharton, “You can turn a smart person into an idiot just by overworking him.”
Nevertheless, most of the real work still requires 50-60 hours a week and I have seen plenty of death-march projects and fear of loss of jobs to offshore has made things much worse.
March 14, 2006
Log Locally and Query Globally
Log locally and query globally I have found the rule “log locally and query globally” quite handy in large distributed applications. However, I still find many projects that try to create a centralized logging server for receiving all log messages. In many cases this adds too much overhead for logging. Though, asynchronous logging can reduce direct overhead, but it is still problematic. This also results in loss of log messages or confusion when the log server crashes. A single logging server often becomes bottleneck for resources such as network, disk and CPU. I admit I have designed some systems where I setup centralized logging service. In some instances I used wrapper on top of UNIX’s syslog server. One of the drawback of syslog was that it drops messages under heavy load. Though, syslog service provides useful logging for system level logs where you don’t want to log system related messages locally especially security violations because if the local machine is compromised you will loose all logging information. However, I have found for application logging, local logging works much better. On the other hand this requires that a centralized service exist that can search logs from all machines. One of
the difficulty with local logging is creating a complete picture of logs. For example a user request goes throught various services running on different servers and for debugging it is important to trace complete business transaction information. In such cases, I have found a transaction correlation id quite useful. For example, each business transaction id can be stored as part of the log messages or user’s session information can be used, which is later
used to create complete picture. Also, it is important that all servers use NTP service and have their clocks synchronized so that the log messages are in right time order. For query, you can create a local searching agent on each machine which is contacted by centralized query system. This way you can query multiple log files simultaneously and return results much more
efficiently.
March 4, 2006
Responsibility vs Accountability
Based on Jim O’ Coplien’s book on organization patterns, you can not give responsibility to anyone. Only someone can accept it, however you can hold someone accountable. According to Kent Beck, accountability entails reporting back on what happened and responsibility means sticking with the problem in trying to solve it. However, you cannot hold someone accountable if you don’t provide enough resources to him to complete his job. I think this last statement is specially true and I have dealt with numerous situations when management never provided tools or resources to finish the job or do the job right. For example, recently I was trying to track an application bug that caused some financial damage. However, due to complexity of the system, the log files were of terabytes. Worse, there isn’t any data-mining tools to query the logs instead you have to do zgrep on huge compressed files and it takes days to search. In the meantime management keeps bugging when will the financial data be ready. This is another example of an organization where daily firefighting prevents anyone to think about any long term solutions to common problems.
March 2, 2006
Glory of Firefighting
Glory of Firefighting
I have observed in a few dysfunctional organizations where firefighting
is the norm and is rewarded. In such organization normal process of software
development is viewed as mundane and people do poor quality job. However,
they use this low quality process to become heros when crisis emerge. In
one of the organization, they created special roles and teams like SWAT
team that handles crisis or firefighting. In these organizations, emphasis
is on firefighting rather than fire-prevention.