Shahzad Bhatti Welcome to my ramblings and rants!

December 4, 2022

Evolving Software Development Model for Building Distributed Systems

Filed under: Business,Technology — Tags: , — admin @ 5:19 pm

1. Overview

Over the last few decades, the software systems has evolved from the mainframe and client-server models to distributed systems and service oriented architecture. In early 2000, Amazon and Netflix forged ahead the industry to adopt microservices by applying Conway’s law and self-organizing teams structure where a small 2-pizza team own entire lifecycle of the microservices including operational responsibilities. The microservice architecture with small, cross-functional and independent structure has helped team agility to develop, test and deploy microservices independently. However, the software systems are becoming increasingly complex with the Cambrian explosion of microservices and the ecosystem of microservices is reaching a boiling point where building new features, releasing the enhancements and operational load from maintaining high availability, scalability, resilience, security, observability, etc are slowing down the development teams and raising the artificial complexity as a result of mixing different concerns.

Following diagram shows difference between monolithic architecture and microservice architecture:


As you can see in above diagram, each microservice is responsible for managing a number of cross-cutting concerns such as AuthN/AuthZ, monitoring, rate-limit, configuration, secret-management, etc., which adds a scope of work for development of each service. Following sections dive into fundamental causes of the complexity that comes with the microservices and a path forward for evolving the development methodology to build these microservices more effectively.

2. Perils in Building Distributed Systems

Following is a list of major pitfalls faced by the development teams when building distributed systems and services:

2.1 Coordinating Feature Development

The feature development becomes more convoluted with increase in dependencies of downstream services and any large change in a single service often requires the API changes or additional capabilities from multiple dependent services. This creates numerous challenges such as prioritization of features development among different teams, coordinating release timelines and making progress in absence of the dependent functionalities in the development environment. This is often tackled with additional personnel for project management and managing dependencies with Gantt charts or other project management tools but it still leads to unexpected delays and miscommunication among team members about the deliverables.

2.2 Low-level Concurrency Controls

The development teams often use imperative languages and apply low-level abstractions to implement distributed services where each request is served by a native thread in a web server. Due to extensive overhead of native threads such as stack size limits, the web server becomes constrained with the maximum number of concurrent requests it can support. In addition, these native threads in the web server often share a common state, which must be protected with a mutex, semaphore or lock to avoid data corruption. These low-level and primitive abstractions couple business logic with the concurrency logic, which add accidental complexity when safeguarding the shared state or communicating between different threads. This problem worsens with the time as the code size increases that results in subtle concurrency related heisenbugs where these bugs may produce incorrect results in the production environment.

2.3 Security

Each microservice requires implementing authentication, authorization, secure communication, key management and other aspects of the security. The development teams generally have to support these security aspects for each service they own and they have to be on top of any security patches and security vulnerabilities. For example, a zero-day log4j vulnerability in December 2021 created a havoc in most organizations as multiple services were affected by the bug and needed a patch immediately. This resulted in large effort by each development team to patch their services and deploy the patched services as soon as possible. Worst, the development teams had to apply patches multiple times because initial bug fixes from the log4j team didn’t fully work, thus further multiplying the work by each development team. With growth of the dependency stack or bill of material for third party libraries in modern applications, the development teams face an overwhelming operational burden and enormous security risk to support their services safely.

2.4 Web Server

In general, each microservice requires a web server, which adds additional computing and administration overhead for deploying and running the service stack. The web server must be running all the time whether the service is receiving requests or not, thus wasting CPU, memory and storage resources needlessly.

2.5 Colocating Multiple Services

The development teams often start with a monolithic style applications that hosts multiple services on a single web server or with segregated application servers hosting multiple services on the same web server to lessen the development and deployment effort. The monolithic and service colocation architecture hinders speed, agility and extensibility as the code becomes complicated, harder to maintain and reasoned due to lack of isolation. In this style of deployment, computing resources can be entirely consumed by a single component or a bug in one service can crash the entire system. As each service may have unique runtime or usage characteristics, it’s also arduous to scale a single service, to plan service capacity or to isolate service failures in a colocated runtime environment.

2.6 Cross Cutting Concerns

Building distributed systems require managing a lot of horizontal concerns such as security, resilience, business continuity, and availability but coupling these common concerns with the business logic results in inconsistencies and higher complexity by different implementations in microservices. Mixing these different concerns with business logic in microservices means each development team will have to solve those concerns independently and any omission or divergence may cause miserable user experience, faulty results, poor protection against load spikes or a security breach in the system.

2.7 Service Discovery

Though, microservices use various synchronous and asynchronous protocols for communicating with other services but they often store the endpoints of other services locally in the service configurations. This adds maintenance and operational burden for maintaining the endpoints for all dependent services in each development, test and production environment. In addition, services may not be able to apply certain containment or access control policies such as not invoking cross-region service to maintain lower latency or sustain a service for disaster recovery.

2.8 Architectural Quality Attributes

The architecture quality attributes include performance, availability sustainability, security, scalability, fault tolerance, performance, resilience, recovery and usability, etc. Each development team not only has to manage these attributes for each service but often requires coordination with other teams when scaling their services so that downstream services can handle additional load or meet the availability/reliability guarantees. Thus, the availability, fault tolerance, capacity management or other architectural concerns become tied with downstream services as an outage in any of those services directly affect upstream services. Thus, improving availability, scalability, fault tolerance or other architecture quality attributes often requires changes from the dependent services, which adds scope of the development work.

2.9 Superfluous Development Work

When a developing a microservice, a development team owns end-to-end development and release process that includes a full software development lifecycle support such as :

  • maintaining build scripts for CI/CD pipelines
  • building automation tools for integration/functional/load/canary tests
  • defining access policies related to throttling/rate-limits
  • implementing consistent error handling, idempotency behavior, contextual information across services
  • adding alarms/metrics/monitoring/observability/notification/logs
  • providing customized personal and system dashboards
  • supporting data encryption and managing secret keys
  • defining security policies related to AuthN/AuthZ/Permissons/ACL
  • defining network policies related to VPN, firewall, load-balancer, network/gateway/routing configuration
  • adding compression, caching, and any other common pre/post processing for services

As a result, any deviations or bugs in implementation of these processes or misconfiguration in underlying infrastructure can lead to conflicting user experience, security gaps and outages. Some organizations maintain lengthy checklists and hefty review processes for applying best practices before the software release but they often miss key learnings from other teams and slow down the development process due to cumbersome release process.

2.10 Yak shaving when developing and testing features

Due to enormous artificial complexity of microservices with abysmal dependency stack, the development teams have to spend inordinate amount of time in setting up a development environment when building a new feature. The feature development requires testing the features using unit tests with a mock behavior of dependent services and using integration tests with real dependent services in a local development environment. However, it’s not always possible to run complete stack locally and fully test the changes for new features, thus the developers are encumbered with finding alternative integration environment where other developers may also be testing their features. All this yak shaving makes the software development awfully tedious and error prone because developers can’t test their features in isolation with a high confidence. This means that development teams find bugs later in phases of the release process, which may require a rollback of feature changes and block additional releases until bugs are fixed in the main/release branch.

3. Path to the Enlightenment

Following are a few recommendations to remedy above pitfalls in the development of distributed systems and microservices:

3.1 Higher level of Development Abstraction

Instead of using low-level imperative languages or low-level concurrency controls, high-level abstractions can be applied to simplify the development of the microservices as follows:

3.1.1 Actor Model

The Actor model was first introduced in 1973 by Carl Hewitt and it provides a high-level abstraction for concurrent computation. An actor uses a mailbox or a queue to store incoming messages and processes one message at a time using local state in a single green thread or a coroutine. An actor can create other actors or send messages without using any lock-based synchronization and blocking. Actors are reactive so they cannot initiate any action on their own, instead they simply react to external stimuli in the form of message passing. Actors provide much better error handling where applications can define a hierarchy of actors with parent/child relationships and a child actor may crash when encountering system errors, which are monitored and supervised by parent actors for failure recovery.


Actor model is supported natively in many languages such Erlang/Elixir, Scala, Swift and Pony and it’s available as a library in many other languages. In these languages, actors generally use green threads, coroutines or a preemptive scheduler to schedule actors with non-blocking I/O operations. As the actors incur much lower overhead compare to native threads, they can be used to implement microservices with much greater scalability and performance. In addition, message passing circumvents the need to guard the shared state as each actors only maintains a local state, which provides more robust and reliable implementation of microservices. Here is an example of actor model in Erlang language:

-export([init/0, add/1, get/0]).

init() ->
    Pid = spawn(fun() -> loop(0) end),
    register(sumActor, Pid).

loop(N) ->
        {add, X} -> loop(N+X);
        {Client, get} ->
            Client ! N,

add(X) ->
    sumActor ! {add, X}.

get() ->
    sumActor ! {self(), get},
    receive Result -> Result end.

In above example, an actor is spawned to run loop function, which uses a tail recursion to receive next message from the queue and then processes it based on the tag of the message such as add or get. The client code uses a symbol sumActor to send a message, which is registered with a local registry. As an actor only maintains local state, microservices may use use external data store and manage a state machine using orchestration based SAGA pattern to trigger next action.

3.1.2 Function as a service (FaaS) and Serverless Computing

Function as a service (FaaS) offers serverless computing to simplify managing physical resources. Cloud vendors offer APIs for AWS Lambda, Google Cloud Functions and Azure Functions to build serverless applications for scalable workloads. There are also open source support for FaaS computing such as OpenFaas and OpenWhisk on top of Kubernetes or OpenShift. These functions resemble actor model as each function is triggered based on an event and is designed with a single responsibility, idempotency and shared nothing principles that can be executed concurrently.


The FaaS and serverless computing can be used to develop microservices where business logic can be embedded within a serverless functions but any platform services such as storage, messaging, caching, SMS can be exposed via Backend as a Service (BaaS). The Backend as a Service (BaaS) adds additional business logic on top of Platform as a Service (PaaS).

3.1.3 Agent based computing

Microservices architecture decouples data access from the business logic and microservices fetch data from the data-store, which incurs a higher overhead if the business logic needs to fetch a lot of data for processing or filtering before generating results. As opposed, agent style computing allow migrating business logic remotely where data or computing resources reside, thus it can process data more efficiently. In a simplest example, an agent may behave like a stored procedure where a function is passed to a data store for executing a business logic, which is executed within the database but other kind of agents may support additional capabilities to gather data from different data stores or sources and then produces desired results after processing the data remotely.

3.2 Service and Schema Registry

The service registry allows microservices to register the endpoints so that other services can look up the endpoints for communication with them instead of storing the endpoints locally. This allows service registry to enforce any authorization and access policies for communication based on geographic location or other constraints. A service registry may also allow registering mock services for testing in a local development environment to facilitate feature development. In addition, the registry may store schema definitions for the API models so that services can validate requests/responses easily or support multiple versions of the API contracts.

3.3 API Router, Reverse-Proxy or Gateway

API router, reverse-proxy or an API gateway are common patterns with microservices for routing, monitoring, versioning, securing and throttling APIs. These patterns can also be used with FaaS architecture where an API gateway may provide these capabilities and eliminate the need to have a web server for each service function. Thus, API gateway or router can result in lowering computing cost for each service and reducing the complexity for maintaining non-functional capabilities or -ilities.

3.4 Virtualization

Virtualization abstracts computer hardware and uses a hypervisor to create multiple virtual computers with different operating systems and applications on top of a single physical computer.

3.4.1 Virtual Machines

The initial implementation of virtualization was based on Virtual Machines for building virtualized computing environments and emulating a physical computer. The virtual machines use a hypervisor to communicate with the physical computer.

virtual machines

3.4.2 Containers

Containers implement virtualization using host operating system instead of a hypervisor, thus provide more light-weight and faster provisioning of computing resources. The containers use platforms such as Docker and Kubernetes to execute applications and services, which are bundled into images based on Open Container Initiative (OCI) standard.


3.4.3 MicroVM

MicroVMs such as Firecracker and crosVM are based on kernel-based VM (KVM) and use hostOS acting as a hypervisor to provide isolation and security. As MicroVMs only include essential features for network, storage, throttling and metadata so they are quick to start and can scale to support multiple VMs with minimal overhead. A number of serverless platforms such as AWS Lambda, appfleet, containerd,, Kata, Koyeb, OpenNebula, Qovery, UniK, and Weave FireKube have adopted Firecracker VM, which offers low overhead for starting a new virtual machine or executing a serverless workload.

micro virtualmachine

3.4.4 WebAssembly

The WebAssembly is a stack-based virtual machine that can run at the edge or in cloud. The applications written Go, C, Rust, AssemblyScript, etc. are compiled into WebAssembly binary and are then executed on a WebAssembly runtime such as extism, faasm, wasmtime, wamr, wasmr and wagi. The WebAssembly supports WebAssembly System Interface (WASI) standard, which provides access to the systems APIs for different operating systems similar to POSIX standard. There is also an active development of WebAssembly Component Model with proposals such as WebIDL bindings and Interface Types. This allows you to write microservices in any supported language, compile the code into WASM binary and then deploy in a managed platform with all support for security, traffic management, observability, etc.


A number of WebAssembly platforms such as teaclave, wasmCloud, fermyon and Lunatic have also adopted Actor model to build a platform for writing distributed applications.

If WASM+WASI existed in 2008, we wouldn’t have needed to created Docker. That’s how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let’s hope WASI is up to the task!

3.5 Instrumentation and Service Binding

In order to reduce service code that deals specifically with non-functional capabilities such as authentication, authorization, logging, monitoring, etc., the business service can be instrumented to provide those capabilities at compile or deployment time. This means that the development team can largely focus on the business requirements and instrumentation takes care of adding metrics, failure reporting, diagnostics, monitoring, etc. without any development work. In addition, any external platform dependencies such as messaging service, orchestration, database, key/value store, caching can be injected into the service dynamically at runtime. The runtime can be configured to provide different implementation for the platform services, e.g. it may use a local Redis server for key/value store in a hosted environment or AWS/Azure’s implementation in a cloud environment.

When deploying services with WebAssembly support, the instrumentation may use WebAssembly libraries for extending the services to support authentication, rate-limiting, observability, monitoring, state management, and other non-functional capabilities, so that the development work can be shortened as shown below:

wasm platform

3.6 Orchestration and Choreography

The orchestration and choreography allows writing microservices that can be easily composed to provide a higher abstractions for business services. In orchestration design, a coordinator manages synchronous communication among different services whereas choreography uses event-driven architecture to communicate asynchronously. An actor model fits naturally with event based architecture for communicating with other actors and external services. However, orchestration services can be used to model complex business processes where a SAGA pattern is used to manage state transitions for different activities. Microservices may also use staged event-driven architecture (SEDA) to decompose complex event-driven services into a set of stages that are connected by different queues, which supports better modularity and code-reuse. SEDA allows enforcing admission control on each event queue and grants flexible scheduling for processing events based on adaptive workload controls and load shedding policies.

3.7 Automation, Continuous Deployment and Infrastructure as a code

Automation is a key to remove any drudgery work during the development process and consolidate common build processes such as continuous integration and deployment for improving the productivity of a development team. The development teams can employ continuous delivery to deploy small and frequent changes by developers. The continuous deployment often uses rolling updates, blue/green deployments or canary deployments to minimize disruption to end users. The monitoring system watches for error rates at each stage of the deployment and automatically rollbacks changes if a problem occurs.

Infrastructure as code (IaC) uses a declarative language to define development, test and production environment, which is managed by the source code management software. These provisioning and configuration logic can be used by CI/CD pipelines to automatically deploy and test environments. Many cloud vendors provide support for IaC such as Azure Resource Manager (ARM), AWS Cloud Development Kit (CDK), Hashicorp Terraform etc to deploy computing resources.

3.8 Proxy Patterns

Following sections shows implementing cross cutting concerns using proxy patterns for building microservices:

3.8.1 Sidecar Model

The sidecar model helps modularity and reusability where an application requires two containers: application container and sidebar container where sidebar container provides additional functionality such as adding SSL proxy for the service, observability, collecting metrics for the application container.

sidecar pattern

The Sidecar pattern generally uses another container to proxy off all traffic, which enforces security, access control, throttling before forwarding the incoming requests to the microservice.

3.8.2 Service Mesh

The service mesh uses a mesh of sidecar proxies to enable:

  • Dynamic request routing for blue-green deployments, canaries, and A/B testing
  • Load balancing based on latency, geographic locations or health checks
  • Service discovery based on a version of the service in an environment
  • TLS/mTLS encryption
  • Authentication and Authorization
  • Keys, certificates and secrets management
  • Rate limiting and throttling
  • State management and PubSub
  • Observability, metrics, monitoring, logging
  • Distributed tracing
  • Traffic management and traffic splitting
  • Circuit breaker and retries using libraries like FinagleStubby, and Hysterix to isolate unhealthy instances and gradually adding them back after successful health checks
  • Error handling and fault tolerance
  • Control plane to manage routing tables, service discovery, load balancer and other service configuration
Service Mesh

The service mesh pattern uses a data plane to host microservices and all incoming and outgoing requests go through a sidecar proxy that implements cross cutting concerns such as security, routing, throttling, etc. The control-plan in mesh network allows administrators to change the behavior of data plane proxies or configuration for data access. Popular service mesh frameworks include Consul, Distributed Application Runtime (Dapr), Envoy, Istio, Linkerd, Kong, Koyeb and Kuma for providing control-plane and data-plane with builtin support for networking, observability, traffic management and security. Dapr mesh network also supports Actor-model using the Orleans Virtual Actor pattern, which leverages the scalability and reliability guarantees of the underlying platform.

3.9 Cell-Based Architecture

A Cell-based Architecture (CBA) allows grouping business functions into single units called “cells”, which provides better scalability, modularity, composibility, disaster-recovery and governance for building microservices. It reduces the number of deployment units for microservices, thus simplifying the artificial complexity in building distributed systems.

cell-based architecture

A cell is an immutable collection of components and services that is independently deployable, manageable, and observable. The components and services in a cell communicate with each other using local network protocols and other cells via network endpoints through a cell gateway or brokers. A cell defines an API interface as part of its definition, which provides better isolation, short radius blast and agility because releases can be rolled out cell by cell. However, in order to maintain the isolation, a cell should be either self contained for all service dependencies or dependent cells should be on the same region so that there is no cross region dependency.

cell-based data-plane

3.10 12-Factor app

The 12-factor app is a set of best practices from Heroku for deploying applications and services in a virtualized environment. It recommends using declarative configuration for automation, enabling continuous deployment and other best practices. Though, these recommendations are a bit dated but most of them still holds except you may consider storing credentials or keys in a secret store or in an encrypted files instead of simply using environment variables.

4. New Software Development Workflow

The architecture patterns, virtual machines, WebAssembly and serverless computing described in above section can be used to simplify the development of microservices and easily support both functional and non-functional requirements of the business needs. Following sections describe how these patterns can be integrated with the software development lifecycle:

4.1 Development

4.1. Adopting WebAssembly as the Lingua Franca

WebAssembly along with its component model and WASI standard has been gaining support and many popular languages now can be compiled into wasm binary. Microservices can be developed in supported languages and ubiquitous WebAssembly support in production environment can reduce the development and maintenance drudgery for the application development teams. In addition, teams can leverage a number of serverless platforms that support WebAssembly such as extism, faasm, netlify, vercel, wasmcloud and wasmedge to reduce the operational load.

4.1.2 Instrumentation and Service Binding

The compiled WASM binary for microservices can be instrumented similar to aspects so that the generated code automatically supports horizontal concerns such as circuit breaker, diagnostics, failure reporting, metrics, monitoring, retries, tracing, etc. without any additional endeavor by the application developer.

In addition, the runtime can use service mesh features to bind external platform services such as event bus, data store, caching, service registry or dependent business services. This simplifies the development effort as development team can use the shared services for developing microservices instead of configuring and deploying infrastructure for each service. The shared infrastructure services support multi-tenancy and distinct namespaces for each microservice so that it can manage its state independently.

4.1.3 Adopting Actor Model for Microservices

As discussed above, the actor model offers a light-weight and highly concurrent model to implement a microservice APIs. The actors based microservices can be coded in any supported high-level language but then compiled into WebAssembly with a WASI support. A number of WebAssembly serverless platforms including Lunatic and wasmCloud already support Actor model while other platforms such as fermyon use http based request handlers, which are invoked for each request similar to actors based message passing. For example, here is a sample actor model in wasmCloud in Rust language though any language with wit-bindgen is supported as well:

#[derive(Debug, Default, Actor, HealthResponder)]
#[services(Actor, HttpServer)]
struct HelloActor {}

impl HttpServer for HelloActor {
    async fn handle_request(
        _ctx: &Context,
        req: &HttpRequest,
    ) -> std::result::Result<HttpResponse, RpcError> {
        let text=form_urlencoded::parse(req.query_string.as_bytes())
            .find(|(n, _)| n == "name")
            .map(|(_, v)| v.to_string())
            .unwrap_or_else(|| "World".to_string());
        Ok(HttpResponse {
            body: format!("Hello {}", text).as_bytes().to_vec(),

The wasmCloud supports Contract-driven design and development (CDD) using Wasmcloud interfaces based on smithy IDL for building microservices and composable systems. There is also a pending work to support OpenFaas with wasmCloud to invoke functions on capability providers with appropriate privileges.

Following example demonstrates similar capability with fermyon, which can be deployed to Fermyon Cloud:

use anyhow::Result;
use spin_sdk::{
    http::{Request, Response},
fn hello_rust(req: Request) -> Result<Response> {
    println!("{:?}", req.headers());
        .header("foo", "bar")
        .body(Some("Hello, Fermyon".into()))?)

Following example shows how Dapr and WasmEdge work together to support lightweight WebAssembly-based microservices in a cloud-native environment:

fn main() -> std::io::Result<()> {
    let port = std::env::var("PORT").unwrap_or(9005.to_string());
    println!("new connection at {}", port);
    let listener = TcpListener::bind(format!("{}", port))?;
    loop {
        let _ = handle_client(listener.accept()?.0);

fn handle_client(mut stream: TcpStream) -> std::io::Result<()> {
  ... ...

fn handle_http(req: Request<Vec<u8>>) -> bytecodec::Result<Response<String>> {
  ... ...

The WasmEdge can also be used with other serverless platforms such as Vercel, Netlify, AWS Lambda, SecondState and Tencent.

4.1.4 Service Composition with Orchestration and Choreography

As described above, actors based microservices can be extended with the orchestration patterns such as SAGA and choreography/event driven architecture patterns such as SEDA to build composable services. These design patterns can be used to build loosely coupled and extensible systems where additional actors and components can be added without changing existing code.

4.2 Deployment and Runtime

4.2.1 Virtual Machines and Serverless Platform

Following diagram shows the evolution of virtualized environments for hosting applications, services, serverless functions and actors:

Evolution of hosting apps and services

In this architecture, the microservices are compiled into wasm binary, instrumented and then deployed in a micro virtual machine.

4.2.2 Sidecar Proxy and Service Mesh

Though, the service code will be instrumented with additional support for error handling, metrics, alerts, monitoring, tracing, etc. before the deployment but we can further enforce access policies, rate limiting, key management, etc. using a sidecar proxy or service mesh patterns:

For example, WasmEdge can be integrated with Dapr service mesh for adding building blocks for state management, event bus, orchestration, observability, traffic routing, and bindings to external services. Similarly, wasmCloud can be extended with additional non-functional capabilities by implementing capability provider. wasmCloud also provides a lattice, self-healing mesh network for simplifying communication between actors and capability providers.

4.2.3 Cellular Deployment

As described above, Cell-based Architecture (CBA) provides better scalability, modularity, composibility and business continuity for building microservices. Following diagram shows how above design with virtual machines, service-mesh and WebAssembly can be extended to support cell-based architecture:

In above architecture, each cell deploys a set of related microservices for an application that persists state in a replicated data store and communicate with other cells with an event-bus. In this model, separate cells are employed to access data-plane services and control-plane services for configuration and administration purpose.

5. Conclusion

The transition of monolithic services towards microservices architecture over last many years has helped development teams to be more agile in building modular and reusable code. However, as teams are building a growing number new microservices, they are also tasked with supporting non-functional requirements for each service such as high availability, capacity planning, scalability, performance, recovery, resilience, security, observability, etc. In addition, each microservice may depend on numerous other microservices, thus a microservice becomes susceptible to scaling/availability limits or security breaches in any of the downstream services. Such tight coupling of horizontal concerns escalates complexity, development undertaking and operational load by each development team resulting in larger time to market for new features and larger risk for outages due to divergence in implementing non-functional concerns. Though, serverless computing, function as a service (Faas) and event-driven compute services have emerged to solve many of these problems but they remain limited in the range of capabilities they offer and lack a common standards across vendors. The advancements in micro virtual machines and containers have created a boon to the serverless platforms such as appfleet, containerd,, Kata, Koyeb, OpenNebula, Qovery, UniK, and Weave FireKube. In addition, widespread adoption of WebAssembly along with its component model and WASI standard are helping these serverless platforms such as extism, faasm, netlify, vercel, wasmcloud and wasmedge to build more modular and reusable components. These serverless platforms allow the development teams to primarily focus on building business features and offload all non-functional concerns to the underlying platforms. Many of these serverless platforms also support service mesh and sidecar patterns so that they can bind platform and dependent services and automatically handle concerns such as throttling, security, state management, secrets, key management, etc. Though, cell-based architecture is still relatively new and is only supported by more matured serverless and cloud platforms, but it further raises scalability, modularity, composibility, business continuity and governance of microservices. As each cell is isolated, it adds agility to deploy code changes to a single cell and use canary tests to validate the changes before deploying code to all cells. Due to such isolated deployment, cell-based architecture reduces the blast radius if a bug is found during validation or other production issues are discovered. Finally, automating continuous deployment processes and applying Infrastructure as code (IaC) can simplify local development and deployment so that developers use the same infrastructure setup for local testing as the production environment. This means that the services can be deployed in any environment consistently, thus reduces any manual configuration or subtle bugs due to misconfigurations.

In summary, the development teams will greatly benefit from the architecture patterns, virtualized environments, WebAssembly and serverless platforms described above so that application developers are not burdened with maintaining horizontal concerns and instead they focus on building core product features, which will be the differentiating factors in the competing markets. These serverless and managed platform not only boosts developer productivity but also lowers the infrastructure cost, operational overhead, cruft development work and the time to market for releasing new features.

September 17, 2012

Tips from Unusually Excellent: The Necessary Nine Skills Required for the Practice of Great Leadership

Filed under: Business — admin @ 2:17 pm

I recently read Unusually Excellent: The Necessary Nine Skills Required for the Practice of Great Leadership. Here are a few tips I enjoyed from the book:


Earning the Right to Lead Through Character

This book shows that in order to gain credibility, you need to be authentic, trustworthy, and have character traits such as courage, integrity, and commitment:

  • Being Authentic
  • Look at Life: Seeing Who You Are
  • Owning Your Past: The Sting of Failure – Adversity demands more of us than normal times do.
  • One Day at a time – An Unexpectedly Bad Day
  • Share the Shame – You can share a couple of your past disappointments with your team mates to connect with them on a personal level
  • Face Time – Meet face to face to build personal relationships
  • The Perception Gap – Get feedback on how others see you
  • The Courage to Listen
  • Honest Feedback – great leaders don’t avoid conflict and give honest feedback but at the same time be authentic and professional.

Being Trustworthy

The book shows that great leaders build a track record of honesty, fairness, and integrity that creates a leadership “equity” within their constituency. Trustworthiness takes precedence over heavyweight attributes like creativity and intelligence.

  • Safely Successful – physical, emotional and professional safety is primarl need.
  • Be honest – match your actions with your words and match those words with the truth we see in the world (no spin).
  • Be vulnerable – showing your weakness or raw emotion
  • Be fair
  • A better place for all – The book recommends building trusted interpersonal relationships that have commitments to work and loyality. On the other hand fear inspires defensive behavior, which leaders can eliminate fear by being transparent, crystal clear, and integrity.
  • A Culture of Trust Is a Culture of Truth – One reason people within enterprises fear telling the truth to each other and to their bosses is that they know the organization cannot properly distinguish between the message and the messenger.
  • Bad News Doesn’t Swim Upstream
  • A Culture of Trust Is a Culture of Innovation – Trust is the basis of safety. Create trust, and you’ll create a safe place to take risks and in turn build culture of innovation. The organizations should not punish “good failure”
  • A Culture of Trust Is a Culture of Performance – you should never punish a good person for delivering bad news—or even, on occasion, bad work.
  • Take Your Pain Quickly and Acutely—and Move On

Being Compelling – Commitment to Winning

The book shows that great leaders evoke the emotion and energy of being involved in a crusade. No one will sacrifice for a project if the leader hasn’t made a full and clear—and public—commitment. Great leaders don’t want to be merely an employee instead they want to be part of a team, working together to create something important.

  • Choice and Obligation – The best, most talented followers are really volunteers, and because of those very attributes they are often in considerable demand elsewhere.
  • Attracting the Best and Brightest – Great leaders engage and listen to people.
  • Keeping Your Best on Board
  • Cheerleader
  • Tell Me the Truth – the best people actually find reality, even if it is bad news, compelling.
  • Keep Me Challenged – Talented people want and need challenging work.
  • No Hard Feelings – leaders must be able to stand in their followers’ shoes and see themselves from that viewpoint.

Competence: Leading on the Field with Skill

  • Leading People Talent to Teams – Hiring great people is arguably the highest-leverage activity that leaders undertake.
  • Seating Chart – talent is useless if the person is not a good match with that role and responsibility and a specific place in the structure of the organization.
  • People First – hire the very best people; only then should you focus on building the right plan for the organization.
  • Engagement – engage people by setting realistic goals with them and fairly rewarding them for meeting or exceeding those expectations.
  • Enrollment
  • Expectations
  • Energy – functional, emotional and career energy.
  • Empowerment – delegate power to other people
  • Retreat to Attack
  • How Has the Nature of Your Enterprise Changed? – As a leader, if you have not prepared your people for that change or you resist that change, you have failed in your responsibilities.
  • Where Is Your Authority or Positional Power Best Used in Leading People? – By carefully setting performance expectations with your key team members, you move the whole game up a notch.
  • What Is Your Plan to Deal with Your Weakest Link? – being aware of poorly performing subordinate and acting on it instead of avoiding it.
  • Will You Distinguish the Bad Performer from a Bad Plan? – think like a venture capitalist, with your project leaders as the entrepreneurs and the project itself a new venture. Have a post-mortem and inquire why project failed.

Leading Strategy – ideas to plans

Leaders need to distinct between leading people, strategy and execution.

  • Process to Plans – plan shows what needs to be done, where as trategy is bigger than plan and includes how things are done and fallback options.
  • The Process: Inclusive and Collaborative – The process must include the best people and the best ideas, from both within and outside the company, and must foster collaborative thinking and constructive, rigorous discussion.
  • Winnowing Out a Plan – solicit ideas from others when you don’t know the domain
  • The Plan: Realistic and Compelling – The book shows that leaders need to be engaged throughout the process to make sure the process moves along with appropriate energy and that the team remains realistic in terms of time, resources, and goals.
  • Stickiness – commitment in the face of adversity

Leading Execution – actions to results

Execution is about results. Leaders need to distinct between leading people, strategy and execution. Execution provides feedback that can be measured against plans.

  • Solve the Hard Problems First – don’t distract yourself with second-tier tasks
  • At the Edges – In order to build high-reliability organizations (e.g. SWAT), you need zero tolerance team execution, which require:
    • reliable communications
    • continuous training
    • standardize and synchronize
    • mission-goal clarity and loyality
    • empower the front line
    • redundancy
  • Leadership Leverage in Execution – The book suggests leading the process and setting the standards for the right goals. This includes leading the design process to create the appropriate metrics, ensuring a winner’s commitment and making sure that attitude permeates the culture.
  • Curb Your Enthusiasm: Focus, Commit, and Deliver – don’t overcommit and follow the rule of “first things first.”
  • It Isn’t Real If You Don’t Measure It – Measuring what matters is an extremely high-leverage opportunity. Use management by objectives (MBO), “as measured by” (AMB) or a key performance indicator (KPI) processes for measureing factors that correlate very highly with winning.
  • Let the Dashboard Drive – Measuring what matters to naturally direct attention, focus, and commitment to the right activities
  • It’s Just Like Pinball: If You Win, You Get to Play Again
    • Winner’s mindset
    • Failing elgantly – No lame excuses
  • Sloppiness – HRO never allow sloppiness because they know it equals death. The book shows that leaders may
    feel like part of being a nice guy, succumbing to that temptation promotes a culture of mediocrity.

  • Performance Feedback – look for data coming back from the field.

Consequence: Creating a Culture, Leaving a Legacy of Values

Trust is the most fragile of assets; at a certain point, different in every situation.

Legacy = Culture + Reputation

A Leader’s Communication

  • Open, Honest Dialogue – The book shows that the ability of leaders to communicate effectively is highest leverage activity in their set of responsibilities and should include:
    • What are we doing? (Vision and mission.)
    • Why are we doing it? (Purpose and goals.)
    • What’s the plan to win?
    • (What’s the strategy here?)
    • How are we doing? (Results and status—health of the business.)
    • What is my part in the game? (What do you expect from me?)
    • What’s in it for me? (Why is this a compelling place for me to be?)
    • How am I doing? (Give me feedback, acknowledgment, appreciation.)

Talking Trust

In order to build trust, leaders not only need to focus on contents but also emotional content of that message and the
connection—the leader’s empathy with the audience.

Checklists and Guideposts

Here are some key points from the book:

  • communication is a core responsibility of leading
  • most of the important things in organizations are the result of the right conversation
  • starving followers from basic information will result in high cost

Here are five C’s for What question leader needs to communicate:

  • A compelling cause
  • Credibility & Competence
  • Character
  • Commitment
  • Contribution

Here are five E’s for Why question leader needs to communicate:

  • Engagement
  • Enrollment
  • Energy
  • Empowerment
  • Endorsement

Here are six C’s for When question leader needs to communicate:

  • Context
  • Confidence
  • Challenge
  • Collaboration
  • Culture
  • Coaching

Here are seven C’s for How question leader needs to communicate:

  • Clarity
  • Consistency
  • Carefulness
  • Courage
  • Conviction
  • Compassion
  • Completion

The Solitary Touch

The book shows that there is really no such thing as a “casual” conversation.

Your 24 × 7 Job

The book shows leader has three basic tasks:

  • Align the interests, energy, and commitment of the team.
  • Reduce fear, confusion, and anxiety.
  • Instill confidence and trust, while rallying support and contributions.

A Leader’s Decision Making Values-Based Choices

The book shows that leader does not need to make most of the decisions, but need to help followers make make better decisions.

Decision Structure

  • What Exactly Are We Deciding?
  • What Flavor Is This Decision? – decisions can be classified as either simple or complex. You need sufficient data to make the decision, otherwise you have to use intuition. Decisions can also be characterized as easy or difficult.
  • When Does This Decision Need to Be Made?

    Total Cycle Time = Time to Decide + Time to Commit + Time to Execute

  • Who Should Make This Decision? – who is best equipped—by skill, experience, proximity
  • Don’t Wait; Decide
  • Chasing Decisions – communicate with followers and empower them to make decisions

A Leader’s Impact The Transfer of Influence from Leader to Follower

Finally, the book shows how to build lasting legacy and reputation:

  • Leader Taking the High Ground
  • Whisper Campaign – use public forums to acknowledge accomplishments, sacrifices and courage. Also, appreciate them in private.
  • All You Leave Behind – using exit interviews to get feedback
  • Collective Memory
  • What to Do
  • Pay attention to change
  • Get More Curious, and Smarter, About Human
  • Nature – leaders tend to gravitate toward the objective and away from the subjective.
  • Give feedback
  • Celebrate success
  • Respect Life Outside of Work
  • Your Greatest Legacy

May 22, 2012

Review of ‘Good Boss, Bad Boss: How to Be the Best… and Learn from the Worst’

Filed under: Business — admin @ 4:27 pm

I recently finished Bob Sutton’s book Good Boss, Bad Boss, who is well known for his book The No Asshole Rule: Building a Civilized Workplace and Surviving One That Isn’t . As most of us, I have many bosses and also manage other people so I have found this book quite useful. Good bosses not only help productivity and work environment but they also reduce stress, diseases or family troubles.

Bob shows that good boses apply Lasorda’s law and use less management, however they don’t ignore their people and help them out. Also, good bosses have mentality of running marathon rather than sprint and they instill grit in followers, where they push them to try a bit harder and be more creative. Bob suggests using small wins and manageable tasks to drive focus and sense of accompllishment in followers. Bob warns against bosses with attitude of toxic tandems, who are self absorbed. Good bosses also back their followers and balance performance and humanity by helping people to do great work and experience pride and dignity.

Here are highlights from Bob’s book:

Take Control

The media generally portrays leaders as heros, but research shows that most bosses have little impact on overall performance of a company. Good bosses use this illusion to their advantage to bring confidence in their followers and increasing odds of their success.

Don’t Dither

Good bosses use crisp language and decide unequivocally, however they are not afraid to change their decisions. They follow the rule of strong opinions that are weakly held.

Get/Give Credit

Bosses get credit no matter what but good bosses also give credit to others. I have worked in environments, where bosses took all the glory and passed shit to their followers. However, everyone wins if boss give credits as much as possible.

Blame yourself

Good bosses also take the heat for team, which builds loyalty of their followers.

Strive to be Wise

Good bosses create balance of over confidence and healthy dose of self doubt. They ask questions and listen instead of talking too much. Wise bosses assume best from their people and show them compassion and love.

Forgive and Remember

Wise bosses forgive and remember the mistakes so that they can learn from them instead of blaming followers or forgetting them altogether.

Safety & Creativity

Wise bosses create safe environment to share ideas and be more creative. They fight for what they believe in but gracefully accept defeat.


Wise bosses ask good questions, listen and ask for help. They show empathy, compassion and gratitude to their followers. They know their flaws and work with other people to compensate for their weaknesses.

Stars & Rotten Apple

Some organizations glorify solo stars, which undermines team collaboration. Good bosses recruit energizers and eliminate bad apples or energy suckers, who undermine constructive actions.

Keep teams together

Good bosses keep teams together. I have found that a new team takes a couple of months to gel, and having worked in project-based teams (which was awful) I take this advice to the heart.

Link Talk & Action

Good bosses say same simple things and build harmony between their actions and words. They empathizes with their customers by eating their own dog food. They try to reduce complexity and use simple principles, strategies, and metrics.

Don’t shirk Dirty Work

Good bosses confront problems directly, which may include personnel problems such as firing low performer or bad apple. They create realistic expectations for followers and make tough decisions, however they make those decisions with understanding, control and compassion.

Squelch your inner Bosshole

Let’s face it, there are plenty bosses who act like assholes. Unfortunately, most of them are unaware of their attitude and habits. Bossholes create negative work environment and cause health problems for their followers. I have worked in companies, where this was cultural issue and the role models of bossholes was passed from top-down. Nevertheless, this is often the cause of employees leaving companies or having heart attacks. Bob suggests a couple of solutions such as tape method to help manage anger.


As we spend most part of day at work, it helps if the work environment and the boss is empathetic. Bob provides a lot of advice to bosses so that they can build better work environment for their followers.

April 26, 2012

How to survive in today’s work environments and businesses

Filed under: Business — admin @ 10:39 pm

In modern work environments and businesses, it is crucial to expedite learning cycle and create a knowledge workplace. In order to shorten the learning cycle, you can apply the scientific approach, which comprises of three stages, i.e., making a hypothesis, testing the hypothesis and validating test results. Generally, the test results leads to another hypothesis, and another cycle of experiment, validation and then publishing results. At the end of each cycle, you learn something knew, thus the shorter your cycle the faster you learn. Following are few variations of this approach used in manufacturing, research and development:


John Boyd revolutionized military aviation strategy by designing lightweight fighter planes that were based on a shorter loop of

  • Observe
  • Orient
  • Decide
  • Act

John showed that light-weight planes that provided quick feedback to pilots proved better in dogfights. It showed that speed of loop beats quality of iteration.

Theory of Constraint

The theyory of constraint creates flow of an activity and finds all constraints from end to end and then tries to remove biggest constraint. It then repeats the process and identifies next biggest constraint and solves that constraint.

Lean Manufacturing

Lean manufacturing is based on Toyota production system that emphasizes waste elimination and creating value for customers. Lean production system stresses reducing activity time from start to finish and continuous improvement or Kaizen.

Lean Software Development

Lean software development applies lessons of Toyota production systems to software development by eliminating waste, reducing in-progress work (inventory), and amplifying learning. It speeds up learning process by short cycle of iterative development.

Lean Startup

The startups generally start with visions or hypothesis that are unproven. Lean startups borrow ideas from Toyota production systems by creating rapid prototypes that test market assumptions and uses customer feedback evolve the product. Just like the scientific approach, it uses a cycle of product-idea or hypothesis, product-development or preparing an experiment, releasing the product, where startups collects data about value to users. The test results help startups adjust the product and another round of tests are followed. The Lean Startups emphasized validated learning that you gain from running actual experiment than just guess work.

Spiral Methodology

Spiral methodology is a software development process that uses iterative development and encourages prototyping and experimenting. Each iteration starts with objectives, constraints, and alternatives, which are then evaluated, developed and then validated.

Scrum Methodology

Scrum is an iterative and incremental agile software methodology for project management. It uses short sprints for software development, where each sprint starts with planning meeting that defines the user stories or features to be delivered, followed by development and release. At the end of sprint, the team holds Sprint review meeting and retrospective to identify impediments so that team can improve the process. The whole process is repeated with another round of short sprints.

Other Agile Methodologies

There are a number of other agile methodologies that also founded on short iterations and incremental development such as Agile Unified Process, Crystal Clear, XP, Feature driven development. These methodologies encourage transparent, collaborative and open work environments, which provide foundation for adoptive and knowledge workplaces.

Test-Driven Development

Test-driven development is a software development process uses a short cycle of development, where developer writes a failing test case for desired functionality, then implements functionality to pass the test and finally refactors the new code. It eliminates the waste by focusing on business functionality that is required and helps build design incrementally. Each cycle of TDD is very short and provides rapid feedback to developer if the code is working.

Integrated Development Environment (IDE)

Modern IDEs are built to shorten development cycle by providing rapid feedback to developer such as syntax warnings, errors and integration to testing, debugging, static analysis, deploying and other tools. The productivity of developer increases when the cycle from edit, build to test, debug or deploy is short.

One-on-One vs Annual Reviews

Unfortunately Annual Reviews are still annual rituals in most companies that provide feedback once a year. Instead weekly one-on-one provide shorter feedback and is more effective.

Opinions vs Data

All of us have plenty of opinions which are nothing more than guesses. Modern development methodologies such as Lean software development or Lean startup encourages data-driven approach by executing short experiments and learning from the experiments.


We live in rapidly changing knowledge economy. We need to learn how to be nimble and to create a culture that speeds up learning process by performing short experiments. Instead of working in vacuum, we need to validate our assumptions and guesses with actual experiments and take data-driven approach to test our hypothesis. This approach is more bottom-up approach but it doesn’t mean that we don’t have a vision. It just means, we are continuously learning, improving and adopting as our environment changes.

April 25, 2011

Review of Guy Kawasaki’s book – “Enchantment: The Art of Changing Hearts, Minds, and Actions”

Filed under: Business — admin @ 12:23 pm

I recently read Guy Kawasaki’s book Enchantment: The Art of Changing Hearts, Minds, and Actions. This book shows how to engage with other people and build better relationships similar to Dale Carnegie’s book How to Win Friends & Influence People. Though, this book covers these topics in more professional context and it includes advice from several other business and management books. As, Guy is also a very savvy social media user, this book covers several tips on using modern networking tools to build personal relationships with others.


Guy describes enchantments as a way of delighting people with a product, service or organization, which is similar to the concept of Customer Delight popular in business literature. Guy suggests to start with a good product or service and fill people with the delight. This also reminded me Tony Hsieh’s book Delivering Happiness: A Path to Profits, Passion, and Purpose.

Likable and Trustworthy

Once you have a good product, you build the enchantments by being likable and trustworthy. The likability chapter covers several pointers such as smile, dress appropriately, firm handshake, accept others, yes attitude, and work in open environment. Guy encourages finding shared interests with other party and creating win-win situation when negotiating. On being trustworthy, Guy suggests giving people benefit of doubt, disclosing interests and positioning yourself. Some of these techniques seemed similar to what I have read from Stephen Covey’s book The 7 Habits of Highly Effective People and from agile development gurus.


Guy gives a great set of tips on preparation before launching a product and suggests the product should be:

  • great
  • deep
  • intelligent
  • complete
  • empower
  • elegant

Product Launch

On launching a new product, Guy suggests telling personal stories, showing courage, planting many seeds and aspiring people by promising a better world. This chapter reminded me of how Steve Jobs promotes Apple products by promising better future, giving great demo, and simplifying the interface.

Overcoming Resistance

On overcoming resistance, Guy suggests creating perception of ubiquity and scarcity and finding a way to agree, which enhances your chances of being likable. I found the chapter on overcoming resistance a bit weak and encourage readers to look at Switch: How to Change Things When Change Is Hard and Fearless Change: Patterns for Introducing New Ideas.

Enchanting Influencers

Guy offers a great practical guidance on enchanting influencers such as working on grassroots, creating intrinsic motivation, paying it forward, and reciprocity. I liked his advice of saying “I know you’d do same for me” instead of saying “You’re welcome” in response to thank-you”.


In order to create a grassroots support of your products or services, Guy recommends creating a product worthy of ecosystem and then lists several tools, which encourage exchange of ideas and collaboration such as user-groups, blogs, conferences, reward system, open architecture. Another key factor for ecosystem is having a diversified team, which different roles such as advocate, skeptic, visionary, adult, evangelist and rain maker.

Push Technology

This is one of best chapters in the book and shows how to use modern push technologies such as Presentations, Email, and Twitter. Guy recommends engaging many people fast and often. He also recommends giving them credit and providing a value for them. On presentations, Guy recommends customizing intro based on audience, selling dreams, dramatizing and rehearsing it. He suggests keeping the presentation short with 10-20-30 rule, where presentation has no more than 10 slides, takes 20 minutes and uses no less than 30-size font. For email, Guy suggests keeping it short (under six sentences) and asking for a specific action.

Pull Technology

On pull technologies, Guy suggests creating a website/blog with good content, refreshing contents frequently and having an about page. On Facebook, Guy suggests having a good landing page and being helpful. On Linked-in, Guy suggests having a great profile and reaching out to others actively.

Enchanting Employees

Guy also provides useful set of pointers on being a good employer such as engaging employees by providing MAP (Mastery, Autonomy, Purpose) and empowering employees to do the right things. He recommends instead of judging actions of others against their intentions, be harsh on yourself and judge your results against their intentions. He also suggests celebrating success and includes tips from Good Boss, Bad Boss: How to Be the Best… and Learn from the Worst such as protecting people from intrusions. Guy cites Michael Lopp’s advice from Managing Humans: Biting and Humorous Tales of a Software Engineering Manager such as setting ambitious goals, enabling, appreciating and providing feedback to the employees.

Enchanting Boss

On enchanting boss, Guy recommends:

  • make your boss look good
  • drop everything when boss asks for something
  • under-promise and over-deliver
  • prototype work by completing part of assignment and asking for feedback
  • show and broadcast progress while giving credit to colleagues who helped
  • form friendship
  • ask for mastership
  • deliver bad news early

Resist Enchanters

Finally on resisting enchanters, Guy suggests looking far in future, knowing your limits, having a skeptic attitude and not falling for example of one.


In this book, Guy Kawasaki provided a good collection of practical advice on building better interpersonal relationships and using tools from social media effectively. It shows that in order to build long lasting relationships, you have to be sincere and always be willing to help others. I found Guy’s pointers on push and pull technologies most helpful as he has created cult of followers on Twitter and Facebook and provided a number of tips from his personal experience.

August 10, 2010

NoSql databases bring “Stored Procedures” back in fashion

Filed under: Business — admin @ 5:26 pm

One of best tip I learned from my post-graduate research in Parallel & Distributed area was to bring the computation closer to the data. However, most applications in the real world are designed as three or more tiers that separate databases from the application server, where the business logic resides. Though, stored procedures have long been used in client server architecture, dataware services, reporting, and other forms to run the business logic closer to the database, but they are generally shunned due to the maintenance issues. I find it interesting that NoSQL databases are bringing back the stored procedures in the form of map/reduce queries. NoSQL databases come in various forms such as key-value stores, document stores, column stores, and graph stores. They are primarily influenced by the Brewer’s CAP Theorem and use BASE (basically available, soft state, eventually consistent) transactions as opposed to ACID (atomicity, consistency, isolation, durability) transactions. NoSQL databases are designed for horizontal scalability and are able to support large data by partitioning it. NoSQL offer rich queries based on map/reduce, which are generally written in javascript or other scripting languages. These queries provide powerful mechanism to define the business logic for filtering or aggregating results, which are then executed inside the database or closer to the data. Thus, NoSQL databases are able to provide much better performance as a side effect if the application logic is transferred to the host where the data resides. Everything old is new again and stored procedures are back in the fashion.


June 3, 2010

A few lessons from Seth Godin’s book – Linchpin: Are You Indispensable?

Filed under: Business — admin @ 10:25 pm

I just finished reading Seth Godin’s new book Linchpin: Are You Indispensable?. Seth shows how the white-collar jobs, which supposed to save the middle class are being eliminated either by machines or outsourcing with cheap labors. He shows that you can either continue to live your life as a faceless cog or choose to become Linchpin. Here are some of the lessons I learned from this book:

Industrial Revolution is Over

The race to make average stuff for average people in huge quantities is almost over.

This book shows the industrial revolution is changing and in order to survive in the new era of economy, you have to become linchpin or indispensable. In last three hundred years, the industrialization began by standardizing the tasks so that it can be performed by easily replaceable labor or so called cogs. It relied on two layers: management and labor, where management breaks production of goods into tiny tasks, which are performed by the labor. The management wins when it can get the most work for the least pay. The system taught workers to follow the instructions and you don’t have to think. Though, that system worked but has been falling apart in the face of competition, outsourcing and globalization. The attendance-based compensation (ABC) is over. Th old American dream that taught to keep your head down, follow instructions, work hard and you will be rewarded is dead. The mass production treats everything such as labor and material as interchangeable. However, in global market, the competition is fierce and cheap strategy doesn’t scale very well.
Instead of easily replaced laborers or cogs, you can choose to become Linchpin by differentiating yourself from the rest and focusing on humanity, connection and art. The web has made it easier to be productive and create or invent. The new American dream is to be remarkable, generous, create art and connect with people.

Education System is a Sham

In capitalist market, the companies make money by hiring obedient and competent workers as cheaply as you can and using productivity advantage to earn more profit. Andrew Carnegie saw that limited amount of education to get them to cooperate. The school system throughout the world encourages mediocre obedience and is driven by fear as when we learn things in fear. Seth shows public school system is designed to prepare us for factories, where we are just replaceable cogs and care little about our jobs or customers. The same factory model created consumer culture that uses consumption as a shortcut to happniess. Instead, school should teach solving interesting problems and leading.

Becoming a Linchpin

In order to become a linchpin or indispensable, you must embrace an artist and genius within you. Seth recommends avoiding asympototic goals such as bowling, where there is a ceiling of how good you can be. Also, for an artist, the economy is not just zero sum game, instead he/she can increase the pie. Seth cites Richard Florida’s survey of top ten reason for employees to do best work as follows:

  • challenge and responsibility
  • flexibility
  • stable work environment
  • money
  • professional development
  • peer recognition
  • stimulatng colleagues and bosses
  • exciting job content
  • organization culture
  • location and community

All of above reasons except money are internal that we can control. Seth encourages readers to find the work that suits your passion. He uses Emotional labor term, originally coined by Arlie Hochschild to connect with the work. Though, you may get a little compensation in return of emotional labor, but you get inward reward. Instead of day’s work for day’s job or the poverty mentality that treats life as zero sum game, you give gift and build bonds. Seth shows that the easier work is to quantify, the less it’s worth and more humanity you bring to your work, the better results you will receive. Seth cites Krulak’s law for building strong relations with your customers, i.e.,

The closer you get to the front, the more power you have over the brand.

Resistance to Change

Seth gives plenty of examples and demonstrates that real artists ship, however shipping is hard due to trashing/tweaking and coordination. According to Seth, the biggest resistance to the change is our lizard brain. He explains how we all have two brains: primeval brain or lizard brain and gray matter or recently developed brain. The lizard brain has animal instincts such as hungry, scared, angry and horny, whereas newer brain allows big thoughts, generosity, speech, and art. Lizard brain seek compfort and obedience, and avoids risks, public speaking and generosity.

Good is enemy of perfect

Seth encourages readers to become excellent and not perfect as art is never defect-free. He cites Bre Pettis, who says that there are three states of being: not knowing, action and completion. He says accept that everything is draft as it helps to get it done.


Exchanging gifts is an ancient tradition. Seth shows that artists who give gifts win as becoming a linchpin is not an act of selfishness. Seth also shows how usury was prohibited in Bible as interest-free loan was kind of gift. This changed when Martin Luther lifted the sanction to get support for the Protestant Reformation. Seth writes:

For the last five hundred years, the best way to succeed has been to treat everyone as a stranger you could do business with.

Seth cites Metcalfe’s law states that the value of a network increases with the square of the number of nodes on the network. The new social media platforms such as Twitter, Facebook, Blogsphere, and Internet is changing the circle of the gift system and he shows that there are three cicles of gifts, the first circle represents true gifts to family and friends. The second circle is for commerce, they pay for souvenir edition and the third circle is your tribe, followers, fans or friendlies.

There is no map

Seeing the future is hard because we are attached to the world and want stability and fear change. Seth gives plenty of examples of record industry and newspaper industry who have been too attached with their legacy model and failed to adjust in the new economy. In order to become linchpin, you need to draw a map and lead instead of being passive. You need to find a job that matches your passion.

Culture of connections

How to Make a Personal Connection with Customers
The industrialization removed human connection between different parties. The social media and Internet is changing that, now companies can connect directly with their customers and receive their input. Often, when companies negotiate with other companies, the key point of distinction is the perceived connection between the prospect and the organization. The salesman who relies only on the script would fail, instead you have to rely on honest signals and genuine gifts to make connections.

Seven attributes of Linchpin

Linchpins are geniuses, artists and givers of gifts, who extert emotional labor and make their own map. Here are seven abilities of the linchin:

  • Providing a unique interface between members of the organization
  • Delivering unique creativity
  • Managing a situation or organization of great complexity
  • Leading customers
  • Inspiring staff
  • Providing deep domain knowledge
  • Possessing a unique talent


We have been in declining economy for a while and many of the white collar and blue collar jobs lost in last few years won’t come back. I found a lot of Seth’s advice similar to agile movement in software development and My Job Went to India. I also wrote about Taylorism in my blog IT Sweatshops, where I deplored Taylorism based command and control structure in a lot of companies even the one that claim to adopt agile methodologies. Seth even says that you don’t need a resume as it hides the fact that you are linchpin. Instead have a project that an employer can see or blog that people can follow. I find this book offers very practical and timely advice for future market. In the job market, You need to differentiate yourself and have a trail of breadcrumbs of your previous work. Being an average is over, instead you have to be a linchpin and live without a map.

April 24, 2010

Favorite fifteen tips from “Rework” book by Jason Fried and DHH

Filed under: Business — admin @ 2:04 pm

I have long been following Jason Fried of 37Signals and read his first book Getting Real. Jason along with DHH have put together many of their ideas from their blog Signal vs. Noise into a new book Rework. I just finished reading it and though it reiterates many ideas from the earlier book “Getting Real” and their blogs, it’s worth re-reading those ideas as many of business companies today still runs on old fallacies. The book consists of thirteen sections and over eighty ideas, here are my favorite ideas from the book:

Failure is not a rite of passage

Overcoming Failure in Life: The Failure Checklist

I have heard the advice from startup folks about “Fail early and fail often.” On the contrary, this book shows that the people who learn from mistakes will make new mistakes, instead success shows what actually works. Another related advice in the book is “Reason to quit”, which shows when you can quit and choose something else. When I read Founders at Work: Stories of Startups’ Early Days, it also showed that most startups don’t stick to their original ideas and move to other ideas based on early feedback.

Planning is Guessing

Key Challenges in Test Estimation and Planning | Trusted Partner for Software Testing, Test ...

This is related to another advice from the book “Your estimates suck” as Planning and Estimation is hard especially in software business. I have written about Software Estimation in my earlier blogs, however most places still equate estimates with commitments. Jason and DHH reminds us again that estimates are just guesses that were made based on the best information available at the time.


This is another unorthodox advice that is contradictory to how most software projects are run. Most companies measure workers’ dedication on how many hours they put even when they are not actually producing desired outcome. This is also common when managers treat estimates as commitments and refuse to admit reality when things change. We are all familiar with iron triangle of schedule/cost/functionality or sometime referred to as cost/quality/schedule or cost/resources/schedule. Often business folks are unwilling to change schedule and functionality, which often requires working late hours. This is also related to Heroism, which I have blogged before and go to sleep, as work-holism can result in sleep deprivation, which reduces creativity and productivity.

Scratch your own itch

Most successful businesses started with hobbies or personal interests or problems and there are tons of examples of this. This advice is also related to eat your own dog food, though not mentioned in this book.

Start making something

Jason and DHH reminds us another great point that ideas are cheap and the real question is how well you execute them.

Draw a line in the sand

One of the key characteristics of Ruby on Rails software that DHH produced is having strong opinions that limits variations. Similarly, 37Signals is known for their simple design and limited features. You can differentiate yourself from others by standing for something.

Outside money is Plan Z

Both DHH and Jason often talked about downside of getting money from venture capitalists and I agree that these days you can start most software startups with minimal money and raising money can be very distracting. Another related tip that “building a flip is building to flop”, which is often what startup founders hope to get out.

Start at the epicenter

This book recommends focusing on your core product. Though, this book briefly mentions this topic but there is a great presentation of Video of Geoffrey Moore at Business of Software 2009 that talks about similar topic. This advice is also related to other tips from the book such as “don’t copy”, “decommoditize your product”, “focus on you instead of they”, i.e., focus on your core strengths and not your competitors.

Focus on what won’t change

This is great advice for building business that will last. I remember when I started working at Amazon, we were told the core values of Amazon that included having a large selection, cheap prices, customer service and everything we built started from outside-in focus, i.e., it started with customers.

Get it out here

This is similar to common advice from the startup and agile community, i.e. release early and release often.

Interruption is the enemy of productivity

Interruption Is God's Invitation | Desiring God

More and more research is showing that our brain can’t focus on one thing at a time, and constant interruption and multi-tasking hampers your productivity. This is also somewhat related to office space is setup as many agile practices encourage more open space with pair programming and I have found that it prevents concentration. I found that private office pattern offered from Organizational Patterns of Agile Software Development provides less interruption.

Meetings are toxic

This is another hallmark idea of 37Signals and the book contains a number of tips on making your productive such as fixed time, fewer people, clear agenda, beginning with a specific problem and ending with action items and making someone responsible for them.

Good enough is fine

37Signals is known for their simple design and fewer features. This is related other advice in the book such as “embrace the constraints”, “throw less at the problem”, “underdo your competitor”, “say no” and “be a curator”. When you have limited resources, you can become more creative. Also, you are better off building half a product, not a half assed product.

Make tiny decisions

The authors encourage to make tiny decisions as big decisions are hard to make and hard to change. This advice is related to other tips such as “decisions are progress”, which encourages you to always make progress and “quick wins”, which encourages you to build momentum by accomplishing small tasks.

Build an audience

The authors encourage to build audience that come back to you by writing blogs, tweets and speaking. This is also related to “sell your by-products”, “emulate chefs”, “emulate drug dealers” and “out-teach your competitors”.


Though, I skipped many gems of advice on hiring, culture and marketing but I suggest you read the book for many practice advice on building a long lasting and successful business.

Powered by WordPress