Shahzad Bhatti

October 25, 2008

My gripes about REST services

Filed under: REST — admin @ 11:17 am

I love REST style services because of their simplicity and ease of testing. I have discussed benefits of REST earlier. I have used a number of distributed technologies over seventeen years such as LU6.2 (CICS) for mainframes, BSD Sockets, RPC, CORBA, RMI, JINI, Messaging Middlewares, SOAP, etc. In most of those technologies, you had to use special libraries to interact with the server. I worked in some organizations where I saw real dependency hell or jar hell, where I had to import dozens of client jars from different groups to talk to those services. REST, on the other hand only relies on http (and security) libraries.

I first wrote REST style service in late 90s before I knew about the REST term. At the time, I worked as a consultant for government DOTs and built CORBA based system to provide traffic data to media agencies. However, people were scraping our website for the data so I built an XML over HTTP service to download the data with some credentials. I think a lot of people used simplicity of HTTP to build similar services. And many of them didn’t understand REST as put forth by Roy Fielding. Over past few years, a lot of people are promoting real style of REST such as Sam Ruby, Stefan Tilkov, Steve Vinosky and Roy Fielding himself. At the same time, they are chastising people who diverge from their vision.

I have tried building REST style services over last few years and recently I have started building new services that will be used for entire organization. As, these services will need to support large number of transactions, performance and scalability are critical. Also, these services need to support batch of requests. In my older blog, I discussed commandments for writing service and wrote about importance of batch requests for scalability. That requirement changes a few things, for example instead of taking advantage of request parameters in HTTP, I had to use XML for input request and had to use XML for response. Though, this style is suitable for POST or PUT where you are expecting to read request as a file but is unnatural for GET types.

Another tenant of REST style services is resource. In this style, you interact with the service similar to how users interact with a web site, i.e., you click on links to go to another page, which returns more links and so on. However, this style in service adds network communication. For example, in one of the service for workflow I had to return active tasks for a particular workflow. In true REST style, I would have to return a hyperlink for each task instead of contents of task and the client would have to ask server for task content by hitting the resource for the task. As, you might guess this adds significant network latency so I am being practical instead of purest and as a result, I am returning task contents. Also, the sevice allows you to pass requests for multiple workfows and returns a single response for all workflows.

I used similar style when using messaging middlewares that allowed serving batches of requests in a single message. I found messaging middlewares offered more flexible options, e.g. you can design an aggregator service that waits for a few seconds for incoming requests and then puts them in a single request to another service that serve them. These kind of batch services are integral for building scalable systems. I have discussed some of these limitations in earlier blog, especially its lack of push based architecture. Though there are some workaround for push such as long-lived http connection that are difficult to use behind firewall, comet style or reverse ajax based communication, which is limited and more recently reverse http, which is too new to apply in real applications. Perhaps, messaging standards like XMPP can fill this gap.

I find that IT industry is largely driven by a single minded attitude where every new technology, language or process is adopted by bandwagon of people who just repeat each other’s words without understanding. For example, I have seen similar attitude from agile folks, folks from object oriented programming, aspect oriented programming, metaprogramming or dynamic programming, design patterns, etc. In the end, I believe you have to be practical instead of purest and as always answer to every question in IT is “it depends”, i.e., there is no single universal solution.

August 14, 2008

Benefits of REST based services

Filed under: REST — admin @ 10:47 pm

I saw Damien Katz blog on REST, I just don’t get it and I was a bit surprised that he doesn’t get REST especially since he wrote CouchDB based on REST. Though, I admit there are a lot of bad examples of REST services that use REST sort of like RPC over POST, but resource oriented services can be quite simple and powerful. REST principles are building blocks for the web and it has proven to be quite scalable and efficient. I have been developing REST based services for a number of years, in some ways before I learned about Roy Fieldings’ thesis and REST principles. Back in 90s, I worked on building traffic sites and used CORBA to subscribe and publish traffic events. We also published that data on the website, but soon we found a number of people were scraping the website so I wrote a simple XML over HTTP service to download the data that other groups can use. I have found following benefits when using REST based services:

  • separating reads from writes. I have worked on large ecommerce and travel website and one of the lesson is to keep your read/query services separate from your transactional services. REST APIs define separate operations for reads and write/updates.
  • caching: you can find tons of off the shelf solutions for caching GET requests including hardware solutions. There are tons of features like ETags and cache headers that provide this feature.
  • compression: Since REST uses HTTP, you can use compression such as gzip. This can improve the performance of the services.
  • idempotency: GET, PUT, DELETE and HEAD are idempotent, which means if designed correctly the request can be retried without any worries about side effects. POST on the other hand is not idempotent and may have side effects.
  • bookmarking: GET requests can be easily bookmarked. It is important not to use GET to change state of application.
  • security: Though, security has been weakest area of REST as compared to SOAP, but HTTPS and simple authentication surfice. Though, there are better standards like oauth.
  • big response size: REST/HTTP is the only service platform that I have seen supports gigabytes of responses. I have done a lot of CORBA based services in 90s, EJBs/SOAP in early 2000s and messaging based services for over ten years. None of those platforms support large size responses.
  • simplicity: I find this is the biggest reason for using REST. I can use browser to call GET based requests and write client in any language.
  • resources: REST response can include URIs for other APIs and client can change state through these resources. You can use XHTML to embed all these resources that can be easily tested with browsers.
  • No need for additional jars: When I used CORBA, EJBs, RMI or JINI, I always had to put client/skeleton jar files. Having worked in large companies where I had to import dozens of these jar files became maintenance problem. With REST, I can simply call the service without importing anything.
  • Error codes: HTTP comes with a number of response codes for real life services including thrashing requests such as server busy (503).
  • Meta data: As opposed to CORBA, JINI, RMI services, I can pass meta data easily as HTTP supports predefined and user-defined headers. These headers can include information on authentication, quality of service, timeout or other context related data. Occasionally, I add Map<String,String> to APIs when I use Java based services, but it polutes pure APIs.

The only real drawbacks I see for REST based services are that they are generally synchronous and blocking which can waste threads (though some of it can be solved with async I/O or event based dispatching). Personally I like to use messaging underneath REST services that provide asynchronous, persistence, and better reliability.

Powered by WordPress