More fun with Erlang

I am working on a prototype of an application that needs to proxy client HTTP requests to different origins. I needed a function that will filter HTTP headers before I forward them. Application is in Erlang, but since I’ve been a big Ruby fan for a while now… Here is a comparison of possible implementations:

Ruby:

  NO_FORWARD = ['Host', 'Content-Length', 'Accept-Encoding']
  
  def filter_headers(headers)
      headers.inject({}) do |result, (name, value)|
         result[name.to_s] = value unless NO_FORWARD.include?(name.to_s)
         result
      end
  end

Now, same code … but in Erlang:


filter_headers(Headers) ->
NoForward = ['Host', 'Content-Length', 'Accept-Encoding'],

Pred = fun({Name, Value}, L) ->
case lists:member(Name, NoForward) of
false ->
Key = case is_atom(Name) of
true -> atom_to_list(Name);
false -> Name
end,
[{Key, Value} | L];
true -> L
end
end,

lists:foldl(Pred, [], Headers).

I think given a bit more effort, each code sample might be even more compact, but even what I have looks pretty readable 🙂

Advertisements

Erlang – elegant, but mind blowing :)

I’ve been playing around with Erlang for a while now. Interesting thing about Erlang is that the language is pretty small and very consistent, I guess it is typical for functional language 🙂 , but in a same time it takes a long time to start thinking in “Erlang way”. Everything is based on pattern matching – function signatures, case and if statements are all just pattern matching. Once you get a hold of it… it is a lot of fun and makes code pretty clean, short and readable.
Here is an example of function that parses ttl – 20s, 10m, 2h :

first we have an entry point:


parse(Ttl) -> parse_ttl(lists:reverse(Ttl)).

And here comes all the fun :


%% Private Functions.

parse_ttl([$s | Ttl]) -> list_to_integer(lists:reverse(Ttl));
parse_ttl([$S | Ttl]) -> parse_ttl([$s | Ttl]);

parse_ttl([$m | Ttl]) -> 60 * list_to_integer(lists:reverse(Ttl));
parse_ttl([$M | Ttl]) -> parse_ttl([$m | Ttl]);

parse_ttl([$h | Ttl]) -> 60* 60 * list_to_integer(lists:reverse(Ttl));
parse_ttl([$H | Ttl]) -> parse_ttl([$h | Ttl]);

parse_ttl(_) -> false.

Since strings in Erlang are just lists of integers, it is very common to apply list functions to strings. The [H | T] notation allows you to separate “head” and a “tail” of the list; in our case first symbol and the rest of the string. And since we have reversed our original string before applying our pattern … it is working pretty well 🙂

Easy Helpers for Sinatra

I am spending more time with Sinatra apps. I think it is a perfect framework to start application with. What I like about Sinatra the most is how easy it is to extend it. Recently I was building a small app, that processes user provided templates and was looking for an easy way to add template components (helpers) to give customers a sort of DSL to work with. What I ended up doing was the following:

 
module Components
  
  components_path = File.expand_path(File.dirname(__FILE__) + '/components')

  files = Dir.new(components_path).select{|f| f =~ /.*\.rb/ } || []
  files.each do |f|
    body = File.new(File.join([components_path, f])).read
    name = File.basename(f, '.rb').capitalize + "Helper"
    module_eval <<-"end;"
      module #{name}
        #{body}
      end  
    end;
    Sinatra.helpers(const_get(name))
  end

end

Link to the file: http://gist.github.com/141220

As you can see this pretty simple Ruby magic allows me to put my “components” into a subdirectory and add them all as Sinatra helpers. Since the body of the component is wrapped into a module, component development becomes very easy. Following code, once saved into components directory will add a search helper to your app:

def search(options = {})
......
end

Pretty simple, but still fun 🙂

Ruby wrapper for Google App Engine URL Fetch service

I am still playing with my Sinatra app, running on GAE. Just in case if do that too… and might need to fetch some external data, following is a wrapper code I’m using :

require 'java'

module UrlFetch
  
  module UF
    import java.net.URL;
    import java.net.URLEncoder;    
    import com.google.appengine.api.urlfetch.HTTPHeader
    import com.google.appengine.api.urlfetch.HTTPMethod
    import com.google.appengine.api.urlfetch.HTTPRequest
    import com.google.appengine.api.urlfetch.HTTPResponse
    import com.google.appengine.api.urlfetch.URLFetchService
    import com.google.appengine.api.urlfetch.URLFetchServiceFactory
    
    Service = URLFetchServiceFactory.getURLFetchService()
  end
  
  module InstanceMethods
    def fetch_url(options = {})
      self.class.fetch_url(options)
    end

  end

  module ClassMethods
  
    # = Fetch URL proxy
    # === Accepted options:
    # :url     - request url
    # :method  - HTTP method ('get', 'post' ..)
    # :headers - hash of request headers
    # :params  - request params. Only valid if the method is post
    #
    # === Response:
    # Rack stype responce:
    # [response_code, headers, body]
    
    def fetch_url(options = {})
      return nil unless (options[:url])
      
      url = UF::URL.new(options[:url])
      request = UF::HTTPRequest.new(url, UF::HTTPMethod.valueOf((options[:method] || 'get').upcase))
      
      options[:headers].each{|name, value| request.addHeader(UF::HTTPHeader.new(name, value))} if options[:headers] && options[:headers].is_a?(Hash)
      
      if options[:method] == 'post' && options[:params]
        payload = options[:params].collect{|name, value| "#{UF::URLEncoder.encode(name, 'UTF-8')}=#{UF::URLEncoder.encode(value, 'UTF-8')}" }
        request.setPayload(payload.to_java_bytes)
      end
      response = UF::Service.fetch(request)
      [
        response.getResponseCode,
        response.getHeaders().inject({}){|hash, header| hash[header.name] = header.value; hash },
        (String.from_java_bytes(response.getContent) if response.getContent)
      ]
    rescue => e
      [500, {}, e.to_s]
    end

  end
  
  def self.included(base)
    base.send :include, InstanceMethods
    base.send :extend,  ClassMethods
  end
  
end

I hope this is will save some typing 🙂

Here is a link to GitHub gist: git clone git://gist.github.com/111006.git gist-111006

Rack::Cache on Google App Engine

It’s been a few weeks now since Google announced Java support on their App Engine infrastructure. Since Java support also adds support for JRuby, it is now possible to deploy Rails or Sinatra or any other Rack based applications to Google. In my case, I am playing with Sinatra 🙂 This is an excelent step-by-step tutorial on how to get started with Sinatra App. If you running your applications on App Engine, you might be interested (at least I was) in reducing your application load by using some sort of page cache. Since Google won’t allow you to write out files, I figured using their MemCachedService might be pretty nice way to work around this issue. Ryan Tomayko has written a very nice Rack middle-ware, Rack::Cache, which provides support for cache control and validation and looked to me like a very nice solution. Following is my extension to Ryan’s work, which will allow to use Rack::Cache with Google Memcache as a meta-data and entity storage.
I have the following code in ./lib/cache.rb file:

require 'java'
require 'rack/cache'
require 'yaml'

module Cache

module MC
import com.google.appengine.api.memcache.Expiration;
import com.google.appengine.api.memcache.MemcacheService;
import com.google.appengine.api.memcache.MemcacheServiceFactory;
import com.google.appengine.api.memcache.Stats;

Service = MemcacheServiceFactory.getMemcacheService
end

module ClassMethods
def clear
MC::Service.clearAll
end

def exists?(key)
MC::Service.contains(key)
end

alias_method :contains?, :exists?

def get(key)
value = MC::Service.get(key)
YAML.load(value) if value
end

def put(key, value, ttl = nil)
expiration = ttl ? MC::Expiration.byDeltaSeconds(ttl) : nil
MC::Service.put(key, value.to_yaml, expiration)
end

def namespace
MC::Service.getNamespace
end

def namespace=(value)
MC::Service.setNamespace(value.to_s)
end

def delete(key)
MC::Service.delete(key)
end
end

module Service
extend Cache::ClassMethods
end

end

module Rack::Cache

class MetaStore

public

class GAEStore < MetaStore
attr_reader :cache

def initialize(options = {})
@cache = Cache::Service
@cache.namespace = options[:namespace] if options[:namespace]
end

def read(key)
key = hexdigest(key)
@cache.get(key) || []
end

def write(key, entries)
key = hexdigest(key)
@cache.put(key, entries)
end

def purge(key)
key = hexdigest(key)
@cache.delete(key)
end

def self.resolve(uri)
self.new(:namespace => uri.host)
end

end

GAECACHE = GAEStore
GAE = GAEStore
end

class EntityStore

public

class GAEStore < EntityStore
attr_reader :cache

def initialize(options = {})
@cache = Cache::Service
@cache.namespace = options[:namespace] if options[:namespace]
end

def exist?(key)
@cache.exists?(key)
end

def read(key)
result = @cache.get(key)
result
end

def open(key)
if data = read(key)
[data]
else
nil
end
end

def write(body)
buf = StringIO.new
key, size = slurp(body){|part| buf.write(part) }
@cache.put(key, buf.string)
[key, size]
end

def self.resolve(uri)
self.new(:namespace => uri.host)
end
end

GAECACHE = GAEStore
GAE = GAEStore

end
end

Add the following into config.ru:

……..

require ‘application’

use Rack::Cache, {:metastore => ‘gae://namespace‘, :entitystore => ‘gae://namespace‘}

run Sinatra::Application

In order to make it work properly, read Ryan’s documentation on all supported options.

I am hoping this is helpful 🙂

Fun with Sinatra

A few days ago I started playing with Sinatra, very small, but very nice Ruby web framework. It takes about 15 minutes to read the book and get started. Since I’ve been using Rails for some time now, I started to wonder: what would it take to implement something similar to Rails respond_to functionality… This is what I’ve got so far:

helpers do

def respond_to(&block)
class << (mappings = {} )
def method_missing(name, &resp)
self[name.to_s] = Proc.new(&resp)
end
end
yield mappings
handler = mappings[params[‘format’]] if params[‘format’]
handler.call if handler
end

end

This will add a helper method, which can be used in all routes. Now, we need to add a before filter to make sure format is added to the request_path:

before do
ext = request.path_info =~ /.*([.]([^.]*))$/ ? $1 : ‘.html’
request.path_info = request.path_info + ext unless $1
content_type ext
end

All there is left to do is to define a route, which will allow for format parameter, and use our helper method:

get ‘/hello.:format’ do
respond_to do |format|
format.html { “You have requested HTML format” }
format.json { { :message => ‘hello’ }.to_json }
format.xml  { “<hello />”}
end
end

Now it looks more like Rails 😉

Software as an engineering discipline

When we think of software, software product definition or a software development process for some reason everyone is trying to invent something new. New methodologies, new analogies … Why do we think it is OK to have an engineering degree when it comes to software, but it is not OK to treat it as an engineering discipline? Why software has to be so different from car manufacturing, hardware manufacturing, etc? A most common argument is that software changes too fast.. Well, it is true, but for a duration of a given release, for a life-cycle of a given software product it doesn’t look like it is changes any more then anything else: car design is changing from year to year, houses get new design ideas…. Why can’t we apply same logic to software design/requirements changes as we do to same sort of changes in car manufacturing? I think good software product goes thru same life-cycle as a good car: High level design, performance criteria definition, quality metrics… and all this gets applied to the product as a whole, each module or sub-system, each component and each class… There also should be a set of industry accepted criteria, which will classify software product as “market legal”, same way as cars and motorcycles get classified as “street legal” only when they implement and comply with industry standards. Well… I am wondering, and that’s what I do :), wouldn’t that make everybody’s life: developers, PMs, managers, customers, consumers , etc much easier?