Ahoy matey, let's plunder 'tis camp

Friday Oktober 7th I visited Arrrrcamp in Gent, Belgium. Arrrrcamp is a conference about Ruby, Rails, Radiant and Rum with a pirate theme, yarrr. The conference offerred a broad spectrum of topics and some well known speakers, like Jonas Nicklas and Corey Haines. A celebration of open source Ruby development with a serious hint of craftsmanship. Here the outline of my day.

First up was Jim Gay, the lead developer for Radiant CMS. Most of his talk was about the new features of Radiant CMS. Since I've never used Radiant, most of his talk was wasted on me. The features he showed were quite impressive. Radiant is a CMS created in Ruby on Rails (of course) and you can use cool RoR features like Sass, Scss or Coffeescript in Radiant. You can create pages and define your CSS and javascript seperately. What's really nice is that they created different editors for this, like an IDE which supports different languages. Jim talked a lot about their upcoming 1.0 release and their struggle to migrate to Rails 3. Migrating a product which has many open source plugins is quite a challenge, since most plugins will not work in Rails 3. Jim explained they cannot just convert to Rails 3: you will have to give users and plugin makers a clear path to migrate along with you. This is why the 1.0 version will not be a Rails 3 version, but will remain on Rails 2.3. However, they are already working hard on the Rails 3 version and in the future, it will be possible to add Radiant as a Rails engine to your project. Wow.

A speaker which I really looked forward to seeing, was Jonas Nicklas, and he certainly didn't disappoint. Jonas is the creator of Capybara (an acceptance test framework for webapplications), Carrierwave (a solution for file uploads in Rails) and during his talk he introduced Evergreen, which integrates Jasmine javascript tests into Rails applications.
Here some of Jonas' quotes:

• Rubyists love testing. Rubyists love programming. Rubyists love Ruby.
• We want beautiful, well structured and tested code.
• More and more code moves to front-end. Backend is more and more a CRUD application with JSON responses.
• jQuery is the PHP of javascript.

Jonas demonstrated testing javascript using Evergreen, which integrates Jasmine javascript testing into your Rails application. This has some advantages: a route is mounted in your application where you can see if your Jasmine tests succeed and you can use Coffeescript to write your Jasmine tests.

He also offered some advice on using Capybara. Capybara is not an integration test framework, it's a tool to simulate user behaviour which you can use to create automated acceptance tests. His advice was:

1. add your own selectors
2. avoid asserting on URL, session state, cookies or application state (!!)
3. avoid too specific selectors
4. make generic asserts

Sound advice, if you ask me. To summarize: use Capybara to test what a user can do and can see, not what the application does. You should test that elsewhere in your test suite.

After a great lunch, I joined Julian Fischer's talk about "three ways to scale towards happiness". Julian had a disadvantage, since the room was next another one where they cranked up the sound so loud we could almost follow that talk instead of Julian's. Beforehand, I was afraid of having to listen to a commercial promotion of Railshoster.de, since he is involved in that company. But Julian's talk was incredibly informative and educational. He started talking about the high hosting prices in the U.S. and the legal issues you face when you host your webapp at a U.S. company. I've never realized that hosting my data in the U.S. can mean that the U.S. government might be allowed to demand access to that data. Do we really want to risk that?

The three ways he compared to scale towards happiness were:

1. cloud hosting
2. cheap datacenter hosting
3. high end datacenter hosting

For each way, he made a detailed comparison of the advantages and disadvantages. Did you know that cloud hosting costs upto 20 times more than other hosting solutions?
At the end of his talk, he explained how to choose between the three options. Each application has it's own characteristics, which, when examined properly, will lead to a logical choice of one the options. You should do a requirement analysis of budget, load (normal and peak load), response time, availability, geographic targeting, estimation of growth rate, legal constraints, technical constraints and competence of inhouse support.Check out his presentation here on how to choose the correct Ruby on Rails hosting.

Next up was Andrew Nesbitt, who showed how to do A/B testing with Split, a Rack based A/B testing framework. He asked us the question: are you testing your users? (eehm, no...) How do you know that you are succeeding, then?
He demonstrated how to do A/B testing using the Split gem. Split uses Redis to save test data, which of course is very fast and scalable. It uses cookies to differentiate users and you can test almost anything, you can swap css, render different templates or partials, set different controller variables or even hack up your javascript.
Using Split looks easy, you can just create an experiment with alternatives and view via a CRUD controller how the experiment is doing. Split also has a dashboard to show results and you can even combine this with Google analytics to get a more detailed view. Disadvantage is that you will code the A/B tests in your product's release code. Andrew also had two memorable quotes:

• Be a good Ruby citizen and contribute to your community!
• If you're not prepared to fail, you're never going to be original.

A pretty long break followed, in which the organisation served free mojito's (yarrrr). To wrap up this enjoyable day, Corey Haines did a talk about fast Rails tests. A phenomenon which most of us only hear about, but never experience. I've heard Corey last year on the RubyAndRails conference in Amsterdam. He talked about software craftsmanship and that was the most insipiring talk I've ever seen. Corey has a great stage appearance, he's relaxed, funny, provocative, opinionated and inspiring.
He showed his way to get fast tests for Rails projects and why you should bother. You should bother, because slow tests will finally stop your developers running tests and it distracts them from the work they should be focussing on. To get faster tests, his main point is to NOT load Rails in your test unless it's absolutely necessary. You can create seperate modules or classes to defer behaviour from the Rails context and use stubs or mocks to test the behaviour outside of Rails.

Corey talked about Test Driven Development, here a summary of some of his statements:

• The fundamental difference between test first and test driven development is, that in test-first you change your tests until workable, where with test-driven, if you discover that something's hard to test, you change your design.
• TESTABILITY = GOOD DESIGN
• Better design is one that is easy to change
• The 3 A's of testing: Arrange, Act and Assert. Use 1 assert per test. Try it out for a while and you'll discover when to use it and when not to.

Especially Corey's opinion on how to create your Rails test suite and his opinion on integration testing led to much discussion. Which for me, actually continued until we were almost back home. Thanks to Corey for another inspiring talk.

When all was said and done: the Arrrrcamp conference is a properly organized event. The venue is small and has small disadvantages, but the pirate theme and excellent food and drinks more than made up for those disadvantages. I haven't seen an uninteresting talk all day, so I will return next year without a doubt. Yarrr!

Ruby on Rails surprise: conditional active_record callbacks

Sometimes you discover hidden features in Rails by trial and error. This week I was refactoring a method into an after_create callback. The method would create an empty user profile when a user was created, thus using an after_create callback would simplify this code. Using the handy create_ method which active_record provides when a belongs_to or has_one relationship is used I've managed to refactor this (quite horrible) piece of code:

class User < ActiveRecord::Base
  has_one :profile

  def create_from_ldap(login)
    #some code left out
    create_personal_belongings
    save
  end

  def create_personal_belongings
    profile = Personal::Profile.new
    profile.id = self.id
    profile.save!
  end
end

into these lines of code:

class User < ActiveRecord::Base
  has_one :profile

  after_create :create_profile

  def create_from_ldap(login)
    #some code left out
    save
  end  
end

Consequence of this change was that a user profile is always created when a user is created. Which is great, because users without a profile should not exist anyway. But it broke some specs, because some specs relied on specifying a certain profile for a user, so we can test some aspects of the user / profile relationship properly.

So what I actually wanted to do, is create a profile after creating a user, but only if the user hasn't got a profile yet.
When using active_record validations I've discovered you can do a conditional validation like this:

class User < ActiveRecord::Base
  validates :email, :presence => true, :email => true, :if => :has_email_enabled?
end

This validates the user's email attribute only if the has_email_enabled? method returns true for the specific user object. (see the active_record_validations documentation for more info) I couldn't find if I can also use conditionals on active_record lifecycle callbacks, so I just tried:

class User < ActiveRecord::Base
  has_one :profile

  after_create :create_profile, :unless => :profile

  def create_from_ldap(login)
    #some code left out
    save
  end  
end

Hey, that works! I believe it's not a documented feature, but it's really nice. I love Rails' adagium "use convention over configuration", because once you know how some stuff works in Rails, you can deduct how the other stuff works. And even better, the chosen conventions usually make sense. To me, at least.

Redis and Phusion Passenger: reconnect on fork

We're nearly ready for a new Coconut production release. Of course, this is the moment when bugs start coming in from the beta stage that are difficult to reproduce. One bug report stated: "sometimes, my widgets do not load". I couldn't reproduce this bug, but it was in the back of my mind the whole week. Suddenly, this afternoon, when I was clicking through a review build, the widgets didn't load. So I immediately pulled the log files and found this Redis error:

Got '1' as initial reply byte. If you're running in a multi-threaded environment, make sure you pass the :thread_safe option when initializing the connection. If you're in a forking environment, such as Unicorn, you need to connect to Redis after forking.

After some research, I've found out that the error was caused by the combination of using Redis and Phusion Passenger. We use Redis as a chained backend store for i18n (see Railscast 256 for  our inspiration), so we can have custom translations for each Coconut instance. Which is a very cool feature, because every customer has his own domain language and we can tweak the translation messages accordingly.

As the error states, Phusion Passenger is a "forking environment" like Unicorn. Phusion Passenger spawns new worker processes when it thinks more processes are needed, it uses what they call a "smart spawning method". Basically, it forks a new thread to increase capacity to handle requests. Normally, the newly created thread will try to use the same Redis connection, which causes problems. What you need to do, is create a new connection when the current process is forked.This is done by creating a new Rails initializer and adding some code for handling Phusion Passenger fork events.

Adding a new Rails initializer is simple: just add an .rb file to config/initializers. Our initializer looks like this:

if defined?(PhusionPassenger)
  PhusionPassenger.on_event(:starting_worker_process) do |forked|
    if forked
      # We're in smart spawning mode. If we're not, we do not need to do anything
      Rails.cache.reset
      I18nRedisBackend.connect
    end
  end
end

You might recognize the Rails.cache.reset from Dalli, which has the same issue if used with Phusion Passenger. The I18nRedisBackend.connect creates a new connection with Redis, like this (note: this code was simplified, to make it more readable):

module I18nRedisBackend
  @@default_i18n_backend = I18n.backend

  def self.connect
    redis = Redis.new
    I18n.backend = I18n::Backend::Chain.new(I18n::Backend::KeyValue.new(redis), @@default_i18n_backend)
  end 
end

To summarize, when Phusion Passenger forks a process to create a new worker thread, it automatically creates a new Redis connection. Problem solved!

thanx to the Phusion Passenger users guide, Appendix C which provided me with the correct code example

Ruby on Rails 3: chaining scopes with lambda's

Today we ran into a really strange bug: the bugreport stated that a blog post was shown in the blog list approximately 15 minutes after creation. Which is really weird, because we've dealt with time zone misery before, but that always applies to hours, not a quarter of an hour. After analysis, we've found out that a chained scope caused the trouble (note: the scopes were simplified for better readability):

class Blog::Post
  scope :published_posts, lambda { where("publication_time < ?", DateTime.now) }
  scope :published_non_rotator_posts, published_posts.where("rotator_position IS NULL")
end

Scopes are evaluated at the moment you call them. The published_posts scope uses a lambda to ensure that DateTime.now is evaluated on each call, instead of being evaluated to the same DateTime value for every call. For the published_non_rotator_posts it's the same. This scope is also evaluated on the first call. Since it doesn't use a lambda expression, the outcome of the chained published_posts scope will have the same value on every next call! The correct code is:

class Blog::Post
  scope :published_posts, lambda { where("publication_time < ?", DateTime.now) }
  scope :published_non_rotator_posts, lambda { published_posts.where("rotator_position IS NULL") }
end

So: when chaining a lambda scope you must also wrap it with a lambda! thanx to this slash dot dash article.