Internet Explorer 8 crashes with jQuery 1.6.2

Can browser tabs crash? Sure they can. If you're using a buggy plugin or add-on, you can probably crash tabs in any browser. But crashing a tab when using only javascript is probably limited to the ones that have a Microsoft brand on them.
Since august, we've been having problems when our users were accessing Coconut on IE8 in a Citrix environment (and in rare cases also outside a Citrix environment). Which is hard to solve: first of all, I did not have a Citrix account. After the IT department fixed that problem, I was able to reproduce the bug, but had no way to analyze it. In Citrix I couldn't access log files, change IE settings or even view the advanced properties tab. My Google results told me I should try disabling all add-ons, but I wasn't allowed to do this either, even though the IT guys made me local admin on Citrix. It seems that Citrix has all kinds of seperate rules which are enforced on all users, and you have to make specific exceptions to the rules. Just telling Citrix: "hey, this is an admin" is not enough.
Fortunately, my colleague discovered that the error message was the same for each user and he found a stackoverflow topic which had a reference to this error message

If you encounter this error: (this the Dutch version of course)

Or this error:

then you should upgrade your jQuery javascript, version 1.6.4 works fine.

jQuery's .html() and Internet Explorer 8

Sometimes it's easy to blame IE for unwanted behaviour. This case is a bit different, in the sense that I think IE's behaviour is understandable, or even correct. I was trying to fix a bug where the contents of an overlay screen would not load in IE8. The contents were fetched using an ajax GET request, after which this piece of code would be executed:

show: function(html) {
  var box = $("#dialog_box");  
  box.html(html);
  box.show();
}

In Firefox or Chrome, this works fine. I verified that the show method was actually called with the html from the GET request. The box.show() call was executed correctly, but somehow the box.html(html) was not. But the method didn't fail either.

After much fiddling around, I've found out that:

Internet Explorer will only insert a HTML string using jQuery's .html() method if every tag in the HTML string is opened and closed properly.

The response from the GET request wasn't valid, it contained one closing div tag too many. IE then refuses to insert the HTML into the DOM. Other browsers seem to have no problem inserting the invalid HTML. So IE8 is correct? Remarkable.

Redis and Phusion Passenger: reconnect on fork

We're nearly ready for a new Coconut production release. Of course, this is the moment when bugs start coming in from the beta stage that are difficult to reproduce. One bug report stated: "sometimes, my widgets do not load". I couldn't reproduce this bug, but it was in the back of my mind the whole week. Suddenly, this afternoon, when I was clicking through a review build, the widgets didn't load. So I immediately pulled the log files and found this Redis error:

Got '1' as initial reply byte. If you're running in a multi-threaded environment, make sure you pass the :thread_safe option when initializing the connection. If you're in a forking environment, such as Unicorn, you need to connect to Redis after forking.

After some research, I've found out that the error was caused by the combination of using Redis and Phusion Passenger. We use Redis as a chained backend store for i18n (see Railscast 256 for  our inspiration), so we can have custom translations for each Coconut instance. Which is a very cool feature, because every customer has his own domain language and we can tweak the translation messages accordingly.

As the error states, Phusion Passenger is a "forking environment" like Unicorn. Phusion Passenger spawns new worker processes when it thinks more processes are needed, it uses what they call a "smart spawning method". Basically, it forks a new thread to increase capacity to handle requests. Normally, the newly created thread will try to use the same Redis connection, which causes problems. What you need to do, is create a new connection when the current process is forked.This is done by creating a new Rails initializer and adding some code for handling Phusion Passenger fork events.

Adding a new Rails initializer is simple: just add an .rb file to config/initializers. Our initializer looks like this:

if defined?(PhusionPassenger)
  PhusionPassenger.on_event(:starting_worker_process) do |forked|
    if forked
      # We're in smart spawning mode. If we're not, we do not need to do anything
      Rails.cache.reset
      I18nRedisBackend.connect
    end
  end
end

You might recognize the Rails.cache.reset from Dalli, which has the same issue if used with Phusion Passenger. The I18nRedisBackend.connect creates a new connection with Redis, like this (note: this code was simplified, to make it more readable):

module I18nRedisBackend
  @@default_i18n_backend = I18n.backend

  def self.connect
    redis = Redis.new
    I18n.backend = I18n::Backend::Chain.new(I18n::Backend::KeyValue.new(redis), @@default_i18n_backend)
  end 
end

To summarize, when Phusion Passenger forks a process to create a new worker thread, it automatically creates a new Redis connection. Problem solved!

thanx to the Phusion Passenger users guide, Appendix C which provided me with the correct code example

Ruby on Rails 3: chaining scopes with lambda's

Today we ran into a really strange bug: the bugreport stated that a blog post was shown in the blog list approximately 15 minutes after creation. Which is really weird, because we've dealt with time zone misery before, but that always applies to hours, not a quarter of an hour. After analysis, we've found out that a chained scope caused the trouble (note: the scopes were simplified for better readability):

class Blog::Post
  scope :published_posts, lambda { where("publication_time < ?", DateTime.now) }
  scope :published_non_rotator_posts, published_posts.where("rotator_position IS NULL")
end

Scopes are evaluated at the moment you call them. The published_posts scope uses a lambda to ensure that DateTime.now is evaluated on each call, instead of being evaluated to the same DateTime value for every call. For the published_non_rotator_posts it's the same. This scope is also evaluated on the first call. Since it doesn't use a lambda expression, the outcome of the chained published_posts scope will have the same value on every next call! The correct code is:

class Blog::Post
  scope :published_posts, lambda { where("publication_time < ?", DateTime.now) }
  scope :published_non_rotator_posts, lambda { published_posts.where("rotator_position IS NULL") }
end

So: when chaining a lambda scope you must also wrap it with a lambda! thanx to this slash dot dash article.

Software has a new quality standard: ISO 25010

You might have missed the news that the ISO 9126 quality standard has been replaced recently by the ISO 25010 quality standard. Since the ISO/IEC wants at least 110 euro's for a PDF containing the new standard (WTF????), I thought I'd summarize the new standard and compare it to the old one.

The 'old' ISO 9126 model described six main characteristics, with a set of subcharacteristics:

1. Functionality - A set of attributes that bear on the existence of a set of functions and their specified properties. The functions are those that satisfy stated or implied needs.
   • Suitability
   • Accuracy
   • Interoperability
   • Security
   • Functionality Compliance
2. Reliability - A set of attributes that bear on the capability of software to maintain its level of performance under stated conditions for a stated period of time.
   • Maturity
   • Fault Tolerance
   • Recoverability
   • Reliability Compliance
3. Usability - A set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by a stated or implied set of users.
   • Understandability
   • Learnability
   • Operability
   • Attractiveness
   • Usability Compliance
4. Efficiency - A set of attributes that bear on the relationship between the level of performance of the software and the amount of resources used, under stated conditions.
   • Time Behaviour
   • Resource Utilisation
   • Efficiency Compliance
5. Maintainability - A set of attributes that bear on the effort needed to make specified modifications.
   • Analyzability
   • Changeability
   • Stability
   • Testability
   • Maintainability Compliance
6. Maintainability - A set of attributes that bear on the effort needed to make specified modifications.
   • Analyzability
   • Changeability
   • Stability
   • Testability
   • Maintainability Compliance
7. Portability - A set of attributes that bear on the ability of software to be transferred from one environment to another.
   • Adaptability
   • Installability
   • Co-Existence
   • Replaceability
   • Portability Compliance

The new model has eight characteristics, instead of six, which are quite similar to the old model:

1. Functional suitability - The degree to which the product provides functions that meet stated and implied needs when the product is used under specified conditions
   • Suitability
   • Accuracy
   • Interoperability
   • Security
   • Compliance
2. Reliability - The degree to which a system or component performs specified functions under specified conditions for a specified period of time.
   • Maturity
   • Fault Tolerance
   • Recoverability
   • Compliance
3. Operability - The degree to which the product has attributes that enable it to be understood, learned, used and attractive to the user, when used under specified conditions
   • Appropriateness
   • Recognisability
   • Ease of use
   • Learnability
   • Attractiveness
   • Technical accessibility
   • Compliance
4. Performance efficiency - The performance relative to the amount of resources used under stated conditions
   • Time Behaviour
   • Resource Utilisation
   • Compliance
5. Security - The degree of protection of information and data so that unauthorized persons or systems cannot read or modify them and authorized persons or systems are not denied access to them
   • Confidentiality
   • Integrity
   • Non-repudiation
   • Accountability
   • Authenticity
   • Compliance
6. Compatibility - The degree to which two or more systems or components can exchange information and/or perform their required functions while sharing the same hardware or software environment
   • Replaceability
   • Co-existence
   • Interoperability
   • Compliance
7. Maintainability - The degree of effectiveness and efficiency with which the product can be modified
   • Modularity
   • Reusability
   • Analyzability
   • Changeability
   • Modification stability
   • Testability
   • Compliance
8. Transferability - The degree to which a system or component can be effectively and efficiently transferred from one hardware, software or other operational or usage environment to another
   • Portability
   • Adaptability
   • Installability
   • Compliance

In the new model, security and compatibility were added as main characteristics. I've always wondered why security wasn't that important for software quality measurement, but now it is. Some subcharacterics were added to the model and a number of them were renamed to more accurate terms. The 25010 quality standard also works a bit different than the 9126 standard. The software product quality model describes the internal and external measures of software quality. Internal measures describe a set of static internal attributes that can be measured. The external measures focuses more on software as a black box and describes external attributes that can be measured.
Besides the software product quality model, the 25010 standard also describes another model, the model of software quality in use:

1. Effectiveness - The accuracy and completeness with which users achieve specified goals
   • Effectiveness
2. Efficiency- The resources expended in relation to the accuracy and completeness with which users achieve goals
   • Efficiency
3. Satisfaction- The degree to which users are satisfied with the experience of using a product in a specified context of use
   • Likability
   • Pleasure
   • Comfort
   • Trust
4. Safety - The degree to which a product or system does not, under specified conditions, lead to a state in which human life, health, property, or the environment is endangered
   • Economic damage risk
   • Health and safety risk
   • Environmental harm risk
5. Usability- The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use
   • Learnability
   • Flexibility
   • Accessability
   • Context conformity

I like the fact they've created a seperate model to emphasize how important quality in use is for a software product. Their motivation to do so might be different, they probably assume the characteristics in this model are "in the eye of the beholder" and thus harder to measure. And harder to agree on a common standard for these characteristics.

To summon it up, the new model has a broader range and is more accurate. I believe this is an improvement to the old model, but why someone would pay more than 100 euro's to be able to study the new model is beyond me.

Highs and lows of "Test automation day 2011"

Last thursday I attended the "Test automation day 2011" in Zeist. Most of the day was a big disappointment. Only the foreign speakers had put some real effort into making an enjoyable presentation. This day proved again that testing is still a boring subject and most speakers did no effort whatsoever to disprove that statement.

Some interesting stuff

Jamie Plower from Bearing Point presented their automated test process, which used some tools like Jenkins (open source automated build server) and Sonar (code metrics for Java, but also PHP, C and C#). An impressive setup, especially how they could create a screencast of a failing functional test. The screencast would capture every click in the browser until the error, which can be an enormous help when hunting down bugs.
Last year an intern in my team created a similar setup, using Cucumber, Capybara, Selenium RC, Selenium Client and Selenium Webdriver. We couldn't get it stable however, although I probably should have tried harder. I am looking forward to try out the RSpec acceptance testing DSL, which was just released in the 1.0 Capybara gem. This development looks exciting, Jeff Kreeftmeijer blogged about this earlier this year.

Another interesting presentation was the closing keynote by Scott Barber. Scott is a performance tester, he shared some of his personal experiences, some of which were quite funny. In performance tests you might, for example, discover that your application is performing excellent. But if you don't use the proper error detection, you might discover that what you have been testing is how fast the 500, 404 or 401 error page loads. Which in most applications (I've confirmed this already in Coconut), is blazing fast!
He gave us ten very good tips on automating performance tests. I'm sure they can help you, so you can download his presentation here.

Some embarrassing stuff

Nearly all native speakers showed up with a boring or less than convincing story. Most embarrasing was the day's host speaker, Mr Bob van den Burgt. He started his keynote by stating that he didn't really know much about test automation. Well Bob, I believe that's probably true, but you shouldn't have told me. It ruined your credibility.
From his acting on stage, I could also conclude that he doesn't have much talent for presenting either. And to top it off, he spoke English with an awful Dutch accent, we call it "steenkolen Engels" in Holland, which can be translated to "broken English". This reminded me of Wim Kok, our former prime minister. Wim Kok was famous around the globe for two things: a) his last name and b) his incredible Dutch accent.

Another presentation @ test automation day was called "Agile and the cloud - the impact of modern IT megatrends on testautomation". Sounds interesting, right? Well no.
First of all, the presentation would be given by Mr Wolfgang Platz, CEO of Tricentis Technology, but he didn't bother to show up. Instead, he left us with his Dutch employee, who didn't actually understand what "Agile" or "Cloud" means. Instead, he presented Tricentis' product Tosca, which you could probably use for test automation, but please don't. They're idiots. I know, the presenter probably did not have a choice when his CEO called him and he wasn't very well prepared for such a gig, but come on. This is insulting.

And why do conferences forget to mention that the presentation is for a product these guys sell? At Microsoft DevDays 2010 I experienced the same thing, having to sit out 45 minutes of product advertising by people who refused to be honest about the shortcomings of their own product. That really pisses me off and next time I will leave the room.

To end on a positive note

There was one Dutch speaker who showed up with an interesting project. Professor Arie van Deursen from the Delft University presented Crawljax, an open source Java tool for automatically crawling and testing modern (Ajax based) web applications. It sounds very promising. Also refreshing to see what a scientific approach to testing web applications can deliver.

After a day of conference I can conclude that test automation is boring and that this conference showed very few new developments. But hey, to me it was comforting to know that my knowledge of automated testing is up-to-date.

Jeff Sutherland seminar

Last week, I was lucky enough to attend to a Jeff Sutherland seminar (read his blog here) at the Dialogues Technology House in Amsterdam. I've been a fan of Scrum since I've encountered the Agile movement at the end of my study in Information Technology. So for me, it's exhilirating to hear someone speak that has been at the cradle of the Agile movement. This man has had an enormous influence on how we create software today and I think it has been a positive influence as well.

Jeff talked about why you should do Scrum, but if you've read Mike Cohn's excellent book 'Succeeding with Agile: Software development using Scrum' you know what he's talking about (and more). I won't bore you with why you should do Scrum (maybe later :-) ), but I have written down some notable statements from Jeff:

• developers should be having fun!
• timesheets reduce productivity by 10 %, throw them out, they're not true anyway
• even a bad scrum is better than a good waterfall
• scrum is like martial art, you first have learn the exact basic moves and if you've got those worked out, you continuously improve your skills and adapt your own style
• if you want high performance, communication in your team is key
• if it's not working, STOP DOING IT!
• your team needs a goal transcending the usual day to day struggle
• scrum is based on truth, transparency, commitment and trust
• key performance indicator is how fast your team fixes the build
• turnover destroys productivity
• in the future there will not be a company in the top 100 who isn't using Scrum
• specialisation will not only slow you down, it will eventually kill you

A statement like "if it's not working, stop doing it" seems quite obvious, but think about your own environment. I'm sure you can find some examples of ineffective behaviour which you are continuing anyway.

The most interesting part of the talk is that Jeff suggests Scrum is also succesfull because it follows human nature. We like helping each other out, it makes us feel better. We need to have a feeling of being useful, having some significance. And we need to have some fun in order to get the right motivation. It made me think about a book I'm reading at the moment, The Art of Happiness by the Dalai Lama. In this book, the Dalai Lama discusses several themes which relate to becoming happy. For the Dalai Lama, the meaning of life is the struggle to become happy. At some point in time, everyone will wonder "is this going to make me happy?" Doing Scrum will enhance the chances of your employees to say "yes, this job will make me happy!"

Capistrano deployment: Cap shell != Bash shell

Somehow everytime I use an open source plugin, I get to the point were stuff just doesn't work and the documentation doesn't contain an answer to my problem. This time I was releasing Coconut to a fresh Redhat 6 server, which was just created in our private cloud.

During cap:deploy, it suddenly stopped while executing the bundle install task. One gem couldn't compile, it threw the much dreaded "incompatible character encodings: UTF-8 and ASCII-8BIT" error. (which you probably recognize if you're using Ruby 1.9.2) The error baffled me, since it doesn't occur on any of our other Redhat servers, nor on any local development or test machine. It seems my new Redhat installation somehow got confused about the default locale or encoding settings. After some google research, I've found out that setting the environment variable LANG to "en_US.UTF-8" would fix this problem. This sets the default encoding for Ruby to UTF-8. Some resources state that setting the LC_CTYPE variable to the same value might also be necessary.

So, I've added the environment variable to the .bashrc of the capistrano user. But to no avail. After digging further (much further than I really wanted) I've found out that the shell which Capistrano uses to deploy is NOT the same as a normal SSH bash shell!. Which was a bit surprising to me, since Capistrano uses SSH. But they have provided a way to configure the cap shell to use the correct environment variables. Just add this to your deploy.rb file:

set :default_environment, {
  :LANG => 'en_US.UTF-8'
}

You can add as much environment variables here as you need. Sweet.