Tuesday 28 June 2011

Highs and lows of "Test automation day 2011"

Last thursday I attended the "Test automation day 2011" in Zeist. Most of the day was a big disappointment. Only the foreign speakers had put some real effort into making an enjoyable presentation. This day proved again that testing is still a boring subject and most speakers did no effort whatsoever to disprove that statement.

Some interesting stuff

Jamie Plower from Bearing Point presented their automated test process, which used some tools like Jenkins (open source automated build server) and Sonar (code metrics for Java, but also PHP, C and C#). An impressive setup, especially how they could create a screencast of a failing functional test. The screencast would capture every click in the browser until the error, which can be an enormous help when hunting down bugs.
Last year an intern in my team created a similar setup, using Cucumber, Capybara, Selenium RC, Selenium Client and Selenium Webdriver. We couldn't get it stable however, although I probably should have tried harder. I am looking forward to try out the RSpec acceptance testing DSL, which was just released in the 1.0 Capybara gem. This development looks exciting, Jeff Kreeftmeijer blogged about this earlier this year.

Another interesting presentation was the closing keynote by Scott Barber. Scott is a performance tester, he shared some of his personal experiences, some of which were quite funny. In performance tests you might, for example, discover that your application is performing excellent. But if you don't use the proper error detection, you might discover that what you have been testing is how fast the 500, 404 or 401 error page loads. Which in most applications (I've confirmed this already in Coconut), is blazing fast!
He gave us ten very good tips on automating performance tests. I'm sure they can help you, so you can download his presentation here.

Some embarrassing stuff

Nearly all native speakers showed up with a boring or less than convincing story. Most embarrasing was the day's host speaker, Mr Bob van den Burgt. He started his keynote by stating that he didn't really know much about test automation. Well Bob, I believe that's probably true, but you shouldn't have told me. It ruined your credibility.
From his acting on stage, I could also conclude that he doesn't have much talent for presenting either. And to top it off, he spoke English with an awful Dutch accent, we call it "steenkolen Engels" in Holland, which can be translated to "broken English". This reminded me of Wim Kok, our former prime minister. Wim Kok was famous around the globe for two things: a) his last name and b) his incredible Dutch accent.

Another presentation @ test automation day was called "Agile and the cloud - the impact of modern IT megatrends on testautomation". Sounds interesting, right? Well no.
First of all, the presentation would be given by Mr Wolfgang Platz, CEO of Tricentis Technology, but he didn't bother to show up. Instead, he left us with his Dutch employee, who didn't actually understand what "Agile" or "Cloud" means. Instead, he presented Tricentis' product Tosca, which you could probably use for test automation, but please don't. They're idiots. I know, the presenter probably did not have a choice when his CEO called him and he wasn't very well prepared for such a gig, but come on. This is insulting.

And why do conferences forget to mention that the presentation is for a product these guys sell? At Microsoft DevDays 2010 I experienced the same thing, having to sit out 45 minutes of product advertising by people who refused to be honest about the shortcomings of their own product. That really pisses me off and next time I will leave the room.

To end on a positive note

There was one Dutch speaker who showed up with an interesting project. Professor Arie van Deursen from the Delft University presented Crawljax, an open source Java tool for automatically crawling and testing modern (Ajax based) web applications. It sounds very promising. Also refreshing to see what a scientific approach to testing web applications can deliver.

After a day of conference I can conclude that test automation is boring and that this conference showed very few new developments. But hey, to me it was comforting to know that my knowledge of automated testing is up-to-date.

Saturday 18 June 2011

Jeff Sutherland seminar

Last week, I was lucky enough to attend to a Jeff Sutherland seminar (read his blog here) at the Dialogues Technology House in Amsterdam. I've been a fan of Scrum since I've encountered the Agile movement at the end of my study in Information Technology. So for me, it's exhilirating to hear someone speak that has been at the cradle of the Agile movement. This man has had an enormous influence on how we create software today and I think it has been a positive influence as well.

Jeff talked about why you should do Scrum, but if you've read Mike Cohn's excellent book 'Succeeding with Agile: Software development using Scrum' you know what he's talking about (and more). I won't bore you with why you should do Scrum (maybe later :-) ), but I have written down some notable statements from Jeff:
  • developers should be having fun!
  • timesheets reduce productivity by 10 %, throw them out, they're not true anyway
  • even a bad scrum is better than a good waterfall
  • scrum is like martial art, you first have learn the exact basic moves and if you've got those worked out, you continuously improve your skills and adapt your own style
  • if you want high performance, communication in your team is key
  • if it's not working, STOP DOING IT!
  • your team needs a goal transcending the usual day to day struggle
  • scrum is based on truth, transparency, commitment and trust
  • key performance indicator is how fast your team fixes the build
  • turnover destroys productivity
  • in the future there will not be a company in the top 100 who isn't using Scrum
  • specialisation will not only slow you down, it will eventually kill you
A statement like "if it's not working, stop doing it" seems quite obvious, but think about your own environment. I'm sure you can find some examples of ineffective behaviour which you are continuing anyway.

The most interesting part of the talk is that Jeff suggests Scrum is also succesfull because it follows human nature. We like helping each other out, it makes us feel better. We need to have a feeling of being useful, having some significance. And we need to have some fun in order to get the right motivation. It made me think about a book I'm reading at the moment, The Art of Happiness by the Dalai Lama. In this book, the Dalai Lama discusses several themes which relate to becoming happy. For the Dalai Lama, the meaning of life is the struggle to become happy. At some point in time, everyone will wonder "is this going to make me happy?" Doing Scrum will enhance the chances of your employees to say "yes, this job will make me happy!"

Friday 17 June 2011

Capistrano deployment: Cap shell != Bash shell

Somehow everytime I use an open source plugin, I get to the point were stuff just doesn't work and the documentation doesn't contain an answer to my problem. This time I was releasing Coconut to a fresh Redhat 6 server, which was just created in our private cloud.

During cap:deploy, it suddenly stopped while executing the bundle install task. One gem couldn't compile, it threw the much dreaded "incompatible character encodings: UTF-8 and ASCII-8BIT" error. (which you probably recognize if you're using Ruby 1.9.2) The error baffled me, since it doesn't occur on any of our other Redhat servers, nor on any local development or test machine. It seems my new Redhat installation somehow got confused about the default locale or encoding settings. After some google research, I've found out that setting the environment variable LANG to "en_US.UTF-8" would fix this problem. This sets the default encoding for Ruby to UTF-8. Some resources state that setting the LC_CTYPE variable to the same value might also be necessary.

So, I've added the environment variable to the .bashrc of the capistrano user. But to no avail. After digging further (much further than I really wanted) I've found out that the shell which Capistrano uses to deploy is NOT the same as a normal SSH bash shell!. Which was a bit surprising to me, since Capistrano uses SSH. But they have provided a way to configure the cap shell to use the correct environment variables. Just add this to your deploy.rb file:

set :default_environment, {
  :LANG => 'en_US.UTF-8'
}

You can add as much environment variables here as you need. Sweet.