Alfred App + RTM

A month or so ago I raved about Alfred App and how impressed I was with it. I've also been a long time user of Remember The Milk to keep lists of the things I should be doing.

The good folks over at Ruk have shown how to bring these two worlds together using a Ruby command line script. This now means entering a TODO/reminder can be as simple as :

alt-space
r Mail Jim about bulk purchase rates of hamsters ^Wed 9am


This really helps my flow through the day - things that would have become interruptions can be quickly added to the TODO list without any real context switch.

If you are a Mac user you really owe it to yourself to go try AlfredApp out.

IPad2 Xoom Initial development with PhoneGap

After a year of envy of my better half having an original iPad, I was lucky enough to pick one up about ten days ago. In this time its already become my go to device for a bunch of scenarios and I'm delighted with it.

With my professional hat on, we have been asked to take a look at building some tablet based applications. Much as I love my IOS based device, Apple keep a very tight reign on whats happens on that platform so I wanted to show the client alternatives.

To demonstrate the alternatives I went shopping for an Android device - initially planning to buy an Android 2.2 based table, the Vega Tablet PC - 512MB.
However when I got to the store, I spotted that they has both the Motorola Xoom and the ACER Iconia Tab A500 Android 3.0 (Honeycomb) tablets in stock. Android 3 is the first version of the OS designed to run on tablets. So I got upsold and bought a Xoom as an alternate platform for the tests.

Initial feelings about the Xoom are very positive.

Hardware

• Noticeably heavier/thicker than iPad 2.
• Screen is great, touch response etc excellent.
• Sound through speakers is awful (although better when in the "stand mode" of portfolio case).
• Camera - front and rear clearly better than iPad2 (although I'm still dubious about cameras for anything other that video conferencing on this type of device).
• Charging - you can't charge over USB it has a power brick - unforgivable - (am considering returning it for the ACER if that is USB chargeable).

Software

• The setup is nice, integration to google applications etc is all very quick.
• Browser is quick (no difference side by side with iPad2 on BBC, Guardian etc).
• Flash video - works fine.
• I've struggled a little with general navigation etc. I don't think its bad, its just different to what I know from Apple side (interesting that colleagues with Android phones also notice its different).
• Bult in apps for mail/calendar etc etc are good - but lack polish.
• I've crashed the OS 3 or 4 times pressing navigation etc while apps are running.
• Notifications - as people have noted, Android 3 nails this - clearly better than IOS.
• App Store is short of tablet specific format apps right now (and Amazon won't let folks purchase in UK on that store yet) - this is clearly going to change fast.
• Now the important stuff - Angry Birds Rio - feels identical to iPad2 version.

A quick note, I bought the Motorola Portfolio case. IMHO it has a huge design flaw, you can't charge the device with the case/screen protected.

General

These are the first of this generation of Android devices, we are going to see dozens of these within a year.For me the Apple user experience is still the benchmark, but its close. This market is going to be fun over the next few years, I hope this pushes both vendors forward - we should all gain.




Development

Initial application development proof of concept had been built using PhoneGap and Sencha Touch. I'd had this running as an IOS bundle for a few days, it took 10 minutes to migrate same code to native Android App. Application has worked with almost identical performance on the Xoom.

Alfred App

I've been a long time user (3+years) of Quick Silver as a task launcher and general tool for keeping hands on keyboard rather than a mouse.

Around 6 weeks ago I decided to trial Alfred, Quick Silver got uninstalled about an hour later. Alfred lets you launch applications, web search, find local docs, open recent files etc etc with amazing ease. If you own a Mac, go try the free version either from http://www.alfredapp.com/ or the Apple Store. Once you are hooked go spend a few pounds to get the PowerPack and support an independent developer who is doing a great job.

ITerm2 - Growl Notification

As a heavy user of the Mac command line toolset, I spend a lot of time logged into remote machines. I've been an avid ITerm user for a long time, it allows me to have radically different looking consoles for QA/EA/Production machines.

I was reading  http://aming-blog.blogspot.com/2011/01/growl-notification-from-iterm-2.html on changes in ITerm2 to allow console commands to use Growl Notifications (including from remote machine). Typical use case would be a large data import/build etc - followed by growl telling you he jobs complete.

One quick spot - the blog above talks about the Growl settings being in Iterm2/Preferences/Advanced - on the nightly I am using they are in Iterm2/Preferences/General/Services.

Add the following to .bash_profile :

growl() { echo -e $'\e]9;'${1}'\007' ; return ; }

You can then perform actions like  :

ant bigTarget; growl "Big Target Complete"

This has the advantage over using something like the say command in that it will update your local growl even if running on a remote box.

Static Hosting Using S3/AWS

We have been gradually moving large amount of our managed and internal services to AWS. EC2/RDS make for a very compelling model. As a software/consultancy house using these services let us concentrate on our core competencies. The benefit of no longer sitting waiting for hardware to fail and ruin someones weekend is hard to beat.

For those wanting to dip a toe in the water, Amazon now support S3 as a static website. i.e. you can link an S3 "bucket" to being a static website. Announcement blog post is here - it boils down to uploading the content to S3 and then pointing a CNAME at it.

You can then simply use a tool like  Cyberduck to allow your creatives to upload the content.

Page Based Testing With Tools Like Selenium - tips

After the grand experiment described below I ported a large number of the smoke tests over, these happily ran on my Mac under both Safari and Firefox 3.

Of course despite my bias for OSX/Linux apparently one or two internet users still use IE. So we probably need to test on that "other" platform too.

To do this I setup three VirtualBox instances, running IE6/7/8 respectively. I then modified our base test to take a set of parameters for test server/port/browser. i.e. make our base test setup the driver as follows :

selenium = new DefaultSelenium(seleniumClientIP, seleniumClientPort, seleniumClientBrowser, baseTestURI);

Then I created a quick ant script to perform all my smoke test on each combination, wrapped it up in a Hudson instance to run it periodically and report any JUnit failures.

All sounds rosy doesn't it?

Whilst the tests all run fine in Safari and Firefox on any of the hosts/virtual machines. It appeared that we are not so lucky with IE. Bottom line was that without intervention tests running in the IE VMs fail about 15% of the time with "Internet explorer has encountered a problem". This modal dialog eventually leads to timeouts in the tests, and the windows/IE instances don't go away without manual intervention. As previously discussed if the tests are flakey or out of date they become an irrelevent distraction.

A lot of internet digging this weekend has led to this thread. This has two salient bits of advice :

a) Even though the IE selenium runs are much slower that those in Safari/Firefox, it can still get into trouble with timing. setting the default selenium speed like this selenium.setSpeed("500") seems to help.

b) If the above doesn't fix all issues, turn off windows error reporting to stop the instances getting hit with the modal dialog. (The tests will still fail, but the instance is recoverable without manual intervention).

Since I put these changes in place, we have not had a single IE fail. Time will tell of course but things certainly look positive so far!

Page Based Testing With Tools Like Selenium

For a while I have been doing ad hoc runtime smoke testing of deployments with tools like Selenium. i.e. running a set of simple tests that exercise a series of pre-defined application points. The scripts where typically recorded using the excellent Selenium IDE then replayed through ant/Junit on one or more target browser platforms.

Not full regression testing but a start - right? Well these tests become a pain to maintain, and unmaintained/unreliable tests are pointless. A typical test would be something like the following :

a) login to the application
b) pick an object from an info set.
c) pass that object through several workflow steps.
d) complete the workflow.
e) log out.

The majority of the tests would be around the differences between step c) the options chosen during workflow.

I tried to take a step back and look at why these became such a pain to maintain. If I had 10 tests scripted as defined above a small change (like how I select a workflow) would effect all 10 test scripts. Multiply this to a hundred or so tests on a project with half a dozen active developers then you quickly have more broken smoke tests than working ones.

So I took a step back to try to look at a new way of writing these sort of tests so that they become less brittle. The method that seems to be working for me is to produce a more "Page" based approach rather than the test "stream" approach described above.
The concept is pretty simple, write a class that offers public methods for the types of operations that a user can perform through a browser on that page. Ensure that the operation returns another page based object.

As a simple example, lets look at a typical login page,it contains two fields userId and password. The Page object looks something like this :

public class LoginPage
{

/** Contains the actual driver we are using. */
private final Selenium selenium;

/**
* Create the page.
* @param driver the driver.
*/
public LoginPage(Selenium selenium)
{
this.selenium = selenium;

assert (selenium.getTitle().equalsIgnoreCase("My Applications Login Page));
}

/** A "good" login. i.e. one we know should allow us into the site.
*
* @param userName userId.
* @param password password to log in with.
* @return HomePage as "logged in".
*/
public HomePage performLogin(String userName, String password)
{

selenium.type("userName", userName);
selenium.type("password", password);

selenium.click("login");
selenium.waitForPageToLoad(SeleniumConstants.MAX_PAGE_LOAD_TIMEOUT);

return new HomePage(selenium);

}
}


then a simple HomePage with a search method might look like this :


public class HomePage
{

/** Contains the actual driver we are using. */
private final Selenium selenium;

/**
* Create the page.
* @param driver the driver.
*/
public HomePage(Selenium selenium)
{
super(selenium);
this.selenium = selenium;
assert (selenium.getTitle().equalsIgnoreCase("My Application Home"));
}

/** Perform a search with the supplied term.
*
* @param searchTerm term to look for.
* @return a search result page.
*/
public SearchResultPage performSearch(String searchTerm)
{

selenium.type("searchTerm", searchTerm);
selenium.click("homeSearchCommand");
selenium.waitForPageToLoad(SeleniumConstants.MAX_PAGE_LOAD_TIMEOUT);

return new SearchResultPage(selenium);

}

}


This is implemented using selenium, but the concept is applicable to any of the other tools. Note that the Home constructor contains a simple assertion that checks that the title of the page is "My Applications Home". This is important because if anything with the login fails and I stay on the login page rather than going to the home page my test will automatically terminate.

The home page again exposes methods like performSearch that again return result pages etc. This form of testing leads to tests that are *really* simple to view/read eg.

public void testPerformLoginThenSearch()
{
// these 3 lines should probably be put into a base/setup method for all tests
selenium = new DefaultSelenium("localhost", 4444, "*safari", "http://myapplication.com/");
selenium.open("/myApp/Login");
selenium.waitForPageToLoad(SeleniumConstants.MAX_PAGE_LOAD_TIMEOUT);

HomePage homePage=new HomePage(selenium);

homePage.performLogin("goul","notthatdaft!");

assertTrue( selenium.getHtmlSource().contains("Welcome back master how may I help"));

SearchResultPage searchResult=homePage.performSearch("banana hammock");
.
.
.
.
}



Something to note, if you know a test should do something different i.e. return a non-standard return page (like login fail - returning you to login) then create another public method - the page flow is the key thing here i.e. add a method like


/** A "bad" login. i.e. one we know should not allow us into the site.
*
* @param userName userId.
* @param password password to log in with.
* @return LoginPage because we are still not logged in.
*/
public LoginPage performLoginExpectingFail(String userName, String password)
{

selenium.type("userName", userName);
selenium.type("password", password);

selenium.click("login");
selenium.waitForPageToLoad(SeleniumConstants.MAX_PAGE_LOAD_TIMEOUT);

return new LoginPage(selenium);

}


Going back to the test view, we can easily hide some of the minor complexity to get started and check page content. We can easily add some utility methods to hide that ugly login and the selenium content. Lets also assume that the SearchResultPage adds methods like routeFirstObject that launches the first returned search object into a named workflow.

Our test now looks like :

public void testSearch()
{
HomePage homePage=login();
checkPageContains("Welcome back master how may I help");

SearchResultPage resultPage=homePage.performSearch("banana hammock");
checkPageContains("Hammock Object");

WorkflowRoutePage workflowRoute=resultPage.routeFirstObject("Author, Review and Translate");
.
.
.
}
This test is now amazingly simple to write, in fact you could probably convince the QA department that they can write the tests if the developers provide the page objects. Any modern IDE is going to give you strongly typed Pages for results of functions, the method names are self explanatory etc.

So how does this solve my initial problem around brittle tests?

Well lets say that how the search on the home page is launched changes (new design/ids for field etc etc.) I only have to fix the HomePage object, and fixing that single object in place means that *all* tests that use it should now run cleanly without needing changes.

I believe this approach gives the following advantages :

a) Page objects can be "grown" as tests are needed. As functionality is added to the page object, everyone using that object gets the advantage over time. i.e. the effort for new tests actually reduces as the cumulative methods are added to Pages.

b) The test scripts are very strongly typed, and are simple to understand. The page approach means the tests obviously model the flow through the application.

c) Tooling like SeleniumIDE can still be used - but take the simple steps and apply them into Page objects.

d) By making all tests that perform a login do it through the LoginPage object, you only have to change the LoginPage object if the layout/operation of login changes. i.e. fix one page object rather than a all the places calling login in scripts.

I've moved about 20 of our "smoke tests" over to this approach so far and coupled with hudson am able to run them on a series of browsers. So far so good, I'll blog further if I've been missing some huge pitfall in this approach.

Turns out this is not an original idea the good folks at WebDriver have a paper almost identical to this. Hopefully this means it is an approach that will really work!

(BTW - WebDriver looks very interesting, the HTMLUnit headless mode is great).