Beginning Cross-Browser Testing of Websites

Why Bother?

Assuming you want to do it properly, testing your site to ensure consistency across a range of browsers is a great deal of work, so why would you bother? Well, the difficultly with websites is that they need to be viewed on a large number of different browsers, on a large number of computers. Without proper cross-browser testing your site will not display consistently or without errors on your users’ computers. This is a bad look and not how you want to present yourself to your site’s customers/visitors.

What I won’t cover in this article, but I may cover another time

  1. Testing with Javascript turned off: This is important for testing accessibility, but that’s not what we are concerned with today.
  2. Mobile Browser, many of the points I cover will be somewhat applicable, but I won’t be covering mobile browsing specifically.
  3. Complete end-to-end QA or functional testing, we are focusing specifically on ensuring a constant, quality experience between different browsers. It is assumed that site quality has already been tested against a reference browser, which is not Internet Explorer. I tend to use Firefox 3.5+, but any newish versions of Safari, Chrome, or Opera would work just as well. The important thing though, is to choose a baseline which has minimal quirks.
  4. Usability studies, Eye-tracking studies, or any other specific Usability testing. I assume any Usability testing has already been completed against the reference browser before compatibility testing begins.

What Causes these Errors?

Different Browsers

The people that created the browsers are only human, and as such all had different views and opinions. In addition to this, even though there are internationally agreed on standards for building web pages (for example HTML is the main language used), there are still differences of interpretation. This leads to seemingly little discrepancies, that are actually very visible to your visitors.

For instance, the web standards don’t say a whole lot on what the default font size is for a web page. Consequently, the browser vendors all pick their own default size. You can end up with your web page displaying noticeably larger in one browser than another. Here’s a list of the most common browsers. You’ll definitely want to be testing IE, Firefox, and Safari. Ideally, you should also be testing Chrome and Opera but, when time is tight, those two frequently fall by the wayside.

Different Browser Versions

As progress is marching along, updated versions of the browsers are getting released, with many old errors fixed, a bunch of new features and, usually, a fistful of new errors to contend with. This means that IE 6 and 7 have a different set of bugs and need to be tested separately, as does 8 and 9. The same goes for Firefox 2 and 3, and Safari 3 and 4. Here’s a reasonably complete list of the most significant versions. Exactly which versions you spend the most time testing will depend on what your visitors are using. If you don’t know what your visitors are using, or haven’t launched your site yet, test as much as your can.

  • IE 6, 7, 8
  • Firefox 3, 3.5
  • Safari 3, 4
  • Chrome 3, 4
  • Opera 10

Whether to Support IE 6 or Not

Internet Explorer 6 is the albatross around all of our necks. It’s now almost nine year old, has a large number of bugs and faults (serious faults, mind you, not minor annoyances), is very slow, and does not support any of the cool bits that all the new browsers do. It is where most of your time and budget you have for testing will be spent. Additionally, fixing IE6 bugs, once they are found, is also time consuming.

A Website in IE6 (left) and Google Chrome (right)

A Website in IE6 (left) and Google Chrome (right)

If it causes so many problems why do we still support it, especially considering its almost 9 years old? Because we are pragmatists — we have to be. Think of it this way, if you ran a store selling imported, handmade Italian scarfs (beautiful quality, feel the fine weave, why wouldn’t you want one?), then one day, for reasons unknown, 30% of customers started paying in Nepalese Rupees.

Your options are to turn away 30% of your business, or take the money and do the currency conversion yourself. It’s not ideal but at least you still come out ahead.

So if it’s that simple why did Google just drop support for IE6? And why is 37 Signals is phasing out support, and Facebook and Digg, among others, are asking users to upgrade? The question that the creators of each of those sites have asked themselves is: does the percentage of IE6 users make up a large enough lump of your total site traffic to justify the additional expense of maintaining IE6?

Most web stats tend to put current usage of IE6 at between 20% and 40%, but your site could be vastly different. Unless the site that you are testing is already live and you can see how many visitors are using IE6, then you’ll need to base your decision on traffic from similar sites.

Different Operating Systems (OS)

Mostly the different browsers work pretty consistently between OSes, but one area where there are quite noticeable differences is in font rendering and availability. Each OS comes with a different set of font. For example, Mac OS X comes with a font called Helvetica, where as Windows comes with a similar looking font called Arial. Windows uses Arial because they didn’t want to pay to license Helvetica, and although they are close there are a number of visual differences that are not hard to spot.

The upshot of this is that if you specify your site should use Helvetica, Windows users will actually see Arial. There are a number of other fonts that are only available on one OS, but not the others.

By the by, if you’re interested in Type or design in general a really wonderful film was made about Helvetica in 2007. Not so coincidentally, 2007 was also Helvetica’s 50th birthday!

Another difference that leads to a lack of visual consistency is fundamental, philosophical differences between how the OSes treat type rendering. These differences tend to mean that text on one OS can look more blurry than on another, will cause paragraphs to be laid out differently, or any number of other differences that will lead to your page looking subtly (or not so subtly) inconsistent on different computers. I’ll avoid going into anymore detail regarding this as it starts getting technical pretty quick, but if anyone is interested here’s a couple of links to get you started:

Bottom Line: the main three ones you need to worry about are:

  • Windows
  • Mac OS
  • Linux

Again, where your time is most spent will depend on what your visitors are using, but it’s a safe bet that you’ll want to spend the most time testing Windows and the least amount testing Linux.

Different Screen Sizes

This one is comparatively simple, usually the site is designed for a minimum screen resolution, currently either 800×600 or 1024×768 pixels are standard, but it depends on your site design. Just make sure that the site looks good (that nothing is cutoff or cropped) at your target resolution in each of the browsers and versions being tested, and that it scales nicely to higher resolutions.

Browser Bugs

As the programmers that created the browsers are only human, we also need to deal with bugs in the browsers themselves. They are unique to each browser and are usually obscure. They typically require almost arcane knowledge of the quirks of the specific browser.

Our biggest problem is IE6 and, to a much lesser degree IE7. IE6 is an enormously buggy browser, and its bugs tend to present in odd, sometimes logic-defying ways. Which is why it’s almost universally hated by web developers everywhere. IE6 bugs tend to be tricky to isolate, and even trickier to fix.

Expect to waste a lot of time on IE6.

A Quick Word on Browser Testing Tools

There are a bunch of nice tools out there to help with testing websites for cross-browser compatibility. A quick search on Google will turn up a swath of them, but one I’m quite fond of is Litmus. Litmus can take screenshots of your site in various Browsers, Browser Versions, and OSes. It won’t help with testing more complex logic, Javascript testing for instance, but it can greatly speed up layout testing.

Types of Errors. What are we Testing for?

When performing general site testing there’s quite a range of things that can go wrong, but when specifically doing cross-browser testing the types of errors we are looking for can be classified into one of two, rather lumpy categories: Display Errors and Logic Errors.

Display Errors

These are usually pretty easy to spot, anytime anyone in your team says anything like “Eh, why is the ad banner displaying over the login screen in IE?” they’ve found a display error. These can be embarrassing but are usually less critical than Logic Errors as they don’t usually prevent bits of the site from working, but they do make a bad impression.

A website in IE6

A website in IE6

Logic Errors

These are particularly problematic as, not only are they trickier to fix, but they also prevent parts of the site from working. This will likely impact the effectiveness of the site to your visitors. Which will in turn affect the effectiveness of the site to your organisation.

Examples of questions to ask yourself when testing for logic errors:

  • Can a User access all necessary functionality?
  • Is functionality working as expected?
  • Does it respond sanely to invalid input (a bad password, for example)?
Logic Error in IE6: In this case a Javascript Error

A Logic Error in IE6: In this case a Javascript Error

Knowing What to Test

The first thing that’s needed is a list of all possible user tasks and all the areas of the site they’ll need to complete those tasks, a site map is useful for the latter, a good understanding of your user roles is critical for the former. What you want to end up with is a list of what you want to test and how to find it on the site. I’ve included a simple example below.

What Where How
User Login Page http://example.com/login From the login form
Quick User Login http://example.com Use the quick user login form in the right column on the home page.
Add to Cart From any product page From the product details area, select a quantity and click the “Add to Cart” button.

Reporting Errors, or “Don’t Make Your Developers Want to kill You”

The thing with recording errors is to give enough detail that whoever has to fix the problem will be able to understand at a glance what happened, and then be able to reproduce it themselves. That means you need to include the following in your error report:

  • What you were trying to do?
  • The page (url) you were trying to do it on?
  • Any error messages that were displayed on the screen.
  • A screenshot of any errors messages, or visual artifacts that were visible as the error occurred.
  • The steps that you used to trigger the error, in enough detail that someone can follow them.

When I say “in enough detail that someone can follow them” I actually mean at least this level of detail:

  1. Type “rolly” into the login input box.
  2. Type ‘123456’ into the password input box.
  3. Click the “Login” button.

Without this level of detail it will be considerably more time-consuming to fix the bug, and your web developers will hate you for making their jobs unnecessarily difficult.

Actually Testing

Work methodically, trying possible combinations, sane data, insane data, recording whats been tested, recording errors.

When testing you must work methodically, and document your progress. As you test each example record what works, and record any errors. Sometimes all that you will need to do will be to glance at a page and make sure the layout matches up with your reference browser. In more complex cases, say a login form or any situation where the visitor needs to interact in a more complex fashion, you’ll need to dig in and actually complete the tasks your visitors would complete.

Using the example of a login form again, it’s important to realise that you need to dig deeper when mimicking your visitors tasks. It isn’t enough to just type a login and password, click the login button and, assuming it doesn’t explode, tick it off your list. You need to identify the major outcomes and test them all.

Here’s a more through example of login form test cases. A lot more could be added, but these cover the most critical cases:

What How Outcome
Sucessful User Login
http://example.com/login
From the login form: Using username “bob” and password “bob” The user should be redirected to their dashboard and a message should display saying “Welcome back”
Failed User Login Due to Invalid Password
http://example.com/login
From the login form: Using username “bob” and password “invalidpassword” The user should be redirected back to the login page, above the login form it should say “Your username or password was incorrect, please try again.”
Failed User Login Due to Invalid Username
http://example.com/login
From the login form: Using username “invalidusername” and password “bob” The user should be redirected back to the login page, above the login form it should say “Your username or password was incorrect, please try again.”
User can’t remember their login/password so they click the “Forgot Password” link near the login button
http://example.com/login
Click the “Forgot Password” link The user should be redirected to the “Reset Your Password” page.

Go Test Your Site Now

Even through I feel like I’ve been typing forever, a lot more could be said about this topic, not to mention related topics like Usability and Conversion Optimisation. Treat this as a basic primer into a much larger topic, now you know enough to know what questions to ask next.

Let me know how you get on, or if you have anymore questions.

Making Currency Conversion Easier for the Web

Converting between different currencies is a common problem on the web. It turns up most frequently for ecommerce systems, but it tends to rear its head anytime money changes hands.

There’s a number of paid and free solutions out there, one of the best free ones I’ve come across is the one in Google Finance. Sometimes you want something you can integrate more closely, or modify more easily.

When providing currency conversion on a website, there’s a couple of common approaches:

  1. You could do the conversion by hooking into an external webservice and pulling back results in realtime everytime someone requests a conversion.
  2. You could display a conversion tool from another site via an iframe. The Google Finance tool I mentioned above is a good candidate for this.
  3. You could pull the exchange rates from an external source and generate a cached version on your server. This method is handy when you can’t afford (or don’t want) the performance hit of having to make a HTTP request everytime you perform conversion. Also, since you can actually access the raw data (again, without a performance hit) you can perform more complex financial calculations then are practical with the other methods.

I’d say number 2 is the most common for the simple case when you just want to allow the user to convert X dollars from currency Y to currency Z. It’s simple and straightforward. Number 3 is best when you want kickass performance, maximum control, or need to provide currency conversion in an environment where you can’t assume that Internet access will be available when the user is accessing your app.

Number 1 is a hydrid of 2 and 3 and has the downsides of both without really capitalising on their strengths. I don’t see much benefit to using that one.

The tool that I’ve written uses method 3. It’s written as a small command line tool. It generates small libraries in the desired target language to quickly calculate exchange rates and perform currency conversion. The library that it generates will only convert from a single currency (that you choose when you generate the library) but can convert currency amounts from that single currency to a host of others.

Helpfully, it can also generate tools in multiple programming languages. Although, it currently only supports Javascript and Ruby.

Why Should I Care?

If we wanted to generate a Javascript library to convert USD amounts to other currencies we could execute:

generate_exchange_rates -c usd -f javascript

By default this will generate a Javascript file at the current path called “usd_exchange_rates.js”, which can be used from Javascript as follows:

// What is the Base Currency of the converter?
exchange_rates.usd.base_currency();
// Get the list of all supported currencies:
exchange_rates.usd.supported_currencies();
// Get the exchange rate of the base currency to NZD.
// Basically how many NZD dollars does 1 USD buy:
exchange_rates.usd.get("NZD");
// Convert 2 USD to NZD:
exchange_rates.usd.convert(2, "NZD");
// Get the raw exchange rates for every supported currency:
exchange_rates.usd.rates();

And this all happens without server callbacks or messy iframes. Neat.

A Quick Word on Currency Sources

The tool can actually support a number of currency sources. It comes with out-of-the-box support for only one, the free exchange rate XML feed from the European Central Bank, but using custom sources is just a matter of writing a custom source class. You should be able to hook in most simple sources in less than 100 lines of Ruby code.

If anyone has any suggestions for sources that should be included in the default package I’d love to hear them.

How Would I Use This with My Shiny New Web App

Once the converter is generated it’s just a matter of linking (in the case of a Javascript converter) or requiring it (for a Ruby one) where you need it, and then using it like a normal library for that language. The generated converters have comments in the source and are quite short so you shouldn’t have any trouble figuring them out.

As for generating the converters, a periodic job running somewhere on the web server (CRON is your best friend here) that just generates a new version over the previous one is all you need. When generating new versions of Javascript converters you may have some caching issues to address before the new version will be loaded by the client’s browser. It just depends on how aggressively your framework is caching Javascript assets.

Issues?

There’s no known issues, there is a few rough edges however.

No Tests Yet

I put this together in one short, ultra fast coding session (just before lunch in fact) and I didn’t pause to write tests. They will come, but probably not immediately as it already works for my purposes.

How Formatters Work is Deeply Hideous

Yup, I’ve got a bunch of ideas how I could tidy this up. But it’s mainly to save me some embarrassment. It works fine, it’s just not pretty.

Enjoy!

If you have any feedback, or issues with this tool let me know and I’ll help where I can. I’d also like to thank Intercity who actually needed currency conversion and partially sponsored the development.

The Joy and Horror of your first Facebook App

As I’ve recently completed my first Facebook Application I thought it might be worthwhile writing up my lessons learnt, in the hope that they will help someone else. Initially I though this post would be a technical discussion of the features and limitations of the API itself, but as soon as I started looking over my notes I realised most of the useful things I’ve actually learnt aren’t actually technical.

So here we go, gather ’round children.

Facebook Apps run on Facebook

No really, they do. The thing is everyone understands this, but I don’t think everyone necessarily appreciates how empowering, and constraining that can be. Usually when building an app, an initial concept is dreamed up, it’s fleshed out through prototyping or specing (or some combination of the two), bugs are fixed, design mistakes are correct, usability is improved, then you launch it.

Now, the misleading thing is that for a Facebook App the same steps are needed, but everyone one of them has an entirely different set of constraints. For example, say you’ve brain stormed a brilliant idea for a Facebook App that you’re convinced will dramatically increase your company’s/product brand visibility, make you rich beyond your wildest dreams, and win you the adoration of the girl (or boy) you always lusted after.

You start prototyping it and you discover that, because of how Facebook allows you access to it’s data, your killer feature can’t work exactly as you envisioned it. So you tweak your concept and throw yourself back into prototyping. Very soon you find yourself up against another wall and you have to stop and subtly change your design again. Pretty soon all the little changes add up and you realise that you idea doesn’t look anything like it did when you started. Maybe thats fine, but maybe that means the business plan needs to change or, if you are building the app for a client, you may need to renegotiate what you’re actually delivering.

Of course now you scoff at me, “It obvious that you’ll need to research the capabilities of any new platform before you finalise any plans”, indeed you chortle with glee at my naiveté. But I’m not convinced that just research is actually enough, I personally read everything I could get my hands on before starting, I even spent several days with my head buried in Facebook’s API WIki. And this was before I put pen to paper or wrote a single line of code.

Why isn’t it enough? Because there is a lot of information out there, but much of it is out of date. I spent an awful lot of time sifting through conflicting info trying to work out which to believe. This isn’t helped by the fact that, although Facebook’s own documentation is very good in some areas, it’s frustratingly vague in others. At the start, It can be very difficult to draw yourself a convincing conceptual picture of how all the various Facebook pieces fit together.

Your next problem is that, just when you’ve figured it out, Facebook goes and changes on you. It’s a very fast moving platform that can leave you, your plans, and your client eating it’s dust if you aren’t paying attention. Agility is important here, there’s nearly always a way forwards but it might not be exactly as you envisioned it at the start.

Eh, but you said it was empowering too, right?

Absolutely. You get to leverage (steal) all the best bits from Facebook and take advantage of all their hard work making their platform work on every browser in existence. You need a Captcha widget to stop your app getting spammed to death? No problem, just use Fb:captcha. It already does most everything you’re likely to need and you don’t need to maintain it. Facebook’s doing that for you. There are pre-written widgets for feed items, comments, editors, user selection, dialogs, and variable cornucopia of other interactive gifts that you can embed into your app with the absolute minimal amount of effort.

Think of how much time you usually spend designing and styling the look and feel or your web apps, then take one tenth of that and that’s probably what you’ll need here. I mean, apart from adding your own highlights and bits of branding, you’re going to end up with something that looks like Facebook. You could make it look completely different from Facebook but you don’t really want to freak out the Facebook user’s, do you? So you’re mostly going to use Facebook’s colours and their styles for buttons and tabs (not to mention headings, links, etc).

There’s plenty of information out there that details how to get the Facebook look, you’d be surprised how far you can get initially just by doing a bit of copy and pasting and focusing on getting the basics (text sizes/fonts, link colours) right. Don’t get me wrong, you’ll still need to put time into spit and polish. You still need good copy. usability, and all the other fun things that go hand in hand with this type of work, but the style issue? Mostly sorted for you. This frees you of a lot of additional cognitive load and allows you to focus on the stylistic details that really allow your app to slap its users in their faces with its own unique awesomeness.

So…in summary

The important thing to remember is that, if this is your first Facebook App, then allow much more time for research and prototyping. If your deadline is tight and you can’t spare the time to learn by doing then you better be agile in your requirements. If you play by the rules that Facebook sets down you can achieve some very cool things, if you don’t you are going to have a very frustrating and fundamentally futile fight ahead of you.

So pick your battles and make the best use of the advantages Facebook can give you, but if you intend to wrestle Facebook into your own vision of how things should work well…don’t say I didn’t warn you.

Quick Search /
Beginning Cross-Browser Testing of Websites
April 7th, 2010

Today I thought I’d write a little about how to test that a website looks and behaves consistently across a range of modern browsers, and how to correctly document the situations when it doesn’t. We’ll discuss issues that can complicate this, including different browser versions, and inconsistencies between operating system.

Making Currency Conversion Easier for the Web
February 14th, 2010

Converting between different currencies is a common problem on the web. It turns up most frequently for ecommerce systems, but it tends to rear its head anytime money changes hands. There’s a number of paid and free solutions out there, but sometimes you want something you can integrate more closely, or modify more easily.
I’ve written a small tool that aims to make your life easier when dealing with currency conversion in your own apps.

The Joy and Horror of your first Facebook App
July 18th, 2009

As I’ve recently completed (well almost) my first Facebook Application I thought it might be worthwhile writing up my lessons learnt, in the hope that they will help someone else. Initially I though this post would be a technical discussion of the features and limitations of the API itself, but as soon as I started looking over my notes I realised most of the useful things I’ve actually learnt aren’t actually technical. So here we go, gather ’round children.