Being a web development studio we would like to include the end-users opinions during the development of every website project we work on. We have been partnering with Testbirds over the last couple of months, to help us achieve this. Testbirds offers testing via the crowd. They have a large number of testers in countries all over the world and offer a variety of user tests (100,000 registered testers on our testing platform, located in 170 countries with over 250,000 devices at your disposal). Until now we have worked with them on card sorting and concept tests, but bug testing is also a type of test that they offer. Personally I am very enthusiastic about the results, so I wrote this blog to give some insights about the process of user testing with Testbirds.

Situation

Indivirtual has been collaborating with Business Lease on their client website since 2014. On this website lease drivers can find information and tools for their lease car, and fleet managers can manage their fleet of lease cars. The lease drivers are the largest group of users on the website, but they use the website only once or twice per year. Therefor, it is important that the website is structured very logical and intuitive so the drivers can easily find what they need. Business Lease is active in six very diverse European countries and it is a challenge to align the websites from a business perspective with each country. With Testbirds we wanted to know if the users have a different experience during their visit of the website based on their country background.

Preparation

Based on the HTML templates (that were built for the website), we created a prototype of the Business Lease client website with click paths, realistic data and user interaction. This was a perfect basis for the test, because the testers could test scenarios from the real world in the actual look and feel of the website. The templates were responsive so we could test them on mobile devices as well. In the statistics we noticed that mobile usage of the website was growing so we requested that 40% of the testers used their smartphone. The test scenario that we created consisted of 5 open question and 10 closed questions and covered most parts of the website but focusing on a maintenance scheduler functionality that we were about to introduce.

The Test

A selected group of testers, matching the profile of the typical lease drivers using the website under normal circumstances, were invited to participate. They can do the testing whenever they want as long as it is within the set time-frame. This particular test lasted for four days and 20 testers participated and finished their test. Apart from answering the questions the testers are requested to add screenshots to give us insights in the way they navigated the website and to record their findings. Testbirds only approves a test result if it meets their standard for clarity and completeness. As a client we saw the results slowly trickling in via the online dashboard provided by Testbirds. After the testing period had passes and all the results were approved by Testbirds they provided us with a report that summarized the test findings.

Results

Even though there were differences between the test users from the two selected countries (Poland and the Netherlands), they differences weren’t as great as we had expected. The results confirmed that we are on the right track, but that there are parts that could be improved as well. The testers came up with useful suggestions for new functionality that will certainly be added to the website in the near future.

This type of crowd testing is an easy way to involve users and get valuable insights quickly. The results come in fast so there is more time to work on improvements!

Roundtable on November 24

Would you like to know more about the advantages of these types of user testing? On the 24th of November we will be hosting a roundtable to look at the challenges of online projects. Gert van Vliet (Indivirtual) and Andrew Maat (Testbirds) will talk you through the major challenges that they face at their clients and inspire you to get more grip on your online projects.