This week some random tests were made using a smart phone, tablet and the Google TV emulator. The idea was to understand how well eGov services work on these devices and how easy it is to access the services. First up were mobile devices, we used a Nexus 4 running Android 4.4, using Firefox which is freely available from the PlayStore.
The website defaults to the mobile version but we found this to be more complicated for what we wanted to test so we opted for the 'full' version to get busy.
While this view makes things a lot harder to click around at least we could start looking into beginning manual testing. We began looking at the services that are being offered.
There are a significant amount of services that are being offered and we realized rather soon that doing any sort of manual testing on a mobile phone for so many services would be almost impossible, not to mention the time and resources that it would take. This is the main reason for using automation. One last screenshot from the Nexus 4:
Next up on the list we used a Nexus 7 tablet, running Android 4.4 and the default Chromium Browser that gets installed with Google devices. Here we pretty much had the same experiences, while it was easier to navigate with a tablet due to the larger screen real estate the underlying problems were the same.
Immediately we started testing some services, with a focus on those that provide online documents to download:
And were surprised to find out that the download failed every time!
Nothing could be opened:
Now the problem could be related to a few things, the website acting up, the tablet, Chrome but who knows. To start we would need to undertake more manual testing since there are a great deal of documents that have been made available to the public. This would take too much time and would be a daunting job as well. Once again I mention automation and the rather obvious need for it.
Last on the list are SMART TVs. What we mean by this is that there are (according to our knowledge) to camps for SMART TVs, the market leader which is Samsung and the SMART TV Alliance which consists of other manufacturers such as Philips, Toshiba, LG, etc. Since at this time we do not actually have a physical SMART TV to test on we went for the Google TV emulator, which could give us somewhat of an idea of what testing is like.
Luckily the entire testing did not go like this, see below for more:
Much better. Lots and lots of screen real estate. Oddly enough the interaction was not that user friendly. In the emulator we only enabled the keyboard of the computer and the mouse, not the simulated remote that can be used. It took us a minute or two to understand that you need to keep the left mouse button pressed in order to then drag the screen down, something that usually gets done with simply scrolling the mouse wheel. While not a deal breaker it was certainly an inconvenience to navigate with a mouse in that way. In 'real life' we would have a remote control but would still need to keep the arrow down button pressed in order to scroll down the page. It is a slow process and testing like this (on the emulator) at least it would take even longer than on a tablet or phone. One last screenshot below:
To summarize we will now begin more formal testing and automate everything as much as we can. Manual testing is simply not time efficient as there are too many devices on the market and the amount of services that need testing are too many, we need tools to help us do that job.
More information on SMART TV testing will be coming in future blog posts.