How To become a eGov Testing Machine developer



Testingmachine is used to test eGovernment services.  The eGovernment services can vary from country to country but generally it grants citizens access to important documents and information. In most EU countries there are additional services like paying property tax that can be done online. Most governments in Europe are leaning in this direction in order to decrease administrative overhead.

Another challenge to overcome will be testing eGov services on mobile platforms, more specifically automating tests under Android. Good news is this is possible using Selenium. The one major obstacle that we face is getting around the smart card login. So far we are unaware of getting this to work under Android unless you patch the kernel and this is of course not an option. More documentation and research is needed concerning this.

Now that we have explained what testingmachine is and what we are using it for we can begin explaining how you can become a developer.

Get source

Getting the code for the project is pretty straightforward.

You can either clone the repository using git with:

git clone

In addition you can download the zip file from the online repository here:

If this is your first time using Git, here is a resource to help you on your way:

Building and installing

  • Go to the tm-vmm directory

cd tm-vmm

  • Configure the software

./configure --prefix <installationdir>

  • Build the software


  • Install the software

sudo make install

  • Verify the installation

<installationdir>/bin/tm-vmm --list-clients


  • Create the directory $HOME/.testingmachine

mkdir $HOME/.testingmachine

  • Create tm-vmm.conf in .testingmachine, typically with your favorite editor (emacs?)

emacs ~/.testingmachine/tm-vmm.conf

In this file you can configure settings you want to use as default in your clients. It is perfectly possible to override these settings in your individual client configurations. For a list of variables, see section Configuration syntax in the full manual of Testing Machine.

Using Virtual Machine Manager

We thought it might make sense to explain very briefly how to use the virtual machine manager, just to get your feet wet and understand a little what is going on before we get to the more interesting stuff:

Starting with guest operating system visible

tm-vmm --start-client <CLIENTNAME>


tm-vmm --start-client Ubuntu-14

Starting with guest operating system headless

tm-vmm --start-client-headless <CLIENTNAME>


tm-vmm --start-client-headless Ubuntu-14

Checking status

tm-vmm --check-client-status <CLIENTNAME>


tm-vmm --check-client-status Ubuntu-14

Becoming a Virtual Machine Manager developer

Software Overview

The Testing Machine is currently made up by Virtual Machine Manager (tm-vmm) and documentation on how to write, execute and automate tests of eGov sites in particular but also other software.

Virtual Machine Manager (vmm)

tm-vmm is made up by bash scripts that let the user manage various virtual machine software in a general way. See the tm-vmm manual for more information.


Directory overview


Directory containing all programs visible to the user.

  • tm-vmm - main program for the management of virtual machines. This script is basically a parser for the functions as stored in the scripts in the scripts directory.

  • tm-vmm-auto - outdated script previously used to automate stuff


A directory containing scripts and other files that can be used while developing Testing Machine. Nothing of this stuff needs be included in the distribution.

  • bin - contains scripts

  • - script to create a new distribution


This is the directory containing the manuals and other documentation. The manuals are written in Markdown format.

  • *.md - Source version of the manuals

  • generated-files - all files generated from the .md files are placed here

  • pics - pictures used in the manuals

  • faq-pics - pictures related to the FAQ


Various examples of how to use Testing Machine. We will remove this directory since we switched to using a separate repository (tm-examples) for this purpose.


This is where the functionality of Testing Machine is stored.

  • functions

  • generate-conf

  • vmm-client-android-specific

  • vmm-client-qemu-specific

  • vmm-client-functions

loggers - directory containing various scripts that implements different logging functionalities.

  • debug

  • debug-logger

  • txt

  • txt-logger

Adding bash code or fixing bugs

Add a new command line option

If you want to a new command line option to tm-vmm you should update the bin/tm-vmm.tmpl file. Make sure to add code to parse the command line and also as an entry in the help printout.

Update the documentation accordingly.

Add bash functions

If you want to fix bugs or add new features add this in the scripts directory. If you add a bash file, make sure to update the Makefile.tmpl accordingly.

Writing documentation

All documentation is written in Markdown format. We use pandoc to generate html and pdf.

Each chapter/section is written in separate files to make them more easily re-usable. Since there's no *include* functionality in Markdown we had used the Makefile to cp/cat together the various manuals. Perhaps not the most optimal way of doing things - but it works. If you have improvement suggestion you're more than welcome.

If you add a markdown file, make sure to update the doc/Makefile.tmpl accordingly.

Building Testing Machine

The scripts and Makefiles in Testing Machine contains variables used to store the installation dir. This is needed to source all bash files.

The scripts and Makefiles are created, with the variables described above updated according to the installation dir, from templates.

Starting with eGov testing

Since you now know how to add functionality to testing machine we thought it would be beneficial to help those who wish to get started with testing web services. Our examples are based on the services that the Province of Bolzano use but it should be easy to understand and modify so you can apply the same thing to some web service you would like to test.

Getting the code for testing eGov services can also be done using Git. There are some example tests that can be run however we first need to install some packages on the host operating system. Use your package manager, here we use apt:

apt-get install openjdk-7-jre-headless ant

Now the environment is setup and we can clone the directory containing the examples

git clone

Navigate to


cd tm-examples/eGov/bolzano

This is an extension of the simple Testing Machine example that can be found in the parent of this directory. It contains some Bolzano-specific tests, and is using Selenium.

The init method (in the file), which is annotated with @BeforeClass, will set up the driver object for us, and it does so before the actual testing start. The actual tests are contained in the file.

Finally, the driver is closed with a method annotated with @AfterClass.

Build and run the project simply by placing yourself in this directory (the directory containing build.xml) and type:

ant -Dtm.seleniumDriver=<Driver>

should be replaced with either AndroidDriver, FirefoxDriver, or ChromeDriver. When ChromeDriver is used, you must set the path to the driver file:

ant -Dtm.seleniumDriver=ChromeDriver<Path>

When AndroidDriver is used, you can set the optional tm.seleniumUrl parameter in this way:

ant -Dtm.seleniumDriver=AndroidDriver -Dtm.seleniumUrl=<URL>

The Ant command will compile the classes and run TestNG, which in turn will generate an HTML summary of the tests in a "testng_output" subdirectory. (If you want to change this directory, all you have to do is to set the -Dtm.testngOutput parameter. The results can now be copied to any machine using SSH.

The tests in the tm-examples/eGov/bolzano performs Selenium based tests that check various things such as links on a page, if a certain text is present, if the header of the page matches the website and so forth. The Selenium documentation has a lot more detail, automating Selenium is covered here but not how to write Selenium tests, that is out of scope

Starting with eGov testing on a virtual machine

Once the host environment has been setup and the guest virtual machine is up and running we can now start testing on the virtual machines that we setup in the previous tests. Lets fetch the necessary Selenium jar that will be needed.


Now we copy this over where it is needed, replace <CLIENTNAME> with the appropriate VBox name

tm-vmm --client-copy-file selenium-server-standalone-2.37.0.jar <CLIENTNAME>:~/selenium-server-standalone-2.37.0.jar

To proceed with the next step we need to make sure that port forwarding is enabled from the host port 4444 to the guest port 4444, this is done in the settings of VirtualBox:

Settings > Network > Advanced > Port Forwarding

VirtualBox will route connections to localhost on the host port to the port on the guest.

Everything is now in place and we can begin to run our tests, once again from the host issue the following command:

ant -Dtm.seleniumDriver=FirefoxDriver -Dtm.seleniumURL=http://localhost:4444/wd/hun -Dtm.testNGOutput=test_results/<CLIENTNAME>-`date +%Y-%m-%d`

The results that have been generated can now be viewed locally, or copied to some remote destination if needed.

scp -r testng_output someuser@someserver:/somedir

To make things even simpler there is a bash script that does all the heavy lifting for us. Not not only that but we can also run the same suite of tests on multiple guest VirtualBox machines that we have setup

In the same directory, tm-examples/eGov/bolzano you can simply run:

tm-vmm --clients-exec “test0;test1;test2”

Now tm-vmm will start the same action for clients test1, test2 etc etc. If you want the script to perform the tests on all clients, simply leave out the client names, or specify the ones you want the tests to run on.


The main part of setting up the environment is now complete. The script can be automated using cron. The actual work here comes in writing the eGov tests which can be rather time consuming but it is well worth it. Every eGov website will be slightly different, we hope however that with the examples we have provided that anyone using this will quickly understand how to become an eGov developer, by expanding on the existing Bash code base and being able to modify the Selenium tests that have been written in Java. To conclude, once the setup with the virtual machines is complete, it is simply a matter of writing tests for Selenium, that can then be run against different browsers on virtual machines without having to change any code in the tests.

Project Communication

Reporting bugs

Try to be as precise as possible when reporting bugs. The more information we get the bigger chance we have of fixing the problem.

Use the mailing list below to report bugs

Getting involved

How to join: clone the repo, try it out -- join the mailing list.

For more information read the developer guidelines.

Mailing List

We have one mailing list for the project:

Join this list here:

If you send emails to this list as a non subscriber chances are it will get lost.

If you want to report a bug:

  • use a github account and add an issue

  • subscribe to the mailing and send the report to the list


Home page

Social Media
No comments yet. Be the first.