Improve certification testsuite, tools and processes for external testing

External testing, that is, the testing performed outside of Canonical
certification labs, is a significant part of all the testing
activities around Ubuntu that is growing on every cycle.

It is becoming more common that people outside the certification team
want to run our certification tools and tests, to either submit
results to Friendly, test their systems, or check if a system would
pass certification.

As part of the continuous improvement practices currently in place,
the certification team will focus on enhancing the user experience for
external testers for both desktop and server, so that they can run
test cases with the maximum coverage (both for hardware and software)
and report problems effectively.

The areas that have been identified to work on during the next cycle
are the following ones:

== Test case usability improvement ==

Some test cases executed internally in the laboratory assume the
presence of devices and configurations needed to run them
smoothly. Unfortunately, when these test cases are executed in a
different environment, they fail and cannot be used unless all the
prerequisites are properly fulfilled.

* We need to take into account that for the external whitelist is
  going to be run outside the lab, so we cannot make assumptions on
  configuration

* Automate as much as possible to get rid of any assumption and also
  run the test cases on any system that matches a minimal set of
  requirements.

* Provide guidelines either in the documentation or the test case
  description to configure the environment properly when full
  automation isn't attainable.

* Improve general usability of the testcases, to allow people less
  familiar with the tools, to run the test suite without having to ask
  questions.

== Documentation ==

Documentation is intended to solve any doubt that could prevent a
testing session from finishing properly with the submission of results
and the reporting of the problems found.

* Create a document that describes the procedure to run test cases
  using the external whitelist in a desktop.

* Update the document for the server that was created in the previous
  cycle. This includes, for example, adding a section about how to
  troubleshoot problems. For instance, what to do if a test case
  doesn't work as expected, the test results cannot be submitted, etc.

* Create training material for TAMs and customers, so that they can
  get some hands-on experience before certifying a device for a given
  release for the first time.

* Provide guidelines either in the documentation or the test case
  description to configure the environment properly when full
  automation isn't attainable.

== Whitelist coverage ==

The different set of test cases between the internal and the external
whitelist, causes that the coverage achieved depends on the whitelist
executed when it shouldn't be the case.

* Review both whitelists and make sure that they provide the same
  amount of coverage.

* When the improvements on test case automation aren't enough to get
  the desired coverage, create alternative and easy to configure test
  cases that can be used to test the same capability/feature in a
  different environment.

== User interface usability ==

The new Qt interface is a step forward in terms of having a modern
interface that looks appealing to users. However, we have to make sure
that it lives up to the expectations of testers.

* Review the interactions required to run test cases and solve any
  issue that prevents the tester from having a good experience that
  encourages collaboration and makes reporting problems easy. For
  example, look at the consistency of buttons when marking a test
  cases as passed/failed/skipped.