QA move to Image Based Updates and click packages

https://wiki.ubuntu.com/ImageBasedUpgrades
https://blueprints.launchpad.net/ubuntu/+spec/qa-s-image-based-updates

New OS updates imply two new things QA-wise:
* testing new OS updates
* adapting QA tests to new OS update (read-only system fs)

Also simultaneously switching over to Click packages as the system is r/o.
Testing new OS updates
3 main tests to add to smoke tests:
* bootstrap
* delta updates:
  * with one delta
  * with two deltas
* full update (uses a full image instead of a delta to update)

Updating goes through a series of reboots, so QA testing will need to detect these reboots

There is a DBus service to trigger the OS updates (that settings app hooks into) that the tests should use too to trigger updates

Adapting QA systems
apt-get install not working (read-only fs); problematic for some tests and to pull common deps such as autopilot

(fginther) adapt upstream merger infrastructure to use Click instead of debian, run autopilot tests on device, trigger merge
- Building (sergio)
  - click chroot builder
    - jenkins plugin similar to pbuilder?
  - licence check hook
  - jenkins job which creates ubuntu touch image
    - create a daily image
    - perform an apt-get update/dist-upgrade prior to building ss
  - build with and without coverage
    - execute unit tests (with coverage)
  - create click package
  - verify symbols don’t link to anything outside of the base image or the click package itself.
  - export click package as a jenkins artifact
- Testing (ginther)
  - Testing through click image
  - Install archived click package on top of provisioned device
  - Pass MP path as an argument to source tests
  - Remove click package to restore environment

Need to adapt autopilot tests to launch Click apps rather than directly running binaries

Provisioning device with tests when using click packages (sergiusens)
======================================================

- Embed the bzr branch somewhere in the click package that we can retrieve.
- Create a set of base emulator dependencies (unity, osk)
- Fetch the click package branch
- Fetch the emulator dependencies
- Bundle the emulators and tests into a python package
- Push to device
- autopilot run [packagename].* (or whatever)

http://pad.ubuntu.com/w7rxxyjvCR

20130813:
We agreed to use the click application name and parse the installed manifest for the user (click list --manifest) autopilot will have a 'launch_click_app(app_name)' simil interface to launch apps and will hook up to the users installed click list to retrieve the version to generate a full app_id

Work items:
Do real smoke testing after performing an update to validate things really work: TODO
Adapt UTAH to test images without using developer-mode: TODO

Work items for ubuntu-13.07:
[sergiusens] phablet-flash flag to pull older images than the latest one: DONE
[sergiusens] phablet-flash flag to be able to pull new images as well as current: DONE

Work items for ubuntu-13.08:
[doanac] Adapt UTAH to handle new images (use developer mode): DONE
[cjohnston] Setup smoke jobs of current apps to use new images: DONE
[javier.collado] Test new OS updates -1 and -2 image delta: INPROGRESS
[pwlars] Agree with doanac in the name of these images and create jobs that make them appear on the dashboard under a new name, for a while there will be results for existing images and new images: DONE
[pwlars] Modify scripts and generate jobs for new images: DONE
[pwlars] Assess the situation with current HW and decide if we need to run new tests on different devices to avoid disruption of old images: DONE
[larry-e-works] Order new hardware if required and attach to phoenix: INPROGRESS
[fginther] Adapt upstream merger infrastructure to use click instead of debian: TODO
[doanac] Launch autopilot tests via a upstart: INPROGRESS
[thomir-deactivatedaccount] Ensute autopilot can be installed on the new images and can run from a click package: TODO
[om26er] Do manual run of autopilot tests on the new images to make sure all pieces are in place (pre-requisite to starting any kind of automation, green light from omer): TODO
[pwlars] Monitor new results for a while until they are stable: INPROGRESS
[barry] Add flag to force full OS update (LP: #1192575): DONE
[stgraber] Will need a new daily-proposed channel on system-image.u.c: TODO
[sergiusens] Seed autopilot in OS images: DONE
[sergiusens] Jenkins click builder plugin: TODO
[sergiusens] Create tool to provision tests for click packages: DONE

Work items for ubuntu-13.09:
[javier.collado] Add new test cases for images updates: TODO
[pwlars] Start running new test cases for updates on regular cadence: TODO
[pwlars] Change dashboard to use new touch_ro version numbers: DONE
Avoid running tests when images are garbage collected: TODO
[doanac] Add support for daily-proposed channel in CI infrastructure: DONE