Research/develop technology for automatic hardware/UI related tests

pitti, 2012-11-23: I tested the current state of autopilot for GTK, and I'm afraid it is still in its infancy; that, and the fact that GTK itself makes introspection hard/impossible makes it rather hard to use autopilot-gtk for GNOME programs right now; it apparently works a lot better for Unity and Qt programs. I filed bug 1082385, bug 1082388, and bug 1082391 for these difficulties.

pitti, 2013-01-09: Tried various approaches to simulate multiple monitors with xrandr:
- Xephyr supports RANDR, but not Xdmx when binding two of them together
- the dummy X.org driver has very limited RANDR support, just for changing resolutions
- our Qemu/KVM packages do not yet support the "-qxl" option to simulate multiple monitors

X.org drivers don't use /sys for detecting/controlling monitors (which we could easily mock), but use direct hardware access, presumably as X.org targets more than just Linux. Mocking that seems impractical. At this point the most practical approach seems to extend the dummy driver to emulate multiple monitors.

pitti, 2013-01-21: Initial patch for Xvfb support for XRandR: http://lists.x.org/archives/xorg-devel/2013-January/035114.html → This seems more practical and portable than the dummy driver.

Work items:
[pitti] umockdev: find a way to intercept generic ioctls (other than usbfs) and relay them to the test suite: POSTPONED
[pitti] umockdev: intercept /dev node access and relay them to the test suite: DONE
[pitti] see to which degree PtP/MtP devices can be simulated in umockdev: DONE
[pitti] umockdump: add option to dump URBs from real devices into files: DONE
[pitti] umockdev: intercept and simulate usbdevfs ioctls from dumped URBs: DONE
[pitti] umockdev: create package and daily PPA: DONE
[pitti] check what it takes to simulate multiple monitors for X.org/randr, to be able to write XRandR and multi-monitor tests: POSTPONED
[pitti] rewrite some apport-gtk tests in autopilot, compare them side by side, check if there is anything that we cannot do with autopilot and discuss/file bugs for those: POSTPONED
[jibel] find a representative from GNOME (like gedit), software-center (like mvo), PS (unity/indicator developer), and maybe UbuntuOne, who want to take part in "UI tests" research: POSTPONED
[jibel] discuss/prototype/evaluate various solutions (ldtp, dogtail, white-box testing with better GTK API, etc.) with upstream representatives: POSTPONED
[jibel] pick an UI test approach that upstream likes and get some tests upstream: POSTPONED

Work items for ubuntu-13.04-month-6:
[pitti] review some simple desktop integration tests with autopilot and evaluate its utility for that: DONE