system images for servers

Registered by Steve Langasek

Follow up on discussions at Canonical sprint about providing system image updates for server environments. Confirm the design, and determine what we intend to implement for utopic.

Blueprint information

Status:
Not started
Approver:
Steve Langasek
Priority:
Undefined
Drafter:
Stéphane Graber
Direction:
Needs approval
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Whiteboard

* explicitly not on the table to disable apt, this will always be an option
* what kind of image could we build for a server system image?
 * not going to cover all use cases; initially, a small image similar to cloud images that we can run workloads on top of
 * sabdfl suggestion to call it the Ubuntu Core image
 * cjwatson says we should be able to produce something quite similar to the current ubuntu core tarballs - but not identical... click, system-image, bootloader e.g.

 * bootloader challenges: can we make this support both UEFI and BIOS?
  * compatibility with older hardware that people are likely to test on, plus VMs, implies BIOS
  * probably install both bootloaders, as the binary bits (grub-pc-bin, grub-efi-amd64-bin, etc.) don't conflict; may need some care with grub-install calls

* on the phone, we reboot to recovery to apply updates.
 * for server, we may want to avoid this
 * the system-image spec does include a 'bootme' flag; for server we might apply the updates live (mounting the rootfs rw only in a namespace, so the rest of the system processes can't see it) and reboot only if required and in any case only after upgrade
 * s-i "reboot" step (really, apply-update) is hookable, so possibly write a separate app for applying updates and then have s-i call this thing. might need small amount of dev around separating application of update from reboot (i.e. you'd still do that on server if a bootme flag exists, but only after apply-update) - probably also need D-Bus API changes.

* should system-image metadata include an indicator to restart apps, as opposed to restarting the whole system?
 * barry suggests that this metadata should be in the app instead
* do we care about distinguishing between service restart and system restart?
 * yes, because the BIOS itself is quite slow on many of these
 * but maybe we could do kexec, CRIU
  * not for the first iteration

* want to support juju local provider as an interface for this?
* install lxc and use it an lxc host

* need to support installing services as click packages
 * this means click packages need to be able to attach to a system hook that lets them provide upstart jobs or systemd units
 * cjwatson proposes this being systemd from the start
 * there is consensus on this
 * systemd units provided by the click package, but should be filtered
  * also needs extended, not just filtered
 * systemd namespacing support allows unsharing, but not mapping

* which services should be click-ified?

(?)

Work Items

Work items for ubuntu-14.09:
[stgraber] Ubuntu core livefs image via a new job, for amd64 only: DONE
[cjwatson] define a core-1 framework: INPROGRESS
[jamesodhunt] Fork initramfs-tools-ubuntu-touch as initramfs-tools-ubuntu-core, rename the various scripts and update scripts/touch to support the different partition layout: DONE
[jamesodhunt] Make a new binary package for ubuntu-core configuration and ship /etc/system-image/writable-paths (copy from lxc-android-config): DONE
[jamesodhunt] Rewrite system-image-upgrader in python to work in main FS context: DONE

Work items for ubuntu-14.10:
[mvo] make ubuntu-core-14.10 framework available on core image: DONE
[jamesodhunt] define partition scheme for usb key and VM: TODO
[jamesodhunt] device tarball split out from image: TODO
[sergiusens] add support for this image type into ubuntu-device-flasher: DONE
[jamesodhunt] support system image upgrade (VM): DONE
[jamesodhunt] support system image upgrade (on USB key): TODO
[jamesodhunt] write integration tests for upgrader: INPROGRESS
[jamesodhunt] put cloud-init in the image: DONE
[mvo] click commandline support for talking to the store (lp:~mvo/click/acquire+sso): INPROGRESS
[beuno] resolve requirement for authentication from click commandline on the server to talk to the store: INPROGRESS

Work items:
[mvo] click hook support for systemd unit files (lp:click-systemd): DONE
[mvo] click hook for cli apps (lp:click-bin-path): DONE
[mvo] click support for acquire clicks itself (lp:~mvo/click/acquire): INPROGRESS
[mvo] click support for talk to the server click store (lp:~mvo/click/repository): INPROGRESS
[mvo] click support for sso store access (lp:~mvo/click/sso): INPROGRESS
[cjwatson] work out BIOS+UEFI support: TODO
[jamesodhunt] port ubuntu_command interface to server so that s-i has something to call to apply the update: DONE
[barry] some si work to separate apply-update and reboot steps (D-Bus API changes, etc.) bug #1381538 : TODO
click support to provide "stuff" to other click packages: TODO
[jamesodhunt] prototype mounting the system read-write in a namespace for updates on a running system: DONE
Also include /etc/system-image/archive-master* into that package (? already in system-image-common): DONE
[jamesodhunt] put necessary files in /var/lib/cloud-image to prevent image in IoT mode from looking for external data sources on boot: DONE