Destroy device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
4 Domains
3
4.1 Create domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
4.2 Domain startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
4.3 Domain shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
4.4 Domain reboot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
4.5 Add device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
4.6 Remove device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
4.7 Domain save, restore, migrate . . . . . . . . . . . . . . . . . . . . . . . .
4
5 Events
5
6 Data
5
6.1 Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
6.2 Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
6.3 Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
6.4 Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
i 1 Introduction
Some notes on using the control registry and events to perform domain and device opera-
tions.
2 Control registry
The control registry is the persistent repository of data for domains and devices. The con-
trol registry is structured as a tree of data values, addressed by path names. For example
/foo.example.com/domain/123/name
could be the path pointing to the name for
domain 123 on node foo.example.com. The registry supports reading and writing data at
paths and enumerating the children of a path.
The control registry also supports metadata, for example
/foo.example.com/domain/123/name/@mtime
is the modication time of the domain name. All metadata names begin with @.
2.1 Events
In order to communicate with each other, and signal changes in the control registry, clients
use events. Each event has
 
topic: pathname identifying the event
 
header: set of string keys and values indicating what the event is about. The values
are usually pathnames of trees in the control registry.
 
data: event content..
For example, a /backend/device/status event indicates the status of the backend
for a device. The header device is the pathname of the device in the control registry. The
data includes the old and new state of the device, to make it easier to see what the state
change is.
Clients subscribe to events, giving a pattern for the topic and subject they are interested
in. Suppose there are events with topics /a/b/c, /a/b/x, /a/b/y/z. A client can subscribe to just
the rst one with topic /a/b/c. It could subscribe to them all with /a/b/*. It could subscribe
to /a/b/c and /a/b/x using /a/b/?. A trailing * matches anything, and a trailing ? matches
one level of slash.
3 Devices
Each device potentially has three components:
 
The control data managed by its device controller running in the control domain
(usually domain 0).
 
The backend data managed its backend driver (often in domain 0 but not necessarily).
 
The frontend data managed by its frontend driver in the domain using the device.
Note that devices might be persistent, and exist even if they have no backend. For exam-
ple an iSCSI device or virtual network interface could exist persistently, without backend
resources. Similarly an abstract disk device might exist without being connected to its
backend.
1 3.1 Virtual block device
A virtual block device is based on a linux block device in some domain, which it exports
to another domain. Its parameters include the device to export, the device id it is exported
as, and the access mode (read or write).
3.2 Virtual network interface
A virtual network interface provides what looks like a network interface to a domain. Its
parameters include the MAC address and MTU of the exported device, and the MAC ad-
dress and interface name of the linux virtual interface used by the backend.
3.3 iSCSI device
An iSCSI device will provide a virtual block device, but it rst has to be connected to its
server, and the local device representing the connection may vary. So there is a mapping
from iSCSI device parameters to the block device exported to a domain.
An iSCSI device controller can do the mapping, creating a block device, and this can
then be attached to a domain. However, for domain save and migration to work, the block
device conguration has to be the original iSCSI conguration, not the lower-level block
device conguration that is created as a result.
3.4 Create device
1. Controller creates device data tree, including backend identier.
2. Controller sends /device/status.
3. Backends receive the status event and use the backend identier and device info to
see if they should deal with it. If not they ignore it. If one decides to handle the event
it creates backend state (port etc.) and sends /backend/device/status.
4. Controller waits for device status from some backend or timeout. If timeout or status
bad sets control state bad and creation fails. If status ok sets control state ok.
Controller waits until device status is set.
Note that we use backend identiers rather than domain ids to indicate the backend to
use. This is to allow more than one backend for a given device type in a domain. Also
device setup does not need to know which domain the backend is in to create the device.
3.5 Congure device
Change the parameters of an existing device.
1. Device controller changes device cong and sends /device/status.
2. Frontend gets status event and updates cong, sends /frontend/device/status.
3. Backend gets status event and updates cong, sends /backend/device/status.
4. Both ends update from the status.
Note that the status events may arrive in either order at frontend or backend, but if the end
examines the device cong this should be ok. (Might have to signal cong change in the
data to make it easy to tell it has changed.)
2 3.6 Destroy device
1. Device controller changes device state to destroyed and sends /device/status.
2. Frontend gets status event and updates cong, sends /frontend/device/status.
3. Backend gets status event and updates cong, sends /backend/device/status.
4. Both ends update from the status.
4 Domains
Domains are currently run-time entities, something like processes. They come and go, and
there is no representation for a domain that is not running. It may make sense to make
domains, or rather virtual machines, persistent entities. This makes it possible to examine
vms whether they are running or not, and to keep track of saved states.
4.1 Create domain
1. Domain controller gets basic domain parameters and creates domain entry in control
registry.
2. Create domain.
3. Make sure controllers for domain image type and device types are available.
4. Get image controller to congure image and build domain.
5. Get device controllers to congure requested devices, create them. When creating a
domain it should be possible to request connection to existing devices, or to request
devices be created and connected.
6. When device creation is complete for all devices, domain is created (but not running).
7. If image creation or any essential device creation failed, domain creation fails.
4.2 Domain startup
Each domain frontend:
1. Enumerate relevant device subtree of the domain.
2. Create frontend state for each device (port, shmem).
3. Send /frontend/device/status for each device.
4. Backends receive status, if handling that device then get connect data for the device
(port, shmem) from frontend tree and connect up. Backend then reports device status
with /backend/device/status
5. Frontend waits for backend device status or timeout on all devices and sets device
frontend status.
6. Frontend reports status for each device with /frontend/device/status.
3 4.3 Domain shutdown
1. Domain controller set control state of the domain to shutdown and sends
/domain/status
.
2. Backends receive, and those with domain devices disconnect them and report sta-
tus with /backend/device/status. Backends delete backend data for their
devices.
3. Domain controller clears frontend ports, destroys domain.
Domains can fail to shutdown properly - the running domain may miss or ignore the shut-
down command, or shutting down may hang. The domain controller needs to handle shut-
down failure.
4.4 Domain reboot
Stopping the domain for reboot is similar to domain shutdown, except that the domain state
is reboot instead of shutdown, and some devices may persist across reboot (consoles for
example). Restarting the domain is similar to domain create and startup except that the
domain id is reused and devices that persisted do not need to be recreated.
The domain controller needs to cope with failure to shutdown or reboot. Recreating the
domain may fail or the domain may crash during reboot.
The domain data tree should exist across reboot, so that the domain data can still be
seen after the running domain has been shutdown and before it has been recreated.
4.5 Add device
1. If device does not exist, create device.
2. Set device frontend domain to the domain being attached to. Send /device/status.
3. On success the domain notices the device from the /device/status event.
4. Frontend brings up the device using the same process as in domain startup.
5. Domain controller waits for frontend status or timeout to determine result.
4.6 Remove device
1. Device controller sets device control state to down and sends /device/status.
2. Frontend receives controller or backend status, tears down frontend and noties sta-
tus.
3. Backend gets the controller or frontend status, tears down backend and noties status.
Both ends for the device will receive a controller status event and a device status event
from the other end - these events can occur in either order.
4.7 Domain save, restore, migrate
Domain save:
1. Domain controller sends save event.
2. Start domain saving (write)