19dd973159
We had a problem with nodepool and RAX where we weren't bringing the network up fast enough because eth0 was waiting around for a DHCP response (that would never come) before glean got started to configure the interface. This seems to be because network.service was started first, and defaulted to trying to bring up the interface with DHCP. The whole ordering of systemd is a bit of a black art. Reading [1] says network-pre.target is a target that may be used to order services before any network interface is configured. It's primary purpose is for usage with firewall services that want to establish a firewall before any network interface is up. It's a passive unit: you cannot start it directly and it is not pulled in by the the network management service, but by the service that wants to run before it. Network management services hence should set After=network-pre.target, but avoid any Wants=network-pre.target or even Requires=network-pre.target. Services that want to be run before the network is configured should place Before=network-pre.target and also set Wants=network-pre.target to pull it in. This way, unless there's actually a service that needs to be ordered before the network is up the target is not pulled in, hence avoiding any unnecessary synchronization point. So we're not a firewall, but it seems like we fit that model of wanting to configure the interface from config-drive. So, as it suggests use the "Wants" and "Before" to get the glean service to run before we try to enable the network. (We probably don't actually need the After=network.target in their either. You can read on [1] that is mostly for shutdown-ordering, so is not relevant to glean, but it doesn't hurt) Honestly, the relationship between udev, systemd service-unit templates, network.service and ifup/ifdown is pretty uncharted territory. But we tried this with Ib9d7cc711186299c77795e5147bf332c56e3aa7c and the centos-minimal node booted and nodepool connected to it fine. Change-Id: I066c0c89a30280af9d7a5d8d1a916daeddc84fa3 |
||
---|---|---|
doc/source | ||
glean | ||
.coveragerc | ||
.gitignore | ||
.gitreview | ||
.mailmap | ||
.testr.conf | ||
CONTRIBUTING.rst | ||
HACKING.rst | ||
LICENSE | ||
MANIFEST.in | ||
README.rst | ||
rebuild-test-output.sh | ||
setup.cfg | ||
setup.py | ||
test-requirements.txt | ||
tox.ini |
README.rst
glean
Glean is a program intended to configure a system based on configuration provided in a configuration drive.
Why would you want glean?
Different cloud providers have different ways of providing networking and other configuration to guest virtual-machines. Many use DHCP but others, notably Rackspace, use configuration provided via a configuration drive.
What does glean do?
Broadly, glean checks for configuration drive based information and, if found, uses that to configure the network. If config-drive is not found, it falls back to configuring any available interfaces with DHCP.
Specifically, it will mount the special block-device with label config-2 and parse the network_info.json and vendor_info.json files within. If appropriate network configuration is found, it will write out network configuration files.
The format of the network_info.json file is documented here. Please note that glean does not implement every feature listed.
If no network info is found there, available interfaces can be probed from /sys/class/net and any that appear to be up will be configured for use with DHCP.
It will also handle authorized_keys and host-name info provided from meta_data.json.
How does glean do this?
systemd environment
glean install will add a udev rules file (99-glean.rules) that triggers on any network device being added. This will run the glean@.service systemd template for the interface specified.
This systemd unit firstly determines if there is already a /etc/sysconfig/network/ configuration for the interface; if so, the interface is considered configured and skipped.
If not, glean is started with the interface specified. The configuration drive is probed to see if network configuration for the interface is available. If so, it will be added, otherwise the interface will configured for DHCP.
How do I use glean?
Glean ships glean-install, a script which install glean into your system startup environment. It should handle sysv, upstart and systemd to cover all major distributions. This should be run once, during install or image build.
The startup environment will be modified as described above to configure any found interfaces.
Differences to cloud-init?
Glean differs to cloud-init mainly in its very reduced dependency footprint. In a dynamic CI environment such as OpenStack, many of the python dependencies for cloud-init can cause conflicts with packages or versions required for testing.
Glean also better supports static IP allocation within config-drive, particuarly important within the Rackspace environment.
More details
- Free software: Apache license
- Documentation: http://docs.openstack.org/infra/glean
- Source: http://git.openstack.org/cgit/openstack-infra/glean
- Bugs: http://storyboard.openstack.org