cloud-init/ChangeLog

0.7.1:
 - sysvinit: fix missing dependency in cloud-init job for RHEL 5.6 
 - config-drive: map hostname to local-hostname (LP: #1061964)
 - landscape: install landscape-client package if not installed.
   only take action if cloud-config is present (LP: #1066115)
 - cc_landscape: restart landscape after install or config (LP: #1070345)
 - multipart/archive.  do not fail on unknown headers in multipart
   mime or cloud-archive config (LP: #1065116).
 - tools/Z99-cloud-locale-test.sh: avoid warning when user's shell is
   zsh (LP: #1073077)
 - fix stack trace when unknown user-data input had unicode (LP: #1075756)
 - split 'apt-update-upgrade' config module into 'apt-configure' and
   'package-update-upgrade-install'.  The 'package-update-upgrade-install'
   will be a cross distro module.
 - Cleanups:
   - Remove usage of paths.join, as all code should run through util helpers
   - Fix pylint complaining about tests folder 'helpers.py' not being found
   - Add a pylintrc file that is used instead options hidden in 'run_pylint'
 - fix bug where cloud-config from user-data could not affect system_info
   settings [revno 703] (LP: #1076811)
 - for write fqdn to system config for rh/fedora [revno 704]
 - add yaml/cloud config examples checking tool [revno 706]
 - Fix the merging of group configuration when that group configuration is a
   dict => members. [revno 707]
 - add yum_add_repo configuration module for adding additional yum repos
 - fix public key importing with config-drive-v2 datasource (LP: #1077700)
 - handle renaming and fixing up of marker names (LP: 1075980) [revno 710]
   this relieves that burden from the distro/packaging.
 - group config: fix how group members weren't being translated correctly
   when the group: [member, member...] format was used (LP: #1077245)
 - sysconfig: fix how the /etc/sysconfig/network should be using the fully
   qualified domain name instead of the partially qualified domain name
   which is used in the ubuntu/debian case (LP: #1076759)
 - fix how string escaping was not working when the string was a unicode
   string which was causing the warning message not to be written
   out (LP: #1075756)
 - for boto > 0.6.0 there was a lazy load of the metadata added, when cloud-init
   runs the usage of this lazy loading is hidden and since that lazy loading 
   will be performed on future attribute access we must traverse the lazy loaded
   dictionary and force it to full expand so that if cloud-init blocks the ec2
   metadata port the lazy loaded dictionary will continue working properly
   instead of trying to make additional url calls which will fail (LP: #1068801)
 - use a set of helper/parsing classes to perform system configuration
   for easier test.  (/etc/sysconfig, /etc/hostname, resolv.conf, /etc/hosts)
 - add power_state_change config module for shutting down stystem after
   cloud-init finishes. (LP: #1064665)
0.7.0:
 - add a 'exception_cb' argument to 'wait_for_url'.  If provided, this
   method will be called back with the exception received and the message.
 - utilize the 'exception_cb' above to modify the oauth timestamp in 
   DataSourceMAAS requests if a 401 or 403 is received. (LP: #978127)
 - catch signals and exit rather than stack tracing
 - if logging fails, enable a fallback logger by patching the logging module
 - do not 'start networking' in cloud-init-nonet, but add
   cloud-init-container job that runs only if in container and emits
   net-device-added (LP: #1031065)
 - search only top level dns for 'instance-data' in DataSourceEc2 (LP: #1040200)
 - add support for config-drive-v2 (LP:#1037567)
 - support creating users, including the default user.
   [Ben Howard] (LP: #1028503)
 - add apt_reboot_if_required to reboot if an upgrade or package installation
   forced the need for one (LP: #1038108)
 - allow distro mirror selection to include availability-zone (LP: #1037727)
 - allow arch specific mirror selection (select ports.ubuntu.com on arm)
   LP: #1028501
 - allow specification of security mirrors (LP: #1006963)
 - add the 'None' datasource (LP: #906669), which will allow jobs
   to run even if there is no "real" datasource found.
 - write ssh authorized keys to console, ssh_authkey_fingerprints
   config module [Joshua Harlow] (LP: #1010582)
 - Added RHEVm and vSphere support as source AltCloud [Joseph VLcek]
 - add write-files module (LP: #1012854)
 - Add setuptools + cheetah to debian package build dependencies (LP: #1022101)
 - Adjust the sysvinit local script to provide 'cloud-init-local' and have
   the cloud-config script depend on that as well.
 - Add the 'bzr' name to all packages built
 - Reduce logging levels for certain non-critical cases to DEBUG instead of the
   previous level of WARNING
 - unified binary that activates the various stages
   - Now using argparse + subcommands to specify the various CLI options
 - a stage module that clearly separates the stages of the different
   components (also described how they are used and in what order in the
   new unified binary)
 - user_data is now a module that just does user data processing while the
   actual activation and 'handling' of the processed user data is done via
   a separate set of files (and modules) with the main 'init' stage being the
   controller of this
   - creation of boot_hook, cloud_config, shell_script, upstart_job version 2
     modules (with classes that perform there functionality) instead of those
     having functionality that is attached to the cloudinit object (which
     reduces reuse and limits future functionality, and makes testing harder)
 - removal of global config that defined paths, shared config, now this is
   via objects making unit testing testing and global side-effects a non issue
 - creation of a 'helpers.py' 
   - this contains an abstraction for the 'lock' like objects that the various 
     module/handler running stages use to avoid re-running a given 
     module/handler for a given frequency. this makes it separated from 
     the actual usage of that object (thus helpful for testing and clear lines
     usage and how the actual job is accomplished)
     - a common 'runner' class is the main entrypoint using these locks to
       run function objects passed in (along with there arguments) and there
       frequency
   - add in a 'paths' object that provides access to the previously global
     and/or config based paths (thus providing a single entrypoint object/type
     that provides path information)
       - this also adds in the ability to change the path when constructing 
       that path 'object' and adding in additional config that can be used to 
       alter the root paths of 'joins' (useful for testing or possibly useful
       in chroots?)
        - config options now avaiable that can alter the 'write_root' and the 
         'read_root' when backing code uses the paths join() function
   - add a config parser subclass that will automatically add unknown sections
     and return default values (instead of throwing exceptions for these cases)
   - a new config merging class that will be the central object that knows
     how to do the common configuration merging from the various configuration
     sources. The order is the following:
     - cli config files override environment config files
       which override instance configs which override datasource
       configs which override base configuration which overrides
       default configuration.
 - remove the passing around of the 'cloudinit' object as a 'cloud' variable
   and instead pass around an 'interface' object that can be given to modules
   and handlers as there cloud access layer while the backing of that
   object can be varied (good for abstraction and testing)
 - use a single set of functions to do importing of modules
 - add a function in which will search for a given set of module names with
   a given set of attributes and return those which are found
 - refactor logging so that instead of using a single top level 'log' that
   instead each component/module can use its own logger (if desired), this
   should be backwards compatible with handlers and config modules that used
   the passed in logger (its still passed in)
   - ensure that all places where exception are caught and where applicable
     that the util logexc() is called, so that no exceptions that may occur
     are dropped without first being logged (where it makes sense for this 
     to happen)
 - add a 'requires' file that lists cloud-init dependencies
   - applying it in package creation (bdeb and brpm) as well as using it
     in the modified setup.py to ensure dependencies are installed when
     using that method of packaging
 - add a 'version.py' that lists the active version (in code) so that code
   inside cloud-init can report the version in messaging and other config files
 - cleanup of subprocess usage so that all subprocess calls go through the
   subp() utility method, which now has an exception type that will provide
   detailed information on python 2.6 and 2.7
 - forced all code loading, moving, chmod, writing files and other system
   level actions to go through standard set of util functions, this greatly 
   helps in debugging and determining exactly which system actions cloud-init is
   performing
 - adjust url fetching and url trying to go through a single function that
   reads urls in the new 'url helper' file, this helps in tracing, debugging
   and knowing which urls are being called and/or posted to from with-in 
   cloud-init code
   - add in the sending of a 'User-Agent' header for all urls fetched that
     do not provide there own header mapping, derive this user-agent from
     the following template, 'Cloud-Init/{version}' where the version is the
     cloud-init version number
 - using prettytable for netinfo 'debug' printing since it provides a standard
   and defined output that should be easier to parse than a custom format
 - add a set of distro specific classes, that handle distro specific actions
   that modules and or handler code can use as needed, this is organized into
   a base abstract class with child classes that implement the shared 
   functionality. config determines exactly which subclass to load, so it can
   be easily extended as needed.
   - current functionality
      - network interface config file writing
      - hostname setting/updating
      - locale/timezone/ setting
      - updating of /etc/hosts (with templates or generically)
      - package commands (ie installing, removing)/mirror finding
      - interface up/down activating
   - implemented a debian + ubuntu subclass
   - implemented a redhat + fedora subclass
 - adjust the root 'cloud.cfg' file to now have distrobution/path specific 
   configuration values in it. these special configs are merged as the normal
   config is, but the system level config is not passed into modules/handlers
   - modules/handlers must go through the path and distro object instead
 - have the cloudstack datasource test the url before calling into boto to 
   avoid the long wait for boto to finish retrying and finally fail when
   the gateway meta-data address is unavailable
 - add a simple mock ec2 meta-data python based http server that can serve a
   very simple set of ec2 meta-data back to callers
      - useful for testing or for understanding what the ec2 meta-data 
        service can provide in terms of data or functionality
 - for ssh key and authorized key file parsing add in classes and util functions
   that maintain the state of individual lines, allowing for a clearer 
   separation of parsing and modification (useful for testing and tracing)
 - add a set of 'base' init.d scripts that can be used on systems that do
   not have full upstart or systemd support (or support that does not match
   the standard fedora/ubuntu implementation)
   - currently these are being tested on RHEL 6.2
 - separate the datasources into there own subdirectory (instead of being
   a top-level item), this matches how config 'modules' and user-data 'handlers'
   are also in there own subdirectory (thus helping new developers and others
   understand the code layout in a quicker manner)
 - add the building of rpms based off a new cli tool and template 'spec' file
   that will templatize and perform the necessary commands to create a source
   and binary package to be used with a cloud-init install on a 'rpm' supporting
   system
   - uses the new standard set of requires and converts those pypi requirements
     into a local set of package requirments (that are known to exist on RHEL
     systems but should also exist on fedora systems)
 - adjust the bdeb builder to be a python script (instead of a shell script) and
   make its 'control' file a template that takes in the standard set of pypi 
   dependencies and uses a local mapping (known to work on ubuntu) to create the
   packages set of dependencies (that should also work on ubuntu-like systems)
 - pythonify a large set of various pieces of code
   - remove wrapping return statements with () when it has no effect
   - upper case all constants used
   - correctly 'case' class and method names (where applicable)
   - use os.path.join (and similar commands) instead of custom path creation
   - use 'is None' instead of the frowned upon '== None' which picks up a large
     set of 'true' cases than is typically desired (ie for objects that have
     there own equality)
   - use context managers on locks, tempdir, chdir, file, selinux, umask, 
     unmounting commands so that these actions do not have to be closed and/or
     cleaned up manually in finally blocks, which is typically not done and will
     eventually be a bug in the future
   - use the 'abc' module for abstract classes base where possible
      - applied in the datasource root class, the distro root class, and the
        user-data v2 root class
 - when loading yaml, check that the 'root' type matches a predefined set of
   valid types (typically just 'dict') and throw a type error if a mismatch
   occurs, this seems to be a good idea to do when loading user config files
 - when forking a long running task (ie resizing a filesytem) use a new util
   function that will fork and then call a callback, instead of having to
   implement all that code in a non-shared location (thus allowing it to be
   used by others in the future)
 - when writing out filenames, go through a util function that will attempt to
   ensure that the given filename is 'filesystem' safe by replacing '/' with
   '_' and removing characters which do not match a given whitelist of allowed
   filename characters
 - for the varying usages of the 'blkid' command make a function in the util
   module that can be used as the single point of entry for interaction with
   that command (and its results) instead of having X separate implementations
 - place the rfc 8222 time formatting and uptime repeated pieces of code in the
   util module as a set of function with the name 'time_rfc2822'/'uptime'
 - separate the pylint+pep8 calling from one tool into two indivudal tools so
   that they can be called independently, add make file sections that can be 
   used to call these independently
 - remove the support for the old style config that was previously located in
   '/etc/ec2-init/ec2-config.cfg', no longer supported!
 - instead of using a altered config parser that added its own 'dummy' section
   on in the 'mcollective' module, use configobj which handles the parsing of
   config without sections better (and it also maintains comments instead of
   removing them)
 - use the new defaulting config parser (that will not raise errors on sections
   that do not exist or return errors when values are fetched that do not exist)
   in the 'puppet' module
 - for config 'modules' add in the ability for the module to provide a list of 
   distro names which it is known to work with, if when ran and the distro being
   used name does not match one of those in this list, a warning will be written
   out saying that this module may not work correctly on this distrobution
 - for all dynamically imported modules ensure that they are fixed up before 
   they are used by ensuring that they have certain attributes, if they do not
   have those attributes they will be set to a sensible set of defaults instead
 - adjust all 'config' modules and handlers to use the adjusted util functions
   and the new distro objects where applicable so that those pieces of code can 
   benefit from the unified and enhanced functionality being provided in that
   util module
 - fix a potential bug whereby when a #includeonce was encountered it would
   enable checking of urls against a cache, if later a #include was encountered
   it would continue checking against that cache, instead of refetching (which
   would likely be the expected case)
 - add a openstack/nova based pep8 extension utility ('hacking.py') that allows
   for custom checks (along with the standard pep8 checks) to occur when running
   'make pep8' and its derivatives
0.6.4:
 - support relative path in AuthorizedKeysFile (LP: #970071).
 - make apt-get update run with --quiet (suitable for logging) (LP: #1012613)
 - cc_salt_minion: use package 'salt-minion' rather than 'salt' (LP: #996166)
 - use yaml.safe_load rather than yaml.load (LP: #1015818)
0.6.3:
 - add sample systemd config files [Garrett Holmstrom]
 - add Fedora support [Garrent Holstrom] (LP: #883286)
 - fix bug in netinfo.debug_info if no net devices available (LP: #883367)
 - use python module hashlib rather than md5 to avoid deprecation warnings.
 - support configuration of mirror based on dns name ubuntu-mirror in
   local domain.
 - support setting of Acquire::HTTP::Proxy via 'apt_proxy'
 - DataSourceEc2: more resilliant to slow metadata service
   - config change: 'retries' dropped, 'max_wait' added, timeout increased
 - close stdin in all cloud-init programs that are launched at boot 
   (LP: #903993)
 - revert management of /etc/hosts to 0.6.1 style (LP: #890501, LP: #871966)
 - write full ssh keys to console for easy machine consumption (LP: #893400)
 - put INSTANCE_ID environment variable in bootcmd scripts
 - add 'cloud-init-per' script for easily running things with a given frequency
 - replace cloud-init-run-module with cloud-init-per
 - support configuration of landscape-client via cloud-config (LP: #857366)
 - part-handlers now get base64 decoded content rather than 2xbase64 encoded
   in the payload parameter. (LP: #874342)
 - add test case framework [Mike Milner] (LP: #890851)
 - fix pylint warnings [Juerg Haefliger] (LP: #914739)
 - add support for adding and deleting CA Certificates [Mike Milner] 
   (LP: #915232)
 - in ci-info lines, use '.' to indicate empty field for easier machine reading
 - support empty lines in "#include" files (LP: #923043)
 - support configuration of salt minions (Jeff Bauer) (LP: #927795)
 - DataSourceOVF: only search for OVF data on ISO9660 filesystems (LP: #898373)
 - DataSourceConfigDrive: support getting data from openstack config drive
   (LP: #857378)
 - DataSourceNoCloud: support seed from external disk of ISO or vfat
   (LP: #857378)
 - DataSourceNoCloud: support inserting /etc/network/interfaces
 - DataSourceMaaS: add data source for Ubuntu Machines as a Service (MaaS)
   (LP: #942061)
 - DataSourceCloudStack: add support for CloudStack datasource [Cosmin Luta]
 - add option 'apt_pipelining' to address issue with S3 mirrors 
   (LP: #948461) [Ben Howard]
 - warn on non-multipart, non-handled user-data [Martin Packman]
 - run resizefs in the background in order to not block boot (LP: #961226)
 - Fix bug in Chef support where validation_key was present in config, but
   'validation_cert' was not (LP: #960547)
 - Provide user friendly message when an invalid locale is set 
   [Ben Howard] (LP: #859814)
 - Support reading cloud-config from kernel command line parameter and
   populating local file with it, which can then provide data for DataSources
 - improve chef examples for working configurations on 11.10 and 12.04 
   [Lorin Hochstein] (LP: #960564)

0.6.2:
 - fix bug where update was not done unless update was explicitly set.
   It would not be run if 'upgrade' or packages were set to be installed
 - fix bug in part-handler code, that prevented working part-handlers
   (LP: #739694)
 - fix bug in resizefs cloud-config that would cause trace based on
   failure of 'blkid /dev/root' (LP: #726938)
 - convert dos formated files to unix for user-scripts, boothooks,
   and upstart jobs (LP: #744965)
 - fix bug in seeding of grub dpkg configuration (LP: #752361) due
   to renamed devices in newer (natty) kernels (/dev/sda1 -> /dev/xvda1)
 - make metadata urls configurable, to support eucalyptus in
   STATIC or SYSTEM modes (LP: #761847)
 - support disabling byobu in cloud-config
 - run cc_ssh as a cloud-init module so it is guaranteed to run before
   ssh starts (LP: #781101)
 - make prefix for keys added to /root/.ssh/authorized_keys configurable
   and add 'no-port-forwarding,no-agent-forwarding,no-X11-forwarding'
   to the default (LP: #798505)
 - make 'cloud-config ready' command configurable (LP: #785551)
 - make fstab fields used to 'fill in' shorthand entries configurable
   This means you do not have to have 'nobootwait' in the values
   (LP: #785542)
 - read /etc/ssh/sshd_config for AuthorizedKeysFile rather than
   assuming ~/.ssh/authorized_keys (LP: #731849)
 - fix cloud-init in ubuntu lxc containers (LP: #800824)
 - sanitize hosts file for system's hostname to 127.0.1.1 (LP: #802637)
 - add chef support (cloudinit/CloudConfig/cc_chef.py) (LP: ##798844)
 - do not give trace on failure to resize in lxc container (LP: #800856)
 - increase the timeout on url gets for "seedfrom" values (LP: #812646)
 - do not write entries for ephemeral0 on t1.micro (LP: #744019)
 - support 'include-once' so that expiring or one-time use urls can
   be used for '#include' to provide sensitive data.
 - support for passing public and private keys to mcollective via cloud-config
 - support multiple staticly configured network devices, as long as
   all of them come up early (LP: #810044)
 - Changes to handling user data mean that:
   * boothooks will now run more than once as they were intended (and as
     bootcmd commands do)
   * cloud-config and user-scripts will be updated from user data every boot
 - Fix issue where 'isatty' would return true for apt-add-repository.
   apt-add-repository would get stdin which was attached to a terminal
   (/dev/console) and would thus hang when running during boot.  (LP: 831505)
   This was done by changing all users of util.subp to have None input unless
   specified
 - Add some debug info to the console when cloud-init runs.
   This is useful if debugging, IP and route information is printed to the 
   console.
 - change the mechanism for handling .ssh/authorized_keys, to update entries
   rather than appending.  This ensures that the authorized_keys that are
   being inserted actually do something (LP: #434076, LP: #833499)
 - log warning on failure to set hostname (LP: #832175)
 - upstart/cloud-init-nonet.conf: wait for all network interfaces to be up
   allow for the possibility of /var/run != /run.
 - DataSourceNoCloud, DataSourceOVF : do not provide a default hostname.
   This way the configured hostname of the system will be used if not provided
   by metadata (LP: #838280)
 - DataSourceOVF: change the default instance id to 'iid-dsovf' from 'nocloud'
 - Improve the OVF documentation, and provide a simple command line
   tool for creating a useful ISO file.

0.6.1:
 - fix bug in fixing permission on /var/log/cloud-init.log (LP: #704509)
 - improve comment strings in rsyslog file tools/21-cloudinit.conf
 - add previous-instance-id and previous-datasource files to datadir
 - add 'datasource' file to instance dir
 - add setting of passwords and enabling/disabling of PasswordAuthentication
   for sshd.  By default no changes are done to sshd.
 - fix for puppet configuration options (LP: #709946) [Ryan Lane]
 - fix pickling of DataSource, which broke seeding.
 - turn resize_rootfs default to True
 - avoid mounts in DataSourceOVF if 'read' on device fails
   'mount /dev/sr0' for an empty virtual cdrom device was taking 18 seconds
 - add 'manual_cache_clean' option to select manual cleaning of
   the /var/lib/cloud/instance/ link, for a data source that might
   not be present on every boot
 - make DataSourceEc2 retries and timeout configurable
 - add helper routines for apt-get update and install
 - add 'bootcmd' like 'runcmd' to cloud-config syntax for running things early
 - move from '#opt_include' in config file format to conf_d.
   ie, now files in /etc/cloud.cfg.d/ is read rather than reading
   '#opt_include <filename>' or '#include <filename>' in cloud.cfg
 - allow /etc/hosts to be written from hosts.tmpl. which allows
   getting local-hostname into /etc/hosts (LP: #720440)
 - better handle startup if there is no eth0 (LP: #714807)
 - update rather than append in puppet config [Marc Cluet]
 - add cloud-config for mcollective [Marc Cluet]
0.6.0:
 - change permissions of /var/log/cloud-init.log to accomodate
   syslog writing to it (LP: #704509)
 - rework of /var/lib/cloud layout
 - remove updates-check (LP: #653220)
 - support resizing / on first boot (enabled by default)
 - added support for running CloudConfig modules at cloud-init time
   rather than cloud-config time, and the new 'cloud_init_modules'
   entry in cloud.cfg to indicate which should run then.
   The driving force behind this was to have the rsyslog module
   able to run before rsyslog even runs so that a restart would
   not be needed (rsyslog on ubuntu runs on 'filesystem')
 - moved setting and updating of hostname to cloud_init_modules
   this allows the user to easily disable these from running.
   This also means:
   - the semaphore name for 'set_hostname' and 'update_hostname'
     changes to 'config_set_hostname' and 'config_update_hostname'
 - added cloud-config option 'hostname' for setting hostname
 - moved upstart/cloud-run-user-script.conf to upstart/cloud-final.conf
 - cloud-final.conf now runs runs cloud-config modules similar
   to cloud-config and cloud-init.
 - LP: #653271 
   - added writing of "boot-finished" to /var/lib/cloud/instance/boot-finished
     this is the last thing done, indicating cloud-init is finished booting
   - writes message to console with timestamp and uptime
 - write ssh keys to console as one of the last things done
   this is to ensure they don't get run off the 'get-console-ouptut' buffer
 - user_scripts run via cloud-final and thus semaphore renamed from
   user_scripts to config_user_scripts
 - add support for redirecting output of cloud-init, cloud-config, cloud-final
   via the config file, or user data config file
 - add support for posting data about the instance to a url (phone_home)
 - add minimal OVF transport (iso) support
 - make DataSources that are attempted dynamic and configurable from
   system config. changen "cloud_type: auto" as configuration for this
   to 'datasource_list: [ "Ec2" ]'.  Each of the items in that list
   must be modules that can be loaded by "DataSource<item>"
 - add 'timezone' option to cloud-config (LP: #645458)
 - Added an additional archive format, that can be used for multi-part
   input to cloud-init.  This may be more user friendly then mime-multipart
   See example in doc/examples/cloud-config-archive.txt (LP: #641504)
 - add support for reading Rightscale style user data (LP: #668400)
   and acting on it in cloud-config (cc_rightscale_userdata.py)
 - make the message on 'disable_root' more clear (LP: #672417)
 - do not require public key if private is given in ssh cloud-config 
   (LP: #648905)