Commit Graph

77 Commits

Author SHA1 Message Date
Takashi Kajinami e6cd3110cd Remove six
Python 2 is no longer supported, thus usage of six can be removed.

Change-Id: I8abf879e304b7b38754242272ecd105a302a0c5f
2023-07-20 11:15:39 +09:00
Zane Bitter bc88eae1d2 Stop using deprecated name for ConfigParser
The name SafeConfigParser has been deprecated since Python 3.2 and is
removed in Python 3.12.

Change-Id: Ifb1c0f525b7624a91e2cd44e1e86dee8d9f9c51d
2023-07-19 10:43:54 +12:00
ljhuang 7c81d649fb Replace deprecated readfp method with read_file
The readfp method has been deprecated since version 3.2 [1].

[1] https://docs.python.org/3/library/configparser.html?highlight=deprecated#configparser.ConfigParser.readfp

Change-Id: Ib1aa6bba91e331f25c65a5905a89de306faa1d5a
2022-07-22 17:19:17 +08:00
maaoyu 5b3dba0c87 Mark support for py38
in 'wallaby' cycle, we should test py38 by default.
Also support tox test on py36 and py38 as that's what
our check job verify on.
ref:
  https://governance.openstack.org/tc/reference/runtimes/victoria.html

Change-Id: I349d80cb1b963bdb30e374994f7c18352f918e3b
2020-11-16 13:26:58 +08:00
Sean McGinnis 5c2557dbaf
Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: Ib6e4e7f89a9990cfb42afa209878812340109ecf
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-06-09 09:59:03 -05:00
Zane Bitter d04bb529dc Fix pep8 errors with later versions of hacking
We are stuck on a very old version of hacking (0.8). In order to move
forward, we need to fix a bunch of things that flake8 will complain about.

Change-Id: If40ac29094b90c5bae63e7423061a190655f50a3
2018-07-18 16:35:03 -04:00
Zane Bitter c583cbf9b2 Python3: Don't use cmp() function
The built-in cmp() function has been removed in Python 3, so don't try to
use it.

Change-Id: Ic62b7032ec6fd555974fc0d818327879d53a8ff2
2018-07-18 16:33:41 -04:00
shizhihui 590b3fb996 Replace 'MagicMock' with 'Mock'
Change-Id: I24e87a1c341f63929f93d7306782e3e9d6b398fb
Partial-Bug: #1475722
2016-09-23 10:57:16 +08:00
Pablo Iranzo Gómez c9f938b9d0
Fix typo
Change-Id: I15deb824a85aa561c2f2e7f4ea6b4b1ce96367f9
2016-06-29 17:42:09 +02:00
Swapnil Kulkarni (coolsvap) 5d62b178c3 Replace deprecated LOG.warn with LOG.warning
LOG.warn is deprecated. It still used in a few places.
Updated to non-deprecated LOG.warning.

Change-Id: I6e8df0e072448fbd4077c4e5d98b2986e9855489
Closes-Bug:#1508442
2016-03-11 13:09:24 +05:30
Shuquan Huang e4503e43cb Replace assertEqual(None, *) with assertIsNone in tests
Replace assertEqual(None, *) with assertIsNone in tests to have
more clear messages in case of failure.

Change-Id: Ic2dca04e7cdd4f837c42b39dd1ce37604c8f101b
Closes-bug: #1280522
2015-12-17 11:20:13 +08:00
Anant Patil 6571e5ab64 Don't run commands given as list on shell
Commands from AWS::CloudFormation::Init, when supplied as list, should
be run with shell=False. Only when commands are given as string, they
are meant to be run on shell.

In principle, we are trying to give least access to the shell to avoid
any inadvertent shell injections.

Change-Id: I3dc6fe0c29a14f75be044846f737e1ade23a6d6b
Closes-Bug: 1498300
2015-09-23 20:22:28 -04:00
Anant Patil 2710bba2cb Convert all internal commands to list
Make all internal commands as list to avoid any possibility of command
line injection. Commands supplied as string are susceptible to
substitution.

All the internal commands are supplied as list to CommandRunner. As a
convention, all the commands must be given as list to subprocess except
the commands read from file, like in case of cfn hooks and commands
section in metadata.

Few internal commands require shell redirects and they will be
implemented in another patch.

Change-Id: Ifabaf44e341144bc85508dc05c76b1d83e41ae44
Partial-Bug: #1312246
2015-09-22 10:37:11 +05:30
Anant Patil f427a69443 Use seteuid instead of su to control privileges
Control the privileges by setting the effective UID before running the
command. Earlier we used to run command using su -c "USER".

Original EUID is restored after running the command. This is required to
run multiple commands in succession with different run-as users.

Change-Id: I414fc6a802f11deb320b43c6d011f802a42c40c9
Partial-Bug: #1312246
2015-09-22 09:35:56 +05:30
Zane Bitter 090a14dd63 Revert "Fix cfntools command injection"
This reverts commit e424af2236.

Splitting command strings that were previously assumed to be interpreted
by the shell at whitespace and then passing them as separate args to
execvp will not work.

Change-Id: I7c37b5852ce9b20e63bdbbaddfb852463548aa90
2015-09-03 18:18:41 -04:00
Anant Patil e424af2236 Fix cfntools command injection
The CommandRunner used to run commands using su command and passing the
actual command to be run as argument to it.

su USER -c <cmd>

This is susceptible to command line injection as noted in the bug.

The fix required to do two things:

1. Pass the command to be run as list instead of a string. This is to
ensure that the actual arguments are passed as arguments to the program
ought to be executed. And by doing so, avoids running any commands
passed in the argument. On the contrary, if the command were passed as a
string to the shell, the arguments could be formed in a way to execute
malicious commands.

2. The CommandRunner runs the command directly and uses setuid to lower
the privileges if needed. If the 'runas' user is other than root, then
its UID is obtained and setuid is invoked to set the real user-id and
effective user-id to the given user.

Change-Id: I654117e994fd38411508dbe9b85d06c28dc0e411
Closes-Bug: #1312246
2015-08-31 15:58:04 +05:30
Anant Patil 20049ea85f Fix unit tests
Fix failing tests.

Co-Authored-By: Sirushti Murugesan <sirushti.murugesan@hp.com>
Change-Id: If44ea49e5d6262f6e6b51dfdfb76754fb7c467d5
2015-08-14 12:38:13 +05:30
Miro Hrončok 57f8ae8997 Ported tests from mox3 to mock to support Python >= 3.3
Change-Id: I55579328adc7003f78e78161aa6e047524a5c805
Related-Bug: 1403214
2015-02-22 22:19:20 +01:00
Miro Hrončok f8796122c5 Python 3 compatibility
* ConfigParser import from six
 * Drop iteritems()
  * To support both Python 2 and 3
 * Encode string before writing it to file
  * To support both Python 2 and 3
 * Use six.string_types
  * To support both Python 2 and 3
 * Use key on Python 3
  * Because cmp is no longer working
 * Add py33 and py34 to tox.ini

Change-Id: I23985be55302cd4ef577919efb51975ecbd9563d
Related-Bug: 1347899
2015-02-21 11:17:53 +01:00
Ryan S. Brown a7ffb71ffd Support dnf when specified or yum is missing
* handle install/upgrade, version checks, and downgrades

* Allow users to specify packages to be installed with dnf

* Use dnf if yum isn't available, letting older cloud-configs work on
  future Fedoras

Change-Id: Ib3ff49cfdd3e545aa199c944c110852700625496
2015-01-15 16:42:20 -05:00
Bruno Bompastor d96f73c4de Fixes cfn-hup hooks functionality
Change-Id: I39ba2cca1f096e866f69fc016f78004ea991c0b6
Closes-Bug: 1387190
2014-12-11 10:28:11 +00:00
Yukinori Sagara 8ef88a6e86 Change wget to curl for using both of them is redundant
heat-cfntools depends on wget and curl. It's redundant.
Since the curl is widely used, replace wget command to curl.

Change-Id: I691bdc046bd72a44c11f25e359c5036ae1a9e86b
Closes-Bug: 1359430
2014-08-24 08:24:01 +09:00
Yukinori Sagara 2e544679e1 Fix cfn_helper test for random hash seed
test_cfn_helper.py has a part of code which enforces the order of packages
or services processes. But the order is non-deterministic.

Change-Id: I37c4abe697fb3391793ce74fc730b127e920710a
Closes-Bug: #1360212
2014-08-23 17:13:10 +00:00
Jenkins 9bd960408f Merge "Add support for package install via zypper" 2014-06-18 04:16:57 +00:00
Jenkins 269c841b47 Merge "Add res_last_path to store last metadata of a resource" 2014-06-18 04:15:30 +00:00
Ethan Lynn 5f7025558c Add res_last_path to store last metadata of a resource
Add res_last_path='/var/cache/heat-cfntools/last_metadata_<resource>'
in Metadata.retrieve() function to store metadata of a resource.
Remove md5 check between current metadata and last metadata, json.load()
will organize json structure.

Change-Id: Ie0c31a748f0add3fcab6a579431a28b60051f601
Closes-Bug: #1205375
Partial-Bug: #1133049
2014-05-31 18:54:55 +08:00
Ethan Lynn 0fab4170ff Update sysvinit function to support ubuntu
In ubuntu system, we can't find 'chkconfig' command, it use
"update-rc.d" or "sysv-rc-conf" for instead.

_handle_sysv_command function will pick up the right command to
enable service for ubuntu or fedora or redhat, and also map systemd
to _handle_sysv_command and remove _handle_systemd_command.

Change-Id: I5b7ceb7541e989f6b11fc1a5acf94275c1d2e75b
Closes-Bug: #1318481
2014-05-29 17:38:39 +08:00
Dirk Mueller 1b9db9948a Add support for package install via zypper
SUSE uses "zypper" for managing packages.

Change-Id: Iac8399e7a4e85e33cad1085f11a08fdb538a96e6
blueprint: heat-cfntools-zypper
2014-05-08 08:20:34 +02:00
Dirk Mueller 3a4e6b5b81 Switch over to mox3
mox3 is the Python 3.x compatible replacement
of mox.

Change-Id: If107d0ebde50d3461505c5f722d53557993f1e7e
2014-03-23 23:37:17 +01:00
Jenkins 31da8a419c Merge "Support of ignoreErrors for commands" 2014-03-05 03:59:11 +00:00
Simon Pasquier 3a0993210d Support of ignoreErrors for commands
cfn-init will now fail immediately if a command with the key
ignoreErrors='false' or without that key fails (eg it returns an exit
code other than 0). This is similar to what the AWS cfn-init script is
doing.

Change-Id: I41bfa36154fa8b16541a6abb489495739b772376
Closes-Bug: #1269476
2014-02-24 13:36:05 +01:00
Simon Pasquier 4ece05a829 cfn-signal provides a unique default id
Unless the parameter is provided, cfn-signal will use the instance UUID
from the Nova metadata as the id sent back to the WaitCondition.
In case the Nova metadata isn't available, it will use the hostname as a
fallback.

Change-Id: I1e5847c7babd7c6295d8c3e21f6cfa110a9b3026
Closes-bug: 1223429
2014-02-18 18:14:12 +01:00
Simon Pasquier 518c02dd0a Log stdout and stderr on non-zero exit status
This helps troubleshooting failures when running cfn-init.

Change-Id: I601506ab05d372307bb160b198a675f048e2e9ff
Closes-Bug: #1269470
2014-02-13 09:24:26 +01:00
Sascha Peilicke 8e7c392920 Sync with global requirements
And fix pep8 issues discovered by hacking update. Remove dependencies
on pep8, pyflakes and flake8. They should be determined by the hacking
dependency implicitely.

Change-Id: I3fefdabcfdc09c28756f5ab0f5a99d12de2d8a3a
2014-01-17 08:59:45 +01:00
Angus Salkeld 090a5a9f4f Only run yum install once per package block
before we would get this:
yum install a
yum install a b
yum install a b c

Now we just get
yum install a b c

Change-Id: I2067922ab03de9488a0cd4e08c8d44c00296cd6a
Closes-bug: #1235796
2013-11-01 17:14:29 +11:00
Angus Salkeld 8cbb79aff2 Add basic yum and apt package install tests
Change-Id: I96e3bdc2566222fb4926e8a8f88a7f6b822e16f7
Partial-bug: #1235796
2013-11-01 17:14:26 +11:00
Steven Dake e8d2522397 Use on-demand yum metadata caching in cfn-init
Use on-demand yum metadata caching to avoid downloading
50MB (Fedora 19) of extra metadata that is not necessary
for correct cfn-init operation.

Reduces time to orchestration by about 23% and cpu
utilization by about 50%.

Full detailed analsys in the launchpad bug.

Change-Id: Id51d6d506d6051b5e83f550ef318f86d84f3c7a7
Closes-Bug: 1235824
2013-10-05 17:24:46 -07:00
Simon Pasquier 63d70cd0ec Implement -k option for cfn-get-metadata
The -k argument can be a top-level key or a nested key in which case
the keys are separated by dots (eg "foo.bar"). In case a key contains a
dot character, it needs to be surrounded by single quotes (eg
"foo.'bar.1'.fred).
If the -k option is not provided, the command prints out the full
metadata structure as before.

Change-Id: Ib05d39672086001b83e8d7f56bc42cc4ba75751c
Fixes: bug #1183299
2013-09-10 15:56:25 +02:00
JUN JIE NAN 2044ef239d Support array values in command
Description of command in CFN User Guide:

Either an array or a string specifying the command to run. If you use
an array, you do not need to escape space characters or enclose
command parameters in quotes.

So we escape double quote first, and enclose each array value in
double quote.

Fixes bug #1211605

Change-Id: I28ecdb0d4b8a12690dddeac4e2398264c6d6f212
2013-08-19 10:55:02 +08:00
JUN JIE NAN 5b83c132a3 Pass cwd and env to test command
Promote get cwd before test command running and pass cwd to test
command.

Fixes bug #1211606

Change-Id: I2e4d3258f6d591d3d0aadffcefa36487c9122023
2013-08-13 10:49:55 +08:00
Steve Baker 28e9aac8d9 Use python 3 style print functions
Required to pass pep8 H233.

Change-Id: I136dd2aa9fa567208a2fe1660b7a9085fde2927e
2013-08-07 13:29:13 +12:00
Angus Salkeld b97fb83dc0 Add a get_tags() method to the Metadata class
Tags are not properly implemented in nova so we pass the tags
to nova as metadata. So we now [w]get the nova metadata.
Since this is called repeatedly we cache the metadata.

We also add the nova instance id (uuid) as a guest tag.

Change-Id: I599f22fd5166e88cb3d21a71ead5f48c5c5a9269
2013-07-31 10:58:20 +10:00
Jason Dunsmore 73cebadc07 Configure Apt to never prompt for user-input.
Fixes bug #1205378

Change-Id: Ie55725e04aa6045b52c33a5cdd4d2be4ac1a6cc3
2013-07-26 10:18:22 -05:00
JUN JIE NAN a4e7c722f7 Add GitHub tarball and zipball support
Github tarball and zipball support was removed in the change set to
pipe handing in sources. The changeset add it in and restructured.

Change-Id: I107f42e9961cd8776161d1f6a2efe9d103aea125
Fixes: bug #1195622
2013-07-01 17:38:26 +08:00
JUN JIE NAN 026146ae4e Using pipe to handle tgz sources
For tgz(or tar.bz2) sources, use pipe like `wget -O -
http://www.example.com/a.tar.gz | tar -xvf -` to save disk space
usage.

Change-Id: I59663aed098e8c96d8a41b2d84200f2a1e43a927
Fixes: bug #1192135
2013-06-26 15:27:51 +08:00
JUN JIE NAN 971c165242 Delete the temporary directory created by sources handler
Register a hook to delete the temporary directory created by sources
handler at program cfn-init exit.

Change-Id: I821195cf510d35f94b1e7656dacc0dfe308ceeb1
Fixes: bug #1191674
2013-06-20 14:18:02 +08:00
Dirk Mueller bc58e5fa70 Use Python 3.x compatible octal literals
Python 3.x deprecated octal literals in the form
0755. Use 0o755  instead which works at least
with Python 2.6 and newer

Change-Id: I70dc33cb674499548732408924aa2ae728e17ea3
2013-06-13 09:24:08 +02:00
Simon Pasquier 2d00ede1e6 cfn-get-metadata: log metadata to standard output
Add display() method to the Metadata class that prints the metadata to
standard output if the metdata has been successfully retrieved (either
from local cache or from the remote server).

Change-Id: Idf6c1aecf2a5204d7cf7fbf3c8d826f750a72785
Fixes: bug #1183298
2013-06-03 14:21:40 +02:00
Jenkins 6a6922ab5a Merge "Fix leaky tempfiles during test suite." 2013-05-16 04:06:40 +00:00
Monty Taylor 5c67cda719 Fix pyflakes/hacking errors.
Change-Id: I6c364240d9e336fc4f38c2f4bc1fea2ae5e91511
2013-05-11 13:41:12 -04:00