Commit Graph

71 Commits

Author SHA1 Message Date
Andreas Jaeger 06e67f317d Retire dox repository
This change removes all content from the repository and adds the usual
README file to point out that the repository is retired following the
process from
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project

Depends-On: https://review.opendev.org/701446
Change-Id: I49ddca0f6c4a5d0e546e22a5515d405f0e869eab
2020-01-07 20:53:40 +01:00
Jenkins f3d7a5f0b1 Merge "Replace assertEqual(None, *) with assertIsNone in tests" 2017-03-22 02:37:23 +00:00
Kalaswan Datta aafa2bb988 Removed utf-8 coding
Change-Id: Icc8a880de7ae2ab1718059814bd3c1ba776c4bf7
2017-01-07 10:10:29 -05:00
zhangguoqing a515b0a900 Replace assertEqual(None, *) with assertIsNone in tests
Replace assertEqual(None, *) with assertIsNone in tests to have
more clear messages in case of failure.

Change-Id: I0dfc37bd9aad42914e1cd4b2bfadb71358d62bf4
Closes-bug: #1280522
2016-01-12 09:27:02 +00:00
Jon Skarpeteig 32443f48fb Remove docker debug output polluting the view
With the -D flag present, the only extra output is of type:
time="2015-12-17T10:50:24+01:00" level=debug msg="framesize: 74"
between every single STDOUT line. This shouldn't be needed, as you would
expect the debug flag to influence dox things, not Docker

Change-Id: I224c9e13f5f2a0775afb0ca5f74dd7abf69adc45
2015-12-17 13:29:03 +01:00
Flavio Percoco dffe39167b Don't prefix the cached image with `_`
Newer docker versions don't allow images' names starting with `_`. This
patch removes the `_` prefix from the code.

Change-Id: Ibbc82f0c8c86c149835f66f84f1664b347ae76c1
2015-08-28 20:57:57 +02:00
Flavio Percoco ea141b9715 Don't run useradd for root
When running dox with an user_map check whether the user is root and
skip the useradd command on image build if so. Most (all?) images have a
root user anyway.

Change-Id: I08eb60254fb437e02c61025d0a7761eb774c11f3
2015-08-18 00:47:29 +02:00
Marc Abramowitz d0d15f966c More informative error when command fails
Instead of:

    ERROR|dox.cmd|Operation failed, aborting dox.

we get:

    ERROR|dox.cmd|Operation failed, aborting dox.
    Traceback (most recent call last):
      File "/Users/marca/dev/git-repos/dox/dox/cmd.py", line 99, in run_dox
        map(run, images)
      File "/Users/marca/dev/git-repos/dox/dox/runner.py", line 209, in run
        self.build_test_image(image, command)
      File "/Users/marca/dev/git-repos/dox/dox/runner.py", line 157, in build_test_image
        shutil.copy(add_file, os.path.join(tempd, add_file))
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 119, in copy
        copyfile(src, dst)
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 82, in copyfile
        with open(src, 'rb') as fsrc:
    IOError: [Errno 2] No such file or directory: 'tests_require.txt'

Change-Id: Icdb25ae2c2f06ffab416d215a7f8b360f9b65c1e
2015-05-13 08:30:12 -07:00
Chmouel Boudjnah 2997c592e7 Add environement target in image name
Image name were named exactly the same no matter which
environement/section we were running.

We now have :

_dox/$project_name_$environment_(base|test)

as image name, the _dox and not dox is cause docker wants a namespace
with at least three chars.

Change-Id: I3ea05c0907fb911aaca96a18215c5a44ce2a6ebc
2014-12-02 19:10:29 +01:00
Chmouel Boudjnah 9f33ae5252 Allow parent section for dox.yaml
Allow section to have parent like tox.ini, i.e:

foo:
   images:
    - image1

foo:bar:
    commands:
     - blah

would have foo:bar getting the image image1 since dependent of foo

Change-Id: I613db9ed9ac16c978ef8a42a604992ce2205bc08
2014-10-27 15:20:36 +00:00
Chmouel Boudjnah 336d174cee Add test for dox.cmd
Change-Id: Ifc765aac66281d60fde6aaed17e7eb8e1f2cf247
2014-10-27 15:19:23 +00:00
Chmouel Boudjnah cea9aa0053 Allow running multiple target with -e
Try to mimics tox by default with allowing things like :

-e section1,section2

Change-Id: Iab98667c5d26aa80fe31227ded9d21983eb7d7eb
TODO: if none was specified run all targets not the default ones
2014-10-16 17:47:14 +00:00
Chmouel Boudjnah af4113abc2 Allow multiple targets in dox.yml
Try to emulate tox -e type of target/environment.

This works natively in tox.ini as long we have commands to run there
(doesn't support inheritance).

For dox.yml the format has been improved, the format looks now like this :

target1:
    images:
    commands:
    add:
    prep:

target2:
    images:
    commands:
    add:
    prep:

we still support old format with single target. 

Follow up patches in this series improve this implementation.

Change-Id: I2a30aaa63fd4e47fd76f29ed5e5cb025ca326016
2014-10-15 14:17:22 +00:00
Chmouel Boudjnah 94fe98a0d0 Add more tests with runner
Change-Id: If3ccae3bf6b2579267a862d12d6149523fed81cd
2014-10-14 21:53:04 +00:00
Chmouel Boudjnah e06b41a069 Use pwd.getpwuid() to get login
For some reasons why but os.getlogin() is broken on containers :

http://stackoverflow.com/questions/4399617/python-os-getlogin-problem

when a user is not controlling the terminal.

Let's use instead pwd.getpwuid instead of os.getlogin which would work
in any cases, and would make the function safer to run under containers.

Change-Id: I0cc62e3d13b0e89a0ea1c166ea3ac06cfcc931b3
2014-10-14 21:47:35 +00:00
Davanum Srinivas 51bd5fede5 Ability to specify mount path mapping from osx->vm->docker
The docker -v command takes the directory structure where the
docker itself is running. So when using with osx using boot2docker
or just a ubuntu with docker, we need to use say sshfs or virtual
box additions to first make the osx file system available on the
operating system where docker is running (vm in boot2docker). Then
we need to be able to specify how the osx directory maps to the
directory on the vm using the new --path-map parameter.

Change-Id: I8543c94accf5e809323f0f348e9729e32d942b80
2014-10-14 12:28:45 -04:00
Jenkins 6d4ae348d8 Merge "--rm needs to be after the image or it would fail" 2014-10-08 21:17:29 +00:00
Chmouel Boudjnah 9e8b302aa7 --rm needs to be after the image or it would fail
When running an image in docker the image needs to be the last argument.

Change-Id: I1157b5a689c80339af2a3e10680d360b332c24f0
2014-10-08 23:00:26 +02:00
Jenkins 0a9ef7bf4e Merge "Fix py3 compatibility" 2014-10-08 20:51:56 +00:00
David Shrewsbury be61a19eff Add option to keep test container
Adds -k/--keep options to allow the user to keep the test container
around instead of deleting it after the given command completes.

Change-Id: If7063c1a93436f4b477adb9b2c869d4cd027205a
2014-10-03 10:13:18 -04:00
Chmouel Boudjnah 52209d8f05 Fix py3 compatibility
the ini.get() thing is weird between python2 and python3 but it seems
that the fourth argument is unnecessary and not implemented in py3
anyway.

Change-Id: Ic42c3ca144ee9dd1aab372bc7d70bdca34cb7a93
2014-10-02 15:18:40 +02:00
Chmouel Boudjnah 44c10cd57c Allow use another usename than current user
Useful for boot2docker

Change-Id: I1e345a8d8ed05d723f419764401ba564ee900c01
2014-10-01 13:02:19 +02:00
David Shrewsbury 0339a34458 Better support for multiple commands
This changes the 'commands' YAML portion to be an array. E.g.:

commands:
  - ls
  - ps
  - echo 'Have you tried turning it off and on again?'

Docker doesn't really support issuing multiple commands to a container
on the command line very well. You have to either munge it with something
like:

    sh -c 'ls && ps'

or place all of the commands to execute into a master script, then execute
that script.

This change proposes the second option and creates a master script if the
number of commands is more than one.

The master script file will be created in a .dox subdirectory of the
current directory, and named master_script.sh. Right now, it is created
as a Bourne shell script. We could easily add support later for different
shells.

If the -c option is used, then the command specified is the only command
that will run.

If -c is NOT used, but extra arguments are supplied, these extra arguments
will be used to replace the {posargs} placeholders in tox.ini files. The
extra args are not currently used elsewhere.

This change REMOVES the ability to add extra_args to the end of the
command when {posargs} is NOT present. That does not make sense in
a multi-command situation.

Change-Id: I78dbc0613f5db22cea0ed714f5f30ef0207fac24
2014-09-26 12:20:54 -04:00
David Shrewsbury a21659d31e Use base class for config file parsers
The configuration file parsers have a defined API, but we aren't
enforcing use of that. This adds an abstract base class that they
all will now inherit from. Additionally, new tests are added to
ensure the classes inherit from the new base class.

Moves tests for files in the config/ directory into their own
test directory.

Fixes the test class name for the ToxIni tests.

Change-Id: I003b4add6060075676da0f23cc37445ea50ffe88
2014-09-25 11:50:00 -04:00
Jenkins 629ef2d69d Merge "Close files after use" 2014-09-25 15:43:08 +00:00
Cedric Brandily 5aeb94e112 Simplify Popen.communicate() result parsing
Popen.communicate returns a tuple (stdoutdata, stderrdata) which implies
the result length check is useless.

Change-Id: Id5ee7328d8e31bf584c9069183213cc8c2a19da1
2014-09-25 14:49:22 +02:00
Cedric Brandily 20e9713de0 Close files after use
Replace the pattern:

  func(open(...))

by the pattern:

  with open(...) as f:
    func(f)

It ensures files are closed after use.

Change-Id: I7cdd08993ae4a094d101a1bf68ce700817e7fb92
2014-09-25 14:36:47 +02:00
Chmouel Boudjnah 022610fe86 Add tests for tox_ini
Change-Id: I3ad3d392b3741deaf8bb98c392214a65eaac5409
2014-09-11 14:11:01 +02:00
Jenkins 320e05e8c2 Merge "Add support for multiple images" 2014-09-09 07:13:52 +00:00
Jenkins 96eb0af5d8 Merge "Checks that docker is installed" 2014-09-09 06:43:08 +00:00
Flavio Percoco de82bcdf77 Add support for multiple images
This patch adds support for running tests on multiple images. Instead of
parsing just 1 image, it attempts to always parse multiple images from
config files and/or CLI.

To run dox on multiple images from the CLI, you'll have to:

    $ dox -i infra/trusty,infra/f20

The tox.ini `docker:images` option is now a comma-separated list of
images, whereas the dox.yml one is an array of images.

Change-Id: I124c299b208be5c31c63187af754705c48de5693
2014-09-09 06:41:52 +00:00
Mehdi Abaakouk 7c08b582de Checks that docker is installed
The current error when docker is missing is not very helpful:

 ERROR: Operation failed, aborting dox.

The debug one doesn't help more, this change start 'docker version'
command earlier to have a clear error message.

Change-Id: I1e6ac247b0a619915600f604d5f316d5f053e8c1
2014-09-08 20:09:41 +02:00
David Shrewsbury 591ae18767 Logging improvements
Add module name to logging lines. Kind of nice to see where they
come from.

Also, INFO level would only appear with the --debug option because
the default level was WARN. This makes INFO appear by default, WARN
is used for --quiet mode.

Change-Id: If9618e08744d3c03d7b78e44cbb52f0b1f040074
2014-09-07 22:11:49 -04:00
Chmouel Boudjnah 00458699fb Fix pep8 rules
Change-Id: Id31476294863779b230f1e46a65a230d9ddf69af
2014-09-07 09:55:31 +02:00
Monty Taylor 5d335dcff2 Merge pull request #2 from Shrews/fix_prep
Fix DoxYaml when 'prep' not present.
2014-09-05 10:56:09 -07:00
Monty Taylor 9b015856d1 Merge pull request #3 from Shrews/fix_runner_logger
Fix logging exception when runner command fails
2014-09-05 10:55:52 -07:00
David Shrewsbury cdf6e30d7f Fix logging exception when runner command fails
There is no 'stderr' attribute for an Exception. Just log the
exception string.
2014-09-05 13:33:03 -04:00
David Shrewsbury 967f5261b2 Fix DoxYaml when 'prep' not present.
If 'prep' is not present in the dox.yml file, processing fails.
Set a default for it.
2014-09-05 13:25:05 -04:00
Chmouel Boudjnah a98eb2bbf5 Don't use slash for docker tags 2014-09-05 17:19:45 +00:00
Chmouel Boudjnah 6626d18a50 set privileged=true when running container
so we are not failing with selinux
2014-09-05 17:17:53 +00:00
Monty Taylor 2521f0bae7 Make dox.yml have a default empty add list 2014-09-05 09:16:15 -07:00
Monty Taylor eb3ab22801 Add missing file 2014-09-05 08:00:22 -07:00
Monty Taylor b2b690edcc Have useradd create the group itself 2014-07-26 21:54:40 -07:00
Monty Taylor 08aeceec1f Add flag to useradd to avoid directory creation 2014-07-26 21:52:16 -07:00
Monty Taylor e6fb92378b Make command line override work better 2014-07-26 21:51:27 -07:00
Monty Taylor 14e70638c3 Split up code to facilitate testing 2014-07-26 21:24:09 -07:00
Monty Taylor dba440739a Change default log level 2014-07-26 11:35:37 -07:00
Monty Taylor 3256d46fe1 Append extra args with a space for dox.yml 2014-07-26 11:28:21 -07:00
Monty Taylor a3a1d478a3 Indent the text wrap lines hanging 2014-07-26 11:26:15 -07:00
Monty Taylor 72c193e5e9 Run tests as host user
testr writes files into the local dir, which is cool and works
because of the bind mount - but if we don't run as the same UID
as the executing user, then we'll write unreadable data into
.testrepository.
2014-07-26 11:23:34 -07:00