Commit Graph

87 Commits

Author SHA1 Message Date
Nam Nguyen Hoai f2befc4168 TrivialFix: Using assertEqual instead of assertEquals
Following OpenStack Style Guidelines[1]: http://docs.openstack.org/developer/hacking/#unit-tests-and-assertraises
[H234] assertEquals() logs a DeprecationWarning in Python 3.x,
use assertEqual() instead. The same goes for assertNotEquals().

Change-Id: I0469bc4d5672d2832b61ec8e3cd283ad06bf380f
2016-09-28 15:10:44 +07:00
Clint Byrum e0853bb40d Fix test decorators
These tests aren't actually run by nose because the private decorator
doesn't properly wrap its test. This is also a useful pattern, and so we
move it into its own file (note that even better would be to use
testtools and fixtures, but that's a bigger refactor).

In addition, we don't need to use the git CLI, when we have gitpython,
which produces cleaner code and doesn't spray stuff all over our stderr.

Finally, the outdir creating decorator wasn't as reusable as I thought
it would be, so removing it for inline code for now.

Change-Id: Ibe1b993235b68a1a59bc398ded9c822a02475570
2016-07-09 15:37:05 -07:00
Clint Byrum f46a162ea2 Add basic test coverage for build_spec
This asserts the basic public surface area of the module

Change-Id: I97f5f228ed55dd0d0b748a47bd88c72c55500102
2016-07-06 16:56:19 -07:00
Clint Byrum 5e5e0086b9 Add test coverage for openstack_git_repo
Just some basic tests to see if we're interacting with git repositories
well.

Change-Id: I796b70ae505ff10111f640d7f530fb4d6f2d5acb
2016-07-06 16:33:32 -07:00
Craig Tracey 56c0208a9c Remove contraints from postinstall_dependencies
postinstall_dependencies is meant for very fine-grained modification of
the resulting virtual environment. If a user wants to use constraints
after the fact they may then specify those constraints in the
postinstall_dependencies list item.

Change-Id: I21390450fc086d80b16c7ab4d1dc43c81ff590af
2016-06-20 13:44:30 -04:00
Jesse Keating 14b8389c53 Introduce upper constraints
This is a global setting that will apply across every project, a
constraint on the versions of things that can be installed via pip.

This is a first pass, adding basic functionality, by allowing a list
of constraint URLs to be provided. Future iterations of this feature may be
tied to a "superrepo" that contains all the projects _and_ the
constraints for those projects all in one place.

Change-Id: Ib5c2dd441b9294b775e755e595486a671d8de57f
2016-06-15 11:07:21 -07:00
Jesse Keating 9b88e54da3 Python3 clean ups
Use pep3217 syntax for octal
This allows us to pass the py34 gate checks. Compatible at least with
python 2.7.6 (on Ubuntu 14.04).

Handle urlparse -> urllib.parse

Update to latest GitPython

Fix bad print statements

Fix test unittest comparisons

Change-Id: I79dac5e2c94f8ce5d522801837dbb6ea5e044f13
2016-05-02 15:39:38 -07:00
Nicola Heald e573a3e1b3 Use comma separated list of projects for --projects 2016-01-27 11:06:06 +00:00
Nicola Heald 9716cfb3ef Add --projects to specify which projects to build
New option --projects (-p) allows you to only build certain projects
from the manifest you're using.

Example:

giftwrap build -m examples/manifest.yml -t package -p glance cinder
2016-01-21 20:13:31 +00:00
Xav Paice 64a6ed4372 Add quotes to package install
postinstall_dependencies (and probably others) need to have the version
specified with == otherwise the command fails due to < and > being known
as redirects.  This adds quotes round the package name, so that bash
doesn't do that.
2015-12-08 15:37:14 +13:00
Craig Tracey d4d383a029 Update test_builders to make test more resilient
Previously, the order of the list of drivers was indeterminate.
Change this from a list comparison to a set comparison.
2015-11-03 22:03:01 -05:00
Craig Tracey f2d79a0db9 Add ability to extend builders via builder drivers
There are times where you may want to extend or modify the way that
giftwrap builds packages. Short of submitting a pull request or
(worse) forking the repository, there was no easy way to do so.

This change introduces builder drivers with stevedore. By merely adding
entry points in the 'giftwrap.builder.drivers', additional
implementations are now possible.
2015-11-03 21:43:34 -05:00
Craig Tracey 503d3f0b7b Add some threading improvements
Two improvements have been added:
- display the name of the thread in the logging
- record the thread exit status and return it upon completion
2015-10-15 12:32:10 -04:00
Paul Czarkowski 68ae12caf3 Merge pull request #54 from blueboxgroup/gitdepth
Fix git depth issues
2015-10-14 21:42:25 -05:00
Craig Tracey c188d6238a Fix git depth issues
Defaulting a getdepth to 1 makes it difficult to switch branches if
the branch is not available in the shallow clone, so make this
optional. Also, instantiate OpenstackGitRepo properly: depth is not
the fourth parameter.
2015-10-14 21:06:46 -04:00
Craig Tracey 37eed319e5 Install pip dependencies iteratively
Installing pip dependencies interatively (vs. concurrently) provides
for interesting manifest syntax. For example, one could specify
an index-url along side a dependency without affecting the other
dependencies in the list.
2015-10-14 21:01:50 -04:00
Craig Tracey 3c0f88249a Docker builds need to be synchronous
There is no point in making Docker builds "parallel" as they are
executed by way of a docker build.
2015-10-12 11:48:19 -04:00
Paul Czarkowski 75098e9c1d Merge pull request #50 from blueboxgroup/postinstall-deps
Support postinstall python dependencies
2015-10-09 22:59:06 -05:00
Craig Tracey 5df2dff2f2 Support postinstall python dependencies
There are cases where you would like for a Python module to be
installed *after* the project itself has been installed. This is now
possible with project.postinstall_dependencies.
2015-10-09 23:49:27 -04:00
Paul Czarkowski 06c94a0542 Merge pull request #49 from blueboxgroup/errant-etc
Only attempt copy when directory exists
2015-10-09 22:33:42 -05:00
Craig Tracey 8dd2e05a30 Only attempt copy when directory exists
This bug is the result of poor whitespace management.  Only copy
etc if etc is present.
2015-10-09 23:02:12 -04:00
Craig Tracey a4d8caf302 Add parallel builds
This change runs builds in parallel. By default all projects run
simultaneously unless --synchronous is specified. This should speed
up build times dramatically.
2015-10-09 22:55:52 -04:00
Paul Czarkowski 94d11ee1b2 revert virtualenv changes 2015-09-21 12:02:32 -05:00
Paul Czarkowski 8fdfc3d024 do not build a venv inside docker
docker is already isolating dependencies inside the images... there is no
need to add the complexity of a venv.
2015-09-19 16:15:23 -05:00
Paul Czarkowski 3c864f4ea7 docker client was trying to use old API version
also needed to pass project details to build to be able
to name the image.
2015-09-19 16:04:38 -05:00
Craig Tracey e88a960ea9 Refactor giftwrap builders
Previously the two builders had operated as separate build procesess.
What this meant is that when one builder would change, the other would
also need to change in order to support build parity.

This change moves all of the build logic into the base Builder abstract
class. Each sub Builder then implements the necessary primitives to
support the build steps outlined by the base class.
2015-09-04 15:11:42 -04:00
Jesse Keating 9abe6da804 Switch from python setup.py to pip for install
Using python setup.py install with a PBR managed system doesn't work. It
hasn't been supported for a while
(http://lists.openstack.org/pipermail/openstack-dev/2013-September/015525.html
and
https://openstack.nimeyo.com/42989/openstack-dev-pbr-support-for-python-setup-py-install).
It's now breaking in our CI due to not installing the required deps,
such as the requests module. By using pip install we get all that we
want, as upstream for PBR suggests.
2015-06-16 15:30:07 -07:00
Craig Tracey 3a60a2c17c Use distutils copy_tree instead of shutil.copytree
This is just more hassle than it is worth. Pull in an additonal
dependency in distutils and use copy_tree instead as it does
precisely what we want.
2015-04-14 23:48:02 -04:00
Craig Tracey 8dbd5e3be8 Only use copytree when dest doesnt exist
shutil.copytree will fail if the destination directory already exists.
It may exist when other dependencies place an entry there. Therefore,
if it already exists, use shutil.copy2 instead.
2015-04-14 22:52:22 -04:00
Craig Tracey bec6c3bde3 Fix nested jinja tags bug
Unfortunately, there was an issue where {{ base_path }} was evaluating
to an empty string following merge of:

https://github.com/blueboxgroup/giftwrap/pull/20

This introduces a loop over jinja rendering that will continue until it
is done.
2015-03-31 16:00:47 -04:00
Craig Tracey 563ee107d4 Merge pull request #23 from dvorak/pbr-freeze
Add support for projects using pbr freeze
2015-03-09 15:41:00 -04:00
Craig Tracey 25e7b27a91 Merge pull request #25 from dvorak/misc-fixups
Fix a typo and left over debug command
2015-03-09 14:53:16 -04:00
Craig Tracey 5e6826b361 Merge pull request #20 from dvorak/install_path-template
Allow overriding install_path format
2015-03-09 14:37:21 -04:00
Clayton O'Neill eab330bd89 Allow overriding clone depth
This changes the default for packages to be unlimited depth, but allows setting
the depth on a per project basis.  Without this change it's not possible to use
a gitref other than 'master'.
2015-03-09 14:18:37 -04:00
Craig Tracey 0bd7df5e9b Fix pep8 failures
pep8 with 1.6.x is now failing as it wants all imports at the
beginning of the file. Unfortunately, this does not work for us in
all circumstances. Therefore, ignore E402 and fix the E402's where
it actually makes sense.
2015-02-12 16:18:57 -05:00
Clayton O'Neill dd43459dcb Fix a typo and left over debug command 2015-02-11 20:32:06 +00:00
Clayton O'Neill 2448ad5113 Add support for projects using pbr freeze
Designate uses pbr freeze instead of pip freeze.  This also explicitly
filters out the project itself when doing the pip install, since pbr
freeze outputs the project itself.
2015-02-09 14:28:16 +00:00
Clayton O'Neill 32b307bc28 Allow overriding install_path format
Normally the install_path defaults to base_path/project_name.  If you wanted
to do anything else, like base_path/project_name-version you'd have to hard
code the version on the install_path variable for every project.  This allows
setting install_path as a templated top level setting.  The default reproduces
the existing base_path/project_name behavior.
2015-02-06 17:49:06 +00:00
Craig Tracey 1ca409bbde Remove whitespace from end of line
Errant extra space.
2015-01-14 16:55:23 -05:00
Craig Tracey 42ee6520de Add configuration data to packages
This change copies $project/etc into the resulting packages. This is useful
for things like no managing rootwrap configs, etc. This is managed with the
new setting 'include_config.'
2015-01-14 16:40:42 -05:00
Craig Tracey 09d428918c Fix broken branch checkout
gitrefs were not working for the package builder. This was due to the
args being one off.
2014-12-19 01:36:20 -05:00
Craig Tracey 7a33cc5c28 Merge pull request #15 from blueboxgroup/distro-target
Update distro target support
2014-12-13 12:14:59 -05:00
Craig Tracey e0e8de0514 Revert package building in tempdirs
I clearly wasnt thinking when I implemented building packages in
tempdirs. The problem is that virtualenvs are not (really) relocatable,
so building in a tempdir is realy a no-go. Now ensure that packages are
being built in their destination paths.
2014-12-13 12:05:03 -05:00
Craig Tracey a1ac1d71d1 Update distro target support
The SUPPORTED_DISTROS dict was backwards, so make it easier to work with.
Also, add regex support for things like "CentOS" vs "CentOS Linux"
2014-12-11 11:27:54 -05:00
Craig Tracey 5bf3865c09 Merge pull request #13 from paulczar/master
Support CentOS
2014-12-11 10:53:04 -05:00
Craig Tracey 979a775e18 Address broken package building due to tempdirs
Recently, support was added to build packages in temporary directories.
This inadvertently led to a bug wherein when the build is finished, and
the tempdir is removed, the packages were removed with it.

This fix adds the output_dir setting.  If set, the packages will be
output to the directory specified, else to the cwd of the giftwrap
process.
2014-12-11 00:56:26 -05:00
Paul Czarkowski b1048a250c Update package.py 2014-12-10 22:14:06 -06:00
Paul Czarkowski 233e514861 Merge pull request #5 from craigtracey/git-depth-info
Add information about the git repo and limit depth
2014-11-20 20:07:57 -06:00
Paul Czarkowski a0ca9ce640 Merge pull request #4 from craigtracey/temp-build-dir
Utilize a temporary build path
2014-11-20 20:06:59 -06:00
Jesse Keating c8ea1c2ece Merge pull request #7 from craigtracey/shell-type
Allow specification of build type from the CLI
2014-11-20 13:39:24 -08:00