x
/
pbrx
Code Issues Proposed changes

Retire repository 

Depends-On: https://review.opendev.org/726461
Needed-By: https://review.opendev.org/726463
Change-Id: I4deefcafc286f60c1dfc64d95ce8369fe322af95
This commit is contained in:
Andreas Jaeger 2020-05-08 21:18:21 +02:00
parent a6172b6b54
commit 56fd982f69
57 changed files with 8 additions and 2290 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.coveragerc
View File

@ -1,6 +0,0 @@
[run]
branch = True
source = pbrx
[report]
ignore_errors = True

59
.gitignore vendored
View File

@ -1,59 +0,0 @@
*.py[cod]
# C extensions
*.so
# Packages
*.egg*
*.egg-info
dist
build
eggs
parts
bin
var
sdist
develop-eggs
.installed.cfg
lib
lib64
# Installer logs
pip-log.txt
# Unit test / coverage reports
cover/
.coverage*
!.coveragerc
.tox
nosetests.xml
.testrepository
.stestr
.venv
# Translations
*.mo
# Mr Developer
.mr.developer.cfg
.project
.pydevproject
# Complexity
output/*.html
output/*/index.html
# Sphinx
doc/build
# pbr generates these
AUTHORS
ChangeLog
# Editors
*~
.*.swp
.*sw?
# Files created by releasenotes build
releasenotes/build

3
.mailmap
View File

@ -1,3 +0,0 @@
# Format is:
# <preferred e-mail> <other e-mail 1>
# <preferred e-mail> <other e-mail 2>

3
.stestr.conf
View File

@ -1,3 +0,0 @@
[DEFAULT]
test_path=./pbrx/tests
top_dir=./

20
.zuul.yaml
View File

@ -1,20 +0,0 @@
- project:
templates:
- docs-on-readthedocs
- publish-to-pypi
vars:
rtd_webhook_id: '42240'
check:
jobs:
- tox-pep8
- tox-py35:
nodeset: ubuntu-xenial
- tox-py36:
nodeset: ubuntu-bionic
gate:
jobs:
- tox-pep8
- tox-py35:
nodeset: ubuntu-xenial
- tox-py36:
nodeset: ubuntu-bionic

17
CONTRIBUTING.rst
View File

@ -1,17 +0,0 @@
If you would like to contribute to the development of OpenStack, you must
follow the steps in this page:
http://docs.openstack.org/infra/manual/developers.html
If you already have a good understanding of how the system works and your
OpenStack accounts are set up, you can skip to the development workflow
section of this documentation to learn how changes to OpenStack should be
submitted for review via the Gerrit tool:
http://docs.openstack.org/infra/manual/developers.html#development-workflow
Pull requests submitted through GitHub will be ignored.
Bugs should be filed on Storyboard:
https://storyboard.openstack.org

4
HACKING.rst
View File

@ -1,4 +0,0 @@
pbrx Style Commandments
===============================================
Read the OpenStack Style Commandments https://docs.openstack.org/hacking/latest/

176
LICENSE
View File

@ -1,176 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

48
README.rst
View File

@ -1,43 +1,11 @@
====
pbrx
====
This project is no longer maintained.
Utilities for projects using `pbr`_.
The contents of this repository are still available in the Git
source code management system. To see the contents of this
repository before it reached its end of life, please check out the
previous commit with "git checkout HEAD^1".
`pbr`_ is very opinionated about how things should be done. As a result,
there are a set of actions that become easy to deal with generically for
any `pbr`_ based project. **pbrx** is a collection of utilities that contain
support for such actions.
For any further questions, please email
openstack-discuss@lists.openstack.org or join #openstack-infra on
Freenode.
.. note::
Each of the utilities has a primary focus of working for projects using
pbr. However, some of them will also work just fine for non-pbr-based
projects. When that is the case, the utility will be marked appropriately.
* Free software: Apache license
* Documentation: https://docs.openstack.org/pbrx/latest
* Source: https://git.openstack.org/cgit/openstack/pbrx
Features
--------
Each utility is implemented as a subcommand on the ``pbrx`` command.
install-siblings
Updates an installation with local from-source versions of dependencies.
For any dependency that the normal installation installed from pip/PyPI,
``install-siblings`` will look for an adjacent git repository that provides
the same package. If one exists, the source version will be installed to
replace the released version. This is done in such a way that any given
``constraints`` will be honored and not get messed up by transitive depends.
build-images
Builds container images from a project's source tree. The ``python:alpine``
base image is used, and dependencies are taken from ``bindep.txt`` for
distro requirements and ``requirements.txt`` for python requirements. A
base image is made for the project itself, and then an additional image
based on the base image for every entry in ``entry_points.console_scripts``
in the ``setup.cfg`` file.
.. _pbr: https://docs.openstack.org/pbr/latest/

2
babel.cfg
View File

@ -1,2 +0,0 @@
[python: **.py]

5
doc/source/cli/index.rst
View File

@ -1,5 +0,0 @@
================================
Command line interface reference
================================
CLI reference of pbrx.

81
doc/source/conf.py
View File

@ -1,81 +0,0 @@
# -*- coding: utf-8 -*-
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import sys
sys.path.insert(0, os.path.abspath('../..'))
# -- General configuration ----------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = [
'sphinx.ext.autodoc',
'openstackdocstheme',
#'sphinx.ext.intersphinx',
]
# autodoc generation is a bit aggressive and a nuisance when doing heavy
# text edit cycles.
# execute "export SPHINX_DEBUG=1" in your terminal to disable
# The suffix of source filenames.
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'pbrx'
copyright = u'2017, OpenStack Developers'
# openstackdocstheme options
repository_name = 'openstack/pbrx'
bug_project = 'pbrx'
bug_tag = ''
# If true, '()' will be appended to :func: etc. cross-reference text.
add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
add_module_names = True
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# -- Options for HTML output --------------------------------------------------
# The theme to use for HTML and HTML Help pages. Major themes that come with
# Sphinx are currently 'default' and 'sphinxdoc'.
# html_theme_path = ["."]
# html_theme = '_theme'
# html_static_path = ['static']
html_theme = 'openstackdocs'
# Output file base name for HTML help builder.
htmlhelp_basename = '%sdoc' % project
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass
# [howto/manual]).
latex_documents = [
('index',
'%s.tex' % project,
u'%s Documentation' % project,
u'OpenStack Developers', 'manual'),
]
# Example configuration for intersphinx: refer to the Python standard library.
#intersphinx_mapping = {'http://docs.python.org/': None}

5
doc/source/configuration/index.rst
View File

@ -1,5 +0,0 @@
=============
Configuration
=============
Configuration of pbrx.

4
doc/source/contributor/contributing.rst
View File

@ -1,4 +0,0 @@
============
Contributing
============
.. include:: ../../../CONTRIBUTING.rst

9
doc/source/contributor/index.rst
View File

@ -1,9 +0,0 @@
===========================
Contributor Documentation
===========================
.. toctree::
:maxdepth: 2
contributing

23
doc/source/index.rst
View File

@ -1,23 +0,0 @@
====================================
Welcome to the documentation of pbrx
====================================
Contents:
.. toctree::
:maxdepth: 2
readme
install/index
contributor/index
configuration/index
cli/index
user/index
reference/index
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

10
doc/source/install/common_configure.rst
View File

@ -1,10 +0,0 @@
2. Edit the ``/etc/pbrx/pbrx.conf`` file and complete the following
actions:
* In the ``[database]`` section, configure database access:
.. code-block:: ini
[database]
...
connection = mysql+pymysql://pbrx:PBRX_DBPASS@controller/pbrx

75
doc/source/install/common_prerequisites.rst
View File

@ -1,75 +0,0 @@
Prerequisites
-------------
Before you install and configure the pbrx service,
you must create a database, service credentials, and API endpoints.
#. To create the database, complete these steps:
* Use the database access client to connect to the database
server as the ``root`` user:
.. code-block:: console
$ mysql -u root -p
* Create the ``pbrx`` database:
.. code-block:: none
CREATE DATABASE pbrx;
* Grant proper access to the ``pbrx`` database:
.. code-block:: none
GRANT ALL PRIVILEGES ON pbrx.* TO 'pbrx'@'localhost' \
IDENTIFIED BY 'PBRX_DBPASS';
GRANT ALL PRIVILEGES ON pbrx.* TO 'pbrx'@'%' \
IDENTIFIED BY 'PBRX_DBPASS';
Replace ``PBRX_DBPASS`` with a suitable password.
* Exit the database access client.
.. code-block:: none
exit;
#. Source the ``admin`` credentials to gain access to
admin-only CLI commands:
.. code-block:: console
$ . admin-openrc
#. To create the service credentials, complete these steps:
* Create the ``pbrx`` user:
.. code-block:: console
$ openstack user create --domain default --password-prompt pbrx
* Add the ``admin`` role to the ``pbrx`` user:
.. code-block:: console
$ openstack role add --project service --user pbrx admin
* Create the pbrx service entities:
.. code-block:: console
$ openstack service create --name pbrx --description "pbrx" pbrx
#. Create the pbrx service API endpoints:
.. code-block:: console
$ openstack endpoint create --region RegionOne \
pbrx public http://controller:XXXX/vY/%\(tenant_id\)s
$ openstack endpoint create --region RegionOne \
pbrx internal http://controller:XXXX/vY/%\(tenant_id\)s
$ openstack endpoint create --region RegionOne \
pbrx admin http://controller:XXXX/vY/%\(tenant_id\)s

9
doc/source/install/get_started.rst
View File

@ -1,9 +0,0 @@
=====================
pbrx service overview
=====================
The pbrx service provides...
The pbrx service consists of the following components:
``pbrx-api`` service
Accepts and responds to end user compute API calls...

17
doc/source/install/index.rst
View File

@ -1,17 +0,0 @@
===============================
pbrx service installation guide
===============================
.. toctree::
:maxdepth: 2
get_started.rst
install.rst
verify.rst
next-steps.rst
The pbrx service (pbrx) provides...
This chapter assumes a working setup of OpenStack following the
`OpenStack Installation Tutorial
<https://docs.openstack.org/project-install-guide/ocata/>`_.

34
doc/source/install/install-obs.rst
View File

@ -1,34 +0,0 @@
.. _install-obs:
Install and configure for openSUSE and SUSE Linux Enterprise
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This section describes how to install and configure the pbrx service
for openSUSE Leap 42.1 and SUSE Linux Enterprise Server 12 SP1.
.. include:: common_prerequisites.rst
Install and configure components
--------------------------------
#. Install the packages:
.. code-block:: console
# zypper --quiet --non-interactive install
.. include:: common_configure.rst
Finalize installation
---------------------
Start the pbrx services and configure them to start when
the system boots:
.. code-block:: console
# systemctl enable openstack-pbrx-api.service
# systemctl start openstack-pbrx-api.service

33
doc/source/install/install-rdo.rst
View File

@ -1,33 +0,0 @@
.. _install-rdo:
Install and configure for Red Hat Enterprise Linux and CentOS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This section describes how to install and configure the pbrx service
for Red Hat Enterprise Linux 7 and CentOS 7.
.. include:: common_prerequisites.rst
Install and configure components
--------------------------------
#. Install the packages:
.. code-block:: console
# yum install
.. include:: common_configure.rst
Finalize installation
---------------------
Start the pbrx services and configure them to start when
the system boots:
.. code-block:: console
# systemctl enable openstack-pbrx-api.service
# systemctl start openstack-pbrx-api.service

31
doc/source/install/install-ubuntu.rst
View File

@ -1,31 +0,0 @@
.. _install-ubuntu:
Install and configure for Ubuntu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This section describes how to install and configure the pbrx
service for Ubuntu 14.04 (LTS).
.. include:: common_prerequisites.rst
Install and configure components
--------------------------------
#. Install the packages:
.. code-block:: console
# apt-get update
# apt-get install
.. include:: common_configure.rst
Finalize installation
---------------------
Restart the pbrx services:
.. code-block:: console
# service openstack-pbrx-api restart

20
doc/source/install/install.rst
View File

@ -1,20 +0,0 @@
.. _install:
Install and configure
~~~~~~~~~~~~~~~~~~~~~
This section describes how to install and configure the
pbrx service, code-named pbrx, on the controller node.
This section assumes that you already have a working OpenStack
environment with at least the following components installed:
.. (add the appropriate services here and further notes)
Note that installation and configuration vary by distribution.
.. toctree::
:maxdepth: 2
install-obs.rst
install-rdo.rst
install-ubuntu.rst

9
doc/source/install/next-steps.rst
View File

@ -1,9 +0,0 @@
.. _next-steps:
Next steps
~~~~~~~~~~
Your OpenStack environment now includes the pbrx service.
To add additional services, see
https://docs.openstack.org/project-install-guide/ocata/.

24
doc/source/install/verify.rst
View File

@ -1,24 +0,0 @@
.. _verify:
Verify operation
~~~~~~~~~~~~~~~~
Verify operation of the pbrx service.
.. note::
Perform these commands on the controller node.
#. Source the ``admin`` project credentials to gain access to
admin-only CLI commands:
.. code-block:: console
$ . admin-openrc
#. List service components to verify successful launch and registration
of each process:
.. code-block:: console
$ openstack pbrx service list

1
doc/source/readme.rst
View File

@ -1 +0,0 @@
.. include:: ../../README.rst

5
doc/source/reference/index.rst
View File

@ -1,5 +0,0 @@
==========
References
==========
References of pbrx.

97
doc/source/user/images.rst
View File

@ -1,97 +0,0 @@
=========================
Building Container Images
=========================
Python projects that declare their distro dependencies using `bindep`_
can be built into container images without any additional duplicate
configuration. The `pbrx` command ``build-images`` does this as minimally
and efficiently as possible. The aim is to produce single-process application
images that container only those things needed at runtime.
When ``pbrx build-images`` is run in a project source directory, the result
will be a base image, named '{project}-base', and then an image for each
entry in ``entry_points.console_scripts`` with ``CMD`` set to that console
script. For instance, in a python project "foo" that provides console scripts
called "foo-manage" and "foo-scheduler", ``pbrx build-images`` will result in
container images called "foo-base", "foo-manage" and "foo-scheduler".
``pbrx build-images`` uses volume mounts during the image build process instead
of copying to prevent wasted energy in getting source code into the image and
in getting artifacts out of the image. This makes it well suited for use on
laptops or in automation that has access to something that behaves like a full
computer but at the moment less well suited for use in unprivileged container
systems. Work will be undertaken to remove this limitation.
Distro Depends
==============
``build-images`` relies on `bindep`_ and ``bindep.txt`` to get the list of
packages to install.
``build-images`` uses the Builder Image pattern so that one image is used to
make wheels of the project and its dependencies, and another to install the
package. Distro packages needed to build wheels of a project or its python
depends from source should be marked with a ``compile`` profile in
``bindep.txt``. Distro packages needed at runtime should not be marked with
a profile.
``build-images`` uses ``python:alpine`` as a base image. There are no plans
or intent to make that configurable since these are application images and
the guest distro only serves to provide Python and c-library depends. To mark
dependencies in ``bindep.txt`` for images, the ``platform:apline`` profile
can be used.
The following is an example bindep file:
::
gcc [compile test platform:rpm platform:apk]
libffi-devel [compile test platform:rpm]
libffi-dev [compile test platform:dpkg platform:apk]
libffi [platform:apk]
libressl-dev [compile test platform:apk]
linux-headers [compile test platform:apk]
make [compile test platform:apk]
musl-dev [compile test platform:apk]
The only library needed at runtime is ``libffi``. The other dependencies are
all marked ``compile`` so will be installed into the build container but
not the final runtime container. `bindep`_ is useful not just for building
containers, so entries for ``libffi-dev`` on debian as well as ``libffi-devel``
on Red Hat are there. Also, this example marks some packages as needed for
``test``. `pbrx` and `bindep`_ appropriately ignore this information.
.. note::
Because of the use of the ``python:alpine`` image, it is not necessary to
list ``python3-dev`` in ``platform:alpine``.
Python Dependencies
===================
``build-images`` uses normal python mechanisms to get python dependencies.
Namely, it runs ``pip install .`` in the mounted source directory.
In most cases this is sufficient, but there are times when a single set of
dependencies for a set of console-scripts might not be appropriate. In this
case, it is possible to add a Python extra entry for a console script to add
additional python dependencies. For instance, this section in ``setup.cfg``:
::
[extras]
zuul_base =
PyMySQL
psycopg2-binary
zuul_executor =
ara
Will cause ``PyMySQL`` and ``psycopg2-binary`` to be installed into the base
image (even though they are optional dependencies for a normal install) and
for ``ara`` to be installed in the ``zuul-executor`` image.
.. note::
It is important to note that underscores must be used in the extras
definition in place of dashes.
.. _bindep: https://docs.openstack.org/infra/bindep/

10
doc/source/user/index.rst
View File

@ -1,10 +0,0 @@
===========
Users guide
===========
Users guide of pbrx.
.. toctree::
siblings
images

92
doc/source/user/siblings.rst
View File

@ -1,92 +0,0 @@
================================
Installation of Sibling Packages
================================
There are times, both in automated testing, and in local development, where
one wants to install versions of a project from git that are referenced in
a requirements file, or that have somehow already been installed into a given
environment.
This can become quite complicated if a constraints file is involved, as the
git versions don't match the versions in the constraints file. But if a
constraints file is in play, it should also be used for the installation of
the git versions of the additional projects so that their transitive depends
may be properly constrained.
To help with this, `pbrx` provides the ``install-siblings`` command. It takes
a list of paths to git repos to attempt to install, as well as an optional
constraints file.
It will only install a git repositoriy if there is already a corresponding
version of the package installed. This way it is safe to have other repos
wind up in the package list, such as if a Zuul job had a Depends-On including
one or more additional packages that were being put in place for other
purposes.
``pbrx siblings`` expects to be run in root source dir of the primary project.
Sibling projects may be given as relative or absolute paths.
For example, assume the following directory structure:
.. code-block:: none
$ tree -ld -L 3
├── git.openstack.org
│   ├── openstack
│   │   ├── keystoneauth
│   │   ├── python-openstackclient
│   │   ├── python-openstacksdk
│   │   ├── requirements
The user is in the ``git.openstack.org/openstack/python-openstackclient`` and
has installed the code into a virtualenv called ``venv``.
``python-openstackclient`` has the following requirements:
.. code-block:: none
keystoneauth1>=3.3.0 # Apache-2.0
openstacksdk>=0.9.19 # Apache-2.0
And in the ``git.openstack.org/openstack/requirements`` directory is a file
called ``upper-constraints.txt`` which contains:
.. code-block:: none
keystoneauth1===3.4.0
openstacksdk===0.11.3
requests===2.18.4
The command:
.. code-block:: none
$ venv/bin/pbrx install-siblings ../keystoneauth
would result in an installation of the contents of ``../keystoneauth``, since
``keystoneauth1`` is already installed and the package name in the
``git.openstack.org/openstack/keystoneauth`` directory is ``keystoneauth1``.
No constraints are given, so any transitive dependencies that are
in ``git.openstack.org/openstack/keystoneauth`` will be potentially installed
unconstrained.
.. code-block:: none
$ venv/bin/pbrx install-siblings -c ../requirements/upper-constraints.txt ../keystoneauth
Will also update ``keystoneauth1``, but will apply constraints properly to
any transitive depends.
.. code-block:: none
$ venv/bin/pbrx install-siblings -c ../requirements/upper-constraints.txt ../keystoneauth ../python-openstacksdk
will install both ``keystoneauth1`` and ``openstacksdk``.
.. code-block:: none
$ venv/bin/pbrx install-siblings -c ../requirements/upper-constraints.txt ../keystoneauth ../python-openstacksdk ../requirements
will also install both ``keystoneauth1`` and ``openstacksdk``. Even though
``git.openstack.org/openstack/requirements`` is itself a python package, since
it is not one of the ``python-openstackclient`` dependencies, it will be
skipped.

18
pbrx/__init__.py
View File

@ -1,18 +0,0 @@
# -*- coding: utf-8 -*-
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import pbr.version
__version__ = pbr.version.VersionInfo("pbrx").version_string()

0
pbrx/cmd/__init__.py
View File

145
pbrx/cmd/main.py
View File

@ -1,145 +0,0 @@
# Copyright 2018 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import argparse
import json
import logging
import logging.config
import os
import sys
try:
import yaml
except ImportError:
yaml = None
import pbr.version
from pbrx import container_images
from pbrx import siblings
log = logging.getLogger("pbrx")
def _read_logging_config_file(filename):
if not os.path.exists(filename):
raise ValueError("Unable to read logging config file at %s", filename)
ext = os.path.splitext(filename)[1]
if ext in (".yml", ".yaml"):
if not yaml:
raise ValueError(
"PyYAML not installed but a yaml logging config was provided."
" Install PyYAML, or convert the config to JSON."
)
return yaml.safe_load(open(filename, "r"))
elif ext == ".json":
return json.load(open(filename, "r"))
return filename
def setup_logging(log_config, debug):
if log_config:
config = _read_logging_config_file(log_config)
if isinstance(config, dict):
logging.config.dictConfig(config)
else:
logging.config.fileConfig(config)
else:
log.addHandler(logging.StreamHandler())
log.setLevel(logging.DEBUG if debug else logging.INFO)
def main():
parser = argparse.ArgumentParser(
description="pbrx: Utilities for projects using pbr"
)
parser.add_argument(
"--version",
action="version",
version=str(pbr.version.VersionInfo("pbrx")),
)
parser.add_argument(
"--debug", help="Emit debug output", action="store_true"
)
parser.add_argument(
"--log-config",
help="Path to a logging config file. Takes precedence over --debug",
)
subparsers = parser.add_subparsers(
title="commands", description="valid commands",
dest="command", help="additional help"
)
cmd_siblings = subparsers.add_parser(
"install-siblings", help="install sibling packages"
)
cmd_siblings.set_defaults(func=siblings.main)
cmd_siblings.add_argument(
"-c,--constraints",
dest="constraints",
help="Path to constraints file",
required=False,
)
cmd_siblings.add_argument(
"projects", nargs="*", help="List of project src dirs to process"
)
cmd_images = subparsers.add_parser(
"build-images", help="build per-process container images"
)
cmd_images.set_defaults(func=container_images.build)
cmd_images.add_argument(
"--prefix",
help="Organization prefix container images will be published to"
)
cmd_images.add_argument(
"--mirror",
help=(
"Base url for an alpine mirror to use. Will be used to replace"
" http://dl-cdn.alpinelinux.org/alpine"),
)
cmd_push = subparsers.add_parser(
"push-images", help="push project container images to a repository"
)
cmd_push.set_defaults(func=container_images.push)
cmd_push.add_argument(
"--prefix",
help="Organization prefix container images will be published to",
required=True
)
args = parser.parse_args()
setup_logging(args.log_config, args.debug)
if not args.command:
parser.print_help()
return 1
try:
return args.func(args)
except Exception as e:
log.exception(str(e))
return 1
return 0
if __name__ == "__main__":
sys.exit(main())

345
pbrx/container_images.py
View File

@ -1,345 +0,0 @@
# Copyright 2018 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
try:
import configparser
except ImportError:
import ConfigParser as configparser
import contextlib
import logging
import os
import tempfile
import sh
ALPINE_MIRROR_BASE = "http://dl-cdn.alpinelinux.org/alpine"
log = logging.getLogger("pbrx.container_images")
class ProjectInfo(object):
def __init__(self):
self.config = configparser.ConfigParser()
self.config.read("setup.cfg")
self.scripts = self._extract_scripts()
self.name = self.config.get("metadata", "name")
def _extract_scripts(self):
console_scripts = self.config.get("entry_points", "console_scripts")
scripts = set()
for line in console_scripts.strip().split("\n"):
parts = line.split("=")
if len(parts) != 2:
continue
scripts.add(parts[0].strip())
return scripts
@property
def base_container(self):
return "{name}-base".format(name=self.name)
class ContainerContext(object):
def __init__(self, base, volumes):
self._base = base
self._volumes = volumes or []
# bind-mount the pip.conf from the host so that any configured
# pypi mirrors will be used inside of the image builds.
if os.path.exists('/etc/pip.conf'):
self._volumes.append('/etc/pip.conf:/etc/pip.conf')
if os.path.exists(os.path.expanduser('~/.config/pip/pip.conf')):
self._volumes.append('{host}:{guest}'.format(
host=os.path.expanduser('~/.config/pip/pip.conf'),
guest='/root/.config/pip/pip.conf'))
self.create()
log.debug(
"Used base image {base} at sha {sha}".format(
base=self._base,
sha=sh.docker.images('-q', self._base).strip(),
))
self._cont = sh.docker.bake("exec", self.run_id, "sh", "-c",
_truncate_exc=False)
def create(self):
vargs = [
"create",
"--rm",
"-it",
"-v",
"{}:/usr/src".format(os.path.abspath(os.curdir)),
"-w",
"/usr/src",
]
for vol in self._volumes:
vargs.append("-v")
vargs.append(vol)
vargs.append(self._base)
vargs.append("sh")
container_id = sh.docker(*vargs).strip()
self.run_id = sh.docker('start', container_id).strip()
log.debug("Started container %s", self.run_id)
def run(self, command):
log.debug("Running: %s", command)
output = self._cont(command)
log.debug(output)
return output
def commit(self, image, tag=None, comment=None):
'''Apply a commit to the current container.
A new local image based on the current container is created with
the commit (name format of "image:tag").
:param str image: The local image name to create. This should
include any prefix (e.g., username/repository) appropriate for
pushing to a registry. If the image already exists, it is
overwritten.
:param str tag: The tag to apply to the new repo. If not supplied,
the docker default "latest" is used.
:param str comment: A commit message to apply to the new repo.
'''
commit_args = []
if comment:
commit_args.append("-c")
commit_args.append(comment)
commit_args.append(self.run_id)
if image:
if tag:
image = ":".join([image, tag])
commit_args.append(image)
log.debug("Committing container %s to %s", self.run_id, image)
sh.docker.commit(*commit_args)
@contextlib.contextmanager
def docker_container(base, image=None, prefix=None, comment=None, volumes=None):
'''Context manager to use for container runs.
This will start a new container, optionally commit it to a new local
image, and remove the container at exit.
:param str base: Name of base image to use for the container.
:param str image: Image name to use for the new local image. If not
supplied, no local image is created.
:param str prefix: Prefix to apply to the new local image name.
:param str comment: Commit message to use for the new local image.
:param list volumes: List of volumes to bind to the container.
'''
container = ContainerContext(base, volumes)
yield container
if image:
if prefix:
image = "/".join([prefix, image])
container.commit(image, comment=comment)
log.debug("Removing container %s", container.run_id)
sh.docker.rm("-f", container.run_id)
def build(args):
info = ProjectInfo()
log.info("Building base python container")
# Create base python container which has distro packages updated
with docker_container("python:alpine", image="python-base") as cont:
if args.mirror:
cont.run("sed -i 's,{old},{new}' /etc/apk/repositories".format(
old=ALPINE_MIRROR_BASE,
new=args.mirror))
cont.run("apk update")
log.info("Building bindep container")
# Create bindep container
with docker_container("python-base", image="bindep") as cont:
cont.run("pip install bindep")
# Use bindep container to get list of packages needed in the final
# container. It returns 1 if there are packages that need to be installed.
log.info("Get list of bindep packages for run")
try:
packages = sh.docker.run(
"--rm",
"-v",
"{pwd}:/usr/src".format(pwd=os.path.abspath(os.curdir)),
"bindep",
"bindep",
"-b",
)
log.debug(packages)
except sh.ErrorReturnCode_1 as e:
packages = e.stdout.decode('utf-8').strip()
try:
log.info("Get list of bindep packages for compile")
compile_packages = sh.docker.run(
"--rm",
"-v",
"{pwd}:/usr/src".format(pwd=os.path.abspath(os.curdir)),
"bindep",
"bindep",
"-b",
"compile",
)
log.debug(compile_packages)
except sh.ErrorReturnCode_1 as e:
compile_packages = e.stdout.decode('utf-8').strip()
packages = packages.replace("\r", "\n").replace("\n", " ")
compile_packages = compile_packages.replace("\r", "\n").replace("\n", " ")
# Make place for the wheels to go
with tempfile.TemporaryDirectory(
dir=os.path.abspath(os.curdir)
) as tmpdir:
try:
# Pass in the directory into ~/.cache/pip so that the built wheel
# cache will persist into the next container. The real wheel cache
# will go there but we'll also put our built wheel there.
tmp_volume = "{tmpdir}:/root/.cache/pip".format(tmpdir=tmpdir)
# Make temporary container that installs all deps to build wheel
# This container also needs git installed for pbr
log.info("Build wheels in python-base container")
with docker_container("python-base", volumes=[tmp_volume]) as cont:
# Make sure wheel cache dir is owned by container user
cont.run("chown -R $(whoami) /root/.cache/pip")
# Add the compile dependencies
cont.run("apk add {compile_packages} git".format(
compile_packages=compile_packages))
# Build a wheel so that we have an install target.
# pip install . in the container context with the mounted
# source dir gets ... exciting.
# We run sdist first to trigger code generation steps such
# as are found in zuul, since the sequencing otherwise
# happens in a way that makes wheel content copying unhappy.
cont.run(
"python setup.py sdist bdist_wheel -d /root/.cache/pip")
# Install with all container-related extras so that we populate
# the wheel cache as needed.
# NOTE(Shrews): The non-prefixed container names are referenced
# in the extras section.
cont.run(
"pip install"
" $(echo /root/.cache/pip/*.whl)[{base},{scripts}]".format(
base=info.base_container.replace('-', '_'),
scripts=','.join(info.scripts).replace('-', '_')))
# Build the final base container. Use dumb-init as the entrypoint
# so that signals and subprocesses work properly.
log.info("Build base container")
with docker_container(
"python-base",
image=info.base_container,
prefix=args.prefix,
volumes=[tmp_volume],
comment='ENTRYPOINT ["/usr/bin/dumb-init", "--"]',
) as cont:
try:
cont.run(
"apk add {packages} dumb-init".format(
packages=packages)
)
cont.run(
"pip install"
" $(echo /root/.cache/pip/*.whl)[{base}]".format(
base=info.base_container.replace('-', '_')))
if args.mirror:
cont.run(
"sed -i 's,{old},{new}'"
" /etc/apk/repositories".format(
old=args.mirror,
new=ALPINE_MIRROR_BASE))
# chown wheel cache back so the temp dir can delete it
cont.run("chown -R {uid} /root/.cache/pip".format(
uid=os.getuid()))
except Exception as e:
print(e.stdout)
raise
# Build a container for each program.
# In the simple-case, it's just an entrypoint commit setting CMD.
# If a Dockerfile exists for the program, use it instead.
# Such a Dockerfile should use:
# FROM {{ base_container }}-base
# This is useful for things like zuul-executor where the full
# story is not possible to express otherwise.
for script in info.scripts:
dockerfile = "Dockerfile.{script}".format(script=script)
if os.path.exists(dockerfile):
log.info(
"Building container for {script} from"
" Dockerfile".format(script=script))
sh.docker.build("-f", dockerfile, "-t", script, ".")
else:
log.info(
"Building container for {script}".format(
script=script))
# We already have a base container built that we'll use
# for the other images, but it may be prefixed.
base = info.base_container
if args.prefix:
base = "/".join([args.prefix, base])
with docker_container(
base,
image=script,
prefix=args.prefix,
volumes=[tmp_volume],
comment='CMD ["/usr/local/bin/{script}"]'.format(
script=script
),
) as cont:
cont.run(
"pip install"
" $(echo /root/.cache/pip/*.whl)[{script}]".format(
script=script.replace('-', '_')))
finally:
# chown wheel cache back so the temp dir can delete it
with docker_container(
"python-base",
volumes=[tmp_volume],
) as cont:
cont.run(
"chown -R {uid} /root/.cache/pip".format(uid=os.getuid()))
def push(args):
'''Push any built images to the registry.
The images built by pbrx's build-image command should already be named
with a prefix (the repository name to push to). It is expected that
we are already logged in to the registry as the user that owns the
repository.
'''
info = ProjectInfo()
unprefixed_image_names = info.scripts.copy()
unprefixed_image_names.add(info.base_container)
for image in unprefixed_image_names:
image = "/".join([args.prefix, image])
log.info("Pushing {image}".format(image=image))
sh.docker.push(image)

260
pbrx/siblings.py
View File

@ -1,260 +0,0 @@
# Copyright (c) 2018 Red Hat
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
try:
import configparser
except ImportError:
import ConfigParser as configparser
import logging
import os
import pkg_resources
import subprocess
import sys
import tempfile
log = logging.getLogger("pbrx")
def get_package_name(setup_cfg):
"""Get package name from a setup.cfg file."""
try:
c = configparser.ConfigParser()
c.read(setup_cfg)
return c.get("metadata", "name")
except Exception:
log.debug("No name in %s", setup_cfg)
return None
def get_requires_file(dist):
"""Get the path to the egg-info requires.txt file for a given dist."""
return os.path.join(
os.path.join(dist.location, dist.project_name + ".egg-info"),
"requires.txt",
)
def get_installed_packages():
"""Get the correct names of the currently installed packages."""
return [f.project_name for f in pkg_resources.working_set]
def pip_command(*args):
"""Execute a pip command in the current python."""
pip_args = [sys.executable, "-m", "pip"] + list(args)
log.debug("Executing %s", " ".join(pip_args))
output = subprocess.check_output(pip_args, stderr=subprocess.STDOUT)
for line in output.decode("utf-8").split("\n"):
log.debug(line)
return output
class Siblings(object):
def __init__(self, name, projects, constraints):
self.name = name
self.projects = projects
self.constraints = constraints
self.packages = {}
self.get_siblings()
log.info(
"Sibling Processing for %s from %s",
self.name,
os.path.abspath(os.path.curdir),
)
def get_siblings(self):
"""Finds all python packages that are there.
From the list of provided source dirs, find all of the ones that are
python projects and return a mapping of their package name to their
src_dir.
We ignore source dirs that are not python packages so that this can
be used with the list of all dependencies from a Zuul job. In the
future we might want to add a flag that causes that to be an error
for local execution.
"""
self.packages = {}
for root in self.projects:
root = os.path.abspath(root)
name = None
setup_cfg = os.path.join(root, "setup.cfg")
found_python = False
if os.path.exists(setup_cfg):
found_python = True
name = get_package_name(setup_cfg)
self.packages[name] = root
if not name and os.path.exists(os.path.join(root, "setup.py")):
found_python = True
# It's a python package but doesn't use pbr, so we need to run
# python setup.py --name to get setup.py to tell us what the
# package name is.
name = subprocess.check_output(
[sys.executable, "setup.py", "--name"],
cwd=root,
stderr=subprocess.STDOUT,
)
if name:
name = name.strip()
self.packages[name] = root
if found_python and not name:
log.info("Could not find package name for %s", root)
else:
log.info("Sibling %s at %s", name, root)
def write_new_constraints_file(self):
"""Write a temporary constraints file excluding siblings.
The git versions of the siblings are not going to match the values
in the constraints file, so write a copy of the constraints file
that doesn't have them in it, then use that when installing them.
"""
constraints_file = tempfile.NamedTemporaryFile(delete=False)
existing_constraints = open(self.constraints, "r")
for line in existing_constraints.read().split("\n"):
package_name = line.split("===")[0]
if package_name in self.packages:
continue
constraints_file.write(line.encode("utf-8"))
constraints_file.write(b"\n")
constraints_file.close()
return constraints_file
def find_sibling_packages(self):
for package_name in get_installed_packages():
log.debug("Found %s python package installed", package_name)
if package_name == self.name:
# We don't need to re-process ourself. We've filtered
# ourselves from the source dir list, but let's be sure
# nothing is weird.
log.debug("Skipping %s because it's us", package_name)
continue
if package_name in self.packages:
log.debug(
"Package %s on system in %s",
package_name,
self.packages[package_name],
)
log.info("Uninstalling %s", package_name)
pip_command("uninstall", "-y", package_name)
yield package_name
def clean_depends(self, installed_siblings):
"""Overwrite the egg-info requires.txt file for siblings.
When we install siblings for a package, we're explicitly saying
we want a local git repository. In some cases, the listed requirement
from the driving project clashes with what the new project reports
itself to be. We know we want the new project, so remove the version
specification from the requires.txt file in the main project's
egg-info dir.
"""
dist = None
for found_dist in pkg_resources.working_set:
if found_dist.project_name == self.name:
dist = found_dist
break
if not dist:
log.debug(
"main project is not installed, skipping requires clean"
)
return
requires_file = get_requires_file(dist)
if not os.path.exists(requires_file):
log.debug("%s file for main project not found", requires_file)
return
new_requires_file = tempfile.NamedTemporaryFile(delete=False)
with open(requires_file, "r") as main_requires:
for line in main_requires.readlines():
found = False
for name in installed_siblings:
if line.startswith(name):
log.debug(
"Replacing %s with %s in requires.txt",
line.strip(),
name,
)
new_requires_file.write(name.encode("utf-8"))
new_requires_file.write(b"\n")
found = True
break
if not found:
new_requires_file.write(line.encode("utf-8"))
os.rename(new_requires_file.name, requires_file)
def process(self):
"""Find and install the given sibling projects."""
installed_siblings = []
package_args = []
for sibling_package in self.find_sibling_packages():
log.info(
"Installing %s from %s",
sibling_package,
self.packages[sibling_package],
)
package_args.append("-e")
package_args.append(self.packages[sibling_package])
installed_siblings.append(sibling_package)
if not package_args:
log.info("Found no sibling packages, nothing to do.")
return
args = ["install"]
if self.constraints:
constraints_file = self.write_new_constraints_file()
args.extend(["-c", constraints_file.name])
args.extend(package_args)
try:
pip_command(*args)
finally:
os.unlink(constraints_file.name)
self.clean_depends(installed_siblings)
def main(args):
if not os.path.exists("setup.cfg"):
log.info("No setup.cfg found, no action needed")
return 0
if not args.projects:
log.info("No sibling projects given, no action needed.")
return 0
if args.constraints and not os.path.exists(args.constraints):
log.info("Constraints file %s was not found", args.constraints)
return 1
# Who are we?
package_name = get_package_name("setup.cfg")
if not package_name:
log.info("No name in main setup.cfg, skipping siblings")
return 0
siblings = Siblings(package_name, args.projects, args.constraints)
siblings.process()
return 0

0
pbrx/tests/__init__.py
View File

22
pbrx/tests/base.py
View File

@ -1,22 +0,0 @@
# -*- coding: utf-8 -*-
# Copyright 2010-2011 OpenStack Foundation
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslotest import base
class TestCase(base.BaseTestCase):
"""Test case base class for all unit tests."""

31
pbrx/tests/test_command.py
View File

@ -1,31 +0,0 @@
# -*- coding: utf-8 -*-
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import fixtures
from pbrx.cmd import main
from pbrx.tests import base
class TestCommand(base.TestCase):
def patch_argv(self, *args):
argv = ["pbrx"]
argv.extend(args)
self.useFixture(fixtures.MonkeyPatch("sys.argv", argv))
def test_no_args(self):
'''Test no arguments to the command'''
self.patch_argv()
main.main()

28
pbrx/tests/test_pbrx.py
View File

@ -1,28 +0,0 @@
# -*- coding: utf-8 -*-
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
test_pbrx
----------------------------------
Tests for `pbrx` module.
"""
from pbrx.tests import base
class TestPbrx(base.TestCase):
def test_something(self):
pass

11
playbooks/container-images/pre.yaml
View File

@ -1,11 +0,0 @@
- hosts: all
roles:
- ensure-docker
- hosts: all
tasks:
- name: Install pbrx software
command: python3 -m pip install src/opendev.org/x/pbrx
become: yes

17
playbooks/container-images/run.yaml
View File

@ -1,17 +0,0 @@
- hosts: all
tasks:
- name: Build prefexed container images
when: pbrx_prefix is defined
command: 'pbrx --debug build-images --prefix={{ pbrx_prefix }}'
args:
chdir: '{{ zuul_work_dir|default(zuul.project.src_dir) }}'
- name: Build unprefixed zuul container images
when: pbrx_prefix is not defined
command: pbrx --debug build-images
args:
chdir: '{{ zuul_work_dir|default(zuul.project.src_dir) }}'
- name: Output image information
command: docker images

0
releasenotes/notes/.placeholder
View File

5
releasenotes/notes/add-build-images-1d68a85158129d3a.yaml
View File

@ -1,5 +0,0 @@
---
features:
- |
Add support for building per-process container images for each console
script entry point defined.

6
releasenotes/notes/add-install-siblings-4c8bfc91a85bf025.yaml
View File

@ -1,6 +0,0 @@
---
features:
- |
Added ``install-siblings`` command to install relevant packages from
nearby git repositories.
some details.

5
releasenotes/notes/push-images-09c0da9bc9c2cc91.yaml
View File

@ -1,5 +0,0 @@
---
features:
- |
Added new push-images command to push any images built by the build-images
command.

0
releasenotes/source/_static/.placeholder
View File

0
releasenotes/source/_templates/.placeholder
View File

280
releasenotes/source/conf.py
View File

@ -1,280 +0,0 @@
# -*- coding: utf-8 -*-
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
# sys.path.insert(0, os.path.abspath('.'))
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'openstackdocstheme',
'reno.sphinxext',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
# source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'pbrx Release Notes'
copyright = u'2017, OpenStack Developers'
# openstackdocstheme options
repository_name = 'openstack/pbrx'
bug_project = 'pbrx'
bug_tag = ''
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
# The full version, including alpha/beta/rc tags.
release = ''
# The short X.Y version.
version = ''
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
# language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
# today = ''
# Else, today_fmt is used as the format for a strftime call.
# today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = []
# The reST default role (used for this markup: `text`) to use for all
# documents.
# default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
# add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
# add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
# show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
# modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
# keep_warnings = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'openstackdocs'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
# html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
# html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
# html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
# html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
# html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
# html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
# html_extra_path = []
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
# html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
# html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
# html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
# html_additional_pages = {}
# If false, no module index is generated.
# html_domain_indices = True
# If false, no index is generated.
# html_use_index = True
# If true, the index is split into individual pages for each letter.
# html_split_index = False
# If true, links to the reST sources are added to the pages.
# html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
# html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
# html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
# html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
# html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'pbrxReleaseNotesdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
# 'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
('index', 'pbrxReleaseNotes.tex',
u'pbrx Release Notes Documentation',
u'OpenStack Foundation', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
# latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
# latex_use_parts = False
# If true, show page references after internal links.
# latex_show_pagerefs = False
# If true, show URL addresses after external links.
# latex_show_urls = False
# Documents to append as an appendix to all manuals.
# latex_appendices = []
# If false, no module index is generated.
# latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'pbrxrereleasenotes',
u'pbrx Release Notes Documentation',
[u'OpenStack Foundation'], 1)
]
# If true, show URL addresses after external links.
# man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'pbrx ReleaseNotes',
u'pbrx Release Notes Documentation',
u'OpenStack Foundation', 'pbrxReleaseNotes',
'One line description of project.',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
# texinfo_appendices = []
# If false, no module index is generated.
# texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
# texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
# texinfo_no_detailmenu = False
# -- Options for Internationalization output ------------------------------
locale_dirs = ['locale/']

8
releasenotes/source/index.rst
View File

@ -1,8 +0,0 @@
============================================
pbrx Release Notes
============================================
.. toctree::
:maxdepth: 1
unreleased

5
releasenotes/source/unreleased.rst
View File

@ -1,5 +0,0 @@
==============================
Current Series Release Notes
==============================
.. release-notes::

6
requirements.txt
View File

@ -1,6 +0,0 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
pbr>=2.0 # Apache-2.0
sh>=1.12

44
setup.cfg
View File

@ -1,44 +0,0 @@
[metadata]
name = pbrx
summary = Utilities for projects using pbr
description-file =
README.rst
author = OpenStack
author-email = openstack-dev@lists.openstack.org
home-page = http://www.openstack.org/
classifier =
Intended Audience :: Information Technology
Intended Audience :: System Administrators
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: 3
Programming Language :: Python :: 3.5
Programming Language :: Python :: 3.6
[entry_points]
console_scripts =
pbrx = pbrx.cmd.main:main
[build_sphinx]
all-files = 1
warning-is-error = 1
source-dir = doc/source
build-dir = doc/build
[upload_sphinx]
upload-dir = doc/build/html
[compile_catalog]
directory = pbrx/locale
domain = pbrx
[update_catalog]
domain = pbrx
output_dir = pbrx/locale
input_file = pbrx/locale/pbrx.pot
[extract_messages]
keywords = _ gettext ngettext l_ lazy_gettext
mapping_file = babel.cfg
output_file = pbrx/locale/pbrx.pot

29
setup.py
View File

@ -1,29 +0,0 @@
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT
import setuptools
# In python < 2.7.4, a lazy loading of package `pbr` will break
# setuptools if some other modules registered functions in `atexit`.
# solution from: http://bugs.python.org/issue15881#msg170215
try:
import multiprocessing # noqa
except ImportError:
pass
setuptools.setup(
setup_requires=['pbr'],
pbr=True)

15
test-requirements.txt
View File

@ -1,15 +0,0 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
hacking>=0.12.0,<0.13 # Apache-2.0
coverage>=4.0,!=4.4 # Apache-2.0
python-subunit>=0.0.18 # Apache-2.0/BSD
sphinx>=1.6.2 # BSD
oslotest>=1.10.0 # Apache-2.0
stestr>=1.0.0 # Apache-2.0
testtools>=1.4.0 # MIT
openstackdocstheme>=1.11.0 # Apache-2.0
# releasenotes
reno>=1.8.0 # Apache-2.0

35
tools/test-siblings.sh
View File

@ -1,35 +0,0 @@
# Quick test script used during development. This should obviously be replaced
# with actual tests.
# However, if you have repos locally in golang organization, and you have:
# git.openstack.org/openstack-dev/pbr
# git.openstack.org/openstack/keystoneauth
# git.openstack.org/openstack/openstacksdk
# git.openstack.org/openstack/pbrx
# git.openstack.org/openstack/python-openstackclient
# git.openstack.org/openstack/requirements
# github.com/requests/requests
#
# You should be able to run this script in the openstack/python-openstackclient
# directory and verify it does the right things.
# openstack/python-openstackclient was selected because it exhibits a complex
# Entrypoints issue.
set -e
BASE=../../..
OPENSTACK=$BASE/git.openstack.org/openstack
for interp in python2 python3 ; do
venv=venv-$interp
rm -rf $venv
virtualenv --python=$interp venv-$interp
# Install python-openstackclient's requirements with constraints
$venv/bin/pip install -c $OPENSTACK/requirements/upper-constraints.txt -r requirements.txt
# Install python-openstackclient itself
$venv/bin/pip install --no-deps -e .
# Install pbrx with this patch
$venv/bin/pip install -e $BASE/git.openstack.org/openstack/pbrx/
# Run siblings
$venv/bin/pbrx --debug install-siblings -c $OPENSTACK/requirements/upper-constraints.txt $OPENSTACK/keystoneauth $OPENSTACK/openstack/python-openstacksdk $BASE/github.com/requests/requests $BASE/git.openstack.org/openstack-dev/pbr/
# openstack help should not break
$venv/bin/openstack help
done

51
tox.ini
View File

@ -1,51 +0,0 @@
[tox]
minversion = 2.0
envlist = py35,py36,pep8
skipsdist = True
[testenv]
usedevelop = True
install_command = pip install -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt} {opts} {packages}
setenv =
VIRTUAL_ENV={envdir}
OS_STDOUT_CAPTURE=1
OS_STDERR_CAPTURE=1
OS_TEST_TIMEOUT=60
deps = -r{toxinidir}/test-requirements.txt
commands = stestr run {posargs}
basepython = python3
[testenv:pep8]
basepython = python3
commands = flake8 {posargs}
[testenv:venv]
basepython = python3
commands = {posargs}
[testenv:cover]
basepython = python3
setenv =
VIRTUAL_ENV={envdir}
PYTHON=coverage run --source pbrx --parallel-mode
commands =
stestr run {posargs}
coverage combine
coverage html -d cover
coverage xml -o cover/coverage.xml
[testenv:docs]
basepython = python3
commands =
sphinx-build -a -E -W -d doc/build/doctrees -b html doc/source doc/build/html
[testenv:releasenotes]
basepython = python3
commands =
sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html
[flake8]
max-line-length = 88
show-source = True
builtins = _
exclude=.venv,.git,.tox,dist,doc,*lib/python*,*egg,build