diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
deleted file mode 100644
index 0e9946c..0000000
--- a/CONTRIBUTING.rst
+++ /dev/null
@@ -1,17 +0,0 @@
-If you would like to contribute to the development of OpenStack, you must
-follow the steps in this page:
-
- http://docs.openstack.org/infra/manual/developers.html
-
-If you already have a good understanding of how the system works and your
-OpenStack accounts are set up, you can skip to the development workflow
-section of this documentation to learn how changes to OpenStack should be
-submitted for review via the Gerrit tool:
-
- http://docs.openstack.org/infra/manual/developers.html#development-workflow
-
-Pull requests submitted through GitHub will be ignored.
-
-Bugs should be filed on Launchpad, not GitHub:
-
- https://bugs.launchpad.net/fuel
diff --git a/HACKING.rst b/HACKING.rst
deleted file mode 100644
index 50a409c..0000000
--- a/HACKING.rst
+++ /dev/null
@@ -1,4 +0,0 @@
-fuel-mirror Style Commandments
-==============================
-
-Read the OpenStack Style Commandments http://docs.openstack.org/developer/hacking/
diff --git a/LICENSE b/LICENSE
deleted file mode 100644
index 8cdb845..0000000
--- a/LICENSE
+++ /dev/null
@@ -1,340 +0,0 @@
- GNU GENERAL PUBLIC LICENSE
- Version 2, June 1991
-
- Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
- 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
- Everyone is permitted to copy and distribute verbatim copies
- of this license document, but changing it is not allowed.
-
- Preamble
-
- The licenses for most software are designed to take away your
-freedom to share and change it. By contrast, the GNU General Public
-License is intended to guarantee your freedom to share and change free
-software--to make sure the software is free for all its users. This
-General Public License applies to most of the Free Software
-Foundation's software and to any other program whose authors commit to
-using it. (Some other Free Software Foundation software is covered by
-the GNU Lesser General Public License instead.) You can apply it to
-your programs, too.
-
- When we speak of free software, we are referring to freedom, not
-price. Our General Public Licenses are designed to make sure that you
-have the freedom to distribute copies of free software (and charge for
-this service if you wish), that you receive source code or can get it
-if you want it, that you can change the software or use pieces of it
-in new free programs; and that you know you can do these things.
-
- To protect your rights, we need to make restrictions that forbid
-anyone to deny you these rights or to ask you to surrender the rights.
-These restrictions translate to certain responsibilities for you if you
-distribute copies of the software, or if you modify it.
-
- For example, if you distribute copies of such a program, whether
-gratis or for a fee, you must give the recipients all the rights that
-you have. You must make sure that they, too, receive or can get the
-source code. And you must show them these terms so they know their
-rights.
-
- We protect your rights with two steps: (1) copyright the software, and
-(2) offer you this license which gives you legal permission to copy,
-distribute and/or modify the software.
-
- Also, for each author's protection and ours, we want to make certain
-that everyone understands that there is no warranty for this free
-software. If the software is modified by someone else and passed on, we
-want its recipients to know that what they have is not the original, so
-that any problems introduced by others will not reflect on the original
-authors' reputations.
-
- Finally, any free program is threatened constantly by software
-patents. We wish to avoid the danger that redistributors of a free
-program will individually obtain patent licenses, in effect making the
-program proprietary. To prevent this, we have made it clear that any
-patent must be licensed for everyone's free use or not licensed at all.
-
- The precise terms and conditions for copying, distribution and
-modification follow.
-
- GNU GENERAL PUBLIC LICENSE
- TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
-
- 0. This License applies to any program or other work which contains
-a notice placed by the copyright holder saying it may be distributed
-under the terms of this General Public License. The "Program", below,
-refers to any such program or work, and a "work based on the Program"
-means either the Program or any derivative work under copyright law:
-that is to say, a work containing the Program or a portion of it,
-either verbatim or with modifications and/or translated into another
-language. (Hereinafter, translation is included without limitation in
-the term "modification".) Each licensee is addressed as "you".
-
-Activities other than copying, distribution and modification are not
-covered by this License; they are outside its scope. The act of
-running the Program is not restricted, and the output from the Program
-is covered only if its contents constitute a work based on the
-Program (independent of having been made by running the Program).
-Whether that is true depends on what the Program does.
-
- 1. You may copy and distribute verbatim copies of the Program's
-source code as you receive it, in any medium, provided that you
-conspicuously and appropriately publish on each copy an appropriate
-copyright notice and disclaimer of warranty; keep intact all the
-notices that refer to this License and to the absence of any warranty;
-and give any other recipients of the Program a copy of this License
-along with the Program.
-
-You may charge a fee for the physical act of transferring a copy, and
-you may at your option offer warranty protection in exchange for a fee.
-
- 2. You may modify your copy or copies of the Program or any portion
-of it, thus forming a work based on the Program, and copy and
-distribute such modifications or work under the terms of Section 1
-above, provided that you also meet all of these conditions:
-
- a) You must cause the modified files to carry prominent notices
- stating that you changed the files and the date of any change.
-
- b) You must cause any work that you distribute or publish, that in
- whole or in part contains or is derived from the Program or any
- part thereof, to be licensed as a whole at no charge to all third
- parties under the terms of this License.
-
- c) If the modified program normally reads commands interactively
- when run, you must cause it, when started running for such
- interactive use in the most ordinary way, to print or display an
- announcement including an appropriate copyright notice and a
- notice that there is no warranty (or else, saying that you provide
- a warranty) and that users may redistribute the program under
- these conditions, and telling the user how to view a copy of this
- License. (Exception: if the Program itself is interactive but
- does not normally print such an announcement, your work based on
- the Program is not required to print an announcement.)
-
-These requirements apply to the modified work as a whole. If
-identifiable sections of that work are not derived from the Program,
-and can be reasonably considered independent and separate works in
-themselves, then this License, and its terms, do not apply to those
-sections when you distribute them as separate works. But when you
-distribute the same sections as part of a whole which is a work based
-on the Program, the distribution of the whole must be on the terms of
-this License, whose permissions for other licensees extend to the
-entire whole, and thus to each and every part regardless of who wrote it.
-
-Thus, it is not the intent of this section to claim rights or contest
-your rights to work written entirely by you; rather, the intent is to
-exercise the right to control the distribution of derivative or
-collective works based on the Program.
-
-In addition, mere aggregation of another work not based on the Program
-with the Program (or with a work based on the Program) on a volume of
-a storage or distribution medium does not bring the other work under
-the scope of this License.
-
- 3. You may copy and distribute the Program (or a work based on it,
-under Section 2) in object code or executable form under the terms of
-Sections 1 and 2 above provided that you also do one of the following:
-
- a) Accompany it with the complete corresponding machine-readable
- source code, which must be distributed under the terms of Sections
- 1 and 2 above on a medium customarily used for software interchange; or,
-
- b) Accompany it with a written offer, valid for at least three
- years, to give any third party, for a charge no more than your
- cost of physically performing source distribution, a complete
- machine-readable copy of the corresponding source code, to be
- distributed under the terms of Sections 1 and 2 above on a medium
- customarily used for software interchange; or,
-
- c) Accompany it with the information you received as to the offer
- to distribute corresponding source code. (This alternative is
- allowed only for noncommercial distribution and only if you
- received the program in object code or executable form with such
- an offer, in accord with Subsection b above.)
-
-The source code for a work means the preferred form of the work for
-making modifications to it. For an executable work, complete source
-code means all the source code for all modules it contains, plus any
-associated interface definition files, plus the scripts used to
-control compilation and installation of the executable. However, as a
-special exception, the source code distributed need not include
-anything that is normally distributed (in either source or binary
-form) with the major components (compiler, kernel, and so on) of the
-operating system on which the executable runs, unless that component
-itself accompanies the executable.
-
-If distribution of executable or object code is made by offering
-access to copy from a designated place, then offering equivalent
-access to copy the source code from the same place counts as
-distribution of the source code, even though third parties are not
-compelled to copy the source along with the object code.
-
- 4. You may not copy, modify, sublicense, or distribute the Program
-except as expressly provided under this License. Any attempt
-otherwise to copy, modify, sublicense or distribute the Program is
-void, and will automatically terminate your rights under this License.
-However, parties who have received copies, or rights, from you under
-this License will not have their licenses terminated so long as such
-parties remain in full compliance.
-
- 5. You are not required to accept this License, since you have not
-signed it. However, nothing else grants you permission to modify or
-distribute the Program or its derivative works. These actions are
-prohibited by law if you do not accept this License. Therefore, by
-modifying or distributing the Program (or any work based on the
-Program), you indicate your acceptance of this License to do so, and
-all its terms and conditions for copying, distributing or modifying
-the Program or works based on it.
-
- 6. Each time you redistribute the Program (or any work based on the
-Program), the recipient automatically receives a license from the
-original licensor to copy, distribute or modify the Program subject to
-these terms and conditions. You may not impose any further
-restrictions on the recipients' exercise of the rights granted herein.
-You are not responsible for enforcing compliance by third parties to
-this License.
-
- 7. If, as a consequence of a court judgment or allegation of patent
-infringement or for any other reason (not limited to patent issues),
-conditions are imposed on you (whether by court order, agreement or
-otherwise) that contradict the conditions of this License, they do not
-excuse you from the conditions of this License. If you cannot
-distribute so as to satisfy simultaneously your obligations under this
-License and any other pertinent obligations, then as a consequence you
-may not distribute the Program at all. For example, if a patent
-license would not permit royalty-free redistribution of the Program by
-all those who receive copies directly or indirectly through you, then
-the only way you could satisfy both it and this License would be to
-refrain entirely from distribution of the Program.
-
-If any portion of this section is held invalid or unenforceable under
-any particular circumstance, the balance of the section is intended to
-apply and the section as a whole is intended to apply in other
-circumstances.
-
-It is not the purpose of this section to induce you to infringe any
-patents or other property right claims or to contest validity of any
-such claims; this section has the sole purpose of protecting the
-integrity of the free software distribution system, which is
-implemented by public license practices. Many people have made
-generous contributions to the wide range of software distributed
-through that system in reliance on consistent application of that
-system; it is up to the author/donor to decide if he or she is willing
-to distribute software through any other system and a licensee cannot
-impose that choice.
-
-This section is intended to make thoroughly clear what is believed to
-be a consequence of the rest of this License.
-
- 8. If the distribution and/or use of the Program is restricted in
-certain countries either by patents or by copyrighted interfaces, the
-original copyright holder who places the Program under this License
-may add an explicit geographical distribution limitation excluding
-those countries, so that distribution is permitted only in or among
-countries not thus excluded. In such case, this License incorporates
-the limitation as if written in the body of this License.
-
- 9. The Free Software Foundation may publish revised and/or new versions
-of the General Public License from time to time. Such new versions will
-be similar in spirit to the present version, but may differ in detail to
-address new problems or concerns.
-
-Each version is given a distinguishing version number. If the Program
-specifies a version number of this License which applies to it and "any
-later version", you have the option of following the terms and conditions
-either of that version or of any later version published by the Free
-Software Foundation. If the Program does not specify a version number of
-this License, you may choose any version ever published by the Free Software
-Foundation.
-
- 10. If you wish to incorporate parts of the Program into other free
-programs whose distribution conditions are different, write to the author
-to ask for permission. For software which is copyrighted by the Free
-Software Foundation, write to the Free Software Foundation; we sometimes
-make exceptions for this. Our decision will be guided by the two goals
-of preserving the free status of all derivatives of our free software and
-of promoting the sharing and reuse of software generally.
-
- NO WARRANTY
-
- 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
-FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
-OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
-PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
-OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
-MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
-TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
-PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
-REPAIR OR CORRECTION.
-
- 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
-WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
-REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
-INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
-OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
-TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
-YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
-PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
-POSSIBILITY OF SUCH DAMAGES.
-
- END OF TERMS AND CONDITIONS
-
- How to Apply These Terms to Your New Programs
-
- If you develop a new program, and you want it to be of the greatest
-possible use to the public, the best way to achieve this is to make it
-free software which everyone can redistribute and change under these terms.
-
- To do so, attach the following notices to the program. It is safest
-to attach them to the start of each source file to most effectively
-convey the exclusion of warranty; and each file should have at least
-the "copyright" line and a pointer to where the full notice is found.
-
- {description}
- Copyright (C) {year} {fullname}
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License along
- with this program; if not, write to the Free Software Foundation, Inc.,
- 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-Also add information on how to contact you by electronic and paper mail.
-
-If the program is interactive, make it output a short notice like this
-when it starts in an interactive mode:
-
- Gnomovision version 69, Copyright (C) year name of author
- Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
- This is free software, and you are welcome to redistribute it
- under certain conditions; type `show c' for details.
-
-The hypothetical commands `show w' and `show c' should show the appropriate
-parts of the General Public License. Of course, the commands you use may
-be called something other than `show w' and `show c'; they could even be
-mouse-clicks or menu items--whatever suits your program.
-
-You should also get your employer (if you work as a programmer) or your
-school, if any, to sign a "copyright disclaimer" for the program, if
-necessary. Here is a sample; alter the names:
-
- Yoyodyne, Inc., hereby disclaims all copyright interest in the program
- `Gnomovision' (which makes passes at compilers) written by James Hacker.
-
- {signature of Ty Coon}, 1 April 1989
- Ty Coon, President of Vice
-
-This General Public License does not permit incorporating your program into
-proprietary programs. If your program is a subroutine library, you may
-consider it more useful to permit linking proprietary applications with the
-library. If this is what you want to do, use the GNU Lesser General
-Public License instead of this License.
-
diff --git a/MAINTAINERS b/MAINTAINERS
index fd5b24e..98a968b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -26,49 +26,6 @@ description:
maintainers:
-- contrib/:
- - name: Bulat Gaifullin
- email: bgaifullin@mirantis.com
- IRC: bgaifullin
-
- - name: Vladimir Kozhukalov
- email: vkozhukalov@mirantis.com
- IRC: kozhukalov
-
-- debian/: &packaging_team
- - name: Mikhail Ivanov
- email: mivanov@mirantis.com
- IRC: mivanov
-
- - name: Artem Silenkov
- email: asilenkov@mirantis.com
- IRC: asilenkov
-
- - name: Alexander Tsamutali
- email: atsamutali@mirantis.com
- IRC: astsmtl
-
- - name: Daniil Trishkin
- email: dtrishkin@mirantis.com
- IRC: dtrishkin
-
- - name: Ivan Udovichenko
- email: iudovichenko@mirantis.com
- IRC: tlbr
-
- - name: Igor Yozhikov
- email: iyozhikov@mirantis.com
- IRC: IgorYozhikov
-
-- packetary/:
- - name: Bulat Gaifullin
- email: bgaifullin@mirantis.com
- IRC: bgaifullin
-
- - name: Vladimir Kozhukalov
- email: vkozhukalov@mirantis.com
- IRC: kozhukalov
-
- perestroika/: &build_team
- name: Dmitry Burmistrov
email: dburmistrov@mirantis.com
@@ -77,5 +34,3 @@ maintainers:
- name: Sergey Kulanov
email: skulanov@mirantis.com
IRC: SergK
-
-- specs/: *packaging_team
diff --git a/MANIFEST.in b/MANIFEST.in
deleted file mode 100644
index 3f60bfe..0000000
--- a/MANIFEST.in
+++ /dev/null
@@ -1,5 +0,0 @@
-include AUTHORS
-include ChangeLog
-include *requirements.txt
-
-global-exclude *.pyc
diff --git a/README.rst b/README.rst
index 6608033..6c8611a 100644
--- a/README.rst
+++ b/README.rst
@@ -2,38 +2,6 @@
Repository structure
====================
-* contrib/fuel_mirror
- It is a command line utility that provides the same functionality
- and user interface as deprecated fuel-createmirror. It provides
- two major features:
- * clone/build mirror (full or partial)
- * update repository configuration in nailgun
- First one is a matter of packetary while second one should be left
- totally up to fuelclient. So this module is to be deprecated soon
- in favor of packetary and fuelclient.
-
- WARNING: It is not designed to be used on 'live' repositories
- that are available to clients during synchronization. That means
- repositories will be inconsistent during the update. Please use these
- scripts in conjunction with snapshots, on inactive repos, etc.
-
-* debian
- Specs for DEB packages.
-
-* doc
- Documentation for packetary module.
-
-* packetary
- It is a Python library and command line utilty that allows
- one to clone and build rpm/deb repositories.
- Features:
- * Common interface for different package-managers.
- * Utility to build dependency graph for package(s).
- * Utility to create mirror of repository according to dependency graph.
-
* perestroika
It is a set shell/python script that are used to build DEB/RPM
packages. These scripts are widely used by Fuel Packaging CI.
-
-* specs
- Specs for RPM packages.
diff --git a/babel.cfg b/babel.cfg
deleted file mode 100644
index 15cd6cb..0000000
--- a/babel.cfg
+++ /dev/null
@@ -1,2 +0,0 @@
-[python: **.py]
-
diff --git a/contrib/fuel_mirror/MANIFEST.in b/contrib/fuel_mirror/MANIFEST.in
deleted file mode 100644
index 5594774..0000000
--- a/contrib/fuel_mirror/MANIFEST.in
+++ /dev/null
@@ -1,8 +0,0 @@
-include AUTHORS
-include ChangeLog
-recursive-include etc *
-
-exclude .gitignore
-exclude .gitreview
-
-global-exclude *.pyc
diff --git a/contrib/fuel_mirror/README.rst b/contrib/fuel_mirror/README.rst
deleted file mode 100644
index 7f67615..0000000
--- a/contrib/fuel_mirror/README.rst
+++ /dev/null
@@ -1,16 +0,0 @@
-===========
-fuel_mirror
-===========
-
-The fuel-mirror is utility, that allows to create local repositories
-with packages are required for the OpenStack deployment.
-
-* Free software: Apache license
-* Documentation: http://docs.openstack.org/developer/fuel-mirror
-* Source: http://git.openstack.org/cgit/openstack/fuel-mirror/
-* Bugs: http://bugs.launchpad.net/fuel
-
-Features
---------
-
-* TODO
diff --git a/contrib/fuel_mirror/babel.cfg b/contrib/fuel_mirror/babel.cfg
deleted file mode 100644
index 15cd6cb..0000000
--- a/contrib/fuel_mirror/babel.cfg
+++ /dev/null
@@ -1,2 +0,0 @@
-[python: **.py]
-
diff --git a/contrib/fuel_mirror/data/centos.yaml b/contrib/fuel_mirror/data/centos.yaml
deleted file mode 100644
index 93c11da..0000000
--- a/contrib/fuel_mirror/data/centos.yaml
+++ /dev/null
@@ -1,55 +0,0 @@
-fuel_release_match:
- version: $openstack_version
- operating_system: CentOS
-
-repos:
- - ¢os
- name: "centos"
- uri: "http://vault.centos.org/7.1.1503/os/x86_64/"
- type: "rpm"
- priority: null
-
- - ¢os_updates
- name: "centos-updates"
- uri: "http://vault.centos.org/7.1.1503/updates/x86_64/"
- type: "rpm"
- priority: null
-
- - &mos
- name: "mos"
- uri: "http://mirror.fuel-infra.org/mos-repos/centos/mos$mos_version-centos7/os/x86_64/"
- type: "rpm"
- priority: null
-
- - &mos_updates
- name: "mos-updates"
- uri: "http://mirror.fuel-infra.org/mos-repos/centos/mos$mos_version-centos7/updates/x86_64/"
- type: "rpm"
- priority: null
-
- - &mos_security
- name: "mos-security"
- uri: "http://mirror.fuel-infra.org/mos-repos/centos/mos$mos_version-centos7/security/x86_64/"
- type: "rpm"
- priority: null
-
- - &mos_holdback
- name: "mos-holdback"
- uri: "http://mirror.fuel-infra.org/mos-repos/centos/mos$mos_version-centos7/holdback/x86_64/"
- type: "rpm"
- priority: null
-
-groups:
- mos:
- - *mos
- - *mos_updates
- - *mos_security
- - *mos_holdback
-
- centos:
- - *centos
- - *centos_updates
-
-
-inheritance:
- centos: mos
diff --git a/contrib/fuel_mirror/data/ubuntu.yaml b/contrib/fuel_mirror/data/ubuntu.yaml
deleted file mode 100644
index 6c51b9b..0000000
--- a/contrib/fuel_mirror/data/ubuntu.yaml
+++ /dev/null
@@ -1,158 +0,0 @@
-# GLOBAL variables
-ubuntu_baseurl: &ubuntu_baseurl http://archive.ubuntu.com/ubuntu
-mos_baseurl: &mos_baseurl http://mirror.fuel-infra.org/mos-repos/ubuntu/$mos_version
-
-fuel_release_match:
- version: $openstack_version
- operating_system: Ubuntu
-
-# Main is a required parameter which defines what repository will be used
-# for images creation and that mirror should contain all packages for minimal
-# system creation.
-repos:
- - &ubuntu
- main: true
- name: "ubuntu"
- uri: *ubuntu_baseurl
- suite: "trusty"
- section: "main multiverse restricted universe"
- type: "deb"
- priority: null
-
- - &ubuntu_updates
- name: "ubuntu-updates"
- uri: *ubuntu_baseurl
- suite: "trusty-updates"
- section: "main multiverse restricted universe"
- type: "deb"
- priority: null
-
- - &ubuntu_security
- name: "ubuntu-security"
- uri: *ubuntu_baseurl
- suite: "trusty-security"
- section: "main multiverse restricted universe"
- type: "deb"
- priority: null
-
- - &mos
- name: "mos"
- uri: *mos_baseurl
- suite: "mos$mos_version"
- section: "main restricted"
- type: "deb"
- priority: 1000
-
- - &mos_updates
- name: "mos-updates"
- uri: *mos_baseurl
- suite: "mos$mos_version-updates"
- section: "main restricted"
- type: "deb"
- priority: 1000
-
- - &mos_security
- name: "mos-security"
- uri: *mos_baseurl
- suite: "mos$mos_version-security"
- section: "main restricted"
- type: "deb"
- priority: 1000
-
- - &mos_holdback
- name: "mos-holdback"
- uri: *mos_baseurl
- suite: "mos$mos_version-holdback"
- section: "main restricted"
- type: "deb"
- priority: 1000
-
-# Packages are required to build bootstrap images for a system.
-# The mirror should contiain such packages in addition to local mirror.
-packages: &packages
- - "acpi-support"
- - "anacron"
- - "aptitude"
- - "atop"
- - "acct"
- - "bash-completion"
- - "bc"
- - "build-essential"
- - "cloud-init"
- - "conntrackd"
- - "cpu-checker"
- - "cpufrequtils"
- - "debconf-utils"
- - "devscripts"
- - "fping"
- - "git"
- - "grub-pc"
- - "htop"
- - "hwloc"
- - "ifenslave"
- - "iperf"
- - "iptables-persistent"
- - "irqbalance"
- - "language-pack-en"
- - "libapache2-mod-fastcgi"
- - "libnss3-tools"
- - "linux-firmware-nonfree"
- - "linux-headers-generic-lts-trusty"
- - "linux-image-generic-lts-trusty"
- - "live-boot"
- - "livecd-rootfs"
- - "mc"
- - "memcached"
- - "monit"
- - "msmtp-mta"
- - "multipath-tools"
- - "multipath-tools-boot"
- - "nginx"
- - "ntp"
- - "openssh-server"
- - "percona-toolkit"
- - "percona-xtrabackup"
- - "pm-utils"
- - "puppet"
- - "python-lesscpy"
- - "python-pip"
- - "rsyslog-gnutls"
- - "rsyslog-relp"
- - "screen"
- - "squashfs-tools"
- - "swift-plugin-s3"
- - "sysfsutils"
- - "sysstat"
- - "telnet"
- - "tmux"
- - "traceroute"
- - "ubuntu-standard"
- - "vim"
- - "virt-what"
- - "xinetd"
- - "xmlstarlet"
- - "tftpd-hpa"
- - "syslinux"
-
-
-groups:
- mos:
- - *mos
- - *mos_updates
- - *mos_security
- - *mos_holdback
-
- ubuntu:
- - *ubuntu
- - *ubuntu_updates
- - *ubuntu_security
-
-
-inheritance:
- ubuntu: mos
-
-osnames:
- mos: ubuntu
-
-requirements:
- ubuntu: *packages
diff --git a/contrib/fuel_mirror/doc/source/conf.py b/contrib/fuel_mirror/doc/source/conf.py
deleted file mode 100644
index 348029b..0000000
--- a/contrib/fuel_mirror/doc/source/conf.py
+++ /dev/null
@@ -1,79 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import os
-import sys
-
-sys.path.insert(0, os.path.abspath('../..'))
-# -- General configuration ----------------------------------------------------
-
-# Add any Sphinx extension module names here, as strings. They can be
-# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = [
- 'sphinx.ext.autodoc',
- #'sphinx.ext.intersphinx',
- 'oslosphinx'
-]
-
-# autodoc generation is a bit aggressive and a nuisance when doing heavy
-# text edit cycles.
-# execute "export SPHINX_DEBUG=1" in your terminal to disable
-
-# The suffix of source filenames.
-source_suffix = '.rst'
-
-# The master toctree document.
-master_doc = 'index'
-
-# General information about the project.
-project = u'fuel_mirror'
-copyright = u'2015, Mirantis, Inc'
-
-# If true, '()' will be appended to :func: etc. cross-reference text.
-add_function_parentheses = True
-
-# If true, the current module name will be prepended to all description
-# unit titles (such as .. function::).
-add_module_names = True
-
-# The name of the Pygments (syntax highlighting) style to use.
-pygments_style = 'sphinx'
-
-# -- Options for HTML output --------------------------------------------------
-
-# The theme to use for HTML and HTML Help pages. Major themes that come with
-# Sphinx are currently 'default' and 'sphinxdoc'.
-# html_theme_path = ["."]
-# html_theme = '_theme'
-# html_static_path = ['static']
-
-# Output file base name for HTML help builder.
-htmlhelp_basename = '%sdoc' % project
-
-# Grouping the document tree into LaTeX files. List of tuples
-# (source start file, target name, title, author, documentclass
-# [howto/manual]).
-latex_documents = [
- ('index',
- '%s.tex' % project,
- u'%s Documentation' % project,
- u'OpenStack Foundation', 'manual'),
-]
-
-# Example configuration for intersphinx: refer to the Python standard library.
-#intersphinx_mapping = {'http://docs.python.org/': None}
diff --git a/contrib/fuel_mirror/doc/source/contributing.rst b/contrib/fuel_mirror/doc/source/contributing.rst
deleted file mode 100644
index 36570d8..0000000
--- a/contrib/fuel_mirror/doc/source/contributing.rst
+++ /dev/null
@@ -1,4 +0,0 @@
-============
-Contributing
-============
-.. include:: ../../../../CONTRIBUTING.rst
diff --git a/contrib/fuel_mirror/doc/source/index.rst b/contrib/fuel_mirror/doc/source/index.rst
deleted file mode 100644
index 2be2378..0000000
--- a/contrib/fuel_mirror/doc/source/index.rst
+++ /dev/null
@@ -1,25 +0,0 @@
-.. fuel_mirror documentation master file, created by
- sphinx-quickstart on Tue Jul 9 22:26:36 2013.
- You can adapt this file completely to your liking, but it should at least
- contain the root `toctree` directive.
-
-Welcome to fuel_mirror's documentation!
-========================================================
-
-Contents:
-
-.. toctree::
- :maxdepth: 2
-
- readme
- installation
- usage
- contributing
-
-Indices and tables
-==================
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
-
diff --git a/contrib/fuel_mirror/doc/source/installation.rst b/contrib/fuel_mirror/doc/source/installation.rst
deleted file mode 100644
index be36bdf..0000000
--- a/contrib/fuel_mirror/doc/source/installation.rst
+++ /dev/null
@@ -1,12 +0,0 @@
-============
-Installation
-============
-
-At the command line::
-
- $ pip install fuel_mirror
-
-Or, if you have virtualenvwrapper installed::
-
- $ mkvirtualenv fuel_mirror
- $ pip install fuel_mirror
diff --git a/contrib/fuel_mirror/doc/source/readme.rst b/contrib/fuel_mirror/doc/source/readme.rst
deleted file mode 100644
index a6210d3..0000000
--- a/contrib/fuel_mirror/doc/source/readme.rst
+++ /dev/null
@@ -1 +0,0 @@
-.. include:: ../../README.rst
diff --git a/contrib/fuel_mirror/doc/source/usage.rst b/contrib/fuel_mirror/doc/source/usage.rst
deleted file mode 100644
index a1879eb..0000000
--- a/contrib/fuel_mirror/doc/source/usage.rst
+++ /dev/null
@@ -1,7 +0,0 @@
-========
-Usage
-========
-
-To use fuel_mirror in a project::
-
- import fuel_createmirror
diff --git a/contrib/fuel_mirror/etc/config.yaml b/contrib/fuel_mirror/etc/config.yaml
deleted file mode 100644
index e6e9576..0000000
--- a/contrib/fuel_mirror/etc/config.yaml
+++ /dev/null
@@ -1,11 +0,0 @@
-threads_num: 10
-ignore_errors_num: 2
-retries_num: 3
-target_dir: "/var/www/nailgun/mirrors"
-pattern_dir: "/usr/share/fuel-mirror"
-base_url: "http://{FUEL_SERVER_IP}:8080/mirrors/"
-
-# uncomment if need
-# http_proxy: null
-# https_proxy: null
-# fuel_server: 10.20.0.2
diff --git a/contrib/fuel_mirror/fuel_mirror/__init__.py b/contrib/fuel_mirror/fuel_mirror/__init__.py
deleted file mode 100644
index 566a696..0000000
--- a/contrib/fuel_mirror/fuel_mirror/__init__.py
+++ /dev/null
@@ -1,26 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import pbr.version
-
-try:
- __version__ = pbr.version.VersionInfo(
- 'fuel_mirror').version_string()
-except Exception as e:
- print("ERROR", e)
- __version__ = "0.0.0-test"
diff --git a/contrib/fuel_mirror/fuel_mirror/app.py b/contrib/fuel_mirror/fuel_mirror/app.py
deleted file mode 100644
index fca7be0..0000000
--- a/contrib/fuel_mirror/fuel_mirror/app.py
+++ /dev/null
@@ -1,157 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from cliff import app
-from cliff.commandmanager import CommandManager
-import yaml
-
-
-import fuel_mirror
-from fuel_mirror.common import accessors
-from fuel_mirror.common import utils
-
-
-class Application(app.App):
- """Main cliff application class.
-
- Performs initialization of the command manager and
- configuration of basic engines.
- """
-
- config = None
- fuel = None
- repo_manager_accessor = None
- sources = None
- versions = None
-
- def build_option_parser(self, description, version, argparse_kwargs=None):
- """Specifies common cmdline arguments."""
- p_inst = super(Application, self)
- parser = p_inst.build_option_parser(description=description,
- version=version,
- argparse_kwargs=argparse_kwargs)
-
- parser.add_argument(
- "--config",
- default="/etc/fuel-mirror/config.yaml",
- metavar="PATH",
- help="Path to config file."
- )
- parser.add_argument(
- "-S", "--fuel-server",
- metavar="FUEL-SERVER",
- help="The public address of Fuel Master."
- )
- parser.add_argument(
- "--fuel-user",
- help="Fuel Master admin login."
- " Alternatively, use env var KEYSTONE_USER)."
- )
- parser.add_argument(
- "--fuel-password",
- help="Fuel Master admin password."
- " Alternatively, use env var KEYSTONE_PASSWORD)."
- )
- return parser
-
- def initialize_app(self, argv):
- """Initialises common options."""
- with open(self.options.config, "r") as stream:
- self.config = yaml.load(stream)
-
- self._initialize_fuel_accessor()
- self._initialize_repo_manager()
-
- def _initialize_repo_manager(self):
- self.repo_manager_accessor = accessors.get_packetary_accessor(
- threads_num=int(self.config.get('threads_num', 0)),
- retries_num=int(self.config.get('retries_num', 0)),
- ignore_errors_num=int(self.config.get('ignore_errors_num', 0)),
- http_proxy=self.config.get('http_proxy'),
- https_proxy=self.config.get('https_proxy'),
- )
-
- def _initialize_fuel_accessor(self):
- fuel_default = utils.get_fuel_settings()
-
- fuel_server = utils.first(
- self.options.fuel_server,
- self.config.get("fuel_server"),
- fuel_default.get("server")
- )
- fuel_user = utils.first(
- self.options.fuel_user,
- fuel_default.get("user")
- )
- fuel_password = utils.first(
- self.options.fuel_password,
- fuel_default.get("password")
- )
-
- if not fuel_server:
- for option in ("mos_version", "openstack_version"):
- if not self.config.setdefault(option, ''):
- self.LOG.warning(
- "The option '{0}' is not defined."
- "Please specify the option 'fuel-server' or '{0}'."
- .format(option)
- )
- return
-
- self.config["base_url"] = self.config["base_url"].format(
- FUEL_SERVER_IP=fuel_server.split(":", 1)[0]
- )
-
- self.fuel = accessors.get_fuel_api_accessor(
- fuel_server,
- fuel_user,
- fuel_password
- )
- fuel_ver = self.fuel.FuelVersion.get_all_data()
- self.config.setdefault(
- 'mos_version', fuel_ver['release']
- )
- self.config.setdefault(
- 'openstack_version', fuel_ver['openstack_version']
- )
-
-
-def main(argv=None):
- """Entry point."""
- return Application(
- description="The utility to create local mirrors.",
- version=fuel_mirror.__version__,
- command_manager=CommandManager("fuel_mirror", convert_underscores=True)
- ).run(argv)
-
-
-def debug(name, cmd_class, argv=None):
- """Helps to debug command."""
- import sys
-
- if argv is None:
- argv = sys.argv[1:]
-
- argv = [name] + argv + ["-v", "-v", "--debug"]
- cmd_mgr = CommandManager("test_fuel_mirror", convert_underscores=True)
- cmd_mgr.add_command(name, cmd_class)
- return Application(
- description="The fuel mirror utility test.",
- version="0.0.1",
- command_manager=cmd_mgr
- ).run(argv)
diff --git a/contrib/fuel_mirror/fuel_mirror/commands/__init__.py b/contrib/fuel_mirror/fuel_mirror/commands/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/contrib/fuel_mirror/fuel_mirror/commands/apply.py b/contrib/fuel_mirror/fuel_mirror/commands/apply.py
deleted file mode 100644
index 634e6b4..0000000
--- a/contrib/fuel_mirror/fuel_mirror/commands/apply.py
+++ /dev/null
@@ -1,203 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import six
-
-from packetary.library.utils import localize_repo_url
-
-from fuel_mirror.commands.base import BaseCommand
-from fuel_mirror.common.utils import is_subdict
-from fuel_mirror.common.utils import lists_merge
-
-
-class ApplyCommand(BaseCommand):
- """Applies local mirrors for Fuel-environments."""
-
- def get_parser(self, prog_name):
- parser = super(ApplyCommand, self).get_parser(prog_name)
- parser.add_argument(
- "--default",
- dest="set_default",
- action="store_true",
- default=False,
- help="Set as default repository."
- )
- parser.add_argument(
- "--replace",
- dest="replace",
- action="store_true",
- default=False,
- help="Replace default repos with generated mirrors."
- )
- parser.add_argument(
- "-e", "--env",
- dest="env", nargs="+",
- help="Fuel environment ID to update, "
- "by default applies for all environments."
- )
-
- return parser
-
- def take_action(self, parsed_args):
- if self.app.fuel is None:
- raise ValueError("Please specify the fuel-server option.")
-
- data = self.load_data(parsed_args)
- base_url = self.app.config["base_url"]
- release_match = data["fuel_release_match"]
- replace_repos = parsed_args.replace
-
- localized_repos = []
- for _, repos in self.get_groups(parsed_args, data):
- for repo_data in repos:
- new_data = repo_data.copy()
- new_data['uri'] = localize_repo_url(
- base_url, repo_data['uri']
- )
- localized_repos.append(new_data)
-
- localized_repos.sort(key=lambda x: not x.pop('main', False))
-
- self.update_clusters(
- parsed_args.env,
- localized_repos,
- release_match,
- replace_repos=replace_repos)
-
- if parsed_args.set_default:
- self.update_release_repos(
- localized_repos,
- release_match,
- replace_repos=replace_repos)
-
- self.app.stdout.write(
- "Operations have been completed successfully.\n"
- )
-
- def update_clusters(self,
- ids,
- repositories,
- release_match,
- replace_repos=False):
- """Applies repositories for existing clusters.
-
- :param ids: the cluster ids.
- :param repositories: the meta information of repositories
- :param release_match: The pattern to check Fuel Release
- """
- self.app.stdout.write("Updating the Cluster repositories...\n")
-
- if ids:
- clusters = self.app.fuel.Environment.get_by_ids(ids)
- else:
- clusters = self.app.fuel.Environment.get_all()
-
- for cluster in clusters:
- releases = six.moves.filter(
- lambda x: is_subdict(release_match, x.data),
- self.app.fuel.Release.get_by_ids([cluster.data["release_id"]])
- )
- if next(releases, None) is None:
- continue
-
- modified = self._update_repository_settings(
- cluster.get_settings_data(),
- repositories,
- replace_repos=replace_repos)
-
- if modified:
- self.app.LOG.info(
- "Try to update the Cluster '%s'",
- cluster.data['name']
- )
- self.app.LOG.debug(
- "The modified cluster attributes: %s",
- modified
- )
- cluster.set_settings_data(modified)
-
- def update_release_repos(self,
- repositories,
- release_match,
- replace_repos=False):
- """Applies repositories for existing default settings.
-
- :param repositories: the meta information of repositories
- :param release_match: The pattern to check Fuel Release
- """
- self.app.stdout.write("Updating the release repositories...\n")
- releases = six.moves.filter(
- lambda x: is_subdict(release_match, x.data),
- self.app.fuel.Release.get_all()
- )
- for release in releases:
- modified = self._update_repository_settings(
- release.data["attributes_metadata"],
- repositories,
- replace_repos=replace_repos)
- if modified:
- release.data["attributes_metadata"] = modified
- self.app.LOG.info(
- "Try to update the Release '%s'",
- release.data['name']
- )
- self.app.LOG.debug(
- "The modified release attributes: %s",
- release.data
- )
- # TODO(need to add method for release object)
- release.connection.put_request(
- release.instance_api_path.format(release.id),
- release.data
- )
-
- def _update_repository_settings(self,
- settings,
- repositories,
- replace_repos=False):
- """Updates repository settings.
-
- :param settings: the target settings
- :param repositories: the meta of repositories
- """
- editable = settings["editable"]
- if 'repo_setup' not in editable:
- self.app.LOG.info('Attributes are read-only.')
- return
-
- repos_attr = editable["repo_setup"]["repos"]
- if replace_repos:
- repos_attr['value'] = repositories
- else:
- lists_merge(repos_attr['value'], repositories, "name")
-
- # NOTE(akostrikov) That assignment is only for informational purpose.
- settings["editable"]["repo_setup"]["repos"] = repos_attr
-
- return settings
-
-
-def debug(argv=None):
- """Helper for debugging Apply command."""
- from fuel_mirror.app import debug
-
- return debug("apply", ApplyCommand, argv)
-
-
-if __name__ == "__main__":
- debug()
diff --git a/contrib/fuel_mirror/fuel_mirror/commands/base.py b/contrib/fuel_mirror/fuel_mirror/commands/base.py
deleted file mode 100644
index 613d177..0000000
--- a/contrib/fuel_mirror/fuel_mirror/commands/base.py
+++ /dev/null
@@ -1,125 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import os.path
-
-from cliff import command
-from jsonschema import validate
-from jsonschema import ValidationError
-import six
-
-from fuel_mirror.common.utils import load_input_data
-from fuel_mirror.schemas.input_data_schema import SCHEMA
-
-
-class BaseCommand(command.Command):
-
- """The Base command for fuel-mirror."""
- REPO_ARCH = "x86_64"
-
- @property
- def stdout(self):
- """Shortcut for self.app.stdout."""
- return self.app.stdout
-
- def get_parser(self, prog_name):
- """Specifies common options."""
- parser = super(BaseCommand, self).get_parser(prog_name)
-
- input_group = parser.add_mutually_exclusive_group(required=True)
- input_group.add_argument(
- '-I', '--input-file',
- metavar='PATH',
- help='The path to file with input data.')
-
- input_group.add_argument(
- '-P', '--pattern',
- metavar='NAME',
- help='The builtin input file name.'
- )
-
- parser.add_argument(
- "-G", "--group",
- dest="groups",
- required=True,
- nargs='+',
- help="The name of repository groups."
- )
- return parser
-
- def resolve_input_pattern(self, pattern):
- """Gets the full path to input file by pattern.
-
- :param pattern: the config file name without ext
- :return: the full path
- """
- return os.path.join(
- self.app.config['pattern_dir'], pattern + ".yaml"
- )
-
- @staticmethod
- def validate_data(data, schema):
- """Validate the input data using jsonschema validation.
-
- :param data: a data to validate represented as a dict
- :param schema: a schema to validate represented as a dict;
- must be in JSON Schema Draft 4 format.
- """
- try:
- validate(data, schema)
- except ValidationError as ex:
- if len(ex.path) > 0:
- join_ex_path = '.'.join(six.text_type(x) for x in ex.path)
- detail = ("Invalid input for field/attribute {0}."
- " Value: {1}. {2}").format(join_ex_path,
- ex.instance, ex.message)
- else:
- detail = ex.message
- raise ValidationError(detail)
-
- def load_data(self, parsed_args):
- """Load the input data.
-
- :param parsed_args: the command-line arguments
- :return: the input data
- """
- if parsed_args.pattern:
- input_file = self.resolve_input_pattern(parsed_args.pattern)
- else:
- input_file = parsed_args.input_file
-
- data = load_input_data(
- input_file,
- mos_version=self.app.config["mos_version"],
- openstack_version=self.app.config["openstack_version"]
- )
- self.validate_data(data, SCHEMA)
- return data
-
- @classmethod
- def get_groups(cls, parsed_args, data):
- """Gets repository groups from input data.
-
- :param parsed_args: the command-line arguments
- :param data: the input data
- :return: the sequence of pairs (group_name, repositories)
- """
- all_groups = data['groups']
- return (
- (x, all_groups[x]) for x in parsed_args.groups if x in all_groups
- )
diff --git a/contrib/fuel_mirror/fuel_mirror/commands/create.py b/contrib/fuel_mirror/fuel_mirror/commands/create.py
deleted file mode 100644
index 484d601..0000000
--- a/contrib/fuel_mirror/fuel_mirror/commands/create.py
+++ /dev/null
@@ -1,79 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from fuel_mirror.commands.base import BaseCommand
-from fuel_mirror.common.url_builder import get_url_builder
-
-
-class CreateCommand(BaseCommand):
- """Creates a new local mirrors."""
-
- def take_action(self, parsed_args):
- """See the Command.take_action."""
- data = self.load_data(parsed_args)
- repos_reqs = data.get('requirements', {})
- inheritance = data.get('inheritance', {})
- target_dir = self.app.config["target_dir"]
-
- total_stats = None
- for group_name, repos in self.get_groups(parsed_args, data):
- url_builder = get_url_builder(repos[0]["type"])
- repo_manager = self.app.repo_manager_accessor(
- repos[0]["type"], self.REPO_ARCH
- )
- if group_name in inheritance:
- child_group = inheritance[group_name]
- dependencies = [
- url_builder.get_repo_url(x)
- for x in data['groups'][child_group]
- ]
- else:
- dependencies = None
-
- stat = repo_manager.clone_repositories(
- [url_builder.get_repo_url(x) for x in repos],
- target_dir,
- dependencies,
- repos_reqs.get(group_name)
- )
-
- if total_stats is None:
- total_stats = stat
- else:
- total_stats += stat
-
- if total_stats is not None:
- self.stdout.write(
- "Packages processed: {0.copied}/{0.total}\n"
- .format(total_stats)
- )
- else:
- self.stdout.write(
- "No packages.\n"
- )
-
-
-def debug(argv=None):
- """Helper for debugging Create command."""
- from fuel_mirror.app import debug
-
- return debug("create", CreateCommand, argv)
-
-
-if __name__ == "__main__":
- debug()
diff --git a/contrib/fuel_mirror/fuel_mirror/common/__init__.py b/contrib/fuel_mirror/fuel_mirror/common/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/contrib/fuel_mirror/fuel_mirror/common/accessors.py b/contrib/fuel_mirror/fuel_mirror/common/accessors.py
deleted file mode 100644
index 35425f6..0000000
--- a/contrib/fuel_mirror/fuel_mirror/common/accessors.py
+++ /dev/null
@@ -1,64 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import functools
-import os
-
-
-def get_packetary_accessor(**kwargs):
- """Gets the configured repository manager.
-
- :param kwargs: The packetary configuration parameters.
- """
-
- import packetary
-
- return functools.partial(
- packetary.RepositoryApi.create,
- packetary.Context(packetary.Configuration(**kwargs))
- )
-
-
-def get_fuel_api_accessor(address=None, user=None, password=None):
- """Gets the fuel client api accessor.
-
- :param address: The address of Fuel Master node.
- :param user: The username to access to the Fuel Master node.
- :param user: The password to access to the Fuel Master node.
- """
- if address:
- host_and_port = address.split(":")
- os.environ["SERVER_ADDRESS"] = host_and_port[0]
- if len(host_and_port) > 1:
- os.environ["LISTEN_PORT"] = host_and_port[1]
-
- if user is not None:
- os.environ["KEYSTONE_USER"] = user
- if password is not None:
- os.environ["KEYSTONE_PASS"] = password
-
- # import fuelclient.ClientAPI after configuring
- # environment variables
- try:
- from fuelclient import objects
- except ImportError:
- raise RuntimeError(
- "fuelclient module seems not installed. "
- "This action requires it to be available."
- )
- return objects
diff --git a/contrib/fuel_mirror/fuel_mirror/common/url_builder.py b/contrib/fuel_mirror/fuel_mirror/common/url_builder.py
deleted file mode 100644
index be13a8d..0000000
--- a/contrib/fuel_mirror/fuel_mirror/common/url_builder.py
+++ /dev/null
@@ -1,59 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-
-def get_url_builder(repotype):
- """Gets the instance of RepoUrlBuilder.
-
- :param repotype: the type of repository: rpm|deb
- :return: the RepoBuilder implementation
- """
- return {
- "deb": AptRepoUrlBuilder,
- "rpm": YumRepoUrlBuilder
- }[repotype]
-
-
-class RepoUrlBuilder(object):
- REPO_FOLDER = "mirror"
-
- @classmethod
- def get_repo_url(cls, repo_data):
- """Gets the url with replaced variable holders.
-
- :param repo_data: the repositories`s meta data
- :return: the full repository`s url
- """
-
-
-class AptRepoUrlBuilder(RepoUrlBuilder):
- """URL builder for apt-repository(es)."""
-
- @classmethod
- def get_repo_url(cls, repo_data):
- return " ".join(
- repo_data[x] for x in ("uri", "suite", "section")
- )
-
-
-class YumRepoUrlBuilder(RepoUrlBuilder):
- """URL builder for Yum repository(es)."""
-
- @classmethod
- def get_repo_url(cls, repo_data):
- return repo_data["uri"]
diff --git a/contrib/fuel_mirror/fuel_mirror/common/utils.py b/contrib/fuel_mirror/fuel_mirror/common/utils.py
deleted file mode 100644
index 855ef00..0000000
--- a/contrib/fuel_mirror/fuel_mirror/common/utils.py
+++ /dev/null
@@ -1,106 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-from string import Template
-
-import six
-import yaml
-
-
-def lists_merge(main, patch, key):
- """Merges the list of dicts with same keys.
-
- >>> lists_merge([{"a": 1, "c": 2}], [{"a": 1, "c": 3}], key="a")
- [{'a': 1, 'c': 3}]
-
- :param main: the main list
- :type main: list
- :param patch: the list of additional elements
- :type patch: list
- :param key: the key for compare
- """
- main_idx = dict(
- (x[key], i) for i, x in enumerate(main)
- )
-
- patch_idx = dict(
- (x[key], i) for i, x in enumerate(patch)
- )
-
- for k in sorted(patch_idx):
- if k in main_idx:
- main[main_idx[k]].update(patch[patch_idx[k]])
- else:
- main.append(patch[patch_idx[k]])
- return main
-
-
-def is_subdict(dict1, dict2):
- """Checks that dict1 is subdict of dict2.
-
- >>> is_subdict({"a": 1}, {'a': 1, 'b': 1})
- True
-
- :param dict1: the candidate
- :param dict2: the super dict
- :return: True if all keys from dict1 are present
- and has same value in dict2 otherwise False
- """
- for k, v in six.iteritems(dict1):
- if k not in dict2 or dict2[k] != v:
- return False
- return True
-
-
-def first(*args):
- """Get first not empty value.
-
- >>> first(0, 1) == next(iter(filter(None, [0, 1])))
- True
-
- :param args: the list of arguments
- :return first value that bool(v) is True, None if not found.
- """
- for arg in args:
- if arg:
- return arg
-
-
-def get_fuel_settings():
- """Gets the fuel settings from astute container, if it is available."""
-
- try:
- with open("/etc/fuel/astute.yaml", "r") as fd:
- settings = yaml.load(fd)
- return {
- "server": settings.get("ADMIN_NETWORK", {}).get("ipaddress"),
- "user": settings.get("FUEL_ACCESS", {}).get("user"),
- "password": settings.get("FUEL_ACCESS", {}).get("password")
- }
- except (OSError, IOError):
- return {}
-
-
-def load_input_data(input_file, **kwargs):
- """Load yaml file and parse it to dict with replacement by kwargs.
-
- :param input_file: name of file to parse fuel mirror template
- :param kwargs: arguments to substitute template
- :return: processed from yaml file dict.
- """
- with open(input_file, "r") as fd:
- return yaml.load(Template(fd.read()).safe_substitute(**kwargs))
diff --git a/contrib/fuel_mirror/fuel_mirror/schemas/__init__.py b/contrib/fuel_mirror/fuel_mirror/schemas/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/contrib/fuel_mirror/fuel_mirror/schemas/input_data_schema.py b/contrib/fuel_mirror/fuel_mirror/schemas/input_data_schema.py
deleted file mode 100644
index 0cf52a1..0000000
--- a/contrib/fuel_mirror/fuel_mirror/schemas/input_data_schema.py
+++ /dev/null
@@ -1,120 +0,0 @@
-SCHEMA = {
- "$schema": "http://json-schema.org/draft-04/schema#",
- "definitions": {
- "DEB_REPO_SCHEMA": {
- "type": "object",
- "required": [
- "name",
- "uri",
- "suite",
- "section"
- ],
- "properties": {
- "name": {
- "type": "string"
- },
- "type": {
- "type": "string",
- "enum": ["deb"]
- },
- "uri": {
- "type": "string"
- },
- "priority": {
- "anyOf": [
- {
- "type": "integer"
- },
- {
- "type": "null"
- }
- ]
- },
- "suite": {
- "type": "string"
- },
- "section": {
- "type": "string"
- },
- }
- },
- "RPM_REPO_SCHEMA": {
- "type": "object",
- "required": [
- "name",
- "uri",
- ],
- "properties": {
- "name": {
- "type": "string"
- },
- "type": {
- "type": "string",
- "enum": ["rpm"]
- },
- "uri": {
- "type": "string"
- },
- "priority": {
- "anyOf": [
- {
- "type": "integer"
- },
- {
- "type": "null"
- }
- ]
- },
- }
- },
- "REPO_SCHEMA": {
- "anyOf":
- [
- {"$ref": "#/definitions/DEB_REPO_SCHEMA"},
- {"$ref": "#/definitions/RPM_REPO_SCHEMA"}
- ]
- },
-
- "REPOS_SCHEMA": {
- "type": "array", "items": {"$ref": "#/definitions/REPO_SCHEMA"}
- }
- },
- "type": "object",
- "required": [
- "groups",
- ],
- "properties": {
- "fuel_release_match": {
- "type": "object",
- "properties": {
- "operating_system": {
- "type": "string"
- }
- },
- "required": [
- "operating_system"
- ]
- },
- "requirements": {
- "type": "object",
- "patternProperties": {
- "^[0-9a-z_-]+$": {"type": "array"}
- },
- "additionalProperties": False,
- },
- "groups": {
- "type": "object",
- "patternProperties": {
- "^[0-9a-z_-]+$": {"$ref": "#/definitions/REPOS_SCHEMA"}
- },
- "additionalProperties": False,
- },
- "inheritance": {
- "type": "object",
- "patternProperties": {
- "^[0-9a-z_-]+$": {"type": "string"}
- },
- "additionalProperties": False,
- }
- }
-}
diff --git a/contrib/fuel_mirror/fuel_mirror/tests/__init__.py b/contrib/fuel_mirror/fuel_mirror/tests/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/contrib/fuel_mirror/fuel_mirror/tests/base.py b/contrib/fuel_mirror/fuel_mirror/tests/base.py
deleted file mode 100644
index 3e5d0c9..0000000
--- a/contrib/fuel_mirror/fuel_mirror/tests/base.py
+++ /dev/null
@@ -1,33 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-try:
- import unittest2 as unittest
-except ImportError:
- import unittest
-
-
-class TestCase(unittest.TestCase):
-
- """Test case base class for all unit tests."""
-
- def assertNotRaises(self, exception, method, *args, **kwargs):
- try:
- method(*args, **kwargs)
- except exception as e:
- self.fail("Unexpected error: {0}".format(e))
diff --git a/contrib/fuel_mirror/fuel_mirror/tests/data/test_centos.yaml b/contrib/fuel_mirror/fuel_mirror/tests/data/test_centos.yaml
deleted file mode 100644
index 11f69bf..0000000
--- a/contrib/fuel_mirror/fuel_mirror/tests/data/test_centos.yaml
+++ /dev/null
@@ -1,22 +0,0 @@
-fuel_release_match:
- operating_system: CentOS
-
-inheritance:
- centos: mos
-
-groups:
- mos:
- - name: "mos"
- type: "rpm"
- uri: "http://localhost/mos$mos_version/x86_64"
- priority: 10
-
- centos:
- - name: "centos"
- type: "rpm"
- uri: "http://localhost/centos/os/x86_64"
- priority: 5
-
-requirements:
- centos:
- - "package_rpm"
diff --git a/contrib/fuel_mirror/fuel_mirror/tests/data/test_config.yaml b/contrib/fuel_mirror/fuel_mirror/tests/data/test_config.yaml
deleted file mode 100644
index 3ba5cd1..0000000
--- a/contrib/fuel_mirror/fuel_mirror/tests/data/test_config.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-threads_num: 1
-ignore_errors_num: 2
-retries_num: 3
-http_proxy: "http://localhost"
-https_proxy: "https://localhost"
-target_dir: "/var/www/"
-base_url: "http://{FUEL_SERVER_IP}:8080/"
diff --git a/contrib/fuel_mirror/fuel_mirror/tests/data/test_invalid_ubuntu.yaml b/contrib/fuel_mirror/fuel_mirror/tests/data/test_invalid_ubuntu.yaml
deleted file mode 100644
index a5071f4..0000000
--- a/contrib/fuel_mirror/fuel_mirror/tests/data/test_invalid_ubuntu.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-fuel_release_match:
- operating_system: Ubuntu
-
-inheritance:
- ubuntu: mos
-
-requirements:
- ubuntu:
- - "package_deb"
diff --git a/contrib/fuel_mirror/fuel_mirror/tests/data/test_ubuntu.yaml b/contrib/fuel_mirror/fuel_mirror/tests/data/test_ubuntu.yaml
deleted file mode 100644
index 9554ef0..0000000
--- a/contrib/fuel_mirror/fuel_mirror/tests/data/test_ubuntu.yaml
+++ /dev/null
@@ -1,27 +0,0 @@
-fuel_release_match:
- operating_system: Ubuntu
-
-inheritance:
- ubuntu: mos
-
-groups:
- mos:
- - name: "mos"
- type: "deb"
- uri: "http://localhost/mos"
- suite: "mos$mos_version"
- section: "main restricted"
- priority: 1000
-
- ubuntu:
- - name: "ubuntu"
- type: "deb"
- main: true
- uri: "http://localhost/ubuntu"
- suite: "trusty"
- section: "main multiverse restricted universe"
- priority: 500
-
-requirements:
- ubuntu:
- - "package_deb"
diff --git a/contrib/fuel_mirror/fuel_mirror/tests/test_accessors.py b/contrib/fuel_mirror/fuel_mirror/tests/test_accessors.py
deleted file mode 100644
index cbc4ea9..0000000
--- a/contrib/fuel_mirror/fuel_mirror/tests/test_accessors.py
+++ /dev/null
@@ -1,92 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import mock
-
-from fuel_mirror.common import accessors
-from fuel_mirror.tests import base
-
-
-class TestAccessors(base.TestCase):
- def test_get_packetary_accessor(self):
- packetary = mock.MagicMock()
- with mock.patch.dict("sys.modules", packetary=packetary):
- accessor = accessors.get_packetary_accessor(
- http_proxy="http://localhost",
- https_proxy="https://localhost",
- retries_num=1,
- threads_num=2,
- ignore_errors_num=3
- )
- accessor("deb")
- accessor("yum")
- packetary.Configuration.assert_called_once_with(
- http_proxy="http://localhost",
- https_proxy="https://localhost",
- retries_num=1,
- threads_num=2,
- ignore_errors_num=3
- )
- packetary.Context.assert_called_once_with(
- packetary.Configuration()
- )
- self.assertEqual(2, packetary.RepositoryApi.create.call_count)
- packetary.RepositoryApi.create.assert_any_call(
- packetary.Context(), "deb"
- )
- packetary.RepositoryApi.create.assert_any_call(
- packetary.Context(), "yum"
- )
-
- @mock.patch("fuel_mirror.common.accessors.os")
- def test_get_fuel_api_accessor(self, os):
- fuelclient = mock.MagicMock()
- patch = {
- "fuelclient": fuelclient,
- "fuelclient.objects": fuelclient.objects
- }
- with mock.patch.dict("sys.modules", patch):
- accessor = accessors.get_fuel_api_accessor(
- "localhost:8080", "guest", "123"
- )
- accessor.Environment.get_all()
-
- os.environ.__setitem__.asseert_any_call(
- "SERVER_ADDRESS", "localhost"
- )
- os.environ.__setitem__.asseert_any_call(
- "LISTEN_PORT", "8080"
- )
- os.environ.__setitem__.asseert_any_call(
- "KEYSTONE_USER", "guest"
- )
- os.environ.__setitem__.asseert_any_call(
- "KEYSTONE_PASS", "123"
- )
- fuelclient.objects.Environment.get_all.assert_called_once_with()
-
- @mock.patch("fuel_mirror.common.accessors.os")
- def test_get_fuel_api_accessor_with_default_parameters(self, os):
- fuelclient = mock.MagicMock()
- patch = {
- "fuelclient": fuelclient,
- "fuelclient.objects": fuelclient.objects
- }
- with mock.patch.dict("sys.modules", patch):
- accessors.get_fuel_api_accessor()
- os.environ.__setitem__.assert_not_called()
diff --git a/contrib/fuel_mirror/fuel_mirror/tests/test_cli_commands.py b/contrib/fuel_mirror/fuel_mirror/tests/test_cli_commands.py
deleted file mode 100644
index ab2bd0b..0000000
--- a/contrib/fuel_mirror/fuel_mirror/tests/test_cli_commands.py
+++ /dev/null
@@ -1,499 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import mock
-import os.path
-import subprocess
-
-from jsonschema import ValidationError
-
-# The cmd2 does not work with python3.5
-# because it tries to get access to the property mswindows,
-# that was removed in 3.5
-subprocess.mswindows = False
-
-from fuel_mirror.commands import apply
-from fuel_mirror.commands import create
-from fuel_mirror.common.utils import load_input_data
-from fuel_mirror.tests import base
-
-
-CONFIG_PATH = os.path.join(
- os.path.dirname(__file__), "data", "test_config.yaml"
-)
-
-UBUNTU_PATH = os.path.join(
- os.path.dirname(__file__), "data", "test_ubuntu.yaml"
-)
-
-CENTOS_PATH = os.path.join(
- os.path.dirname(__file__), "data", "test_centos.yaml"
-)
-
-INVALID_DATA_PATH = os.path.join(
- os.path.dirname(__file__), "data", "test_invalid_ubuntu.yaml"
-)
-
-
-# TODO(akostrikov) lists_merge we are using is not stable so we have to use
-# different local repos in cases with merge and without it.
-# We pass sorted by priority list, but in lists_merge we sort it by key.
-# As we are aiming to use existing repos as primary source - it is not issue.
-def local_repos(mirror_host='10.25.0.10:8080', name_postfix='', reverse=True):
- mirror_lists = load_input_data(UBUNTU_PATH, mos_version=1)
- sorted_repos = reduce(lambda x, y: x + y, mirror_lists['groups'].values())
- sorted_repos.sort(key=lambda x: x['priority'], reverse=reverse)
- for repo in sorted_repos:
- repo.pop('main', None)
- repo['name'] = repo['name'] + name_postfix
- repo['uri'] = repo['uri'].replace('localhost', mirror_host)
- return sorted_repos
-
-
-def mirror_repos():
- return local_repos(mirror_host='mirror.com:8080', name_postfix='-mirror')
-
-
-@mock.patch.multiple(
- "fuel_mirror.app",
- accessors=mock.DEFAULT
-)
-class TestCliCommands(base.TestCase):
- common_argv = [
- "--config", CONFIG_PATH,
- "--fuel-server=10.25.0.10",
- "--fuel-user=test",
- "--fuel-password=test1"
- ]
-
- def start_cmd(self, cmd, argv, data_file):
- cmd.debug(
- argv + self.common_argv + ["--input-file", data_file]
- )
-
- def _setup_fuel_versions(self, fuel_mock):
- fuel_mock.FuelVersion.get_all_data.return_value = {
- "release": "1",
- "openstack_version": "2"
- }
-
- def _create_fuel_release(self, fuel_mock, osname, repos=None):
- if repos is None:
- repos = []
- release = mock.MagicMock(data={
- "name": "test release",
- "operating_system": osname,
- "attributes_metadata": {
- "editable": {"repo_setup": {"repos": {"value": repos}}}
- }
- })
-
- fuel_mock.Release.get_by_ids.return_value = [release]
- fuel_mock.Release.get_all.return_value = [release]
- return release
-
- def _create_fuel_env(self, fuel_mock, repos=None):
- if repos is None:
- repos = []
- env = mock.MagicMock(data={
- "name": "test",
- "release_id": 1
- })
- env.get_settings_data.return_value = {
- "editable": {"repo_setup": {"repos": {"value": repos}}}
- }
- fuel_mock.Environment.get_by_ids.return_value = [env]
- fuel_mock.Environment.get_all.return_value = [env]
- return env
-
- def test_create_mos_ubuntu(self, accessors):
- self._setup_fuel_versions(accessors.get_fuel_api_accessor())
- packetary = accessors.get_packetary_accessor()
-
- self.start_cmd(create, ["--group", "mos"], UBUNTU_PATH)
- accessors.get_packetary_accessor.assert_called_with(
- threads_num=1,
- ignore_errors_num=2,
- retries_num=3,
- http_proxy="http://localhost",
- https_proxy="https://localhost",
- )
- packetary.assert_called_with("deb", "x86_64")
- api = packetary()
- api.clone_repositories.assert_called_once_with(
- ['http://localhost/mos mos1 main restricted'],
- '/var/www/',
- None, None
- )
-
- def test_create_partial_ubuntu(self, accessors):
- self._setup_fuel_versions(accessors.get_fuel_api_accessor())
- packetary = accessors.get_packetary_accessor()
-
- self.start_cmd(create, ["--group", "ubuntu"], UBUNTU_PATH)
- accessors.get_packetary_accessor.assert_called_with(
- threads_num=1,
- ignore_errors_num=2,
- retries_num=3,
- http_proxy="http://localhost",
- https_proxy="https://localhost",
- )
- packetary.assert_called_with("deb", "x86_64")
- api = packetary()
- api.clone_repositories.assert_called_once_with(
- ['http://localhost/ubuntu trusty '
- 'main multiverse restricted universe'],
- '/var/www/',
- ['http://localhost/mos mos1 main restricted'],
- ['package_deb']
- )
-
- def test_create_mos_centos(self, accessors):
- self._setup_fuel_versions(accessors.get_fuel_api_accessor())
- packetary = accessors.get_packetary_accessor()
-
- self.start_cmd(create, ["--group", "mos"], CENTOS_PATH)
- accessors.get_packetary_accessor.assert_called_with(
- threads_num=1,
- ignore_errors_num=2,
- retries_num=3,
- http_proxy="http://localhost",
- https_proxy="https://localhost",
- )
- packetary.assert_called_with("rpm", "x86_64")
- api = packetary()
- api.clone_repositories.assert_called_once_with(
- ['http://localhost/mos1/x86_64'],
- '/var/www/',
- None, None
- )
-
- def test_create_partial_centos(self, accessors):
- self._setup_fuel_versions(accessors.get_fuel_api_accessor())
- packetary = accessors.get_packetary_accessor()
-
- self.start_cmd(create, ["--group", "centos"], CENTOS_PATH)
- accessors.get_packetary_accessor.assert_called_with(
- threads_num=1,
- ignore_errors_num=2,
- retries_num=3,
- http_proxy="http://localhost",
- https_proxy="https://localhost",
- )
- packetary.assert_called_with("rpm", "x86_64")
- api = packetary()
- api.clone_repositories.assert_called_once_with(
- ['http://localhost/centos/os/x86_64'],
- '/var/www/',
- ['http://localhost/mos1/x86_64'],
- ["package_rpm"]
- )
-
- def test_apply_for_ubuntu_based_env(self, accessors):
- fuel = accessors.get_fuel_api_accessor()
- self._setup_fuel_versions(fuel)
- env = self._create_fuel_env(fuel)
- self._create_fuel_release(fuel, "Ubuntu")
- self.start_cmd(
- apply, ['--group', 'mos', 'ubuntu', '--env', '1'],
- UBUNTU_PATH
- )
- accessors.get_fuel_api_accessor.assert_called_with(
- "10.25.0.10", "test", "test1"
- )
- fuel.FuelVersion.get_all_data.assert_called_once_with()
- env.set_settings_data.assert_called_with(
- {
- 'editable': {
- 'repo_setup': {
- 'repos': {'value': local_repos()}
- }
- }
- }
- )
-
- def test_with_existing_mirrors(self, accessors):
- fuel = accessors.get_fuel_api_accessor()
- self._setup_fuel_versions(fuel)
- env = self._create_fuel_env(fuel, repos=mirror_repos())
- self._create_fuel_release(fuel, "Ubuntu", repos=mirror_repos())
- self.start_cmd(
- apply, ['--group', 'mos', 'ubuntu', '--env', '1'],
- UBUNTU_PATH
- )
- accessors.get_fuel_api_accessor.assert_called_with(
- "10.25.0.10", "test", "test1"
- )
- fuel.FuelVersion.get_all_data.assert_called_once_with()
- env.set_settings_data.assert_called_with(
- {
- 'editable': {
- 'repo_setup': {
- 'repos': {'value': mirror_repos() + local_repos()}
- }
- }
- }
- )
-
- def test_replace_existing_mirrors_with_local(self, accessors):
- fuel = accessors.get_fuel_api_accessor()
- self._setup_fuel_versions(fuel)
- env = self._create_fuel_env(fuel, repos=mirror_repos())
- self._create_fuel_release(fuel, "Ubuntu", repos=mirror_repos())
- self.start_cmd(
- apply, ['--group', 'mos', 'ubuntu', '--env', '1', '--replace'],
- UBUNTU_PATH
- )
- accessors.get_fuel_api_accessor.assert_called_with(
- "10.25.0.10", "test", "test1"
- )
- fuel.FuelVersion.get_all_data.assert_called_once_with()
- env.set_settings_data.assert_called_with(
- {
- 'editable': {
- 'repo_setup': {
- 'repos': {'value': local_repos(reverse=False)}
- }
- }
- }
- )
-
- def test_apply_for_centos_based_env(self, accessors):
- fuel = accessors.get_fuel_api_accessor()
- self._setup_fuel_versions(fuel)
- env = self._create_fuel_env(fuel)
- self._create_fuel_release(fuel, "CentOS")
- self.start_cmd(
- apply, ['--group', 'mos', 'centos', '--env', '1'],
- CENTOS_PATH
- )
- accessors.get_fuel_api_accessor.assert_called_with(
- "10.25.0.10", "test", "test1"
- )
- fuel.FuelVersion.get_all_data.assert_called_once_with()
- env.set_settings_data.assert_called_with(
- {'editable': {'repo_setup': {'repos': {'value': [
- {
- 'priority': 5,
- 'name': 'centos',
- 'type': 'rpm',
- 'uri': 'http://10.25.0.10:8080/centos/os/x86_64'
- },
- {
- 'priority': 10,
- 'name': 'mos',
- 'type': 'rpm',
- 'uri': 'http://10.25.0.10:8080/mos1/x86_64'
- }]
- }}}}
- )
-
- def test_apply_for_ubuntu_release(self, accessors):
- fuel = accessors.get_fuel_api_accessor()
- self._setup_fuel_versions(fuel)
- env = self._create_fuel_env(fuel)
- release = self._create_fuel_release(fuel, "Ubuntu")
- self.start_cmd(
- apply, ['--group', 'mos', 'ubuntu', '--default'],
- UBUNTU_PATH
- )
- accessors.get_fuel_api_accessor.assert_called_with(
- "10.25.0.10", "test", "test1"
- )
- fuel.FuelVersion.get_all_data.assert_called_once_with()
- self.assertEqual(1, env.set_settings_data.call_count)
- release.connection.put_request.assert_called_once_with(
- release.instance_api_path.format(),
- {
- 'name': "test release",
- 'operating_system': 'Ubuntu',
- 'attributes_metadata': {
- 'editable': {'repo_setup': {'repos': {'value': [
- {
- 'name': 'mos',
- 'priority': 1000,
- 'suite': 'mos1',
- 'section': 'main restricted',
- 'type': 'deb',
- 'uri': 'http://10.25.0.10:8080/mos'
- },
- {
- 'name': 'ubuntu',
- 'priority': 500,
- 'suite': 'trusty',
- 'section': 'main multiverse restricted universe',
- 'type': 'deb',
- 'uri': 'http://10.25.0.10:8080/ubuntu'
- }
- ]}}}
- }
- }
- )
-
- def test_update_release_and_cluster_repositories(self, accessors):
- fuel = accessors.get_fuel_api_accessor()
- self._setup_fuel_versions(fuel)
- env = self._create_fuel_env(fuel, repos=mirror_repos())
- release = \
- self._create_fuel_release(fuel, "Ubuntu", repos=mirror_repos())
- self.start_cmd(
- apply, ['--group', 'mos', 'ubuntu', '--default'],
- UBUNTU_PATH
- )
- accessors.get_fuel_api_accessor.assert_called_with(
- "10.25.0.10", "test", "test1"
- )
- fuel.FuelVersion.get_all_data.assert_called_once_with()
- expected_repos = mirror_repos() + local_repos()
- env.set_settings_data.assert_called_with(
- {
- 'editable': {
- 'repo_setup': {
- 'repos': {'value': expected_repos}
- }
- }
- }
- )
- release.connection.put_request.assert_called_once_with(
- release.instance_api_path.format(),
- {
- 'name': "test release",
- 'operating_system': 'Ubuntu',
- 'attributes_metadata': {
- 'editable': {'repo_setup': {'repos': {
- 'value': expected_repos
- }}}
- }
- }
- )
-
- def test_replace_release_and_cluster_repositories(self, accessors):
- fuel = accessors.get_fuel_api_accessor()
- self._setup_fuel_versions(fuel)
- env = self._create_fuel_env(fuel, repos=mirror_repos())
- release = \
- self._create_fuel_release(fuel, "Ubuntu", repos=mirror_repos())
- self.start_cmd(
- apply, ['--group', 'mos', 'ubuntu', '--default', '--replace'],
- UBUNTU_PATH
- )
- accessors.get_fuel_api_accessor.assert_called_with(
- "10.25.0.10", "test", "test1"
- )
- fuel.FuelVersion.get_all_data.assert_called_once_with()
- expected_repos = local_repos(reverse=False)
- env.set_settings_data.assert_called_with(
- {
- 'editable': {
- 'repo_setup': {
- 'repos': {'value': expected_repos}
- }
- }
- }
- )
- release.connection.put_request.assert_called_once_with(
- release.instance_api_path.format(),
- {
- 'name': "test release",
- 'operating_system': 'Ubuntu',
- 'attributes_metadata': {
- 'editable': {'repo_setup': {'repos': {
- 'value': expected_repos
- }}}
- }
- }
- )
-
- def test_apply_for_centos_release(self, accessors):
- fuel = accessors.get_fuel_api_accessor()
- self._setup_fuel_versions(fuel)
- env = self._create_fuel_env(fuel)
- release = self._create_fuel_release(fuel, "CentOS")
- self.start_cmd(
- apply, ['--group', 'mos', 'centos', '--default'],
- CENTOS_PATH
- )
- accessors.get_fuel_api_accessor.assert_called_with(
- "10.25.0.10", "test", "test1"
- )
- fuel.FuelVersion.get_all_data.assert_called_once_with()
- self.assertEqual(1, env.set_settings_data.call_count)
- release.connection.put_request.assert_called_once_with(
- release.instance_api_path.format(),
- {
- 'name': "test release",
- 'operating_system': 'CentOS',
- 'attributes_metadata': {
- 'editable': {'repo_setup': {'repos': {'value': [
- {
- 'name': 'centos',
- 'priority': 5,
- 'type': 'rpm',
- 'uri': 'http://10.25.0.10:8080/centos/os/x86_64'
- },
- {
- 'name': 'mos',
- 'priority': 10,
- 'type': 'rpm',
- 'uri': 'http://10.25.0.10:8080/mos1/x86_64'
- },
- ]}}}
- }
- }
- )
-
- @mock.patch("fuel_mirror.app.utils.get_fuel_settings")
- def test_apply_fail_if_no_fuel_address(self, m_get_settings, accessors):
- m_get_settings.return_value = {}
- with self.assertRaisesRegexp(
- ValueError, "Please specify the fuel-server option"):
- apply.debug(
- ["--config", CONFIG_PATH, "-G", "mos", "-I", UBUNTU_PATH]
- )
- self.assertFalse(accessors.get_fuel_api_accessor.called)
-
- @mock.patch("fuel_mirror.app.utils.get_fuel_settings")
- def test_create_without_fuel_address(self, m_get_settings, accessors):
- m_get_settings.return_value = {}
- packetary = accessors.get_packetary_accessor()
- create.debug(
- ["--config", CONFIG_PATH, "-G", "mos", "-I", UBUNTU_PATH]
- )
- self.assertFalse(accessors.get_fuel_api_accessor.called)
- accessors.get_packetary_accessor.assert_called_with(
- threads_num=1,
- ignore_errors_num=2,
- retries_num=3,
- http_proxy="http://localhost",
- https_proxy="https://localhost",
- )
- packetary.assert_called_with("deb", "x86_64")
- api = packetary()
- api.clone_repositories.assert_called_once_with(
- ['http://localhost/mos mos main restricted'],
- '/var/www/',
- None,
- None
- )
-
- @mock.patch("fuel_mirror.app.utils.get_fuel_settings")
- def test_create_with_invalid_data(self, m_get_settings, accessors):
- self.assertRaises(
- ValidationError, create.debug, ["--config", CONFIG_PATH, "-G",
- "mos", "-I", INVALID_DATA_PATH]
- )
diff --git a/contrib/fuel_mirror/fuel_mirror/tests/test_url_builder.py b/contrib/fuel_mirror/fuel_mirror/tests/test_url_builder.py
deleted file mode 100644
index aad93b9..0000000
--- a/contrib/fuel_mirror/fuel_mirror/tests/test_url_builder.py
+++ /dev/null
@@ -1,70 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from fuel_mirror.common import url_builder
-from fuel_mirror.tests import base
-
-
-class TestUrlBuilder(base.TestCase):
- def test_get_url_builder(self):
- self.assertTrue(issubclass(
- url_builder.get_url_builder("deb"),
- url_builder.AptRepoUrlBuilder
- ))
- self.assertTrue(issubclass(
- url_builder.get_url_builder("rpm"),
- url_builder.YumRepoUrlBuilder
- ))
- with self.assertRaises(KeyError):
- url_builder.get_url_builder("unknown")
-
-
-class TestAptUrlBuilder(base.TestCase):
- @classmethod
- def setUpClass(cls):
- cls.builder = url_builder.get_url_builder("deb")
- cls.repo_data = {
- "name": "ubuntu",
- "suite": "trusty",
- "section": "main restricted",
- "type": "deb",
- "uri": "http://localhost/ubuntu"
- }
-
- def test_get_repo_url(self):
- self.assertEqual(
- "http://localhost/ubuntu trusty main restricted",
- self.builder.get_repo_url(self.repo_data)
- )
-
-
-class TestYumUrlBuilder(base.TestCase):
- @classmethod
- def setUpClass(cls):
- cls.builder = url_builder.get_url_builder("rpm")
- cls.repo_data = {
- "name": "centos",
- "type": "rpm",
- "uri": "http://localhost/os/x86_64"
- }
-
- def test_get_repo_url(self):
- self.assertEqual(
- "http://localhost/os/x86_64",
- self.builder.get_repo_url(self.repo_data)
- )
diff --git a/contrib/fuel_mirror/fuel_mirror/tests/test_utils.py b/contrib/fuel_mirror/fuel_mirror/tests/test_utils.py
deleted file mode 100644
index 1a48ebb..0000000
--- a/contrib/fuel_mirror/fuel_mirror/tests/test_utils.py
+++ /dev/null
@@ -1,114 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import mock
-import six
-
-from fuel_mirror.common import utils
-from fuel_mirror.tests import base
-
-
-class DictAsObj(object):
- def __init__(self, d):
- self.__dict__.update(d)
-
- def __eq__(self, other):
- return self.__dict__ == other.__dict__
-
-
-class TestUtils(base.TestCase):
- def test_lists_merge(self):
- main = [{"a": 1, "b": 2, "c": 0}, {"a": 2, "b": 3, "c": 1}]
- patch = [{"a": 2, "b": 4}, {"a": 3, "b": 5}]
- utils.lists_merge(
- main,
- patch,
- key="a"
- )
- self.assertItemsEqual(
- [{"a": 1, "b": 2, "c": 0},
- {"a": 2, "b": 4, "c": 1},
- {"a": 3, "b": 5}],
- main
- )
-
- def test_first(self):
- self.assertEqual(
- 1,
- utils.first(0, 1, 0),
- )
- self.assertEqual(
- 1,
- utils.first(None, [], '', 1),
- )
- self.assertIsNone(
- utils.first(None, [], 0, ''),
- )
- self.assertIsNone(
- utils.first(),
- )
-
- def test_is_subdict(self):
- self.assertFalse(utils.is_subdict({"c": 1}, {"a": 1, "b": 1}))
- self.assertFalse(utils.is_subdict({"a": 1, "b": 2}, {"a": 1, "b": 1}))
- self.assertFalse(
- utils.is_subdict({"a": 1, "b": 1, "c": 2}, {"a": 1, "b": 1})
- )
- self.assertFalse(
- utils.is_subdict({"a": 1, "b": None}, {"a": 1})
- )
- self.assertTrue(utils.is_subdict({}, {"a": 1}))
- self.assertTrue(utils.is_subdict({"a": 1}, {"a": 1, "b": 1}))
- self.assertTrue(utils.is_subdict({"a": 1, "b": 1}, {"a": 1, "b": 1}))
-
- @mock.patch("fuel_mirror.common.utils.open")
- def test_get_fuel_settings(self, m_open):
- m_open().__enter__.side_effect = [
- six.StringIO(
- 'ADMIN_NETWORK:\n'
- ' ipaddress: "10.20.0.4"\n'
- 'FUEL_ACCESS:\n'
- ' user: "test"\n'
- ' password: "test_pwd"\n',
- ),
- OSError
- ]
-
- self.assertEqual(
- {
- "server": "10.20.0.4",
- "user": "test",
- "password": "test_pwd",
- },
- utils.get_fuel_settings()
- )
-
- self.assertEqual(
- {},
- utils.get_fuel_settings()
- )
-
- @mock.patch("fuel_mirror.common.utils.yaml")
- @mock.patch("fuel_mirror.common.utils.open")
- def test_load_input_data(self, open_mock, yaml_mock):
- data = "$param1: $param2"
- open_mock().__enter__().read.return_value = data
- v = utils.load_input_data("data.yaml", param1="key", param2="value")
- open_mock.assert_called_with("data.yaml", "r")
- yaml_mock.load.assert_called_once_with("key: value")
- self.assertIs(yaml_mock.load(), v)
diff --git a/contrib/fuel_mirror/fuel_mirror/tests/test_validate_config.py b/contrib/fuel_mirror/fuel_mirror/tests/test_validate_config.py
deleted file mode 100644
index 6093367..0000000
--- a/contrib/fuel_mirror/fuel_mirror/tests/test_validate_config.py
+++ /dev/null
@@ -1,158 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import os.path
-
-from jsonschema import validate
-from jsonschema import ValidationError
-import yaml
-
-from fuel_mirror.schemas.input_data_schema import SCHEMA
-from fuel_mirror.tests import base
-
-
-DATA_DIR = os.path.join(os.path.dirname(__file__), "..", "..", "data")
-
-
-class TestValidateConfigs(base.TestCase):
-
- def test_validate_data_files(self):
- for f in os.listdir(DATA_DIR):
- with open(os.path.join(DATA_DIR, f), "r") as fd:
- data = yaml.load(fd)
- self.assertNotRaises(ValidationError, validate, data, SCHEMA)
- self.assertIn("groups", data)
- self.assertIn("fuel_release_match", data)
-
- def test_validate_fail_with_empty_data(self):
- self.assertRaises(ValidationError, validate, {}, SCHEMA)
-
- def test_validate_fail_without_groups(self):
- invalid_data = {
- "requirements": {
- "ubuntu": ["package_deb"]
- }
- }
- self.assertRaisesRegexp(
- ValidationError, "'groups' is a required property", validate,
- invalid_data, SCHEMA)
-
- def test_invalid_requirements_in_pattern_properies(self):
- invalid_data = {
- "requirements": {
- "ubun.tu": ["package_deb"]
- },
- "groups": {
- }
- }
- self.assertRaisesRegexp(
- ValidationError, "'ubun.tu' was unexpected", validate,
- invalid_data, SCHEMA)
-
- def test_invalid_requirements_type_array(self):
- invalid_data = {
- "requirements": {
- "ubuntu": "package_deb"
- },
- "groups": {
- }
- }
- self.assertRaisesRegexp(
- ValidationError, "'package_deb' is not of type 'array'", validate,
- invalid_data, SCHEMA)
-
- def test_invalid_inheritens_in_pattern_properies(self):
- invalid_data = {
- "inheritance": {
- "ubun.tu": "mos"
- },
- "groups": {
- }
- }
- self.assertRaisesRegexp(
- ValidationError, "'ubun.tu' was unexpected", validate,
- invalid_data, SCHEMA)
-
- def test_invalid_inheritens_type_string(self):
- invalid_data = {
- "inheritance": {
- "ubuntu": 123
- },
- "groups": {
- }
- }
- self.assertRaisesRegexp(
- ValidationError, "123 is not of type 'string'", validate,
- invalid_data, SCHEMA)
-
- def test_invalid_groups_in_pattern_properies(self):
- invalid_data = {
- "groups": {
- "mo.s": []
- }
- }
- self.assertRaisesRegexp(
- ValidationError, "'mo.s' was unexpected", validate,
- invalid_data, SCHEMA)
-
- def test_invalid_groups_type_array(self):
- invalid_data = {
- "groups": {
- "mos": "string"
- }
- }
- self.assertRaisesRegexp(
- ValidationError, "'string' is not of type 'array'", validate,
- invalid_data, SCHEMA)
-
- def test_without_name_in_groups_array(self):
- invalid_data = {
- "groups": {
- "mos": [
- {
- 'type': 'deb',
- 'uri': 'http://localhost/mos',
- 'priority': None,
- 'suite': 'mos$mos_version',
- 'section': 'main restricted'
- }
- ]
- }
- }
- self.assertRaisesRegexp(
- ValidationError, "is not valid under any of the given schemas",
- validate, invalid_data, SCHEMA)
-
- def test_with_invalid_type_in_groups_array(self):
- invalid_data = {
- "groups": {
- "mos": [
- {
- 'name': 'mos',
- 'type': 'adf',
- 'uri': 'http://localhost/mos',
- 'priority': None,
- 'suite': 'mos$mos_version',
- 'section': 'main restricted'
- }
- ]
- }
- }
- self.assertRaisesRegexp(
- ValidationError, "is not valid under any of the given schemas",
- validate, invalid_data, SCHEMA)
diff --git a/contrib/fuel_mirror/requirements.txt b/contrib/fuel_mirror/requirements.txt
deleted file mode 100644
index a69fea2..0000000
--- a/contrib/fuel_mirror/requirements.txt
+++ /dev/null
@@ -1,12 +0,0 @@
-# The order of packages is significant, because pip processes them in the order
-# of appearance. Changing the order has an impact on the overall integration
-# process, which may cause wedges in the gate later.
-
-pbr>=0.8
-Babel>=1.3
-cliff>=1.7.0
-six>=1.5.2
-PyYAML>=3.10
-packetary>=0.1.0
-python-fuelclient>=7.0.0
-jsonschema>=2.3.0
diff --git a/contrib/fuel_mirror/scripts/fuel-createmirror b/contrib/fuel_mirror/scripts/fuel-createmirror
deleted file mode 100755
index bc68dbb..0000000
--- a/contrib/fuel_mirror/scripts/fuel-createmirror
+++ /dev/null
@@ -1,80 +0,0 @@
-#!/bin/bash
-
-echo "This script is DEPRECATED. Please use fuel-mirror utility!"
-
-# This shell script was wraps the fuel-mirror utility to provide backward compatibility
-# with previous version of tool.
-
-usage() {
-cat <=0.10.0
-
-coverage>=3.6
-discover
-python-subunit>=0.0.18
-sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2
-oslosphinx>=2.5.0 # Apache-2.0
-oslotest>=1.10.0 # Apache-2.0
-testrepository>=0.0.18
-testscenarios>=0.4
-testtools>=1.4.0
-cliff>=1.7.0
-six>=1.5.2
diff --git a/contrib/fuel_mirror/tox.ini b/contrib/fuel_mirror/tox.ini
deleted file mode 100644
index 0a57b39..0000000
--- a/contrib/fuel_mirror/tox.ini
+++ /dev/null
@@ -1,35 +0,0 @@
-[tox]
-minversion = 1.6
-envlist = py34,py27,py26,pep8
-skipsdist = True
-
-[testenv]
-usedevelop = True
-install_command = pip install -U {opts} {packages}
-setenv =
- VIRTUAL_ENV={envdir}
-deps = -r{toxinidir}/test-requirements.txt
-commands = python setup.py test --slowest --testr-args='{posargs:fuel_mirror}'
-
-[testenv:pep8]
-commands = flake8
-
-[testenv:venv]
-commands = {posargs}
-
-[testenv:cover]
-commands = python setup.py test --coverage --testr-args='{posargs:fuel_mirror}'
-
-[testenv:docs]
-commands = python setup.py build_sphinx
-
-[testenv:debug]
-commands = oslo_debug_helper {posargs}
-
-[flake8]
-# E123, E125 skipped as they are invalid PEP-8.
-
-show-source = True
-ignore = E123,E125
-builtins = _
-exclude=.venv,.git,.tox,dist,doc,*openstack/common*,*lib/python*,*egg,build
diff --git a/debian/changelog b/debian/changelog
deleted file mode 100644
index fb1f193..0000000
--- a/debian/changelog
+++ /dev/null
@@ -1,17 +0,0 @@
-fuel-mirror (10.0.0-1) experimental; urgency=medium
-
- * Bump version to 10.0
-
- -- Sergey Kulanov Mon, 21 Mar 2016 13:29:18 +0200
-
-fuel-mirror (9.0.0-1) experimental; urgency=low
-
- * Bump version to 9.0
-
- -- Sergey Kulanov Thu, 17 Dec 2015 16:39:55 +0200
-
-fuel-mirror (8.0.0-1) experimental; urgency=low
-
- * Initial release.
-
- -- bgaifullin Fri, 27 Nov 2015 00:28:26 +0300
diff --git a/debian/compat b/debian/compat
deleted file mode 100644
index ec63514..0000000
--- a/debian/compat
+++ /dev/null
@@ -1 +0,0 @@
-9
diff --git a/debian/control b/debian/control
deleted file mode 100644
index 4598767..0000000
--- a/debian/control
+++ /dev/null
@@ -1,54 +0,0 @@
-Source: fuel-mirror
-Section: Utilities
-Priority: extra
-Maintainer: Mirantis Product
-Build-Depends: debhelper (>= 9),
- dh-python,
- openstack-pkg-tools (>= 23~),
- python-all,
- python-pbr (>= 0.8),
- python-setuptools
-Standards-Version: 3.9.6
-Homepage: mirantis.com
-
-Package: fuel-mirror
-Architecture: all
-Section: python
-Depends: python-babel,
- python-cliff (>= 1.7.0),
- python-packetary (= ${binary:Version}),
- python-pbr (>= 0.8),
- python-six,
- python-yaml,
- python-tz,
- ${python:Depends}
-Recommends: python-fuelclient (>= 7.0.0)
-Description: Utility to create RPM and DEB mirror
- Provides two commands fuel-mirror and fuel-createmirror.
- Second one is for backward compatibility with the previous
- generation of the utility. These commands could be used
- to create local copies of MOS and upstream deb and rpm
- repositories.
-
-Package: python-packetary
-Architecture: all
-Depends: createrepo,
- python-babel,
- python-bintrees (>= 2.0.2),
- python-chardet,
- python-cliff (>= 1.7.0),
- python-debian (>= 0.1.21),
- python-eventlet (>= 0.15),
- python-lxml,
- python-pbr (>= 0.8),
- python-six,
- python-stevedore (>= 1.1.0),
- python-tz,
- ${python:Depends}
-Description: Library allows to build and clone deb and rpm repos
- Provides object model and API for dealing with deb
- and rpm repositories. One can use this framework to
- implement operations like building repository
- from a set of packages, clone repository, find package
- dependencies, mix repositories, pull out a subset of
- packages into a separate repository, etc.
diff --git a/debian/copyright b/debian/copyright
deleted file mode 100644
index 2fa4ac7..0000000
--- a/debian/copyright
+++ /dev/null
@@ -1,43 +0,0 @@
-Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
-Upstream-Name: fuel-mirror
-Source: git://github.com/openstack/fuel-mirror.git
-
-Files: debian/*
-Copyright: (c) 2014, Mirantis
-License: GPL-2
-
-Files: *
-Copyright: (c) 2014, Mirantis
-License: Apache-2
-
-License: Apache-2
- Licensed under the Apache License, Version 2.0 (the "License"); you may not
- use this file except in compliance with the License. You may obtain a copy of
- the License at:
- .
- http://www.apache.org/licenses/LICENSE-2.0
- .
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
- WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
- License for the specific language governing permissions and limitations under
- the License.
- .
- On Debian-based systems the full text of the Apache version 2.0 license can be
- found in /usr/share/common-licenses/Apache-2.0.
-
-License: GPL-2
- Licensed under the GPL License, Version 2.0 (the "License"); you may not
- use this file except in compliance with the License. You may obtain a copy of
- the License at:
- .
- http://www.opensource.org/licenses/GPL-2.0
- .
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
- WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
- License for the specific language governing permissions and limitations under
- the License.
- .
- On Debian-based systems the full text of the GPL version 2.0 license can be
- found in /usr/share/common-licenses/GPL-2.
diff --git a/debian/fuel-mirror.install b/debian/fuel-mirror.install
deleted file mode 100644
index e8eabae..0000000
--- a/debian/fuel-mirror.install
+++ /dev/null
@@ -1,2 +0,0 @@
-contrib/fuel_mirror/etc/config.yaml /etc/fuel-mirror
-contrib/fuel_mirror/scripts/fuel-createmirror /usr/bin/
diff --git a/debian/rules b/debian/rules
deleted file mode 100755
index 21c7f69..0000000
--- a/debian/rules
+++ /dev/null
@@ -1,39 +0,0 @@
-#!/usr/bin/make -f
-
-PYTHONS:=$(shell pyversions -vr)
-
-include /usr/share/openstack-pkg-tools/pkgos.make
-export OSLO_PACKAGE_VERSION=$(shell dpkg-parsechangelog | grep Version: | cut -d' ' -f2 | sed -e 's/^[[:digit:]]*://' -e 's/[-].*//' -e 's/~/.0/' | head -n 1)
-
-%:
- dh $@ --buildsystem=python_distutils --with python2
-
-override_dh_clean:
- rm -rf build
- dh_clean -O--buildsystem=python_distutils
-
-
-override_dh_auto_install:
- set -e ; for pyvers in $(PYTHONS); do \
- python$$pyvers setup.py install --install-layout=deb \
- --root $(CURDIR)/debian/python-packetary; \
- done
- set -e ; cd contrib/fuel_mirror/; \
- for pyvers in $(PYTHONS); do \
- python$$pyvers ./setup.py install --install-layout=deb \
- --root $(CURDIR)/debian/fuel-mirror; \
- done
-
-override_dh_fixperms:
- set -e; chmod 755 $(CURDIR)/debian/fuel-mirror/usr/bin/fuel-createmirror
-
-override_dh_python2:
- dh_python2 --no-guessing-deps
-
-override_dh_installcatalogs:
-override_dh_installemacsen override_dh_installifupdown:
-override_dh_installinfo override_dh_installmenu override_dh_installmime:
-override_dh_installmodules override_dh_installlogcheck:
-override_dh_installpam override_dh_installppp override_dh_installudev override_dh_installwm:
-override_dh_installxfonts override_dh_gconf override_dh_icons override_dh_perl override_dh_usrlocal:
-override_dh_installgsettings:
\ No newline at end of file
diff --git a/debian/source/format b/debian/source/format
deleted file mode 100644
index 46ebe02..0000000
--- a/debian/source/format
+++ /dev/null
@@ -1 +0,0 @@
-3.0 (quilt)
\ No newline at end of file
diff --git a/doc/__init__.py b/doc/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/doc/source/__init__.py b/doc/source/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/doc/source/conf.py b/doc/source/conf.py
deleted file mode 100755
index 12fc8a7..0000000
--- a/doc/source/conf.py
+++ /dev/null
@@ -1,79 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import os
-import sys
-
-sys.path.insert(0, os.path.abspath('../..'))
-# -- General configuration ----------------------------------------------------
-
-# Add any Sphinx extension module names here, as strings. They can be
-# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = [
- 'sphinx.ext.autodoc',
- #'sphinx.ext.intersphinx',
- 'oslosphinx'
-]
-
-# autodoc generation is a bit aggressive and a nuisance when doing heavy
-# text edit cycles.
-# execute "export SPHINX_DEBUG=1" in your terminal to disable
-
-# The suffix of source filenames.
-source_suffix = '.rst'
-
-# The master toctree document.
-master_doc = 'index'
-
-# General information about the project.
-project = u'packetary'
-copyright = u'2015, Mirantis Inc.'
-
-# If true, '()' will be appended to :func: etc. cross-reference text.
-add_function_parentheses = True
-
-# If true, the current module name will be prepended to all description
-# unit titles (such as .. function::).
-add_module_names = True
-
-# The name of the Pygments (syntax highlighting) style to use.
-pygments_style = 'sphinx'
-
-# -- Options for HTML output --------------------------------------------------
-
-# The theme to use for HTML and HTML Help pages. Major themes that come with
-# Sphinx are currently 'default' and 'sphinxdoc'.
-# html_theme_path = ["."]
-# html_theme = '_theme'
-# html_static_path = ['static']
-
-# Output file base name for HTML help builder.
-htmlhelp_basename = '%sdoc' % project
-
-# Grouping the document tree into LaTeX files. List of tuples
-# (source start file, target name, title, author, documentclass
-# [howto/manual]).
-latex_documents = [
- ('index',
- '%s.tex' % project,
- u'%s Documentation' % project,
- u'OpenStack Foundation', 'manual'),
-]
-
-# Example configuration for intersphinx: refer to the Python standard library.
-#intersphinx_mapping = {'http://docs.python.org/': None}
diff --git a/doc/source/contributing.rst b/doc/source/contributing.rst
deleted file mode 100644
index 1728a61..0000000
--- a/doc/source/contributing.rst
+++ /dev/null
@@ -1,4 +0,0 @@
-============
-Contributing
-============
-.. include:: ../../CONTRIBUTING.rst
diff --git a/doc/source/index.rst b/doc/source/index.rst
deleted file mode 100644
index 46d6225..0000000
--- a/doc/source/index.rst
+++ /dev/null
@@ -1,25 +0,0 @@
-.. packetary documentation master file, created by
- sphinx-quickstart on Tue Jul 9 22:26:36 2013.
- You can adapt this file completely to your liking, but it should at least
- contain the root `toctree` directive.
-
-Welcome to packetary's documentation!
-========================================================
-
-Contents:
-
-.. toctree::
- :maxdepth: 2
-
- readme
- installation
- usage
- contributing
-
-Indices and tables
-==================
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
-
diff --git a/doc/source/installation.rst b/doc/source/installation.rst
deleted file mode 100644
index 532b3f1..0000000
--- a/doc/source/installation.rst
+++ /dev/null
@@ -1,12 +0,0 @@
-============
-Installation
-============
-
-At the command line::
-
- $ pip install packetary
-
-Or, if you have virtualenvwrapper installed::
-
- $ mkvirtualenv packetary
- $ pip install packetary
diff --git a/doc/source/readme.rst b/doc/source/readme.rst
deleted file mode 100644
index a6210d3..0000000
--- a/doc/source/readme.rst
+++ /dev/null
@@ -1 +0,0 @@
-.. include:: ../../README.rst
diff --git a/doc/source/usage.rst b/doc/source/usage.rst
deleted file mode 100644
index c061172..0000000
--- a/doc/source/usage.rst
+++ /dev/null
@@ -1,7 +0,0 @@
-========
-Usage
-========
-
-To use packetary in a project::
-
- import packetary
diff --git a/openstack-common.conf b/openstack-common.conf
deleted file mode 100644
index c0bb65e..0000000
--- a/openstack-common.conf
+++ /dev/null
@@ -1,6 +0,0 @@
-[DEFAULT]
-
-# The list of modules to copy from oslo-incubator.git
-
-# The base module to hold the copy of openstack.common
-base=packetary
diff --git a/packetary/__init__.py b/packetary/__init__.py
deleted file mode 100644
index 9ee18e2..0000000
--- a/packetary/__init__.py
+++ /dev/null
@@ -1,33 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import pbr.version
-
-from packetary.api import Configuration
-from packetary.api import Context
-from packetary.api import RepositoryApi
-
-
-__all__ = [
- "Configuration",
- "Context",
- "RepositoryApi",
-]
-
-__version__ = pbr.version.VersionInfo(
- 'packetary').version_string()
diff --git a/packetary/api.py b/packetary/api.py
deleted file mode 100644
index dd7d98d..0000000
--- a/packetary/api.py
+++ /dev/null
@@ -1,233 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import logging
-
-import six
-
-from packetary.controllers import RepositoryController
-from packetary.library.connections import ConnectionsManager
-from packetary.library.executor import AsynchronousSection
-from packetary.objects import Index
-from packetary.objects import PackageRelation
-from packetary.objects import PackagesTree
-from packetary.objects.statistics import CopyStatistics
-
-
-logger = logging.getLogger(__package__)
-
-
-class Configuration(object):
- """The configuration holder."""
-
- def __init__(self, http_proxy=None, https_proxy=None,
- retries_num=0, threads_num=0,
- ignore_errors_num=0, retry_interval=0):
- """Initialises.
-
- :param http_proxy: the url of proxy for connections over http,
- no-proxy will be used if it is not specified
- :param https_proxy: the url of proxy for connections over https,
- no-proxy will be used if it is not specified
- :param retries_num: the number of retries on errors
- :param retry_interval: the time between retries (in seconds)
- :param threads_num: the max number of active threads
- :param ignore_errors_num: the number of errors that may occurs
- before stop processing
- """
-
- self.http_proxy = http_proxy
- self.https_proxy = https_proxy
- self.ignore_errors_num = ignore_errors_num
- self.retries_num = retries_num
- self.retry_interval = retry_interval
- self.threads_num = threads_num
-
-
-class Context(object):
- """The infra-objects holder."""
-
- def __init__(self, config):
- """Initialises.
-
- :param config: the configuration
- """
- self._connection = ConnectionsManager(
- proxy=config.http_proxy,
- secure_proxy=config.https_proxy,
- retries_num=config.retries_num,
- retry_interval=config.retry_interval
- )
- self._threads_num = config.threads_num
- self._ignore_errors_num = config.ignore_errors_num
-
- @property
- def connection(self):
- """Gets the connection."""
- return self._connection
-
- def async_section(self, ignore_errors_num=None):
- """Gets the execution scope.
-
- :param ignore_errors_num: custom value for ignore_errors_num,
- the class value is used if omitted.
- """
- if ignore_errors_num is None:
- ignore_errors_num = self._ignore_errors_num
-
- return AsynchronousSection(self._threads_num, ignore_errors_num)
-
-
-class RepositoryApi(object):
- """Provides high-level API to operate with repositories."""
-
- def __init__(self, controller):
- """Initialises.
-
- :param controller: the repository controller.
- """
- self.controller = controller
-
- @classmethod
- def create(cls, config, repotype, repoarch):
- """Creates the repository API instance.
-
- :param config: the configuration
- :param repotype: the kind of repository(deb, yum, etc)
- :param repoarch: the architecture of repository (x86_64 or i386)
- """
- context = config if isinstance(config, Context) else Context(config)
- return cls(RepositoryController.load(context, repotype, repoarch))
-
- def get_packages(self, origin, debs=None, requirements=None):
- """Gets the list of packages from repository(es).
-
- :param origin: The list of repository`s URLs
- :param debs: the list of repository`s URL to calculate list of
- dependencies, that will be used to filter packages.
- :param requirements: the list of package relations,
- to resolve the list of mandatory packages.
- :return: the set of packages
- """
- repositories = self._get_repositories(origin)
- return self._get_packages(repositories, debs, requirements)
-
- def clone_repositories(self, origin, destination, debs=None,
- requirements=None, keep_existing=True,
- include_source=False, include_locale=False):
- """Creates the clones of specified repositories in local folder.
-
- :param origin: The list of repository`s URLs
- :param destination: the destination folder path
- :param debs: the list of repository`s URL to calculate list of
- dependencies, that will be used to filter packages.
- :param requirements: the list of package relations,
- to resolve the list of mandatory packages.
- :param keep_existing: If False - local packages that does not exist
- in original repo will be removed.
- :param include_source: if True, the source packages
- will be copied as well.
- :param include_locale: if True, the locales
- will be copied as well.
- :return: count of copied and total packages.
- """
- repositories = self._get_repositories(origin)
- packages = self._get_packages(repositories, debs, requirements)
- mirrors = self.controller.clone_repositories(
- repositories, destination, include_source, include_locale
- )
-
- package_groups = dict((x, set()) for x in repositories)
- for pkg in packages:
- package_groups[pkg.repository].add(pkg)
-
- stat = CopyStatistics()
- for repo, packages in six.iteritems(package_groups):
- mirror = mirrors[repo]
- logger.info("copy packages from - %s", repo)
- self.controller.copy_packages(
- mirror, packages, keep_existing, stat.on_package_copied
- )
- return stat
-
- def get_unresolved_dependencies(self, origin, main=None):
- """Gets list of unresolved dependencies for repository(es).
-
- :param origin: The list of repository`s URLs
- :param main: The main repository(es) URL
- :return: list of unresolved dependencies
- """
- packages = PackagesTree()
- self.controller.load_packages(
- self._get_repositories(origin),
- packages.add
- )
-
- if main is not None:
- base = Index()
- self.controller.load_packages(
- self._get_repositories(main),
- base.add
- )
- else:
- base = None
-
- return packages.get_unresolved_dependencies(base)
-
- def _get_repositories(self, urls):
- """Gets the set of repositories by url."""
- repositories = set()
- self.controller.load_repositories(urls, repositories.add)
- return repositories
-
- def _get_packages(self, repositories, master, requirements):
- """Gets the list of packages according to master and requirements."""
- if master is None and requirements is None:
- packages = set()
- self.controller.load_packages(repositories, packages.add)
- return packages
-
- packages = PackagesTree()
- self.controller.load_packages(repositories, packages.add)
- if master is not None:
- main_index = Index()
- self.controller.load_packages(
- self._get_repositories(master),
- main_index.add
- )
- else:
- main_index = None
-
- return packages.get_minimal_subset(
- main_index,
- self._parse_requirements(requirements)
- )
-
- @staticmethod
- def _parse_requirements(requirements):
- """Gets the list of relations from requirements.
-
- :param requirements: the list of requirement in next format:
- 'name [cmp version]|[alt [cmp version]]'
- """
- if requirements is not None:
- return set(
- PackageRelation.from_args(
- *(x.split() for x in r.split("|"))) for r in requirements
- )
- return set()
diff --git a/packetary/cli/__init__.py b/packetary/cli/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/packetary/cli/app.py b/packetary/cli/app.py
deleted file mode 100644
index 341f6ff..0000000
--- a/packetary/cli/app.py
+++ /dev/null
@@ -1,104 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from cliff import app
-from cliff.commandmanager import CommandManager
-
-import packetary
-
-
-class Application(app.App):
- """Main cliff application class.
-
- Performs initialization of the command manager and
- configuration of basic engines.
- """
-
- def build_option_parser(self, description, version, argparse_kwargs=None):
- """Specifies global options."""
- p_inst = super(Application, self)
- parser = p_inst.build_option_parser(description=description,
- version=version,
- argparse_kwargs=argparse_kwargs)
-
- parser.add_argument(
- "--ignore-errors-num",
- type=int,
- default=2,
- metavar="NUMBER",
- help="The number of errors that can be ignored."
- )
- parser.add_argument(
- "--retries-num",
- type=int,
- default=5,
- metavar="NUMBER",
- help="The number of retries."
- )
- parser.add_argument(
- "--retry-interval",
- type=int,
- default=2,
- metavar="SECONDS",
- help="The minimal time between retries in seconds."
- )
- parser.add_argument(
- "--threads-num",
- default=3,
- type=int,
- metavar="NUMBER",
- help="The number of threads."
- )
- parser.add_argument(
- "--http-proxy",
- default=None,
- metavar="http://username:password@proxy_host:proxy_port",
- help="The URL of http proxy."
- )
- parser.add_argument(
- "--https-proxy",
- default=None,
- metavar="https://username:password@proxy_host:proxy_port",
- help="The URL of https proxy."
- )
- return parser
-
-
-def main(argv=None):
- return Application(
- description="The utility manages packages and repositories.",
- version=packetary.__version__,
- command_manager=CommandManager("packetary", convert_underscores=True)
- ).run(argv)
-
-
-def debug(name, cmd_class, argv=None):
- """Helper for debugging single command without package installation."""
- import sys
-
- if argv is None:
- argv = sys.argv[1:]
-
- argv = [name] + argv + ["-v", "-v", "--debug"]
- cmd_mgr = CommandManager("test_packetary", convert_underscores=True)
- cmd_mgr.add_command(name, cmd_class)
- return Application(
- description="The utility manages packages and repositories.",
- version="0.0.1",
- command_manager=cmd_mgr
- ).run(argv)
diff --git a/packetary/cli/commands/__init__.py b/packetary/cli/commands/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/packetary/cli/commands/base.py b/packetary/cli/commands/base.py
deleted file mode 100644
index bb672cf..0000000
--- a/packetary/cli/commands/base.py
+++ /dev/null
@@ -1,185 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import abc
-
-from cliff import command
-import six
-
-from packetary.cli.commands.utils import make_display_attr_getter
-from packetary.cli.commands.utils import read_lines_from_file
-from packetary import RepositoryApi
-
-
-@six.add_metaclass(abc.ABCMeta)
-class BaseRepoCommand(command.Command):
- """Super class for packetary commands."""
-
- @property
- def stdout(self):
- """Shortcut for self.app.stdout."""
- return self.app.stdout
-
- def get_parser(self, prog_name):
- """Specifies common options."""
- parser = super(BaseRepoCommand, self).get_parser(prog_name)
- parser.add_argument(
- '-t',
- '--type',
- type=str,
- choices=['deb', 'rpm'],
- metavar='TYPE',
- default='deb',
- help='The type of repository.')
-
- parser.add_argument(
- '-a',
- '--arch',
- type=str,
- choices=["x86_64", "i386"],
- metavar='ARCHITECTURE',
- default="x86_64",
- help='The target architecture.')
-
- origin_gr = parser.add_mutually_exclusive_group(required=True)
- origin_gr.add_argument(
- '-o', '--origin-url',
- nargs="+",
- dest='origins',
- type=six.text_type,
- metavar='URL',
- help='Space separated list of URLs of origin repositories.')
-
- origin_gr.add_argument(
- '-O', '--origin-file',
- type=read_lines_from_file,
- dest='origins',
- metavar='FILENAME',
- help='The path to file with URLs of origin repositories.')
-
- return parser
-
- def take_action(self, parsed_args):
- """See the Command.take_action.
-
- :param parsed_args: the command-line arguments
- :return: the result of take_repo_action
- :rtype: object
- """
- return self.take_repo_action(
- RepositoryApi.create(
- self.app_args, parsed_args.type, parsed_args.arch
- ),
- parsed_args
- )
-
- @abc.abstractmethod
- def take_repo_action(self, api, parsed_args):
- """Takes action on repository.
-
- :param api: the RepositoryApi instance
- :param parsed_args: the command-line arguments
- :return: the action result
- """
-
-
-class BaseProduceOutputCommand(BaseRepoCommand):
- columns = None
-
- def get_parser(self, prog_name):
- parser = super(BaseProduceOutputCommand, self).get_parser(prog_name)
-
- group = parser.add_argument_group(
- title='output formatter',
- description='output formatter options',
- )
- group.add_argument(
- '-c', '--column',
- nargs='+',
- choices=self.columns,
- dest='columns',
- metavar='COLUMN',
- default=[],
- help='Space separated list of columns to include.',
- )
- group.add_argument(
- '-s',
- '--sort-columns',
- type=str,
- nargs='+',
- choices=self.columns,
- metavar='SORT_COLUMN',
- default=[self.columns[0]],
- help='Space separated list of keys for sorting '
- 'the data.'
- )
- group.add_argument(
- '--sep',
- type=six.text_type,
- metavar='ROW SEPARATOR',
- default=six.text_type('; '),
- help='The row separator.'
- )
-
- return parser
-
- def produce_output(self, parsed_args, data):
- indexes = dict(
- (c, i) for i, c in enumerate(self.columns)
- )
- sort_index = [indexes[c] for c in parsed_args.sort_columns]
- if isinstance(data, list):
- data.sort(key=lambda x: [x[i] for i in sort_index])
- else:
- data = sorted(data, key=lambda x: [x[i] for i in sort_index])
-
- if parsed_args.columns:
- include_index = [
- indexes[c] for c in parsed_args.columns
- ]
- data = ((row[i] for i in include_index) for row in data)
- columns = parsed_args.columns
- else:
- columns = self.columns
-
- stdout = self.stdout
- sep = parsed_args.sep
-
- # header
- stdout.write("# ")
- stdout.write(sep.join(columns))
- stdout.write("\n")
-
- for row in data:
- stdout.write(sep.join(row))
- stdout.write("\n")
-
- def run(self, parsed_args):
- # Use custom output producer.
- # cliff.lister with default formatters does not work
- # with large arrays of data, because it does not support streaming
- # TODO(implement custom formatter)
-
- formatter = make_display_attr_getter(self.columns)
- data = six.moves.map(formatter, self.take_action(parsed_args))
- self.produce_output(parsed_args, data)
- return 0
-
- @abc.abstractmethod
- def take_repo_action(self, driver, parsed_args):
- """See Command.take_repo_action."""
diff --git a/packetary/cli/commands/clone.py b/packetary/cli/commands/clone.py
deleted file mode 100644
index fb9c3dc..0000000
--- a/packetary/cli/commands/clone.py
+++ /dev/null
@@ -1,115 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from packetary.cli.commands.base import BaseRepoCommand
-from packetary.cli.commands.utils import read_lines_from_file
-
-
-class CloneCommand(BaseRepoCommand):
- """Clones the specified repository to local folder."""
-
- def get_parser(self, prog_name):
- parser = super(CloneCommand, self).get_parser(prog_name)
-
- parser.add_argument(
- "-d", "--destination",
- required=True,
- help="The path to the destination folder."
- )
- parser.add_argument(
- "--clean",
- dest="keep_existing",
- action='store_false',
- default=True,
- help="Remove packages that does not exist in origin repo."
- )
-
- parser.add_argument(
- "--sources",
- action='store_true',
- default=False,
- help="Also copy source packages."
- )
-
- parser.add_argument(
- "--locales",
- action='store_true',
- default=False,
- help="Also copy localisation files."
- )
-
- bootstrap_group = parser.add_mutually_exclusive_group(required=False)
- bootstrap_group.add_argument(
- "-b", "--bootstrap",
- nargs='+',
- dest='bootstrap',
- metavar='PACKAGE [OP VERSION]',
- help="Space separated list of package relations, "
- "to resolve the list of mandatory packages."
- )
- bootstrap_group.add_argument(
- "-B", "--bootstrap-file",
- type=read_lines_from_file,
- dest='bootstrap',
- metavar='FILENAME',
- help="Path to the file with list of package relations, "
- "to resolve the list of mandatory packages."
- )
-
- requires_group = parser.add_mutually_exclusive_group(required=False)
- requires_group.add_argument(
- '-r', '--requires-url',
- nargs="+",
- dest='requires',
- metavar='URL',
- help="Space separated list of repository`s URL to calculate list "
- "of dependencies, that will be used to filter packages")
-
- requires_group.add_argument(
- '-R', '--requires-file',
- type=read_lines_from_file,
- dest='requires',
- metavar='FILENAME',
- help="The path to the file with list of repository`s URL "
- "to calculate list of dependencies, "
- "that will be used to filter packages")
- return parser
-
- def take_repo_action(self, api, parsed_args):
- stat = api.clone_repositories(
- parsed_args.origins,
- parsed_args.destination,
- parsed_args.requires,
- parsed_args.bootstrap,
- parsed_args.keep_existing,
- parsed_args.sources,
- parsed_args.locales
- )
- self.stdout.write(
- "Packages copied: {0.copied}/{0.total}.\n".format(stat)
- )
-
-
-def debug(argv=None):
- """Helper to debug the Clone command."""
- from packetary.cli.app import debug
- debug("clone", CloneCommand, argv)
-
-
-if __name__ == "__main__":
- debug()
diff --git a/packetary/cli/commands/packages.py b/packetary/cli/commands/packages.py
deleted file mode 100644
index 7f02bd7..0000000
--- a/packetary/cli/commands/packages.py
+++ /dev/null
@@ -1,93 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from packetary.cli.commands.base import BaseProduceOutputCommand
-from packetary.cli.commands.utils import read_lines_from_file
-
-
-class ListOfPackages(BaseProduceOutputCommand):
- """Gets the list of packages from repository(es)."""
-
- columns = (
- "name",
- "repository",
- "version",
- "filename",
- "filesize",
- "checksum",
- "obsoletes",
- "provides",
- "requires",
- )
-
- def get_parser(self, prog_name):
- parser = super(ListOfPackages, self).get_parser(prog_name)
-
- bootstrap_group = parser.add_mutually_exclusive_group(required=False)
- bootstrap_group.add_argument(
- "-b", "--bootstrap",
- nargs='+',
- dest='bootstrap',
- metavar='PACKAGE [OP VERSION]',
- help="Space separated list of package relations, "
- "to resolve the list of mandatory packages."
- )
- bootstrap_group.add_argument(
- "-B", "--bootstrap-file",
- type=read_lines_from_file,
- dest='bootstrap',
- metavar='FILENAME',
- help="Path to the file with list of package relations, "
- "to resolve the list of mandatory packages."
- )
-
- requires_group = parser.add_mutually_exclusive_group(required=False)
- requires_group.add_argument(
- '-r', '--requires-url',
- nargs="+",
- dest='requires',
- metavar='URL',
- help="Space separated list of repository`s URL to calculate list "
- "of dependencies, that will be used to filter packages")
-
- requires_group.add_argument(
- '-R', '--requires-file',
- type=read_lines_from_file,
- dest='requires',
- metavar='FILENAME',
- help="The path to the file with list of repository`s URL "
- "to calculate list of dependencies, "
- "that will be used to filter packages")
- return parser
-
- def take_repo_action(self, api, parsed_args):
- return api.get_packages(
- parsed_args.origins,
- parsed_args.requires,
- parsed_args.bootstrap,
- )
-
-
-def debug(argv=None):
- """Helper to debug the ListOfPackages command."""
- from packetary.cli.app import debug
- debug("packages", ListOfPackages, argv)
-
-
-if __name__ == "__main__":
- debug()
diff --git a/packetary/cli/commands/unresolved.py b/packetary/cli/commands/unresolved.py
deleted file mode 100644
index a4ce6f3..0000000
--- a/packetary/cli/commands/unresolved.py
+++ /dev/null
@@ -1,68 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from packetary.cli.commands.base import BaseProduceOutputCommand
-from packetary.cli.commands.utils import read_lines_from_file
-
-
-class ListOfUnresolved(BaseProduceOutputCommand):
- """Gets the list of external dependencies for repository(es)."""
-
- columns = (
- "name",
- "version",
- "alternative",
- )
-
- def get_parser(self, prog_name):
- parser = super(ListOfUnresolved, self).get_parser(prog_name)
- main_group = parser.add_mutually_exclusive_group(required=False)
- main_group.add_argument(
- '-m', '--main-url',
- nargs="+",
- dest='main',
- metavar='URL',
- help='Space separated list of URLs of repository(es) '
- ' that are used to resolve dependencies.')
-
- main_group.add_argument(
- '-M', '--main-file',
- type=read_lines_from_file,
- dest='main',
- metavar='FILENAME',
- help='The path to the file, that contains '
- 'list of URLs of repository(es) '
- ' that are used to resolve dependencies.')
- return parser
-
- def take_repo_action(self, api, parsed_args):
- return api.get_unresolved_dependencies(
- parsed_args.origins,
- parsed_args.main,
- )
-
-
-def debug(argv=None):
- """Helper to debug the ListOfUnresolved command."""
-
- from packetary.cli.app import debug
- debug("unresolved", ListOfUnresolved, argv)
-
-
-if __name__ == "__main__":
- debug()
diff --git a/packetary/cli/commands/utils.py b/packetary/cli/commands/utils.py
deleted file mode 100644
index de89158..0000000
--- a/packetary/cli/commands/utils.py
+++ /dev/null
@@ -1,72 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import operator
-
-import six
-
-
-def read_lines_from_file(filename):
- """Reads lines from file.
-
- Note: the line starts with '#' will be skipped.
-
- :param filename: the path of target file
- :return: the list of lines from file
- """
- with open(filename, 'r') as f:
- return [
- x
- for x in six.moves.map(operator.methodcaller("strip"), f)
- if x and not x.startswith("#")
- ]
-
-
-def get_object_attrs(obj, attrs):
- """Gets object attributes as list.
-
- :param obj: the target object
- :param attrs: the list of attributes
- :return: list of values from specified attributes.
- """
- return [getattr(obj, f) for f in attrs]
-
-
-def get_display_value(value):
- """Get the displayable string for value.
-
- :param value: the target value
- :return: the displayable string for value
- """
- if value is None:
- return u"-"
-
- if isinstance(value, list):
- return u", ".join(six.text_type(x) for x in value)
- return six.text_type(value)
-
-
-def make_display_attr_getter(attrs):
- """Gets formatter to convert attributes of object in displayable format.
-
- :param attrs: the list of attributes
- :return: the formatter (callable object)
- """
- return lambda x: [
- get_display_value(v) for v in get_object_attrs(x, attrs)
- ]
diff --git a/packetary/controllers/__init__.py b/packetary/controllers/__init__.py
deleted file mode 100644
index a34146c..0000000
--- a/packetary/controllers/__init__.py
+++ /dev/null
@@ -1,23 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from packetary.controllers.repository import RepositoryController
-
-__all__ = [
- "RepositoryController"
-]
diff --git a/packetary/controllers/repository.py b/packetary/controllers/repository.py
deleted file mode 100644
index da67e2a..0000000
--- a/packetary/controllers/repository.py
+++ /dev/null
@@ -1,171 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import logging
-import os
-
-import six
-import stevedore
-
-
-logger = logging.getLogger(__package__)
-
-urljoin = six.moves.urllib.parse.urljoin
-
-
-class RepositoryController(object):
- """Implements low-level functionality to communicate with drivers."""
-
- _drivers = None
-
- def __init__(self, context, driver, arch):
- self.context = context
- self.driver = driver
- self.arch = arch
-
- @classmethod
- def load(cls, context, driver_name, repoarch):
- """Creates the repository manager.
-
- :param context: the context
- :param driver_name: the name of required driver
- :param repoarch: the architecture of repository (x86_64 or i386)
- """
- if cls._drivers is None:
- cls._drivers = stevedore.ExtensionManager(
- "packetary.drivers", invoke_on_load=True
- )
- try:
- driver = cls._drivers[driver_name].obj
- except KeyError:
- raise NotImplementedError(
- "The driver {0} is not supported yet.".format(driver_name)
- )
- return cls(context, driver, repoarch)
-
- def load_repositories(self, urls, consumer):
- """Loads the repository objects from url.
-
- :param urls: the list of repository urls.
- :param consumer: the callback to consume objects
- """
- if isinstance(urls, six.string_types):
- urls = [urls]
-
- connection = self.context.connection
- for parsed_url in self.driver.parse_urls(urls):
- self.driver.get_repository(
- connection, parsed_url, self.arch, consumer
- )
-
- def load_packages(self, repositories, consumer):
- """Loads packages from repository.
-
- :param repositories: the repository object
- :param consumer: the callback to consume objects
- """
- connection = self.context.connection
- for r in repositories:
- self.driver.get_packages(connection, r, consumer)
-
- def assign_packages(self, repository, packages, keep_existing=True):
- """Assigns new packages to the repository.
-
- It replaces the current repository`s packages.
-
- :param repository: the target repository
- :param packages: the set of new packages
- :param keep_existing:
- if True, all existing packages will be kept as is.
- if False, all existing packages, that are not included
- to new packages will be removed.
- """
-
- if not isinstance(packages, set):
- packages = set(packages)
- else:
- packages = packages.copy()
-
- if keep_existing:
- consume_exist = packages.add
- else:
- def consume_exist(package):
- if package not in packages:
- filepath = os.path.join(
- package.repository.url, package.filename
- )
- logger.info("remove package - %s.", filepath)
- os.remove(filepath)
-
- self.driver.get_packages(
- self.context.connection, repository, consume_exist
- )
- self.driver.rebuild_repository(repository, packages)
-
- def copy_packages(self, repository, packages, keep_existing, observer):
- """Copies packages to repository.
-
- :param repository: the target repository
- :param packages: the set of packages
- :param keep_existing: see assign_packages for more details
- :param observer: the package copying process observer
- """
- with self.context.async_section() as section:
- for package in packages:
- section.execute(
- self._copy_package, repository, package, observer
- )
- self.assign_packages(repository, packages, keep_existing)
-
- def clone_repositories(self, repositories, destination,
- source=False, locale=False):
- """Creates copy of repositories.
-
- :param repositories: the origin repositories
- :param destination: the target folder
- :param source: If True, the source packages will be copied too.
- :param locale: If True, the localisation will be copied too.
- :return: the mapping origin to cloned repository.
- """
- mirros = dict()
- destination = os.path.abspath(destination)
- with self.context.async_section(0) as section:
- for r in repositories:
- section.execute(
- self._fork_repository,
- r, destination, source, locale, mirros
- )
- return mirros
-
- def _fork_repository(self, r, destination, source, locale, mirrors):
- """Creates clone of repository and stores it in mirrors."""
- new_repository = self.driver.fork_repository(
- self.context.connection, r, destination, source, locale
- )
- mirrors[r] = new_repository
-
- def _copy_package(self, target, package, observer):
- """Synchronises remote file to local fs."""
- dst_path = os.path.join(target.url, package.filename)
- src_path = urljoin(package.repository.url, package.filename)
- bytes_copied = self.context.connection.retrieve(
- src_path, dst_path, size=package.filesize
- )
- if package.filesize < 0:
- package.filesize = bytes_copied
- observer(bytes_copied)
diff --git a/packetary/drivers/__init__.py b/packetary/drivers/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/packetary/drivers/base.py b/packetary/drivers/base.py
deleted file mode 100644
index 3175acb..0000000
--- a/packetary/drivers/base.py
+++ /dev/null
@@ -1,82 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import abc
-import logging
-
-import six
-
-
-@six.add_metaclass(abc.ABCMeta)
-class RepositoryDriverBase(object):
- """The super class for Repository Drivers.
-
- For implementing support of new type of repository:
- - inherit this class
- - implement all abstract methods
- - register implementation in 'packetary.drivers' namespace
- """
- def __init__(self):
- self.logger = logging.getLogger(__package__)
-
- @abc.abstractmethod
- def parse_urls(self, urls):
- """Parses the repository url.
-
- :return: the sequence of parsed urls
- """
-
- @abc.abstractmethod
- def get_repository(self, connection, url, arch, consumer):
- """Loads the repository meta information from URL.
-
- :param connection: the connection manager instance
- :param url: the repository`s url
- :param arch: the repository`s architecture
- :param consumer: the callback to consume result
- """
-
- @abc.abstractmethod
- def get_packages(self, connection, repository, consumer):
- """Loads packages from repository.
-
- :param connection: the connection manager instance
- :param repository: the repository object
- :param consumer: the callback to consume result
- """
-
- @abc.abstractmethod
- def fork_repository(self, connection, repository, destination,
- source=False, locale=False):
- """Creates the new repository with same metadata.
-
- :param connection: the connection manager instance
- :param repository: the source repository
- :param destination: the destination folder
- :param source: copy source files
- :param locale: copy localisation
- :return: The copy of repository
- """
-
- @abc.abstractmethod
- def rebuild_repository(self, repository, packages):
- """Re-builds the repository.
-
- :param repository: the target repository
- :param packages: the set of packages
- """
diff --git a/packetary/drivers/deb_driver.py b/packetary/drivers/deb_driver.py
deleted file mode 100644
index 030f318..0000000
--- a/packetary/drivers/deb_driver.py
+++ /dev/null
@@ -1,374 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from contextlib import closing
-import copy
-import datetime
-import fcntl
-import gzip
-import os
-
-from debian import deb822
-from debian import debfile
-from debian.debian_support import Version
-import six
-
-from packetary.drivers.base import RepositoryDriverBase
-from packetary.library.checksum import composite as checksum_composite
-from packetary.library.streams import GzipDecompress
-from packetary.library import utils
-from packetary.objects import FileChecksum
-from packetary.objects import Package
-from packetary.objects import PackageRelation
-from packetary.objects import Repository
-
-
-_OPERATORS_MAPPING = {
- '>>': 'gt',
- '<<': 'lt',
- '=': 'eq',
- '>=': 'ge',
- '<=': 'le',
-}
-
-_ARCHITECTURES = {
- "x86_64": "amd64",
- "i386": "i386",
- "source": "Source",
- "amd64": "x86_64",
-}
-
-_PRIORITIES = {
- "required": 1,
- "important": 2,
- "standard": 3,
- "optional": 4,
- "extra": 5
-}
-
-# Order is important
-_REPOSITORY_FILES = [
- "Packages",
- "Release",
- "Packages.gz"
-]
-
-# TODO(should be configurable)
-_MANDATORY_PRIORITY = 3
-
-_CHECKSUM_METHODS = (
- "MD5Sum",
- "SHA1",
- "SHA256"
-)
-
-_checksum_collector = checksum_composite('md5', 'sha1', 'sha256')
-
-
-class DebRepositoryDriver(RepositoryDriverBase):
- def parse_urls(self, urls):
- """Overrides method of superclass."""
- for url in urls:
- try:
- tokens = iter(x for x in url.split(" ") if x)
- base, suite = next(tokens), next(tokens)
- components = list(tokens)
- except StopIteration:
- raise ValueError("Invalid url: {0}".format(url))
-
- base = base.rstrip("/")
- if base.endswith("/dists"):
- base = base[:-6]
-
- # TODO(Flat Repository Format[1])
- # [1] https://wiki.debian.org/RepositoryFormat
- for component in components:
- yield (base, suite, component)
-
- def get_repository(self, connection, url, arch, consumer):
- """Overrides method of superclass."""
-
- base, suite, component = url
- release = self._get_url_of_metafile(
- (base, suite, component, arch), "Release"
- )
- deb_release = deb822.Release(connection.open_stream(release))
- consumer(Repository(
- name=(deb_release["Archive"], deb_release["Component"]),
- architecture=arch,
- origin=deb_release["origin"],
- url=base + "/"
- ))
-
- def get_packages(self, connection, repository, consumer):
- """Overrides method of superclass."""
- index = self._get_url_of_metafile(repository, "Packages.gz")
- stream = GzipDecompress(connection.open_stream(index))
- self.logger.info("loading packages from %s ...", repository)
- pkg_iter = deb822.Packages.iter_paragraphs(stream)
- counter = 0
- for dpkg in pkg_iter:
- try:
- consumer(Package(
- repository=repository,
- name=dpkg["package"],
- version=Version(dpkg['version']),
- filesize=int(dpkg.get('size', -1)),
- filename=dpkg["filename"],
- checksum=FileChecksum(
- md5=dpkg.get("md5sum"),
- sha1=dpkg.get("sha1"),
- sha256=dpkg.get("sha256"),
- ),
- mandatory=self._is_mandatory(dpkg),
- # Recommends are installed by default (since Lucid)
- requires=self._get_relations(
- dpkg, "depends", "pre-depends", "recommends"
- ),
- # The deb does not have obsoletes section
- obsoletes=[],
- provides=self._get_relations(dpkg, "provides"),
- ))
- except KeyError as e:
- self.logger.error(
- "Malformed index %s - %s: %s",
- repository, six.text_type(dpkg), six.text_type(e)
- )
- raise
- counter += 1
-
- self.logger.info("loaded: %d packages from %s.", counter, repository)
-
- def rebuild_repository(self, repository, packages):
- """Overrides method of superclass."""
- basedir = utils.get_path_from_url(repository.url)
- index_file = utils.get_path_from_url(
- self._get_url_of_metafile(repository, "Packages")
- )
- utils.ensure_dir_exist(os.path.dirname(index_file))
- index_gz = index_file + ".gz"
- count = 0
- with open(index_file, "wb") as fd1:
- with closing(gzip.open(index_gz, "wb")) as fd2:
- writer = utils.composite_writer(fd1, fd2)
- for pkg in packages:
- filename = os.path.join(basedir, pkg.filename)
- with closing(debfile.DebFile(filename)) as deb:
- debcontrol = deb.debcontrol()
- debcontrol.setdefault("Origin", repository.origin)
- debcontrol["Size"] = str(pkg.filesize)
- debcontrol["Filename"] = pkg.filename
- for k, v in six.moves.zip(_CHECKSUM_METHODS, pkg.checksum):
- debcontrol[k] = v
- writer(debcontrol.dump())
- writer("\n")
- count += 1
- self.logger.info("saved %d packages in %s", count, repository)
- self._update_suite_index(repository)
-
- def fork_repository(self, connection, repository, destination,
- source=False, locale=False):
- # TODO(download gpk)
- # TODO(sources and locales)
- new_repo = copy.copy(repository)
- new_repo.url = utils.localize_repo_url(destination, repository.url)
- packages_file = utils.get_path_from_url(
- self._get_url_of_metafile(new_repo, "Packages")
- )
- release_file = utils.get_path_from_url(
- self._get_url_of_metafile(new_repo, "Release")
- )
- self.logger.info(
- "clone repository %s to %s", repository, new_repo.url
- )
- utils.ensure_dir_exist(os.path.dirname(release_file))
-
- release = deb822.Release()
- release["Origin"] = repository.origin
- release["Label"] = repository.origin
- release["Archive"] = repository.name[0]
- release["Component"] = repository.name[1]
- release["Architecture"] = _ARCHITECTURES[repository.architecture]
- with open(release_file, "wb") as fd:
- release.dump(fd)
-
- open(packages_file, "ab").close()
- gzip.open(packages_file + ".gz", "ab").close()
- return new_repo
-
- def _update_suite_index(self, repository):
- """Updates the Release file in the suite."""
- path = os.path.join(
- utils.get_path_from_url(repository.url),
- "dists", repository.name[0]
- )
- release_path = os.path.join(path, "Release")
- self.logger.info(
- "added repository suite release file: %s", release_path
- )
- with open(release_path, "a+b") as fd:
- fcntl.flock(fd.fileno(), fcntl.LOCK_EX)
- try:
- fd.seek(0)
- release = deb822.Release(fd)
- self._add_to_release(release, repository)
- for m in _CHECKSUM_METHODS:
- release.setdefault(m, [])
-
- self._add_files_to_release(
- release, path, self._get_metafiles(repository)
- )
-
- fd.truncate(0)
- release.dump(fd)
- finally:
- fcntl.flock(fd.fileno(), fcntl.LOCK_UN)
-
- def _get_relations(self, dpkg, *names):
- """Gets the package relations.
-
- :param dpkg: the debian-package object
- :type dpkg: deb822.Packages
- :param names: the relation names
- :return: the list of PackageRelation objects
- """
- relations = list()
- for name in names:
- for variants in dpkg.relations[name]:
- relation = PackageRelation.from_args(
- *(self._unparse_relation(v) for v in variants)
- )
- if relation is not None:
- relations.append(relation)
- return relations
-
- def _get_metafiles(self, repository):
- """Gets the sequence of metafiles for repository."""
- return (
- utils.get_path_from_url(
- self._get_url_of_metafile(repository, filename)
- )
- for filename in _REPOSITORY_FILES
-
- )
-
- @staticmethod
- def _unparse_relation(relation):
- """Gets the relation parameters.
-
- :param relation: the deb822.Releation object
- :return: tuple(name, version_compare, version_edge)
- """
- name = relation['name']
- version = relation.get("version")
- if version is None:
- return name, None
- else:
- return name, _OPERATORS_MAPPING[version[0]], version[1]
-
- @staticmethod
- def _is_mandatory(dpkg):
- """Checks that package is mandatory.
-
- :param dpkg: the debian-package object
- :type dpkg: deb822.Packages
- """
- if dpkg.get("essential") == "yes":
- return True
-
- return _PRIORITIES.get(
- dpkg.get("priority"), _MANDATORY_PRIORITY + 1
- ) < _MANDATORY_PRIORITY
-
- @staticmethod
- def _get_url_of_metafile(repo_or_comps, filename):
- """Gets the URL of meta-file.
-
- :param repo_or_comps: the repository object or
- tuple(baseurl, suite, component, architecture)
- :param filename: the name of meta-file
- """
- if isinstance(repo_or_comps, Repository):
- baseurl = repo_or_comps.url
- suite, component = repo_or_comps.name
- arch = repo_or_comps.architecture
- else:
- baseurl, suite, component, arch = repo_or_comps
-
- return "/".join((
- baseurl.rstrip("/"), "dists", suite, component,
- "binary-" + _ARCHITECTURES[arch],
- filename
- ))
-
- @staticmethod
- def _add_to_release(release, repository):
- """Adds repository information to debian release.
-
- :param release: the deb822.Release instance
- :param repository: the repository object
- """
-
- # reset the date
- release["Date"] = datetime.datetime.now().strftime(
- "%a, %d %b %Y %H:%M:%S %Z"
- )
- release.setdefault("Origin", repository.origin)
- release.setdefault("Label", repository.origin)
- release.setdefault("Suite", repository.name[0])
- release.setdefault("Codename", repository.name[0].split("-", 1)[0])
- release.setdefault("Description", "The packages repository.")
-
- keys = ("Architectures", "Components")
- values = (repository.architecture, repository.name[1])
- for key, value in six.moves.zip(keys, values):
- if key in release:
- release[key] = utils.append_token_to_string(
- release[key],
- value
- )
- else:
- release[key] = value
-
- @staticmethod
- def _add_files_to_release(release, basepath, files):
- """Adds information about meta files to debian release.
-
- :param release: the deb822.Release instance
- :param basepath: the suite folder path
- :param files: the sequence of files
- """
-
- files_info = utils.get_size_and_checksum_for_files(
- files, _checksum_collector
- )
- for filepath, size, cs in files_info:
- fname = filepath[len(basepath) + 1:]
- size = six.text_type(size)
- for m, checksum in six.moves.zip(_CHECKSUM_METHODS, cs):
- for v in release[m]:
- if v["name"] == fname:
- v[m] = checksum
- v["size"] = size
- break
- else:
- release[m].append(deb822.Deb822Dict({
- m: checksum,
- "size": size,
- "name": fname
- }))
diff --git a/packetary/drivers/rpm_driver.py b/packetary/drivers/rpm_driver.py
deleted file mode 100644
index 8453e6e..0000000
--- a/packetary/drivers/rpm_driver.py
+++ /dev/null
@@ -1,283 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import copy
-import multiprocessing
-import os
-import shutil
-
-import createrepo
-import lxml.etree as etree
-import six
-
-from packetary.drivers.base import RepositoryDriverBase
-from packetary.library.streams import GzipDecompress
-from packetary.library import utils
-from packetary.objects import FileChecksum
-from packetary.objects import Package
-from packetary.objects import PackageRelation
-from packetary.objects import PackageVersion
-from packetary.objects import Repository
-
-
-urljoin = six.moves.urllib.parse.urljoin
-
-# TODO(configurable option for drivers)
-_CORE_GROUPS = ("core", "base")
-
-_MANDATORY_TYPES = ("mandatory", "default")
-
-# The namespaces are used in metadata xml of repository
-_NAMESPACES = {
- "main": "http://linux.duke.edu/metadata/common",
- "md": "http://linux.duke.edu/metadata/repo",
- "rpm": "http://linux.duke.edu/metadata/rpm"
-}
-
-
-class CreaterepoCallBack(object):
- """Callback object for createrepo"""
- def __init__(self, logger):
- self.logger = logger
-
- def errorlog(self, msg):
- """Error log output."""
- self.logger.error(msg)
-
- def log(self, msg):
- """Logs message."""
- self.logger.info(msg)
-
- def progress(self, item, current, total):
- """"Progress bar."""
- pass
-
-
-class RpmRepositoryDriver(RepositoryDriverBase):
- def parse_urls(self, urls):
- """Overrides method of superclass."""
- return (url.rstrip("/") for url in urls)
-
- def get_repository(self, connection, url, arch, consumer):
- name = utils.get_path_from_url(url, False)
- consumer(Repository(
- name=name,
- url=url + "/",
- architecture=arch,
- origin=""
- ))
-
- def get_packages(self, connection, repository, consumer):
- """Overrides method of superclass."""
- baseurl = repository.url
- repomd = urljoin(baseurl, "repodata/repomd.xml")
- self.logger.debug("repomd: %s", repomd)
-
- repomd_tree = etree.parse(connection.open_stream(repomd))
- mandatory = self._get_mandatory_packages(
- self._load_db(
- connection, baseurl, repomd_tree, "group_gz", "group"
- )
- )
- primary_db = self._load_db(connection, baseurl, repomd_tree, "primary")
- if primary_db is None:
- raise ValueError("Malformed repository: {0}".format(repository))
-
- counter = 0
- for tag in primary_db.iterfind("./main:package", _NAMESPACES):
- try:
- name = tag.find("./main:name", _NAMESPACES).text
- consumer(Package(
- repository=repository,
- name=tag.find("./main:name", _NAMESPACES).text,
- version=self._unparse_version_attrs(
- tag.find("./main:version", _NAMESPACES).attrib
- ),
- filesize=int(
- tag.find("./main:size", _NAMESPACES)
- .attrib.get("package", -1)
- ),
- filename=tag.find(
- "./main:location", _NAMESPACES
- ).attrib["href"],
- checksum=self._get_checksum(tag),
- mandatory=name in mandatory,
- requires=self._get_relations(tag, "requires"),
- obsoletes=self._get_relations(tag, "obsoletes"),
- provides=self._get_relations(tag, "provides")
- ))
- except (ValueError, KeyError) as e:
- self.logger.error(
- "Malformed tag %s - %s: %s",
- repository, etree.tostring(tag), six.text_type(e)
- )
- raise
- counter += 1
- self.logger.info("loaded: %d packages from %s.", counter, repository)
-
- def rebuild_repository(self, repository, packages):
- """Overrides method of superclass."""
- basepath = utils.get_path_from_url(repository.url)
- self.logger.info("rebuild repository in %s", basepath)
- md_config = createrepo.MetaDataConfig()
- try:
- md_config.workers = multiprocessing.cpu_count()
- md_config.directory = str(basepath)
- md_config.update = True
- mdgen = createrepo.MetaDataGenerator(
- config_obj=md_config, callback=CreaterepoCallBack(self.logger)
- )
- mdgen.doPkgMetadata()
- mdgen.doRepoMetadata()
- mdgen.doFinalMove()
- except createrepo.MDError as e:
- err_msg = six.text_type(e)
- self.logger.exception(
- "failed to create yum repository in %s: %s",
- basepath,
- err_msg
- )
- shutil.rmtree(
- os.path.join(md_config.outputdir, md_config.tempdir),
- ignore_errors=True
- )
- raise RuntimeError(
- "Failed to create yum repository in {0}."
- .format(err_msg))
-
- def fork_repository(self, connection, repository, destination,
- source=False, locale=False):
- # TODO(download gpk)
- # TODO(sources and locales)
- new_repo = copy.copy(repository)
- new_repo.url = utils.localize_repo_url(destination, repository.url)
- self.logger.info(
- "clone repository %s to %s", repository, new_repo.url
- )
- utils.ensure_dir_exist(new_repo.url)
- self.rebuild_repository(new_repo, set())
- return new_repo
-
- def _load_db(self, connection, baseurl, repomd, *aliases):
- """Loads database.
-
- :param connection: the connection object
- :param baseurl: the base repository URL
- :param repomd: the parsed metadata of repository
- :param aliases: the aliases of database name
- :return: parsed database file or None if db does not exist
- """
-
- for dbname in aliases:
- self.logger.debug("loading %s database...", dbname)
- node = repomd.find(
- "./md:data[@type='{0}']".format(dbname), _NAMESPACES
- )
- if node is not None:
- break
- else:
- return
-
- url = urljoin(
- baseurl,
- node.find("./md:location", _NAMESPACES).attrib["href"]
- )
- self.logger.debug("loading %s - %s...", dbname, url)
- stream = connection.open_stream(url)
- if url.endswith(".gz"):
- stream = GzipDecompress(stream)
- return etree.parse(stream)
-
- def _get_mandatory_packages(self, groups_db):
- """Get the set of mandatory package names.
-
- :param groups_db: the parsed groups database
- """
- package_names = set()
- if groups_db is None:
- return package_names
- count = 0
- for name in _CORE_GROUPS:
- result = groups_db.xpath("./group/id[text()='{0}']".format(name))
- if len(result) == 0:
- self.logger.warning("the group '%s' is not found.", name)
- continue
- group = result[0].getparent()
- for t in _MANDATORY_TYPES:
- xpath = "./packagelist/packagereq[@type='{0}']".format(t)
- for tag in group.iterfind(xpath):
- package_names.add(tag.text)
- count += 1
- self.logger.info("detected %d mandatory packages.", count)
- return package_names
-
- def _get_relations(self, pkg_tag, name):
- """Gets package relations by name from package tag.
-
- :param pkg_tag: the xml-tag with package description
- :param name: the relations name
- :return: list of PackageRelation objects
- """
- relations = list()
- append = relations.append
- tags_iter = pkg_tag.iterfind(
- "./main:format/rpm:%s/rpm:entry" % name,
- _NAMESPACES
- )
- for elem in tags_iter:
- append(PackageRelation.from_args(
- self._unparse_relation_attrs(elem.attrib)
- ))
-
- return relations
-
- def _get_checksum(self, pkg_tag):
- """Gets checksum from package tag."""
- checksum = dict.fromkeys(("md5", "sha1", "sha256"), None)
- checksum_tag = pkg_tag.find("./main:checksum", _NAMESPACES)
- checksum[checksum_tag.attrib["type"]] = checksum_tag.text
- return FileChecksum(**checksum)
-
- def _unparse_relation_attrs(self, attrs):
- """Gets the package relation from attributes.
-
- :param attrs: the relation tag attributes
- :return tuple(name, version_op, version_edge)
- """
- if "flags" not in attrs:
- return attrs['name'], None
-
- return (
- attrs['name'],
- attrs["flags"].lower(),
- self._unparse_version_attrs(attrs)
- )
-
- @staticmethod
- def _unparse_version_attrs(attrs):
- """Gets the package version from attributes.
-
- :param attrs: the relation tag attributes
- :return: the PackageVersion object
- """
-
- return PackageVersion(
- int(attrs.get("epoch", 0)),
- attrs.get("ver", "0.0").split("."),
- attrs.get("rel", "0").split(".")
- )
diff --git a/packetary/library/__init__.py b/packetary/library/__init__.py
deleted file mode 100644
index 84e9ae6..0000000
--- a/packetary/library/__init__.py
+++ /dev/null
@@ -1,21 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import eventlet
-
-eventlet.monkey_patch()
diff --git a/packetary/library/checksum.py b/packetary/library/checksum.py
deleted file mode 100644
index d8b6b73..0000000
--- a/packetary/library/checksum.py
+++ /dev/null
@@ -1,84 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import functools
-import hashlib
-
-
-class _HashComposite(object):
- """Combines several hash methods."""
-
- def __init__(self, hash_objects):
- self.hash_objects = hash_objects
-
- def update(self, data):
- """Updates the hash objects with the string arg.
-
- For more details see doc of hashlib.update.
- """
- for o in self.hash_objects:
- o.update(data)
-
- def hexdigest(self):
- """Returns the list of appropriate hexdigests of hash_objects.
-
- For more details see doc of hashlib.hexdigest.
- """
- return [o.hexdigest() for o in self.hash_objects]
-
-
-def _new_composite(methods):
- """Creates new composite method."""
-
- def wrapper():
- return _HashComposite([x() for x in methods])
- return wrapper
-
-
-def _checksum(method):
- """Makes function to calculate checksum for stream."""
- @functools.wraps(method)
- def calculate(stream, chunksize=16 * 1024):
- """Calculates checksum for binary stream.
-
- :param stream: file-like object opened in binary mode.
- :return: the checksum of content in terms of method.
- """
-
- s = method()
- while True:
- chunk = stream.read(chunksize)
- if not chunk:
- break
- s.update(chunk)
- return s.hexdigest()
- return calculate
-
-
-md5 = _checksum(hashlib.md5)
-
-sha1 = _checksum(hashlib.sha1)
-
-sha256 = _checksum(hashlib.sha256)
-
-
-def composite(*methods):
- """Calculate several checksum at one time."""
- return _checksum(_new_composite(
- [getattr(hashlib, x) for x in methods]
- ))
diff --git a/packetary/library/connections.py b/packetary/library/connections.py
deleted file mode 100644
index 49b6c9b..0000000
--- a/packetary/library/connections.py
+++ /dev/null
@@ -1,307 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import errno
-import logging
-import os
-import six
-import six.moves.http_client as http_client
-import six.moves.urllib.request as urllib
-import six.moves.urllib_error as urlerror
-import time
-
-from packetary.library.streams import StreamWrapper
-from packetary.library.utils import ensure_dir_exist
-
-
-logger = logging.getLogger(__package__)
-
-
-RETRYABLE_ERRORS = (http_client.HTTPException, IOError)
-
-
-class RangeError(urlerror.URLError):
- pass
-
-
-class RetryableRequest(urllib.Request):
- MAX_TIMEOUT = 5
-
- offset = 0
- retries_left = 1
- retry_interval = 0
- start_time = 0
-
- def get_retry_interval(self):
- """Calculates progressive retry interval in seconds.
-
- :return: the time to wait before start retry
- """
- # we uses progressive timeout between retries,
- # the greatest number of retry will have greatest timeout
- # but limited with max_delay
- coef = max(self.MAX_TIMEOUT - self.retries_left, 1)
- timeout = self.retry_interval * coef
- return min(timeout, self.MAX_TIMEOUT)
-
-
-class ResumableResponse(StreamWrapper):
- """The http-response wrapper to add resume ability.
-
- Allows to resume read from same position if connection is lost.
- """
-
- def __init__(self, request, response, opener):
- """Initialises.
-
- :param request: the original http request
- :param response: the original http response
- :param opener: the instance of urllib.OpenerDirector
- """
- super(ResumableResponse, self).__init__(response)
- self.request = request
- self.opener = opener
-
- def read_chunk(self, chunksize):
- """Overrides super class method."""
- while 1:
- try:
- chunk = self.stream.read(chunksize)
- self.request.offset += len(chunk)
- return chunk
- except RETRYABLE_ERRORS as e:
- # TODO(check hashsums)
- response = self.opener.error(
- self.request.get_type(), self.request,
- self.stream, 502, six.text_type(e), self.stream.info()
- )
- self.stream = response.stream
-
-
-class RetryHandler(urllib.HTTPRedirectHandler):
- """urllib Handler to add ability for retrying on server errors."""
-
- def redirect_request(self, req, fp, code, msg, headers, newurl):
- new_req = urllib.HTTPRedirectHandler.redirect_request(
- self, req, fp, code, msg, headers, newurl
- )
- if new_req is not None:
- # We use class assignment for casting new request to type
- # RetryableRequest
- new_req.__class__ = RetryableRequest
- new_req.retries_left = req.retries_left
- new_req.offset = req.offset
- new_req.start_time = req.start_time
- new_req.retry_interval = req.retry_interval
- return new_req
-
- @staticmethod
- def http_request(request):
- """Initialises http request.
-
- :param request: the instance of RetryableRequest
- :return: the request
- """
- logger.debug("start request: %s", request.get_full_url())
- if request.offset > 0:
- request.add_header('Range', 'bytes=%d-' % request.offset)
- request.start_time = time.time()
- return request
-
- def http_response(self, request, response):
- """Wraps response in a ResumableResponse.
-
- Checks that partial request completed successfully.
- :param request: the instance of RetryableRequest
- :param response: the response object
- :return: ResumableResponse if success otherwise same response
- """
- code, msg = response.getcode(), response.msg
-
- if 300 <= code < 400:
- # the redirect group, pass to next handler as is
- return response
-
- # the server should response partial content if range is specified
- if request.offset > 0 and code != 206:
- raise RangeError(msg)
-
- if code >= 400:
- logger.error(
- "request failed: %s - %d(%s), retries left - %d.",
- request.get_full_url(), code, msg, request.retries_left - 1
- )
- if is_retryable_http_error(code) and request.retries_left > 0:
- time.sleep(request.get_retry_interval())
- request.retries_left -= 1
- response = self.parent.open(request)
- # pass response to next handler as is.
- return response
-
- logger.debug(
- "request completed: %s - %d (%s), duration - %d ms.",
- request.get_full_url(), response.getcode(), response.msg,
- int((time.time() - request.start_time) * 1000)
- )
-
- return ResumableResponse(request, response, self.parent)
-
- https_request = http_request
- https_response = http_response
-
-
-def is_retryable_http_error(code):
- """Checks that http error can be retried.
-
- :param code: the HTTP_CODE
- :return: True if request can be retried otherwise False
- """
- return code >= http_client.INTERNAL_SERVER_ERROR
-
-
-class ConnectionsManager(object):
- """The connections manager."""
-
- def __init__(self, proxy=None, secure_proxy=None,
- retries_num=0, retry_interval=0):
- """Initialises.
-
- :param proxy: the url of proxy for http-connections
- :param secure_proxy: the url of proxy for https-connections
- :param retries_num: the number of allowed retries
- :param retry_interval: the time between retries (in seconds)
- """
- if proxy:
- proxies = {
- "http": proxy,
- "https": secure_proxy or proxy,
- }
- else:
- proxies = None
-
- self.retries_num = retries_num
- self.retry_interval = retry_interval
- self.opener = urllib.build_opener(
- RetryHandler(),
- urllib.ProxyHandler(proxies)
- )
-
- def make_request(self, url, offset=0):
- """Makes new http request.
-
- :param url: the remote file`s url
- :param offset: the number of bytes from the beginning,
- that will be skipped
- :return: The new http request
- """
-
- if url.startswith("/"):
- url = "file://" + url
-
- request = RetryableRequest(url)
- request.retries_left = self.retries_num
- request.retry_interval = self.retry_interval
- request.offset = offset
- return request
-
- def open_stream(self, url, offset=0):
- """Opens remote file for streaming.
-
- :param url: the remote file`s url
- :param offset: the number of bytes from the beginning,
- that will be skipped
- """
-
- request = self.make_request(url, offset)
- while 1:
- try:
- return self.opener.open(request)
- except (RangeError, urlerror.HTTPError):
- raise
- except RETRYABLE_ERRORS as e:
- if request.retries_left <= 0:
- raise
- request.retries_left -= 1
- logger.exception(
- "Failed to open url - %s: %s. retries left - %d.",
- url, six.text_type(e), request.retries_left
- )
- time.sleep(request.get_retry_interval())
-
- def retrieve(self, url, filename, **attributes):
- """Downloads remote file.
-
- :param url: the remote file`s url
- :param filename: the target filename on local filesystem
- :param attributes: the file attributes, like size, hashsum, etc.
- :return: the count of actually copied bytes
- """
- offset = 0
- try:
- stats = os.stat(filename)
- expected_size = attributes.get('size', -1)
- if expected_size == stats.st_size:
- # TODO(check hashsum)
- return 0
-
- if stats.st_size < expected_size:
- offset = stats.st_size
- except OSError as e:
- if e.errno != errno.ENOENT:
- raise
- ensure_dir_exist(os.path.dirname(filename))
-
- logger.info("download: %s from the offset: %d", url, offset)
-
- fd = os.open(filename, os.O_CREAT | os.O_WRONLY)
- try:
- return self._copy_stream(fd, url, offset)
- except RangeError:
- if offset == 0:
- raise
- logger.warning(
- "Failed to resume download, starts from the beginning: %s",
- url
- )
- return self._copy_stream(fd, url, 0)
- finally:
- os.fsync(fd)
- os.close(fd)
-
- def _copy_stream(self, fd, url, offset):
- """Copies remote file to local.
-
- :param fd: the file`s descriptor
- :param url: the remote file`s url
- :param offset: the number of bytes from the beginning,
- that will be skipped
- :return: the count of actually copied bytes
- """
-
- source = self.open_stream(url, offset)
- os.ftruncate(fd, offset)
- os.lseek(fd, offset, os.SEEK_SET)
- chunk_size = 16 * 1024
- size = 0
- while 1:
- chunk = source.read(chunk_size)
- if not chunk:
- break
- os.write(fd, chunk)
- size += len(chunk)
- return size
diff --git a/packetary/library/executor.py b/packetary/library/executor.py
deleted file mode 100644
index 3e5ef47..0000000
--- a/packetary/library/executor.py
+++ /dev/null
@@ -1,93 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from __future__ import with_statement
-
-import logging
-import sys
-
-import six
-
-from eventlet.greenpool import GreenPool
-
-
-logger = logging.getLogger(__package__)
-
-
-class AsynchronousSection(object):
- """Allows calling function asynchronously with waiting on exit."""
-
- MIN_POOL_SIZE = 1
-
- def __init__(self, size=0, ignore_errors_num=0):
- """Initialises.
-
- :param size: the max number of parallel tasks
- :param ignore_errors_num:
- number of errors which does not stop the execution
- """
-
- self.executor = GreenPool(max(size, self.MIN_POOL_SIZE))
- self.ignore_errors_num = ignore_errors_num
- self.errors = []
- self.tasks = set()
-
- def __enter__(self):
- self.errors[:] = []
- return self
-
- def __exit__(self, etype, *_):
- self.wait(etype is not None)
-
- def execute(self, func, *args, **kwargs):
- """Calls function asynchronously."""
- if 0 <= self.ignore_errors_num < len(self.errors):
- raise RuntimeError("Too many errors.")
-
- gt = self.executor.spawn(func, *args, **kwargs)
- self.tasks.add(gt)
- gt.link(self.on_complete)
- return gt
-
- def on_complete(self, gt):
- """Callback to handle task completion."""
-
- try:
- gt.wait()
- except Exception as e:
- logger.error("Task failed: %s", six.text_type(e))
- self.errors.append(sys.exc_info())
- finally:
- self.tasks.discard(gt)
-
- def wait(self, ignore_errors=False):
- """Waits until all tasks will be completed.
-
- Do not use directly, will be called from context manager.
- """
- self.executor.waitall()
- if len(self.errors) > 0:
- for exc_info in self.errors:
- logger.exception("error details.", exc_info=exc_info)
-
- self.errors[:] = []
- if not ignore_errors:
- raise RuntimeError(
- "Operations completed with errors.\n"
- "See log for more details."
- )
diff --git a/packetary/library/streams.py b/packetary/library/streams.py
deleted file mode 100644
index 5c6f4bf..0000000
--- a/packetary/library/streams.py
+++ /dev/null
@@ -1,133 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import zlib
-
-
-class StreamWrapper(object):
- """Helper class to implement stream wrappers.
-
- It is base-class for Streamers,
- that provides functionality to transform stream on the fly.
- The wrapped stream may return data more that required,
- the extra read data will be kept in the internal buffer till
- next read.
- """
-
- CHUNK_SIZE = 1024
-
- def __init__(self, stream):
- """Initializes.
-
- :param stream: file-like object opened in binary mode.
- """
- self.stream = stream
- self.unread_tail = b""
-
- def __getattr__(self, item):
- return getattr(self.stream, item)
-
- def _read_tail(self):
- tmp = self.unread_tail
- self.unread_tail = b""
- return tmp
-
- def _align_chunk(self, chunk, size):
- self.unread_tail = chunk[size:]
- return chunk[:size]
-
- def read_chunk(self, chunksize):
- """Overrides this method to change default behaviour."""
- return self.stream.read(chunksize)
-
- def read(self, size=-1):
- result = self._read_tail()
- if size < 0:
- while True:
- chunk = self.read_chunk(self.CHUNK_SIZE)
- if not chunk:
- break
- result += chunk
- else:
- if len(result) > size:
- result = self._align_chunk(result, size)
- size -= len(result)
- while size > 0:
- chunk = self.read_chunk(self.CHUNK_SIZE)
- if not chunk:
- break
- if len(chunk) > size:
- chunk = self._align_chunk(chunk, size)
- size -= len(chunk)
- result += chunk
- return result
-
- def readline(self):
- pos = self.unread_tail.find(b"\n")
- if pos >= 0:
- line = self._align_chunk(self.unread_tail, pos + 1)
- else:
- line = self._read_tail()
- while True:
- chunk = self.read_chunk(self.CHUNK_SIZE)
- if not chunk:
- break
- pos = chunk.find(b"\n")
- if pos >= 0:
- line += self._align_chunk(chunk, pos + 1)
- break
- line += chunk
- return line
-
- def readlines(self):
- while True:
- line = self.readline()
- if not line:
- break
- yield line
-
- def __iter__(self):
- return self.readlines()
-
-
-class GzipDecompress(StreamWrapper):
- """The decompress stream."""
-
- def __init__(self, stream):
- super(GzipDecompress, self).__init__(stream)
- # Magic parameter makes zlib module understand gzip header
- # http://stackoverflow.com/questions/1838699/how-can-i-decompress-a-gzip-stream-with-zlib
- # This works on cpython and pypy, but not jython.
- self.decompress = zlib.decompressobj(16 + zlib.MAX_WBITS)
-
- def read_chunk(self, chunksize):
- if self.decompress.unconsumed_tail:
- uncompressed = self.decompress.decompress(
- self.decompress.unconsumed_tail, chunksize
- )
- if uncompressed:
- return uncompressed
-
- while True:
- chunk = self.stream.read(chunksize)
- if not chunk:
- break
- uncompressed = self.decompress.decompress(chunk, chunksize)
- if uncompressed:
- return uncompressed
- return self.decompress.flush()
diff --git a/packetary/library/utils.py b/packetary/library/utils.py
deleted file mode 100644
index 350892f..0000000
--- a/packetary/library/utils.py
+++ /dev/null
@@ -1,115 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from __future__ import with_statement
-
-import errno
-import os
-
-import six
-
-urlparse = six.moves.urllib.parse.urlparse
-
-
-def append_token_to_string(tokens, token):
- """Adds new token to space separated list of tokens.
-
- :param tokens: the 'sep' separated list
- :param token: new item
- """
- values = tokens.split()
- if token not in values:
- values.append(token)
- values.sort()
- return ' '.join(values)
- return tokens
-
-
-def composite_writer(*args):
- """Makes helper, that writes into several files simultaneously.
-
- :param args: the list of file objects
- :return: the callable object - writer
- """
- def write(text):
- """Writes simultaneously to all files with utf-8 encoding control.
-
- :param text: the text, that needs to write
- """
- if isinstance(text, six.text_type):
- text = text.encode("utf-8")
- for arg in args:
- arg.write(text)
- return write
-
-
-def get_size_and_checksum_for_files(files, checksum_algo):
- """Gets the path, size and checksum for files.
-
- :param files: the sequence of files
- :param checksum_algo: the checksum calculator
- :return the sequence of tuples(filename, size, checksum)
- """
-
- for filename in files:
- with open(filename, "rb") as fd:
- size = os.fstat(fd.fileno()).st_size
- checksum = checksum_algo(fd)
- yield filename, size, checksum
-
-
-def get_path_from_url(url, ensure_file=True):
- """Get the path from the URL.
-
- :param url: the URL
- :param ensure_file: If True, ensure that scheme is "file"
- :return: the path component from URL
- :raises ValueError
- """
-
- comps = urlparse(url, scheme="file")
- if ensure_file and comps.scheme != "file":
- raise ValueError(
- "The absolute path is expected, actual have: {0}.".format(url)
- )
- if os.sep != "/":
- return comps.path.replace("/", os.sep)
- return comps.path
-
-
-def localize_repo_url(localurl, repo_url):
- """Gets local repository url.
-
- :param localurl: the base local URL
- :param repo_url: the origin URL of repository
- :return: localurl + get_path_from_url(repo_url)
- """
- return localurl.rstrip("/") + urlparse(repo_url).path
-
-
-def ensure_dir_exist(path):
- """Creates directory if it does not exist.
-
- :param path: the full path to directory
- """
-
- try:
- os.makedirs(path)
- except OSError as e:
- if e.errno != errno.EEXIST:
- raise
diff --git a/packetary/objects/__init__.py b/packetary/objects/__init__.py
deleted file mode 100644
index e1b4e5b..0000000
--- a/packetary/objects/__init__.py
+++ /dev/null
@@ -1,38 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from packetary.objects.index import Index
-from packetary.objects.package import FileChecksum
-from packetary.objects.package import Package
-from packetary.objects.package_relation import PackageRelation
-from packetary.objects.package_relation import VersionRange
-from packetary.objects.package_version import PackageVersion
-from packetary.objects.packages_tree import PackagesTree
-from packetary.objects.repository import Repository
-
-
-__all__ = [
- "FileChecksum",
- "Index",
- "Package",
- "PackageRelation",
- "PackagesTree",
- "PackageVersion",
- "Repository",
- "VersionRange",
-]
diff --git a/packetary/objects/base.py b/packetary/objects/base.py
deleted file mode 100644
index 2f3cf14..0000000
--- a/packetary/objects/base.py
+++ /dev/null
@@ -1,62 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import abc
-import six
-
-
-@six.add_metaclass(abc.ABCMeta)
-class ComparableObject(object):
- """Superclass for objects, that should be comparable.
-
- Note: because python3 does not support __cmp__ slot, use
- cmp method to implement all of compare methods.
- """
-
- @abc.abstractmethod
- def cmp(self, other):
- """Compares with other object.
-
- :return: value is negative if if self < other, zero if self == other
- strictly positive if x > y
- """
-
- def __lt__(self, other):
- return self.cmp(other) < 0
-
- def __le__(self, other):
- return self.cmp(other) <= 0
-
- def __gt__(self, other):
- return self.cmp(other) > 0
-
- def __ge__(self, other):
- return self.cmp(other) >= 0
-
- def __eq__(self, other):
- if other is self:
- return True
- return isinstance(other, type(self)) and self.cmp(other) == 0
-
- def __ne__(self, other):
- if other is self:
- return False
- return not isinstance(other, type(self)) or self.cmp(other) != 0
-
- def __cmp__(self, other):
- return self.cmp(other)
diff --git a/packetary/objects/index.py b/packetary/objects/index.py
deleted file mode 100644
index fc43fcd..0000000
--- a/packetary/objects/index.py
+++ /dev/null
@@ -1,208 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from bintrees import FastRBTree
-from collections import defaultdict
-import functools
-import operator
-import six
-
-
-def _make_operator(direction, op):
- """Makes search operator from low-level operation and search direction."""
- return functools.partial(direction, condition=op)
-
-
-def _start_upperbound(versions, version, condition):
- """Gets all versions from [start, version] that meet condition.
-
- :param versions: the tree of versions.
- :param version: the required version
- :param condition: condition for search
- :return: the list of found versions
- """
-
- result = list(versions.value_slice(None, version))
- try:
- bound = versions.ceiling_item(version)
- if condition(bound[0], version):
- result.append(bound[1])
- except KeyError:
- pass
- return result
-
-
-def _lowerbound_end(versions, version, condition):
- """Gets all versions from [version, end] that meet condition.
-
- :param versions: the tree of versions.
- :param version: the required version
- :param condition: condition for search
- :return: the list of found versions
- """
- result = []
- items = iter(versions.item_slice(version, None))
- bound = next(items, None)
- if bound is None:
- return result
- if condition(bound[0], version):
- result.append(bound[1])
- result.extend(x[1] for x in items)
- return result
-
-
-def _equal(tree, version):
- """Gets the package with specified version."""
- if version in tree:
- return [tree[version]]
- return []
-
-
-def _any(tree, _):
- """Gets the package with max version."""
- return list(tree.values())
-
-
-class Index(object):
- """The search index for packages.
-
- Builds three search-indexes:
- - index of packages with versions.
- - index of virtual packages (provides).
- - index of obsoleted packages (obsoletes).
-
- Uses to find package by name and range of versions.
- """
-
- operators = {
- None: _any,
- "lt": _make_operator(_start_upperbound, operator.lt),
- "le": _make_operator(_start_upperbound, operator.le),
- "gt": _make_operator(_lowerbound_end, operator.gt),
- "ge": _make_operator(_lowerbound_end, operator.ge),
- "eq": _equal,
- }
-
- def __init__(self):
- self.packages = defaultdict(FastRBTree)
- self.obsoletes = defaultdict(FastRBTree)
- self.provides = defaultdict(FastRBTree)
-
- def __iter__(self):
- """Iterates over all packages including versions."""
- return self.get_all()
-
- def __len__(self, _reduce=six.functools.reduce):
- """Returns the total number of packages with versions."""
- return _reduce(
- lambda x, y: x + len(y),
- six.itervalues(self.packages),
- 0
- )
-
- def get_all(self):
- """Gets sequence from all of packages including versions."""
-
- for versions in six.itervalues(self.packages):
- for version in versions.values():
- yield version
-
- def find(self, name, version):
- """Finds the package by name and range of versions.
-
- :param name: the package`s name.
- :param version: the range of versions.
- :return: the package if it is found, otherwise None
- """
- candidates = self.find_all(name, version)
- if len(candidates) > 0:
- return candidates[-1]
- return None
-
- def find_all(self, name, version):
- """Finds the packages by name and range of versions.
-
- :param name: the package`s name.
- :param version: the range of versions.
- :return: the list of suitable packages
- """
-
- if name in self.packages:
- candidates = self._find_versions(
- self.packages[name], version
- )
- if len(candidates) > 0:
- return candidates
-
- if name in self.obsoletes:
- return self._resolve_relation(
- self.obsoletes[name], version
- )
-
- if name in self.provides:
- return self._resolve_relation(
- self.provides[name], version
- )
- return []
-
- def add(self, package):
- """Adds new package to indexes.
-
- :param package: the package object.
- """
- self.packages[package.name][package.version] = package
- key = package.name, package.version
-
- for obsolete in package.obsoletes:
- self.obsoletes[obsolete.name][key] = obsolete
-
- for provide in package.provides:
- self.provides[provide.name][key] = provide
-
- def _resolve_relation(self, relations, version):
- """Resolve relation according to relations index.
-
- :param relations: the index of relations
- :param version: the range of versions
- :return: package if found, otherwise None
- """
- for key, candidate in relations.iter_items(reverse=True):
- if candidate.version.has_intersection(version):
- return [self.packages[key[0]][key[1]]]
- return []
-
- @staticmethod
- def _find_versions(versions, version):
- """Searches accurate version.
-
- Search for the highest version out of intersection
- of existing and required range of versions.
-
- :param versions: the existing versions
- :param version: the required range of versions
- :return: package if found, otherwise None
- """
-
- try:
- op = Index.operators[version.op]
- except KeyError:
- raise ValueError(
- "Unsupported operation: {0}"
- .format(version.op)
- )
- return op(versions, version.edge)
diff --git a/packetary/objects/package.py b/packetary/objects/package.py
deleted file mode 100644
index cb5a7d7..0000000
--- a/packetary/objects/package.py
+++ /dev/null
@@ -1,80 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from collections import namedtuple
-
-from packetary.objects.base import ComparableObject
-
-
-FileChecksum = namedtuple("FileChecksum", ("md5", "sha1", "sha256"))
-
-
-class Package(ComparableObject):
- """Structure to describe package object."""
-
- def __init__(self, repository, name, version, filename,
- filesize, checksum, mandatory=False,
- requires=None, provides=None, obsoletes=None):
- """Initialises.
-
- :param name: the package`s name
- :param version: the package`s version
- :param filename: the package`s relative filename
- :param filesize: the package`s file size
- :param checksum: the package`s checksum
- :param requires: the package`s requirements(optional)
- :param provides: the package`s provides(optional)
- :param obsoletes: the package`s obsoletes(optional)
- :param mandatory: indicates that package is mandatory
- """
-
- self.repository = repository
- self.name = name
- self.version = version
- self.filename = filename
- self.checksum = checksum
- self.filesize = filesize
- self.requires = requires or []
- self.provides = provides or []
- self.obsoletes = obsoletes or []
- self.mandatory = mandatory
-
- def __copy__(self):
- """Creates shallow copy of package."""
- return Package(**self.__dict__)
-
- def __str__(self):
- return "{0} {1}".format(self.name, self.version)
-
- def __unicode__(self):
- return u"{0} {1}".format(self.name, self.version)
-
- def __hash__(self):
- return hash((self.name, self.version))
-
- def cmp(self, other):
- """Compares with other Package object."""
- if self.name < other.name:
- return -1
- if self.name > other.name:
- return 1
- if self.version < other.version:
- return -1
- if self.version > other.version:
- return 1
- return 0
diff --git a/packetary/objects/package_relation.py b/packetary/objects/package_relation.py
deleted file mode 100644
index d24fe9b..0000000
--- a/packetary/objects/package_relation.py
+++ /dev/null
@@ -1,157 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import operator
-
-
-class VersionRange(object):
- """Describes the range of versions.
-
- Range of version is compare operation and edge.
- the compare operation can be one of:
- equal, greater, less, greater or equal, less or equal.
- """
-
- __slots__ = ["op", "edge"]
-
- def __init__(self, op=None, edge=None):
- """Initialises.
-
- :param op: the name of operator to compare.
- :param edge: the edge of versions.
- """
- self.op = op
- self.edge = edge
-
- def __contains__(self, point):
- return getattr(operator, self.op)(point, self.edge)
-
- def __hash__(self):
- return hash((self.op, self.edge))
-
- def __eq__(self, other):
- if not isinstance(other, VersionRange):
- return False
-
- return self.op == other.op and \
- self.edge == other.edge
-
- def __str__(self):
- if self.edge is not None:
- return "{0} {1}".format(self.op, self.edge)
- return "any"
-
- def __unicode__(self):
- if self.edge is not None:
- return u"{0} {1}".format(self.op, self.edge)
- return u"any"
-
- def has_intersection(self, other):
- """Checks that 2 ranges has intersection.
-
- :param other: the candidate to check
- :return: True if intersection exists, otherwise False
- :raise TypeError: when other does not instance of VersionRange
- """
-
- if not isinstance(other, VersionRange):
- raise TypeError(
- "Unorderable type and {0}"
- .format(type(other))
- )
-
- if self.op is None or other.op is None:
- return True
-
- if self.op[0] == other.op[0]:
- if self.op == 'eq':
- return self.edge == other.edge
- # the intersection is -inf or +inf
- return True
- if self.edge == other.edge:
- # need to cover case < a and >= a
- return self.edge in other and other.edge in self
- # all other cases
- return self.edge in other or other.edge in self
-
-
-class PackageRelation(object):
- """Describes the package`s relation.
-
- Relation includes the name of required package
- and range of versions that satisfies requirement.
- """
-
- __slots__ = ["name", "version", "alternative"]
-
- def __init__(self, name, version=None, alternative=None):
- """Initialises.
-
- :param name: the name of required package
- :param version: the version range of required package
- :param alternative: the alternative relation
- """
- self.name = name
- self.version = VersionRange() if version is None else version
- self.alternative = alternative
-
- @classmethod
- def from_args(cls, *args):
- """Construct relation from list of arguments.
-
- :param args: the list of tuples(name, [version_op, version_edge])
- """
- if len(args) == 0:
- return None
-
- head = args[0]
- name = head[0]
- version = VersionRange(*head[1:])
- alternative = cls.from_args(*args[1:])
- return cls(name, version, alternative)
-
- def __iter__(self):
- """Iterates over alternatives."""
- r = self
- while r is not None:
- yield r
- r = r.alternative
-
- def __hash__(self):
- return hash((self.name, self.version))
-
- def __eq__(self, other):
- if not isinstance(other, PackageRelation):
- return False
-
- return self.name == other.name and \
- self.version == other.version
-
- def __str__(self):
- if self.alternative is None:
- return "{0} ({1})".format(self.name, self.version)
- return "{0} ({1}) | {2}".format(
- self.name, self.version, self.alternative
- )
-
- def __unicode__(self):
- if self.alternative is None:
- return u"{0} ({1})".format(self.name, self.version)
- return u"{0} ({1}) | {2}".format(
- self.name, self.version, self.alternative
- )
diff --git a/packetary/objects/package_version.py b/packetary/objects/package_version.py
deleted file mode 100644
index 50ead20..0000000
--- a/packetary/objects/package_version.py
+++ /dev/null
@@ -1,127 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from packetary.objects.base import ComparableObject
-
-
-class PackageVersion(ComparableObject):
- """The Package version."""
-
- __slots__ = ["epoch", "version", "release"]
-
- def __init__(self, epoch, version, release):
- self.epoch = int(epoch)
- self.version = tuple(version)
- self.release = tuple(release)
-
- @classmethod
- def from_string(cls, text):
- """Constructs from string.
-
- :param text: the version in format '[{epoch-}]-{version}-{release}'
- """
- components = text.split("-")
- if len(components) > 2:
- epoch = components[0]
- components = components[1:]
- else:
- epoch = 0
- return cls(epoch, components[0].split("."), components[1].split("."))
-
- def cmp(self, other):
- if not isinstance(other, PackageVersion):
- other = PackageVersion.from_string(str(other))
-
- if not isinstance(other, PackageVersion):
- raise TypeError
- if self.epoch < other.epoch:
- return -1
- if self.epoch > other.epoch:
- return 1
-
- res = self._cmp_version_part(self.version, other.version)
- if res != 0:
- return res
- return self._cmp_version_part(self.release, other.release)
-
- def __eq__(self, other):
- if other is self:
- return True
- return self.cmp(other) == 0
-
- def __str__(self):
- return "{0}-{1}-{2}".format(
- self.epoch,
- ".".join(str(x) for x in self.version),
- ".".join(str(x) for x in self.release)
- )
-
- @classmethod
- def _order(cls, x):
- """Return an integer value for character x"""
- if x.isdigit():
- return int(x) + 1
- if x.isalpha():
- return ord(x)
- return ord(x) + 256
-
- @classmethod
- def _cmp_version_string(cls, version1, version2):
- """Compares two versions as string."""
- la = [cls._order(x) for x in version1]
- lb = [cls._order(x) for x in version2]
- while la or lb:
- a = 0
- b = 0
- if la:
- a = la.pop(0)
- if lb:
- b = lb.pop(0)
- if a < b:
- return -1
- elif a > b:
- return 1
- return 0
-
- @classmethod
- def _cmp_version_part(cls, version1, version2):
- """Compares two versions."""
- ver1_it = iter(version1)
- ver2_it = iter(version2)
- while True:
- v1 = next(ver1_it, None)
- v2 = next(ver2_it, None)
-
- if v1 is None or v2 is None:
- if v1 is not None:
- return 1
- if v2 is not None:
- return -1
- return 0
-
- if v1.isdigit() and v2.isdigit():
- a = int(v1)
- b = int(v2)
- if a < b:
- return -1
- if a > b:
- return 1
- else:
- r = cls._cmp_version_string(v1, v2)
- if r != 0:
- return r
diff --git a/packetary/objects/packages_tree.py b/packetary/objects/packages_tree.py
deleted file mode 100644
index a0f7d83..0000000
--- a/packetary/objects/packages_tree.py
+++ /dev/null
@@ -1,129 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import warnings
-
-from packetary.objects.index import Index
-
-
-class UnresolvedWarning(UserWarning):
- """Warning about unresolved depends."""
- pass
-
-
-class PackagesTree(Index):
- """Helper class to deal with dependency graph."""
-
- def __init__(self):
- super(PackagesTree, self).__init__()
- self.mandatory_packages = []
-
- def add(self, package):
- super(PackagesTree, self).add(package)
- # store all mandatory packages in separated list for quick access
- if package.mandatory:
- self.mandatory_packages.append(package)
-
- def get_unresolved_dependencies(self, base=None):
- """Gets the set of unresolved dependencies.
-
- :param base: the base index to resolve dependencies
- :return: the set of unresolved depends.
- """
- external = self.__get_unresolved_dependencies(self)
- if base is None:
- return external
-
- unresolved = set()
- for relation in external:
- for rel in relation:
- if base.find(rel.name, rel.version) is not None:
- break
- else:
- unresolved.add(relation)
- return unresolved
-
- def get_minimal_subset(self, main, requirements):
- """Gets the minimal work subset.
-
- :param main: the main index, to complete requirements.
- :param requirements: additional requirements.
- :return: The set of resolved depends.
- """
-
- unresolved = set()
- resolved = set()
- if main is None:
- def pkg_filter(*_):
- pass
- else:
- pkg_filter = main.find
- self.__get_unresolved_dependencies(main, requirements)
-
- stack = list()
- stack.append(requirements)
-
- # add all mandatory packages
- for pkg in self.mandatory_packages:
- resolved.add(pkg)
- stack.append(pkg.requires)
-
- while len(stack) > 0:
- required = stack.pop()
- for require in required:
- for rel in require:
- if rel not in unresolved:
- if pkg_filter(rel.name, rel.version) is not None:
- break
- # use all packages that meets depends
- candidates = self.find_all(rel.name, rel.version)
- for cand in candidates:
- if cand not in resolved:
- resolved.add(cand)
- stack.append(cand.requires)
- if len(candidates) > 0:
- break
- else:
- unresolved.add(require)
- msg = "Unresolved depends: {0}".format(require)
- warnings.warn(UnresolvedWarning(msg))
-
- return resolved
-
- @staticmethod
- def __get_unresolved_dependencies(index, unresolved=None):
- """Gets the set of unresolved dependencies.
-
- :param index: the search index.
- :param unresolved: the known list of unresolved packages.
- :return: the set of unresolved depends.
- """
-
- if unresolved is None:
- unresolved = set()
-
- for pkg in index:
- for require in pkg.requires:
- for rel in require:
- if rel not in unresolved:
- candidate = index.find(rel.name, rel.version)
- if candidate is not None and candidate != pkg:
- break
- else:
- unresolved.add(require)
- return unresolved
diff --git a/packetary/objects/repository.py b/packetary/objects/repository.py
deleted file mode 100644
index f2302e7..0000000
--- a/packetary/objects/repository.py
+++ /dev/null
@@ -1,48 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-
-class Repository(object):
- """Structure to describe repository object."""
-
- def __init__(self, name, url, architecture, origin):
- """Initialises.
-
- :param name: the repository`s name, may be tuple of strings
- :param url: the repository`s URL
- :param architecture: the repository`s architecture
- :param origin: the repository`s origin
- """
- self.name = name
- self.url = url
- self.architecture = architecture
- self.origin = origin
-
- def __str__(self):
- if isinstance(self.name, tuple):
- return ".".join(self.name)
- return self.name or self.url
-
- def __unicode__(self):
- if isinstance(self.name, tuple):
- return u".".join(self.name)
- return self.name or self.url
-
- def __copy__(self):
- """Creates shallow copy of package."""
- return Repository(**self.__dict__)
diff --git a/packetary/objects/statistics.py b/packetary/objects/statistics.py
deleted file mode 100644
index 50bf252..0000000
--- a/packetary/objects/statistics.py
+++ /dev/null
@@ -1,47 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import copy
-
-
-class CopyStatistics(object):
- """The statistics of packages copying"""
- def __init__(self):
- # the number of copied packages
- self.copied = 0
- # the number of total packages
- self.total = 0
-
- def on_package_copied(self, bytes_copied):
- """Proceed next copied package."""
- if bytes_copied > 0:
- self.copied += 1
- self.total += 1
-
- def __iadd__(self, other):
- if not isinstance(other, CopyStatistics):
- raise TypeError
-
- self.copied += other.copied
- self.total += other.total
- return self
-
- def __add__(self, other):
- result = copy.copy(self)
- result += other
- return result
diff --git a/packetary/tests/__init__.py b/packetary/tests/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/packetary/tests/base.py b/packetary/tests/base.py
deleted file mode 100644
index 74f4445..0000000
--- a/packetary/tests/base.py
+++ /dev/null
@@ -1,27 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-try:
- import unittest2 as unittest
-except ImportError:
- import unittest
-
-
-class TestCase(unittest.TestCase):
-
- """Test case base class for all unit tests."""
diff --git a/packetary/tests/data/Packages b/packetary/tests/data/Packages
deleted file mode 100644
index b1dcf13..0000000
--- a/packetary/tests/data/Packages
+++ /dev/null
@@ -1,19 +0,0 @@
-Package: test
-Source: test.src
-Version: 1.1.1-1~u14.04+test
-Architecture: all
-Maintainer: Test
-Installed-Size: 3509
-Homepage: http://localhost/
-Priority: required
-Section: web
-Filename: pool/main/t/test.deb
-Size: 100
-Depends: test2 (>= 0.8.16~exp9)|tes2-old, test3
-Pre-Depends: test-main
-Provides: file
-Replaces: test-old
-SHA256: 14d6e308d8699b7f9ba2fe1ef778c0e38cf295614d308039d687b6b097d50859
-SHA1: 402bd18c145ae3b5344edf07f246be159397fd40
-MD5sum: 1ae09f80109f40dfbfaf3ba423c8625a
-Description: test package
diff --git a/packetary/tests/data/Release b/packetary/tests/data/Release
deleted file mode 100644
index a54fe08..0000000
--- a/packetary/tests/data/Release
+++ /dev/null
@@ -1,6 +0,0 @@
-Archive: trusty
-Version: 14.04
-Component: main
-Origin: Ubuntu
-Label: Ubuntu
-Architecture: amd64
diff --git a/packetary/tests/data/groups.xml b/packetary/tests/data/groups.xml
deleted file mode 100644
index 4fadf5e..0000000
--- a/packetary/tests/data/groups.xml
+++ /dev/null
@@ -1,17 +0,0 @@
-
-
-
-
- core
-
- test1
- test2
-
-
-
- other
-
- test1
-
-
-
diff --git a/packetary/tests/data/primary.xml b/packetary/tests/data/primary.xml
deleted file mode 100644
index 3be4161..0000000
--- a/packetary/tests/data/primary.xml
+++ /dev/null
@@ -1,62 +0,0 @@
-
-
- test1
- x86_64
-
- e8ed9e0612e813491ed5e7c10502a39e43ec665afd1321541dea211202707a65
- Test package
- Test package
- Test
- http://localhost/
-
-
-
-
- GPLv2 with exceptions
- CentOS
- System Environment/Daemons
- worker1.bsys.centos.org
- 389-ds-base-1.3.3.1-13.el7.src.rpm
-
-
-
-
-
-
-
-
-
-
-
-
-
- test2
- x86_64
-
- e8ed9e0612e813491ed5e7c10502a39e43ec665afd1321541dea211202707a65
- Test package
- Test package
- Test
- http://localhost/
-
-
-
-
- GPLv2 with exceptions
- CentOS
- System Environment/Daemons
- worker1.bsys.centos.org
- 389-ds-base-1.3.3.1-13.el7.src.rpm
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/packetary/tests/data/repomd.xml b/packetary/tests/data/repomd.xml
deleted file mode 100644
index 990ff06..0000000
--- a/packetary/tests/data/repomd.xml
+++ /dev/null
@@ -1,20 +0,0 @@
-
-
- 1427842153
-
- 32fa7089953ace14f4a6e722bd33c353fcb94d9678d8a062a3b028e54042319c
- 32fa7089953ace14f4a6e722bd33c353fcb94d9678d8a062a3b028e54042319c
-
- 1427842225
- 2528031
- 23175717
-
-
- 689a2ef671fe1c2245539e9c7b90e9dcd1236f4a0dd376512cfd509531a2b70d
- bb7a4b6a6ccc8b4875b569359aedf67f9678cd56da7f372c134200265e276951
-
- 1427842225
- 2528031
- 23175717
-
-
diff --git a/packetary/tests/data/repomd2.xml b/packetary/tests/data/repomd2.xml
deleted file mode 100644
index 198ef47..0000000
--- a/packetary/tests/data/repomd2.xml
+++ /dev/null
@@ -1,20 +0,0 @@
-
-
- 1427842153
-
- 32fa7089953ace14f4a6e722bd33c353fcb94d9678d8a062a3b028e54042319c
- 32fa7089953ace14f4a6e722bd33c353fcb94d9678d8a062a3b028e54042319c
-
- 1427842225
- 2528031
- 23175717
-
-
- 689a2ef671fe1c2245539e9c7b90e9dcd1236f4a0dd376512cfd509531a2b70d
- bb7a4b6a6ccc8b4875b569359aedf67f9678cd56da7f372c134200265e276951
-
- 1427842225
- 2528031
- 23175717
-
-
diff --git a/packetary/tests/stubs/__init__.py b/packetary/tests/stubs/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/packetary/tests/stubs/executor.py b/packetary/tests/stubs/executor.py
deleted file mode 100644
index 051454c..0000000
--- a/packetary/tests/stubs/executor.py
+++ /dev/null
@@ -1,29 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-
-class Executor(object):
- def __enter__(self):
- return self
-
- def __exit__(self, *_):
- return False
-
- @staticmethod
- def execute(f, *args, **kwargs):
- return f(*args, **kwargs)
diff --git a/packetary/tests/stubs/generator.py b/packetary/tests/stubs/generator.py
deleted file mode 100644
index 5c196bd..0000000
--- a/packetary/tests/stubs/generator.py
+++ /dev/null
@@ -1,52 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from packetary import objects
-
-
-def gen_repository(name="test", url="file:///test",
- architecture="x86_64", origin="Test"):
- """Helper to create Repository object with default attributes."""
- return objects.Repository(name, url, architecture, origin)
-
-
-def gen_relation(name="test", version=None, alternative=None):
- """Helper to create PackageRelation object with default attributes."""
- return objects.PackageRelation(
- name=name,
- version=objects.VersionRange(*(version or [])),
- alternative=alternative
- )
-
-
-def gen_package(idx=1, **kwargs):
- """Helper to create Package object with default attributes."""
- repository = gen_repository()
- name = kwargs.setdefault("name", "package{0}".format(idx))
- kwargs.setdefault("repository", repository)
- kwargs.setdefault("version", 1)
- kwargs.setdefault("checksum", objects.FileChecksum("1", "2", "3"))
- kwargs.setdefault("filename", "{0}.pkg".format(name))
- kwargs.setdefault("filesize", 1)
- for relation in ("requires", "provides", "obsoletes"):
- if relation not in kwargs:
- kwargs[relation] = [gen_relation(
- "{0}{1}".format(relation, idx), ["le", idx + 1]
- )]
-
- return objects.Package(**kwargs)
diff --git a/packetary/tests/stubs/helpers.py b/packetary/tests/stubs/helpers.py
deleted file mode 100644
index f4ac845..0000000
--- a/packetary/tests/stubs/helpers.py
+++ /dev/null
@@ -1,66 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-from contextlib import closing
-import gzip
-import mock
-import six
-
-
-class CallbacksAdapter(mock.MagicMock):
- """Helper to return data through callback."""
-
- def __call__(self, *args, **kwargs):
- if len(args) > 0:
- callback = args[-1]
- else:
- callback = None
-
- if not callable(callback):
- return super(CallbacksAdapter, self).__call__(*args, **kwargs)
-
- args = args[:-1]
- data = super(CallbacksAdapter, self).__call__(*args, **kwargs)
-
- if isinstance(data, list):
- for d in data:
- callback(d)
- else:
- callback(data)
-
-
-class Buffer(object):
- """Helper to hide BytesIO methods."""
-
- def __init__(self, io):
- self.io = io
- self.reset()
-
- def reset(self):
- self.io.seek(0)
-
- def read(self, s=-1):
- return self.io.read(s)
-
-
-def get_compressed(stream):
- """Gets compressed stream."""
- compressed = six.BytesIO()
- with closing(gzip.GzipFile(fileobj=compressed, mode="wb")) as gz:
- gz.write(stream.read())
- return Buffer(compressed)
diff --git a/packetary/tests/test_checksums.py b/packetary/tests/test_checksums.py
deleted file mode 100644
index 8b0a7c5..0000000
--- a/packetary/tests/test_checksums.py
+++ /dev/null
@@ -1,52 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import six
-
-from packetary.library import checksum
-from packetary.tests import base
-
-
-class TestChecksum(base.TestCase):
- def test_checksum(self):
- stream = six.BytesIO(b"line1\nline2\nline3\n")
- checksums = {
- checksum.md5: "cc3d5ed5fda53dfa81ea6aa951d7e1fe",
- checksum.sha1: "8c84f6f36dd2230d3e9c954fa436e5fda90b1957",
- checksum.sha256: "66663af9c7aa341431a8ee2ff27b72"
- "abd06c9218f517bb6fef948e4803c19e03"
- }
- for chunksize in (8, 256):
- for algo, expected in six.iteritems(checksums):
- stream.seek(0)
- self.assertEqual(
- expected, algo(stream, chunksize)
- )
-
- def test_composite(self):
- stream = six.BytesIO(b"line1\nline2\nline3\n")
- result = checksum.composite('md5', 'sha1', 'sha256')(stream)
- self.assertEqual(
- [
- "cc3d5ed5fda53dfa81ea6aa951d7e1fe",
- "8c84f6f36dd2230d3e9c954fa436e5fda90b1957",
- "66663af9c7aa341431a8ee2ff27b72"
- "abd06c9218f517bb6fef948e4803c19e03"
- ],
- result
- )
diff --git a/packetary/tests/test_cli_commands.py b/packetary/tests/test_cli_commands.py
deleted file mode 100644
index 28594b1..0000000
--- a/packetary/tests/test_cli_commands.py
+++ /dev/null
@@ -1,154 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import mock
-import subprocess
-
-# The cmd2 does not work with python3.5
-# because it tries to get access to the property mswindows,
-# that was removed in 3.5
-subprocess.mswindows = False
-
-from packetary.cli.commands import clone
-from packetary.cli.commands import packages
-from packetary.cli.commands import unresolved
-from packetary.tests import base
-from packetary.tests.stubs.generator import gen_package
-from packetary.tests.stubs.generator import gen_relation
-from packetary.tests.stubs.generator import gen_repository
-from packetary.tests.stubs.helpers import CallbacksAdapter
-
-
-@mock.patch.multiple(
- "packetary.api",
- RepositoryController=mock.DEFAULT,
- ConnectionsManager=mock.DEFAULT,
- AsynchronousSection=mock.MagicMock()
-)
-@mock.patch(
- "packetary.cli.commands.base.BaseRepoCommand.stdout"
-)
-class TestCliCommands(base.TestCase):
- common_argv = [
- "--ignore-errors-num=3",
- "--threads-num=8",
- "--retries-num=10",
- "--retry-interval=1",
- "--http-proxy=http://proxy",
- "--https-proxy=https://proxy"
- ]
-
- clone_argv = [
- "-o", "http://localhost/origin",
- "-d", ".",
- "-r", "http://localhost/requires",
- "-b", "test-package",
- "-t", "deb",
- "-a", "x86_64",
- "--clean",
- ]
-
- packages_argv = [
- "-o", "http://localhost/origin",
- "-t", "deb",
- "-a", "x86_64"
- ]
-
- unresolved_argv = [
- "-o", "http://localhost/origin",
- "-t", "deb",
- "-a", "x86_64"
- ]
-
- def start_cmd(self, cmd, argv):
- cmd.debug(argv + self.common_argv)
-
- def check_context(self, context, ConnectionsManager):
- self.assertEqual(3, context._ignore_errors_num)
- self.assertEqual(8, context._threads_num)
- self.assertIs(context._connection, ConnectionsManager.return_value)
- ConnectionsManager.assert_called_once_with(
- proxy="http://proxy",
- secure_proxy="https://proxy",
- retries_num=10,
- retry_interval=1
- )
-
- def test_clone_cmd(self, stdout, RepositoryController, **kwargs):
- ctrl = RepositoryController.load()
- ctrl.copy_packages = CallbacksAdapter()
- ctrl.load_repositories = CallbacksAdapter()
- ctrl.load_packages = CallbacksAdapter()
- ctrl.copy_packages.return_value = [1, 0]
- repo = gen_repository()
- ctrl.load_repositories.side_effect = [repo, gen_repository()]
- ctrl.load_packages.side_effect = [
- gen_package(repository=repo),
- gen_package()
- ]
- self.start_cmd(clone, self.clone_argv)
- RepositoryController.load.assert_called_with(
- mock.ANY, "deb", "x86_64"
- )
- self.check_context(
- RepositoryController.load.call_args[0][0], **kwargs
- )
- stdout.write.assert_called_once_with(
- "Packages copied: 1/2.\n"
- )
-
- def test_get_packages_cmd(self, stdout, RepositoryController, **kwargs):
- ctrl = RepositoryController.load()
- ctrl.load_packages = CallbacksAdapter()
- ctrl.load_packages.return_value = gen_package(
- name="test1",
- filesize=1,
- requires=None,
- obsoletes=None,
- provides=None
- )
- self.start_cmd(packages, self.packages_argv)
- RepositoryController.load.assert_called_with(
- mock.ANY, "deb", "x86_64"
- )
- self.check_context(
- RepositoryController.load.call_args[0][0], **kwargs
- )
- self.assertIn(
- "test1; test; 1; test1.pkg; 1;",
- stdout.write.call_args_list[3][0][0]
- )
-
- def test_get_unresolved_cmd(self, stdout, RepositoryController, **kwargs):
- ctrl = RepositoryController.load()
- ctrl.load_packages = CallbacksAdapter()
- ctrl.load_packages.return_value = gen_package(
- name="test1",
- requires=[gen_relation("test2")]
- )
- self.start_cmd(unresolved, self.unresolved_argv)
- RepositoryController.load.assert_called_with(
- mock.ANY, "deb", "x86_64"
- )
- self.check_context(
- RepositoryController.load.call_args[0][0], **kwargs
- )
- self.assertIn(
- "test2; any; -",
- stdout.write.call_args_list[3][0][0]
- )
diff --git a/packetary/tests/test_command_utils.py b/packetary/tests/test_command_utils.py
deleted file mode 100644
index 0022208..0000000
--- a/packetary/tests/test_command_utils.py
+++ /dev/null
@@ -1,75 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import mock
-
-from packetary.cli.commands import utils
-from packetary.tests import base
-
-
-class Dummy(object):
- pass
-
-
-class TestCommandUtils(base.TestCase):
- @mock.patch("packetary.cli.commands.utils.open")
- def test_read_lines_from_file(self, open_mock):
- open_mock().__enter__.return_value = [
- "line1\n",
- " # comment\n",
- "line2 \n"
- ]
-
- self.assertEqual(
- ["line1", "line2"],
- utils.read_lines_from_file("test.txt")
- )
-
- def test_get_object_attrs(self):
- obj = Dummy()
- obj.attr_int = 0
- obj.attr_str = "text"
- obj.attr_none = None
- self.assertEqual(
- [0, "text", None],
- utils.get_object_attrs(obj, ["attr_int", "attr_str", "attr_none"])
- )
-
- def test_get_display_value(self):
- self.assertEqual(u"", utils.get_display_value(""))
- self.assertEqual(u"-", utils.get_display_value(None))
- self.assertEqual(u"0", utils.get_display_value(0))
- self.assertEqual(u"", utils.get_display_value([]))
- self.assertEqual(
- u"1, a, None",
- utils.get_display_value([1, "a", None])
- )
- self.assertEqual(u"1", utils.get_display_value(1))
-
- def test_make_display_attr_getter(self):
- obj = Dummy()
- obj.attr_int = 0
- obj.attr_str = "text"
- obj.attr_none = None
- formatter = utils.make_display_attr_getter(
- ["attr_int", "attr_str", "attr_none"]
- )
- self.assertEqual(
- [u"0", u"text", u"-"],
- formatter(obj)
- )
diff --git a/packetary/tests/test_connections.py b/packetary/tests/test_connections.py
deleted file mode 100644
index c80b03d..0000000
--- a/packetary/tests/test_connections.py
+++ /dev/null
@@ -1,323 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import mock
-import six
-import time
-
-from packetary.library import connections
-from packetary.tests import base
-
-
-@mock.patch("packetary.library.connections.logger")
-class TestConnectionManager(base.TestCase):
- def _check_proxies(self, manager, http_proxy, https_proxy):
- for h in manager.opener.handlers:
- if isinstance(h, connections.urllib.ProxyHandler):
- self.assertEqual(
- (http_proxy, https_proxy),
- (h.proxies["http"], h.proxies["https"])
- )
- break
- else:
- self.fail("ProxyHandler should be in list of handlers.")
-
- def test_set_proxy(self, _):
- manager = connections.ConnectionsManager(proxy="http://localhost")
- self._check_proxies(
- manager, "http://localhost", "http://localhost"
- )
- manager = connections.ConnectionsManager(
- proxy="http://localhost", secure_proxy="https://localhost")
- self._check_proxies(
- manager, "http://localhost", "https://localhost"
- )
- manager = connections.ConnectionsManager(retries_num=2)
- self.assertEqual(2, manager.retries_num)
- for h in manager.opener.handlers:
- if isinstance(h, connections.RetryHandler):
- break
- else:
- self.fail("RetryHandler should be in list of handlers.")
-
- @mock.patch("packetary.library.connections.urllib.build_opener")
- def test_make_request(self, *_):
- manager = connections.ConnectionsManager(retries_num=2)
- request = manager.make_request("/test/file", 0)
- self.assertIsInstance(request, connections.RetryableRequest)
- self.assertEqual("file:///test/file", request.get_full_url())
- self.assertEqual(0, request.offset)
- self.assertEqual(2, request.retries_left)
- request2 = manager.make_request("http://server/path", 100)
- self.assertEqual("http://server/path", request2.get_full_url())
- self.assertEqual(100, request2.offset)
-
- @mock.patch("packetary.library.connections.urllib.build_opener")
- def test_open_stream(self, *_):
- manager = connections.ConnectionsManager(retries_num=2)
- manager.open_stream("/test/file")
- self.assertEqual(1, manager.opener.open.call_count)
- args = manager.opener.open.call_args[0]
- self.assertIsInstance(args[0], connections.RetryableRequest)
- self.assertEqual(2, args[0].retries_left)
-
- @mock.patch("packetary.library.connections.urllib.build_opener")
- def test_retries_on_io_error(self, _, logger):
- manager = connections.ConnectionsManager(retries_num=2)
- manager.opener.open.side_effect = [
- IOError("I/O error"),
- mock.MagicMock()
- ]
- manager.open_stream("/test/file")
- self.assertEqual(2, manager.opener.open.call_count)
- logger.exception.assert_called_with(
- "Failed to open url - %s: %s. retries left - %d.",
- "/test/file", "I/O error", 1
- )
-
- manager.opener.open.side_effect = IOError("I/O error")
- with self.assertRaises(IOError):
- manager.open_stream("/test/file")
- logger.exception.assert_called_with(
- "Failed to open url - %s: %s. retries left - %d.",
- "/test/file", "I/O error", 0
- )
-
- @mock.patch.multiple(
- "packetary.library.connections.urllib.HTTPHandler",
- http_request=mock.DEFAULT,
- http_open=mock.DEFAULT
- )
- def test_retries_on_50x(self, logger, http_open, http_request):
- request = connections.RetryableRequest("http:///localhost/file1.txt")
- request.retries_left = 1
- http_request.return_value = request
- response_mock = mock.MagicMock(code=501, msg="not found")
- response_mock.getcode.return_value = response_mock.code
- http_open.return_value = response_mock
- manager = connections.ConnectionsManager(retries_num=2)
- with self.assertRaises(connections.urlerror.HTTPError) as trapper:
- manager.open_stream("http:///localhost/file1.txt")
- self.assertEqual(501, trapper.exception.code)
- self.assertEqual(2, http_request.call_count)
- for retry_num in six.moves.range(1):
- logger.error.assert_any_call(
- "request failed: %s - %d(%s), retries left - %d.",
- mock.ANY, 501, mock.ANY, retry_num
- )
-
- @mock.patch("packetary.library.connections.urllib.build_opener")
- def test_raise_other_errors(self, *_):
- manager = connections.ConnectionsManager()
- manager.opener.open.side_effect = \
- connections.urlerror.HTTPError("", 500, "", {}, None)
-
- with self.assertRaises(connections.urlerror.URLError):
- manager.open_stream("/test/file")
-
- self.assertEqual(1, manager.opener.open.call_count)
-
- @mock.patch("packetary.library.connections.urllib.build_opener")
- @mock.patch("packetary.library.connections.time")
- def test_progressive_delay_between_request(self, time_mock, *_):
- manager = connections.ConnectionsManager(
- retries_num=6, retry_interval=1
- )
- manager.opener.open.side_effect = IOError("I/O Error")
-
- with self.assertRaises(IOError):
- manager.open_stream("/test/file")
-
- self.assertEqual(7, manager.opener.open.call_count)
- self.assertEqual(
- [1, 1, 2, 3, 4, 5],
- [x[0][0] for x in time_mock.sleep.call_args_list]
- )
-
- @mock.patch("packetary.library.connections.urllib.build_opener")
- @mock.patch("packetary.library.connections.ensure_dir_exist")
- @mock.patch("packetary.library.connections.os")
- def test_retrieve_from_offset(self, os, *_):
- manager = connections.ConnectionsManager()
- os.stat.return_value = mock.MagicMock(st_size=10)
- os.open.return_value = 1
- response = mock.MagicMock()
- manager.opener.open.return_value = response
- response.read.side_effect = [b"test", b""]
- manager.retrieve("/file/src", "/file/dst", size=20)
- os.lseek.assert_called_once_with(1, 10, os.SEEK_SET)
- os.ftruncate.assert_called_once_with(1, 10)
- self.assertEqual(1, os.write.call_count)
- os.fsync.assert_called_once_with(1)
- os.close.assert_called_once_with(1)
-
- @mock.patch("packetary.library.connections.urllib.build_opener")
- @mock.patch("packetary.library.connections.ensure_dir_exist")
- @mock.patch("packetary.library.connections.os")
- def test_retrieve_non_existence(self, os, *_):
- manager = connections.ConnectionsManager()
- os.stat.side_effect = OSError(2, "")
- os.open.return_value = 1
- response = mock.MagicMock()
- manager.opener.open.return_value = response
- response.read.side_effect = [b"test", b""]
- manager.retrieve("/file/src", "/file/dst", size=20)
- os.lseek.assert_called_once_with(1, 0, os.SEEK_SET)
- os.ftruncate.assert_called_once_with(1, 0)
- self.assertEqual(1, os.write.call_count)
- os.fsync.assert_called_once_with(1)
- os.close.assert_called_once_with(1)
-
- @mock.patch("packetary.library.connections.urllib.build_opener",
- new=mock.MagicMock())
- @mock.patch("packetary.library.connections.ensure_dir_exist",
- new=mock.MagicMock())
- @mock.patch("packetary.library.connections.os")
- def test_retrieve_from_offset_fail(self, os, logger):
- manager = connections.ConnectionsManager(retries_num=2)
- os.stat.return_value = mock.MagicMock(st_size=10)
- os.open.return_value = 1
- response = mock.MagicMock()
- manager.opener.open.side_effect = [
- connections.RangeError("error"), response
- ]
- response.read.side_effect = [b"test", b""]
- manager.retrieve("/file/src", "/file/dst", size=20)
- logger.warning.assert_called_once_with(
- "Failed to resume download, starts from the beginning: %s",
- "/file/src"
- )
- os.lseek.assert_called_once_with(1, 0, os.SEEK_SET)
- os.ftruncate.assert_called_once_with(1, 0)
- self.assertEqual(1, os.write.call_count)
- os.fsync.assert_called_once_with(1)
- os.close.assert_called_once_with(1)
-
-
-@mock.patch("packetary.library.connections.logger")
-class TestRetryHandler(base.TestCase):
- def setUp(self):
- super(TestRetryHandler, self).setUp()
- self.handler = connections.RetryHandler()
- self.handler.add_parent(mock.MagicMock())
-
- def test_start_request(self, logger):
- request = mock.MagicMock()
- request.offset = 0
- request.get_full_url.return_value = "/file/test"
- request = self.handler.http_request(request)
- request.start_time <= time.time()
- logger.debug.assert_called_with("start request: %s", "/file/test")
- request.offset = 1
- request = self.handler.http_request(request)
- request.add_header.assert_called_once_with('Range', 'bytes=1-')
-
- def test_handle_response(self, logger):
- request = mock.MagicMock()
- request.offset = 0
- request.start_time.__rsub__.return_value = 0.01
- request.get_full_url.return_value = "/file/test"
- response = mock.MagicMock()
- response.getcode.return_value = 200
- response.msg = "test"
- r = self.handler.http_response(request, response)
- self.assertIsInstance(r, connections.ResumableResponse)
- logger.debug.assert_called_with(
- "request completed: %s - %d (%s), duration - %d ms.",
- "/file/test", 200, "test", 10
- )
-
- def test_handle_partial_response(self, _):
- request = mock.MagicMock()
- request.offset = 1
- request.get_full_url.return_value = "/file/test"
- response = mock.MagicMock()
- response.getcode.return_value = 200
- response.msg = "test"
- with self.assertRaises(connections.RangeError):
- self.handler.http_response(request, response)
- response.getcode.return_value = 206
- self.handler.http_response(request, response)
-
- def test_handle_error(self, logger):
- request = mock.MagicMock(retries_left=1, retry_interval=0, offset=0)
- request.get_retry_interval.return_value = 0
- request.get_full_url.return_value = "/test"
- response_mock = mock.MagicMock(code=500, msg="error")
- response_mock.getcode.return_value = response_mock.code
- self.handler.http_response(request, response_mock)
- logger.error.assert_called_with(
- "request failed: %s - %d(%s), retries left - %d.",
- "/test", 500, "error", 0
- )
- self.handler.http_response(request, response_mock)
- self.handler.parent.open.assert_called_once_with(request)
-
- @mock.patch(
- 'packetary.library.connections.urllib.'
- 'HTTPRedirectHandler.redirect_request'
- )
- def test_redirect_request(self, redirect_mock, _):
- redirect_mock.return_value = connections.urllib.Request(
- 'http://localhost/'
- )
- req = mock.MagicMock(retries_left=10, retry_interval=5, offset=100)
- new_req = self.handler.redirect_request(req, -1, 301, "", {}, "")
- self.assertIsInstance(new_req, connections.RetryableRequest)
- self.assertEqual(req.retries_left, new_req.retries_left)
- self.assertEqual(req.retry_interval, new_req.retry_interval)
- self.assertEqual(req.offset, new_req.offset)
- redirect_mock.return_value = None
- self.assertIsNone(
- self.handler.redirect_request(req, -1, 301, "", {}, "")
- )
-
-
-class TestResumeableResponse(base.TestCase):
- def setUp(self):
- super(TestResumeableResponse, self).setUp()
- self.request = mock.MagicMock()
- self.opener = mock.MagicMock()
- self.stream = mock.MagicMock()
-
- def test_resume_read(self):
- self.request.offset = 0
- response = connections.ResumableResponse(
- self.request,
- self.stream,
- self.opener
- )
- self.stream.read.side_effect = [
- b"chunk1", IOError(), b"chunk2", b""
- ]
- self.opener.error.return_value = response
- data = response.read()
- self.assertEqual(b"chunk1chunk2", data)
- self.assertEqual(12, self.request.offset)
- self.assertEqual(1, self.opener.error.call_count)
-
- def test_read(self):
- self.request.offset = 0
- response = connections.ResumableResponse(
- self.request,
- six.BytesIO(b"line1\nline2\nline3\n"),
- self.opener
- )
- self.assertEqual(
- b"line1\nline2\nline3\n", response.read()
- )
diff --git a/packetary/tests/test_deb_driver.py b/packetary/tests/test_deb_driver.py
deleted file mode 100644
index a8ea21b..0000000
--- a/packetary/tests/test_deb_driver.py
+++ /dev/null
@@ -1,270 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import mock
-import os.path as path
-import six
-
-
-from packetary.drivers import deb_driver
-from packetary.library.utils import localize_repo_url
-from packetary.tests import base
-from packetary.tests.stubs.generator import gen_package
-from packetary.tests.stubs.generator import gen_repository
-from packetary.tests.stubs.helpers import get_compressed
-
-
-PACKAGES = path.join(path.dirname(__file__), "data", "Packages")
-RELEASE = path.join(path.dirname(__file__), "data", "Release")
-
-
-class TestDebDriver(base.TestCase):
- @classmethod
- def setUpClass(cls):
- super(TestDebDriver, cls).setUpClass()
- cls.driver = deb_driver.DebRepositoryDriver()
-
- def setUp(self):
- self.connection = mock.MagicMock()
-
- def test_parse_urls(self):
- self.assertItemsEqual(
- [
- ("http://host", "trusty", "main"),
- ("http://host", "trusty", "restricted"),
- ],
- self.driver.parse_urls(
- ["http://host/dists/ trusty main restricted"]
- )
- )
- self.assertItemsEqual(
- [("http://host", "trusty", "main")],
- self.driver.parse_urls(
- ["http://host/dists trusty main"]
- )
- )
- self.assertItemsEqual(
- [("http://host", "trusty", "main")],
- self.driver.parse_urls(
- ["http://host/ trusty main"]
- )
- )
- self.assertItemsEqual(
- [
- ("http://host", "trusty", "main"),
- ("http://host2", "trusty", "main"),
- ],
- self.driver.parse_urls(
- [
- "http://host/ trusty main",
- "http://host2/dists/ trusty main",
- ]
- )
- )
-
- def test_get_repository(self):
- repos = []
- with open(RELEASE, "rb") as stream:
- self.connection.open_stream.return_value = stream
- self.driver.get_repository(
- self.connection,
- ("http://host", "trusty", "main"),
- "x86_64",
- repos.append
- )
- self.connection.open_stream.assert_called_once_with(
- "http://host/dists/trusty/main/binary-amd64/Release"
- )
- self.assertEqual(1, len(repos))
- repo = repos[0]
- self.assertEqual(("trusty", "main"), repo.name)
- self.assertEqual("Ubuntu", repo.origin)
- self.assertEqual("x86_64", repo.architecture)
- self.assertEqual("http://host/", repo.url)
-
- def test_get_packages(self):
- packages = []
- repo = gen_repository(name=("trusty", "main"), url="http://host/")
- with open(PACKAGES, "rb") as s:
- self.connection.open_stream.return_value = get_compressed(s)
- self.driver.get_packages(
- self.connection,
- repo,
- packages.append
- )
-
- self.connection.open_stream.assert_called_once_with(
- "http://host/dists/trusty/main/binary-amd64/Packages.gz",
- )
- self.assertEqual(1, len(packages))
- package = packages[0]
- self.assertEqual("test", package.name)
- self.assertEqual("1.1.1-1~u14.04+test", package.version)
- self.assertEqual(100, package.filesize)
- self.assertEqual(
- deb_driver.FileChecksum(
- '1ae09f80109f40dfbfaf3ba423c8625a',
- '402bd18c145ae3b5344edf07f246be159397fd40',
- '14d6e308d8699b7f9ba2fe1ef778c0e3'
- '8cf295614d308039d687b6b097d50859'),
- package.checksum
- )
- self.assertEqual(
- "pool/main/t/test.deb", package.filename
- )
- self.assertTrue(package.mandatory)
- self.assertItemsEqual(
- [
- 'test-main (any)',
- 'test2 (ge 0.8.16~exp9) | tes2-old (any)',
- 'test3 (any)'
- ],
- (str(x) for x in package.requires)
- )
- self.assertItemsEqual(
- ["file (any)"],
- (str(x) for x in package.provides)
- )
- self.assertEqual([], package.obsoletes)
-
- @mock.patch.multiple(
- "packetary.drivers.deb_driver",
- deb822=mock.DEFAULT,
- debfile=mock.DEFAULT,
- fcntl=mock.DEFAULT,
- gzip=mock.DEFAULT,
- utils=mock.DEFAULT,
- os=mock.DEFAULT,
- open=mock.DEFAULT
- )
- def test_rebuild_repository(self, os, debfile, deb822, fcntl,
- gzip, utils, open):
- repo = gen_repository(name=("trusty", "main"), url="file:///repo")
- package = gen_package(name="test", repository=repo)
- os.path.join = lambda *x: "/".join(x)
- utils.get_path_from_url = lambda x: x[7:]
-
- files = [
- mock.MagicMock(), # Packages, w
- mock.MagicMock(), # Release, a+b
- mock.MagicMock(), # Packages, rb
- mock.MagicMock(), # Release, rb
- mock.MagicMock() # Packages.gz, rb
- ]
- open.side_effect = files
- self.driver.rebuild_repository(repo, [package])
- open.assert_any_call(
- "/repo/dists/trusty/main/binary-amd64/Packages", "wb"
- )
- gzip.open.assert_called_once_with(
- "/repo/dists/trusty/main/binary-amd64/Packages.gz", "wb"
- )
- debfile.DebFile.assert_called_once_with("/repo/test.pkg")
-
- @mock.patch.multiple(
- "packetary.drivers.deb_driver",
- deb822=mock.DEFAULT,
- gzip=mock.DEFAULT,
- open=mock.DEFAULT,
- os=mock.DEFAULT,
- utils=mock.DEFAULT
- )
- def test_fork_repository(self, deb822, gzip, open, os, utils):
- os.path.sep = "/"
- os.path.join = lambda *x: "/".join(x)
- utils.get_path_from_url = lambda x: x
- utils.localize_repo_url = localize_repo_url
- repo = gen_repository(
- name=("trusty", "main"), url="http://localhost/test/"
- )
- files = [
- mock.MagicMock(),
- mock.MagicMock()
- ]
- open.side_effect = files
- new_repo = self.driver.fork_repository(self.connection, repo, "/root")
- self.assertEqual(repo.name, new_repo.name)
- self.assertEqual(repo.architecture, new_repo.architecture)
- self.assertEqual(repo.origin, new_repo.origin)
- self.assertEqual("/root/test/", new_repo.url)
- utils.ensure_dir_exist.assert_called_once_with(os.path.dirname())
- open.assert_any_call(
- "/root/test/dists/trusty/main/binary-amd64/Release", "wb"
- )
- open.assert_any_call(
- "/root/test/dists/trusty/main/binary-amd64/Packages", "ab"
- )
- gzip.open.assert_called_once_with(
- "/root/test/dists/trusty/main/binary-amd64/Packages.gz", "ab"
- )
-
- @mock.patch.multiple(
- "packetary.drivers.deb_driver",
- fcntl=mock.DEFAULT,
- gzip=mock.DEFAULT,
- open=mock.DEFAULT,
- os=mock.DEFAULT,
- utils=mock.DEFAULT
- )
- def test_update_suite_index(
- self, os, fcntl, gzip, open, utils):
- repo = gen_repository(name=("trusty", "main"), url="/repo")
- files = [
- mock.MagicMock(), # Release, a+b
- mock.MagicMock(), # Packages, rb
- mock.MagicMock(), # Release, rb
- mock.MagicMock() # Packages.gz, rb
- ]
- files[0].items.return_value = [
- ("SHA1", "invalid 1 main/binary-amd64/Packages\n"),
- ("Architectures", "i386"),
- ("Components", "restricted"),
- ]
- os.path.join = lambda *x: "/".join(x)
- open().__enter__.side_effect = files
- utils.get_path_from_url.return_value = "/root"
- utils.append_token_to_string.side_effect = [
- "amd64 i386", "main restricted"
- ]
-
- utils.get_size_and_checksum_for_files.return_value = (
- (
- "/root/dists/trusty/main/binary-amd64/{0}".format(name),
- 10,
- (k + "_value" for k in deb_driver._CHECKSUM_METHODS)
- )
- for name in deb_driver._REPOSITORY_FILES
- )
- self.driver._update_suite_index(repo)
- open.assert_any_call("/root/dists/trusty/Release", "a+b")
- files[0].seek.assert_called_once_with(0)
- files[0].truncate.assert_called_once_with(0)
- files[0].write.assert_any_call(six.b("Architectures: amd64 i386\n"))
- files[0].write.assert_any_call(six.b("Components: main restricted\n"))
- for k in deb_driver._CHECKSUM_METHODS:
- files[0].write.assert_any_call(six.b(
- '{0}:\n'
- ' {1} 10 main/binary-amd64/Packages\n'
- ' {1} 10 main/binary-amd64/Release\n'
- ' {1} 10 main/binary-amd64/Packages.gz\n'
- .format(k, k + "_value")
- ))
- open.assert_any_call("/root/dists/trusty/Release", "a+b")
- print([x.fileno() for x in files])
- fcntl.flock.assert_any_call(files[0].fileno(), fcntl.LOCK_EX)
- fcntl.flock.assert_any_call(files[0].fileno(), fcntl.LOCK_UN)
diff --git a/packetary/tests/test_executor.py b/packetary/tests/test_executor.py
deleted file mode 100644
index 3fff595..0000000
--- a/packetary/tests/test_executor.py
+++ /dev/null
@@ -1,65 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import mock
-import threading
-import time
-
-from packetary.library import executor
-from packetary.tests import base
-
-
-def _raise_value_error(*_):
- raise ValueError("error")
-
-
-@mock.patch("packetary.library.executor.logger")
-class TestAsynchronousSection(base.TestCase):
- def setUp(self):
- super(TestAsynchronousSection, self).setUp()
- self.results = []
-
- def test_isolation(self, _):
- section1 = executor.AsynchronousSection()
- section2 = executor.AsynchronousSection()
- event = threading.Event()
- section1.execute(event.wait)
- section2.execute(time.sleep, 0)
- section2.wait()
- event.set()
- section1.wait()
-
- def test_ignore_errors(self, logger):
- section = executor.AsynchronousSection(ignore_errors_num=1)
- section.execute(_raise_value_error)
- section.execute(time.sleep, 0)
- section.wait(ignore_errors=True)
- logger.exception.assert_called_with(
- "error details.", exc_info=mock.ANY
- )
-
- def test_fail_if_too_many_errors(self, _):
- section = executor.AsynchronousSection(size=1, ignore_errors_num=0)
- section.execute(_raise_value_error)
- time.sleep(0) # switch context
- with self.assertRaisesRegexp(RuntimeError, "Too many errors"):
- section.execute(time.sleep, 0)
-
- with self.assertRaisesRegexp(
- RuntimeError, "Operations completed with errors"):
- section.wait(ignore_errors=False)
diff --git a/packetary/tests/test_index.py b/packetary/tests/test_index.py
deleted file mode 100644
index 4348bc3..0000000
--- a/packetary/tests/test_index.py
+++ /dev/null
@@ -1,191 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import six
-
-from packetary.objects.index import Index
-
-from packetary import objects
-from packetary.tests import base
-from packetary.tests.stubs.generator import gen_package
-from packetary.tests.stubs.generator import gen_relation
-
-
-class TestIndex(base.TestCase):
- def test_add(self):
- index = Index()
- index.add(gen_package(version=1))
- self.assertIn("package1", index.packages)
- self.assertIn(1, index.packages["package1"])
- self.assertIn("obsoletes1", index.obsoletes)
- self.assertIn("provides1", index.provides)
-
- index.add(gen_package(version=2))
- self.assertEqual(1, len(index.packages))
- self.assertIn(1, index.packages["package1"])
- self.assertIn(2, index.packages["package1"])
- self.assertEqual(1, len(index.obsoletes))
- self.assertEqual(1, len(index.provides))
-
- def test_find(self):
- index = Index()
- p1 = gen_package(version=1)
- p2 = gen_package(version=2)
- index.add(p1)
- index.add(p2)
-
- self.assertIs(
- p1,
- index.find("package1", objects.VersionRange("eq", 1))
- )
- self.assertIs(
- p2,
- index.find("package1", objects.VersionRange())
- )
- self.assertIsNone(
- index.find("package1", objects.VersionRange("gt", 2))
- )
-
- def test_find_all(self):
- index = Index()
- p11 = gen_package(idx=1, version=1)
- p12 = gen_package(idx=1, version=2)
- p21 = gen_package(idx=2, version=1)
- p22 = gen_package(idx=2, version=2)
- index.add(p11)
- index.add(p12)
- index.add(p21)
- index.add(p22)
-
- self.assertItemsEqual(
- [p11, p12],
- index.find_all("package1", objects.VersionRange())
- )
- self.assertItemsEqual(
- [p21, p22],
- index.find_all("package2", objects.VersionRange("le", 2))
- )
-
- def test_find_newest_package(self):
- index = Index()
- p1 = gen_package(idx=1, version=2)
- p2 = gen_package(idx=2, version=2)
- p2.obsoletes.append(
- gen_relation(p1.name, ["lt", p1.version])
- )
- index.add(p1)
- index.add(p2)
-
- self.assertIs(
- p1, index.find(p1.name, objects.VersionRange("eq", p1.version))
- )
- self.assertIs(
- p2, index.find(p1.name, objects.VersionRange("eq", 1))
- )
-
- def test_find_top_down(self):
- index = Index()
- p1 = gen_package(version=1)
- p2 = gen_package(version=2)
- index.add(p1)
- index.add(p2)
- self.assertIs(
- p2,
- index.find("package1", objects.VersionRange("le", 2))
- )
- self.assertIs(
- p1,
- index.find("package1", objects.VersionRange("lt", 2))
- )
- self.assertIsNone(
- index.find("package1", objects.VersionRange("lt", 1))
- )
-
- def test_find_down_up(self):
- index = Index()
- p1 = gen_package(version=1)
- p2 = gen_package(version=2)
- index.add(p1)
- index.add(p2)
- self.assertIs(
- p2,
- index.find("package1", objects.VersionRange("ge", 2))
- )
- self.assertIs(
- p2,
- index.find("package1", objects.VersionRange("gt", 1))
- )
- self.assertIsNone(
- index.find("package1", objects.VersionRange("gt", 2))
- )
-
- def test_find_accurate(self):
- index = Index()
- p1 = gen_package(version=1)
- p2 = gen_package(version=2)
- index.add(p1)
- index.add(p2)
- self.assertIs(
- p1,
- index.find("package1", objects.VersionRange("eq", 1))
- )
- self.assertIsNone(
- index.find("package1", objects.VersionRange("eq", 3))
- )
-
- def test_find_obsolete(self):
- index = Index()
- p1 = gen_package(version=1)
- index.add(p1)
-
- self.assertIs(
- p1, index.find("obsoletes1", objects.VersionRange("le", 2))
- )
- self.assertIsNone(
- index.find("obsoletes1", objects.VersionRange("gt", 2))
- )
-
- def test_find_provides(self):
- index = Index()
- p1 = gen_package(version=1)
- p2 = gen_package(version=2)
- index.add(p1)
- index.add(p2)
-
- self.assertIs(
- p2, index.find("provides1", objects.VersionRange("ge", 2))
- )
- self.assertIsNone(
- index.find("provides1", objects.VersionRange("gt", 2))
- )
-
- def test_len(self):
- index = Index()
- for i in six.moves.range(3):
- index.add(gen_package(idx=i + 1))
- self.assertEqual(3, len(index))
-
- for i in six.moves.range(3):
- index.add(gen_package(idx=i + 1, version=2))
- self.assertEqual(6, len(index))
- self.assertEqual(3, len(index.packages))
-
- for i in six.moves.range(3):
- index.add(gen_package(idx=i + 1, version=2))
- self.assertEqual(6, len(index))
- self.assertEqual(3, len(index.packages))
diff --git a/packetary/tests/test_library_utils.py b/packetary/tests/test_library_utils.py
deleted file mode 100644
index b3a8d0b..0000000
--- a/packetary/tests/test_library_utils.py
+++ /dev/null
@@ -1,112 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import mock
-
-from packetary.library import utils
-from packetary.tests import base
-
-
-class TestLibraryUtils(base.TestCase):
- def test_append_token_to_string(self):
- self.assertEqual(
- "v1 v2 v3",
- utils.append_token_to_string("v2 v3", "v1")
- )
- self.assertEqual(
- "v1",
- utils.append_token_to_string("", "v1")
- )
- self.assertEqual(
- "v1 v2 v3 v4",
- utils.append_token_to_string('v1\tv2 v3', "v4")
- )
- self.assertEqual(
- "v1 v2 v3",
- utils.append_token_to_string('v1 v2 v3', "v1")
- )
-
- def test_composite_writer(self):
- fds = [
- mock.MagicMock(),
- mock.MagicMock()
- ]
- writer = utils.composite_writer(*fds)
- writer(u"text1")
- writer(b"text2")
- for fd in fds:
- fd.write.assert_any_call(b"text1")
- fd.write.assert_any_call(b"text2")
-
- @mock.patch.multiple(
- "packetary.library.utils",
- os=mock.DEFAULT,
- open=mock.DEFAULT
- )
- def test_get_size_and_checksum_for_files(self, os, open):
- files = [
- "/file1.txt",
- "/file2.txt"
- ]
- os.fstat.side_effect = [
- mock.MagicMock(st_size=1),
- mock.MagicMock(st_size=2)
- ]
- r = list(utils.get_size_and_checksum_for_files(
- files, mock.MagicMock(side_effect=["1", "2"])
- ))
- self.assertEqual(
- [("/file1.txt", 1, "1"), ("/file2.txt", 2, "2")],
- r
- )
-
- def test_get_path_from_url(self):
- self.assertEqual(
- "/a/f.txt",
- utils.get_path_from_url("/a/f.txt")
- )
-
- self.assertEqual(
- "/a/f.txt",
- utils.get_path_from_url("file:///a/f.txt?size=1")
- )
-
- with self.assertRaises(ValueError):
- utils.get_path_from_url("http:///a/f.txt")
-
- self.assertEqual(
- "/f.txt",
- utils.get_path_from_url("http://host/f.txt", False)
- )
-
- @mock.patch("packetary.library.utils.os")
- def test_ensure_dir_exist(self, os):
- os.makedirs.side_effect = [
- True,
- OSError(utils.errno.EEXIST, ""),
- OSError(utils.errno.EACCES, ""),
- ValueError()
- ]
- utils.ensure_dir_exist("/nonexisted")
- os.makedirs.assert_called_with("/nonexisted")
- utils.ensure_dir_exist("/existed")
- os.makedirs.assert_called_with("/existed")
- with self.assertRaises(OSError):
- utils.ensure_dir_exist("/private")
- with self.assertRaises(ValueError):
- utils.ensure_dir_exist(1)
diff --git a/packetary/tests/test_objects.py b/packetary/tests/test_objects.py
deleted file mode 100644
index 39dbc6e..0000000
--- a/packetary/tests/test_objects.py
+++ /dev/null
@@ -1,234 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import copy
-import six
-
-from packetary.objects import PackageRelation
-from packetary.objects import PackageVersion
-from packetary.objects import VersionRange
-
-
-from packetary.tests import base
-from packetary.tests.stubs import generator
-
-
-class TestObjectBase(base.TestCase):
- def check_copy(self, origin):
- clone = copy.copy(origin)
- self.assertIsNot(origin, clone)
- self.assertEqual(origin, clone)
- origin_name = origin.name
- origin.name += "1"
- self.assertEqual(
- origin_name,
- clone.name
- )
-
- def check_ordering(self, *args):
- for i in six.moves.range(len(args) - 1, 1, -1):
- self.assertLess(args[i - 1], args[i])
- self.assertGreater(args[i], args[i - 1])
-
- def check_equal(self, o1, o11, o2):
- self.assertEqual(o1, o11)
- self.assertEqual(o11, o1)
- self.assertNotEqual(o1, o2)
- self.assertNotEqual(o2, o1)
- self.assertNotEqual(o1, None)
-
- def check_hashable(self, o1, o2):
- d = dict()
- d[o1] = o2
- d[o2] = o1
-
- self.assertIs(o2, d[o1])
- self.assertIs(o1, d[o2])
-
-
-class TestPackageObject(TestObjectBase):
- def test_copy(self):
- self.check_copy(generator.gen_package(name="test1"))
-
- def test_ordering(self):
- self.check_ordering([
- generator.gen_package(name="test1", version=1),
- generator.gen_package(name="test1", version=2),
- generator.gen_package(name="test2", version=1),
- generator.gen_package(name="test2", version=2)
- ])
-
- def test_equal(self):
- self.check_equal(
- generator.gen_package(name="test1", version=1),
- generator.gen_package(name="test1", version=1),
- generator.gen_package(name="test2", version=1)
- )
-
- def test_hashable(self):
- self.check_hashable(
- generator.gen_package(name="test1", version=1),
- generator.gen_package(name="test2", version=1),
- )
- self.check_hashable(
- generator.gen_package(name="test1", version=1),
- generator.gen_package(name="test1", version=2),
- )
-
-
-class TestRepositoryObject(base.TestCase):
- def test_copy(self):
- origin = generator.gen_repository()
- clone = copy.copy(origin)
- self.assertEqual(clone.name, origin.name)
- self.assertEqual(clone.architecture, origin.architecture)
-
- def test_str(self):
- self.assertEqual(
- "a.b",
- str(generator.gen_repository(name=("a", "b")))
- )
- self.assertEqual(
- "/a/b/",
- str(generator.gen_repository(name="", url="/a/b/"))
- )
- self.assertEqual(
- "a",
- str(generator.gen_repository(name="a", url="/a/b/"))
- )
-
-
-class TestRelationObject(TestObjectBase):
- def test_equal(self):
- self.check_equal(
- generator.gen_relation(name="test1"),
- generator.gen_relation(name="test1"),
- generator.gen_relation(name="test2")
- )
-
- def test_hashable(self):
- self.check_hashable(
- generator.gen_relation(name="test1"),
- generator.gen_relation(name="test1", version=["le", 1])
- )
-
- def test_from_args(self):
- r = PackageRelation.from_args(
- ("test", "le", 2), ("test2",), ("test3",)
- )
- self.assertEqual("test", r.name)
- self.assertEqual("le", r.version.op)
- self.assertEqual(2, r.version.edge)
- self.assertEqual("test2", r.alternative.name)
- self.assertEqual(VersionRange(), r.alternative.version)
- self.assertEqual("test3", r.alternative.alternative.name)
- self.assertEqual(VersionRange(), r.alternative.alternative.version)
- self.assertIsNone(r.alternative.alternative.alternative)
-
- def test_iter(self):
- it = iter(PackageRelation.from_args(
- ("test", "le", 2), ("test2", "ge", 3))
- )
- self.assertEqual("test", next(it).name)
- self.assertEqual("test2", next(it).name)
- with self.assertRaises(StopIteration):
- next(it)
-
-
-class TestVersionRange(TestObjectBase):
- def test_equal(self):
- self.check_equal(
- VersionRange("eq", 1),
- VersionRange("eq", 1),
- VersionRange("le", 1)
- )
-
- def test_hashable(self):
- self.check_hashable(
- VersionRange(op="le"),
- VersionRange(op="le", edge=3)
- )
-
- def __check_intersection(self, assertion, cases):
- for data in cases:
- v1 = VersionRange(*data[0])
- v2 = VersionRange(*data[1])
- assertion(
- v1.has_intersection(v2), msg="%s and %s" % (v1, v2)
- )
- assertion(
- v2.has_intersection(v1), msg="%s and %s" % (v2, v1)
- )
-
- def test_have_intersection(self):
- cases = [
- (("eq", 2), ("eq", 2)),
- (("eq", 2), ("lt", 3)),
- (("eq", 2), ("gt", 1)),
- (("lt", 2), ("gt", 1)),
- (("lt", 2), ("lt", 3)),
- (("lt", 2), ("lt", 2)),
- (("lt", 2), ("le", 2)),
- (("gt", 2), ("gt", 1)),
- (("gt", 2), ("lt", 3)),
- (("gt", 2), ("ge", 2)),
- (("gt", 2), ("gt", 2)),
- (("ge", 2), ("le", 2)),
- ((None, None), ("eq", 2)),
- ]
- self.__check_intersection(self.assertTrue, cases)
-
- def test_does_not_have_intersection(self):
- cases = [
- (("eq", 2), ("eq", 1)),
- (("eq", 2), ("lt", 2)),
- (("eq", 2), ("gt", 2)),
- (("eq", 2), ("gt", 3)),
- (("eq", 2), ("lt", 1)),
- (("lt", 2), ("ge", 2)),
- (("lt", 2), ("gt", 3)),
- (("gt", 2), ("le", 2)),
- (("gt", 2), ("lt", 1)),
- ]
- self.__check_intersection(self.assertFalse, cases)
-
- def test_intersection_is_typesafe(self):
- with self.assertRaises(TypeError):
- VersionRange("eq", 1).has_intersection(("eq", 1))
-
-
-class TestPackageVersion(base.TestCase):
- def test_get_from_string(self):
- ver = PackageVersion.from_string("1.0-22")
- self.assertEqual(0, ver.epoch)
- self.assertEqual(('1', '0'), ver.version)
- self.assertEqual(('22',), ver.release)
-
- ver2 = PackageVersion.from_string("1-11.0-2")
- self.assertEqual(1, ver2.epoch)
- self.assertEqual(('11', '0'), ver2.version)
- self.assertEqual(('2',), ver2.release)
-
- def test_compare(self):
- ver1 = PackageVersion.from_string("6.3-31.5")
- ver2 = PackageVersion.from_string("13.9-16.12")
- self.assertLess(ver1, ver2)
- self.assertGreater(ver2, ver1)
- self.assertEqual(ver1, ver1)
- self.assertLess(ver1, "6.3-40")
- self.assertGreater(ver1, "6.3-31.4a")
diff --git a/packetary/tests/test_packages_tree.py b/packetary/tests/test_packages_tree.py
deleted file mode 100644
index f7e936e..0000000
--- a/packetary/tests/test_packages_tree.py
+++ /dev/null
@@ -1,135 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import warnings
-
-from packetary.objects import Index
-from packetary.objects import PackagesTree
-from packetary.tests import base
-from packetary.tests.stubs import generator
-
-
-class TestPackagesTree(base.TestCase):
- def setUp(self):
- super(TestPackagesTree, self).setUp()
-
- def test_get_unresolved_dependencies(self):
- ptree = PackagesTree()
- ptree.add(generator.gen_package(
- 1, requires=[generator.gen_relation("unresolved")]))
- ptree.add(generator.gen_package(2, requires=None))
- ptree.add(generator.gen_package(
- 3, requires=[generator.gen_relation("package1")]
- ))
- ptree.add(generator.gen_package(
- 4,
- requires=[generator.gen_relation("loop")],
- obsoletes=[generator.gen_relation("loop", ["le", 1])]
- ))
-
- unresolved = ptree.get_unresolved_dependencies()
- self.assertItemsEqual(
- ["loop", "unresolved"],
- (x.name for x in unresolved)
- )
-
- def test_get_unresolved_dependencies_with_main(self):
- ptree = PackagesTree()
- ptree.add(generator.gen_package(
- 1, requires=[generator.gen_relation("unresolved")]))
- ptree.add(generator.gen_package(2, requires=None))
- ptree.add(generator.gen_package(
- 3, requires=[generator.gen_relation("package1")]
- ))
- ptree.add(generator.gen_package(
- 4,
- requires=[generator.gen_relation("package5")]
- ))
- main = Index()
- main.add(generator.gen_package(5, requires=[
- generator.gen_relation("package6")
- ]))
-
- unresolved = ptree.get_unresolved_dependencies(main)
- self.assertItemsEqual(
- ["unresolved"],
- (x.name for x in unresolved)
- )
-
- def test_get_minimal_subset_with_master(self):
- ptree = PackagesTree()
- ptree.add(generator.gen_package(1, requires=None))
- ptree.add(generator.gen_package(2, requires=None))
- ptree.add(generator.gen_package(3, requires=None))
- ptree.add(generator.gen_package(
- 4, requires=[generator.gen_relation("package1")]
- ))
-
- master = Index()
- master.add(generator.gen_package(1, requires=None))
- master.add(generator.gen_package(
- 5,
- requires=[generator.gen_relation(
- "package10",
- alternative=generator.gen_relation("package4")
- )]
- ))
-
- unresolved = set([generator.gen_relation("package3")])
- resolved = ptree.get_minimal_subset(master, unresolved)
- self.assertItemsEqual(
- ["package3", "package4"],
- (x.name for x in resolved)
- )
-
- def test_get_minimal_subset_without_master(self):
- ptree = PackagesTree()
- ptree.add(generator.gen_package(1, requires=None))
- ptree.add(generator.gen_package(2, requires=None))
- ptree.add(generator.gen_package(
- 3, requires=[generator.gen_relation("package1")]
- ))
- unresolved = set([generator.gen_relation("package3")])
- resolved = ptree.get_minimal_subset(None, unresolved)
- self.assertItemsEqual(
- ["package3", "package1"],
- (x.name for x in resolved)
- )
-
- def test_mandatory_packages_always_included(self):
- ptree = PackagesTree()
- ptree.add(generator.gen_package(1, requires=None, mandatory=True))
- ptree.add(generator.gen_package(2, requires=None))
- ptree.add(generator.gen_package(3, requires=None))
- unresolved = set([generator.gen_relation("package3")])
- resolved = ptree.get_minimal_subset(None, unresolved)
- self.assertItemsEqual(
- ["package3", "package1"],
- (x.name for x in resolved)
- )
-
- def test_warning_if_unresolved(self):
- ptree = PackagesTree()
- ptree.add(generator.gen_package(
- 1, requires=None))
-
- with warnings.catch_warnings(record=True) as log:
- ptree.get_minimal_subset(
- None, [generator.gen_relation("package2")]
- )
- self.assertIn("package2", str(log[0]))
diff --git a/packetary/tests/test_repository_api.py b/packetary/tests/test_repository_api.py
deleted file mode 100644
index 7ea5599..0000000
--- a/packetary/tests/test_repository_api.py
+++ /dev/null
@@ -1,257 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import mock
-
-from packetary.api import Configuration
-from packetary.api import Context
-from packetary.api import RepositoryApi
-from packetary.tests import base
-from packetary.tests.stubs import generator
-from packetary.tests.stubs.helpers import CallbacksAdapter
-
-
-class TestRepositoryApi(base.TestCase):
- def test_get_packages_as_is(self):
- controller = CallbacksAdapter()
- pkg = generator.gen_package(name="test")
- controller.load_packages.side_effect = [
- pkg
- ]
- api = RepositoryApi(controller)
- packages = api.get_packages("file:///repo1")
- self.assertEqual(1, len(packages))
- package = packages.pop()
- self.assertIs(pkg, package)
-
- def test_get_packages_with_depends_resolving(self):
- controller = CallbacksAdapter()
- controller.load_packages.side_effect = [
- [
- generator.gen_package(idx=1, requires=None),
- generator.gen_package(
- idx=2, requires=[generator.gen_relation("package1")]
- ),
- generator.gen_package(
- idx=3, requires=[generator.gen_relation("package1")]
- ),
- generator.gen_package(idx=4, requires=None),
- generator.gen_package(idx=5, requires=None),
- ],
- generator.gen_package(
- idx=6, requires=[generator.gen_relation("package2")]
- ),
- ]
-
- api = RepositoryApi(controller)
- packages = api.get_packages([
- "file:///repo1", "file:///repo2"
- ],
- "file:///repo3", ["package4"]
- )
-
- self.assertEqual(3, len(packages))
- self.assertItemsEqual(
- ["package1", "package4", "package2"],
- (x.name for x in packages)
- )
- controller.load_repositories.assert_any_call(
- ["file:///repo1", "file:///repo2"]
- )
- controller.load_repositories.assert_any_call(
- "file:///repo3"
- )
-
- def test_clone_repositories_as_is(self):
- controller = CallbacksAdapter()
- repo = generator.gen_repository(name="repo1")
- packages = [
- generator.gen_package(name="test1", repository=repo),
- generator.gen_package(name="test2", repository=repo)
- ]
- mirror = generator.gen_repository(name="mirror")
- controller.load_repositories.return_value = repo
- controller.load_packages.return_value = packages
- controller.clone_repositories.return_value = {repo: mirror}
- controller.copy_packages.return_value = [0, 1]
- api = RepositoryApi(controller)
- stats = api.clone_repositories(
- ["file:///repo1"], "/mirror", keep_existing=True
- )
- self.assertEqual(2, stats.total)
- self.assertEqual(1, stats.copied)
- controller.copy_packages.assert_called_once_with(
- mirror, set(packages), True
- )
-
- def test_copy_minimal_subset_of_repository(self):
- controller = CallbacksAdapter()
- repo1 = generator.gen_repository(name="repo1")
- repo2 = generator.gen_repository(name="repo2")
- repo3 = generator.gen_repository(name="repo3")
- mirror1 = generator.gen_repository(name="mirror1")
- mirror2 = generator.gen_repository(name="mirror2")
- pkg_group1 = [
- generator.gen_package(
- idx=1, requires=None, repository=repo1
- ),
- generator.gen_package(
- idx=1, version=2, requires=None, repository=repo1
- ),
- generator.gen_package(
- idx=2, requires=None, repository=repo1
- )
- ]
- pkg_group2 = [
- generator.gen_package(
- idx=4,
- requires=[generator.gen_relation("package1")],
- repository=repo2,
- mandatory=True,
- )
- ]
- pkg_group3 = [
- generator.gen_package(
- idx=3, requires=None, repository=repo1
- )
- ]
- controller.load_repositories.side_effect = [[repo1, repo2], repo3]
- controller.load_packages.side_effect = [
- pkg_group1 + pkg_group2 + pkg_group3,
- generator.gen_package(
- idx=6,
- repository=repo3,
- requires=[generator.gen_relation("package2")]
- )
- ]
- controller.clone_repositories.return_value = {
- repo1: mirror1, repo2: mirror2
- }
- controller.copy_packages.return_value = 1
- api = RepositoryApi(controller)
- api.clone_repositories(
- ["file:///repo1", "file:///repo2"], "/mirror",
- ["file:///repo3"],
- keep_existing=True
- )
- controller.copy_packages.assert_any_call(
- mirror1, set(pkg_group1), True
- )
- controller.copy_packages.assert_any_call(
- mirror2, set(pkg_group2), True
- )
- self.assertEqual(2, controller.copy_packages.call_count)
-
- def test_get_unresolved(self):
- controller = CallbacksAdapter()
- pkg = generator.gen_package(
- name="test", requires=[generator.gen_relation("test2")]
- )
- controller.load_packages.side_effect = [
- pkg
- ]
- api = RepositoryApi(controller)
- r = api.get_unresolved_dependencies("file:///repo1")
- controller.load_repositories.assert_called_once_with("file:///repo1")
- self.assertItemsEqual(
- ["test2"],
- (x.name for x in r)
- )
-
- def test_get_unresolved_with_main(self):
- controller = CallbacksAdapter()
- pkg1 = generator.gen_package(
- name="test1", requires=[
- generator.gen_relation("test2"),
- generator.gen_relation("test3")
- ]
- )
- pkg2 = generator.gen_package(
- name="test2", requires=[generator.gen_relation("test4")]
- )
- controller.load_packages.side_effect = [
- pkg1, pkg2
- ]
- api = RepositoryApi(controller)
- r = api.get_unresolved_dependencies("file:///repo1", "file:///repo2")
- controller.load_repositories.assert_any_call("file:///repo1")
- controller.load_repositories.assert_any_call("file:///repo2")
- self.assertItemsEqual(
- ["test3"],
- (x.name for x in r)
- )
-
- def test_parse_requirements(self):
- requirements = RepositoryApi._parse_requirements(
- ["p1 le 2 | p2 | p3 ge 2"]
- )
-
- expected = generator.gen_relation(
- "p1",
- ["le", '2'],
- generator.gen_relation(
- "p2",
- None,
- generator.gen_relation(
- "p3",
- ["ge", '2']
- )
- )
- )
- self.assertEqual(1, len(requirements))
- self.assertEqual(
- list(expected),
- list(requirements.pop())
- )
-
-
-class TestContext(base.TestCase):
- @classmethod
- def setUpClass(cls):
- cls.config = Configuration(
- threads_num=2,
- ignore_errors_num=3,
- retries_num=5,
- retry_interval=5,
- http_proxy="http://localhost",
- https_proxy="https://localhost"
- )
-
- @mock.patch("packetary.api.ConnectionsManager")
- def test_initialise_connection_manager(self, conn_manager):
- context = Context(self.config)
- conn_manager.assert_called_once_with(
- proxy="http://localhost",
- secure_proxy="https://localhost",
- retries_num=5,
- retry_interval=5
- )
-
- self.assertIs(
- conn_manager(),
- context.connection
- )
-
- @mock.patch("packetary.api.AsynchronousSection")
- def test_asynchronous_section(self, async_section):
- context = Context(self.config)
- s = context.async_section()
- async_section.assert_called_with(2, 3)
- self.assertIs(s, async_section())
- context.async_section(0)
- async_section.assert_called_with(2, 0)
diff --git a/packetary/tests/test_repository_contoller.py b/packetary/tests/test_repository_contoller.py
deleted file mode 100644
index 9eb6cba..0000000
--- a/packetary/tests/test_repository_contoller.py
+++ /dev/null
@@ -1,145 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import copy
-import mock
-import six
-
-from packetary.controllers import RepositoryController
-from packetary.tests import base
-from packetary.tests.stubs.executor import Executor
-from packetary.tests.stubs.generator import gen_package
-from packetary.tests.stubs.generator import gen_repository
-from packetary.tests.stubs.helpers import CallbacksAdapter
-
-
-class TestRepositoryController(base.TestCase):
- def setUp(self):
- self.driver = mock.MagicMock()
- self.context = mock.MagicMock()
- self.context.async_section.return_value = Executor()
- self.ctrl = RepositoryController(self.context, self.driver, "x86_64")
-
- def test_load_fail_if_unknown_driver(self):
- with self.assertRaisesRegexp(NotImplementedError, "unknown_driver"):
- RepositoryController.load(
- self.context,
- "unknown_driver",
- "x86_64"
- )
-
- @mock.patch("packetary.controllers.repository.stevedore")
- def test_load_driver(self, stevedore):
- stevedore.ExtensionManager.return_value = {
- "test": mock.MagicMock(obj=self.driver)
- }
- RepositoryController._drivers = None
- controller = RepositoryController.load(self.context, "test", "x86_64")
- self.assertIs(self.driver, controller.driver)
-
- def test_load_repositories(self):
- self.driver.parse_urls.return_value = ["test1"]
- consumer = mock.MagicMock()
- self.ctrl.load_repositories("file:///test1", consumer)
- self.driver.parse_urls.assert_called_once_with(["file:///test1"])
- self.driver.get_repository.assert_called_once_with(
- self.context.connection, "test1", "x86_64", consumer
- )
- for url in [six.u("file:///test1"), ["file:///test1"]]:
- self.driver.reset_mock()
- self.ctrl.load_repositories(url, consumer)
- if not isinstance(url, list):
- url = [url]
- self.driver.parse_urls.assert_called_once_with(url)
-
- def test_load_packages(self):
- repo = mock.MagicMock()
- consumer = mock.MagicMock()
- self.ctrl.load_packages([repo], consumer)
- self.driver.get_packages.assert_called_once_with(
- self.context.connection, repo, consumer
- )
-
- @mock.patch("packetary.controllers.repository.os")
- def test_assign_packages(self, os):
- repo = gen_repository(url="/test/repo")
- packages = [
- gen_package(name="test1", repository=repo),
- gen_package(name="test2", repository=repo)
- ]
- existed_packages = [
- gen_package(name="test3", repository=repo),
- gen_package(name="test2", repository=repo)
- ]
-
- os.path.join = lambda *x: "/".join(x)
- self.driver.get_packages = CallbacksAdapter()
- self.driver.get_packages.return_value = existed_packages
- self.ctrl.assign_packages(repo, packages, True)
- os.remove.assert_not_called()
- all_packages = set(packages + existed_packages)
- self.driver.rebuild_repository.assert_called_once_with(
- repo, all_packages
- )
- self.driver.rebuild_repository.reset_mock()
- self.ctrl.assign_packages(repo, packages, False)
- self.driver.rebuild_repository.assert_called_once_with(
- repo, set(packages)
- )
- os.remove.assert_called_once_with("/test/repo/test3.pkg")
-
- def test_copy_packages(self):
- repo = gen_repository(url="file:///repo/")
- packages = [
- gen_package(name="test1", repository=repo, filesize=10),
- gen_package(name="test2", repository=repo, filesize=-1)
- ]
- target = gen_repository(url="/test/repo")
- self.context.connection.retrieve.side_effect = [0, 10]
- observer = mock.MagicMock()
- self.ctrl.copy_packages(target, packages, True, observer)
- observer.assert_has_calls([mock.call(0), mock.call(10)])
- self.context.connection.retrieve.assert_any_call(
- "file:///repo/test1.pkg",
- "/test/repo/test1.pkg",
- size=10
- )
- self.context.connection.retrieve.assert_any_call(
- "file:///repo/test2.pkg",
- "/test/repo/test2.pkg",
- size=-1
- )
- self.driver.rebuild_repository.assert_called_once_with(
- target, set(packages)
- )
-
- @mock.patch("packetary.controllers.repository.os")
- def test_clone_repository(self, os):
- os.path.abspath.return_value = "/root/repo"
- repos = [
- gen_repository(name="test1"),
- gen_repository(name="test2")
- ]
- clones = [copy.copy(x) for x in repos]
- self.driver.fork_repository.side_effect = clones
- mirrors = self.ctrl.clone_repositories(repos, "./repo")
- for r in repos:
- self.driver.fork_repository.assert_any_call(
- self.context.connection, r, "/root/repo", False, False
- )
- self.assertEqual(mirrors, dict(zip(repos, clones)))
diff --git a/packetary/tests/test_rpm_driver.py b/packetary/tests/test_rpm_driver.py
deleted file mode 100644
index 47dc6fb..0000000
--- a/packetary/tests/test_rpm_driver.py
+++ /dev/null
@@ -1,211 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import mock
-import os.path as path
-import sys
-
-import six
-
-from packetary.library.utils import localize_repo_url
-from packetary.objects import FileChecksum
-from packetary.tests import base
-from packetary.tests.stubs.generator import gen_repository
-from packetary.tests.stubs.helpers import get_compressed
-
-
-REPOMD = path.join(path.dirname(__file__), "data", "repomd.xml")
-
-REPOMD2 = path.join(path.dirname(__file__), "data", "repomd2.xml")
-
-PRIMARY_DB = path.join(path.dirname(__file__), "data", "primary.xml")
-
-GROUPS_DB = path.join(path.dirname(__file__), "data", "groups.xml")
-
-
-class TestRpmDriver(base.TestCase):
- @classmethod
- def setUpClass(cls):
- cls.createrepo = sys.modules["createrepo"] = mock.MagicMock()
- # import driver class after patching sys.modules
- from packetary.drivers import rpm_driver
-
- super(TestRpmDriver, cls).setUpClass()
- cls.driver = rpm_driver.RpmRepositoryDriver()
-
- def setUp(self):
- self.createrepo.reset_mock()
- self.connection = mock.MagicMock()
-
- def test_parse_urls(self):
- self.assertItemsEqual(
- [
- "http://host/centos/os",
- "http://host/centos/updates"
- ],
- self.driver.parse_urls([
- "http://host/centos/os",
- "http://host/centos/updates/",
- ])
- )
-
- def test_get_repository(self):
- repos = []
-
- self.driver.get_repository(
- self.connection,
- "http://host/centos/os/x86_64",
- "x86_64",
- repos.append
- )
-
- self.assertEqual(1, len(repos))
- repo = repos[0]
- self.assertEqual("/centos/os/x86_64", repo.name)
- self.assertEqual("", repo.origin)
- self.assertEqual("x86_64", repo.architecture)
- self.assertEqual("http://host/centos/os/x86_64/", repo.url)
-
- def test_get_packages(self):
- streams = []
- for conv, fname in zip(
- (lambda x: six.BytesIO(x.read()),
- get_compressed, get_compressed),
- (REPOMD, GROUPS_DB, PRIMARY_DB)
- ):
- with open(fname, "rb") as s:
- streams.append(conv(s))
-
- packages = []
- self.connection.open_stream.side_effect = streams
- self.driver.get_packages(
- self.connection,
- gen_repository("test", url="http://host/centos/os/x86_64/"),
- packages.append
- )
- self.connection.open_stream.assert_any_call(
- "http://host/centos/os/x86_64/repodata/repomd.xml"
- )
- self.connection.open_stream.assert_any_call(
- "http://host/centos/os/x86_64/repodata/groups.xml.gz"
- )
- self.connection.open_stream.assert_any_call(
- "http://host/centos/os/x86_64/repodata/primary.xml.gz"
- )
- self.assertEqual(2, len(packages))
- package = packages[0]
- self.assertEqual("test1", package.name)
- self.assertEqual("1.1.1.1-1.el7", package.version)
- self.assertEqual(100, package.filesize)
- self.assertEqual(
- FileChecksum(
- None,
- None,
- 'e8ed9e0612e813491ed5e7c10502a39e'
- '43ec665afd1321541dea211202707a65'),
- package.checksum
- )
- self.assertEqual(
- "Packages/test1.rpm", package.filename
- )
- self.assertItemsEqual(
- ['test2 (eq 0-1.1.1.1-1.el7)'],
- (str(x) for x in package.requires)
- )
- self.assertItemsEqual(
- ["file (any)"],
- (str(x) for x in package.provides)
- )
- self.assertItemsEqual(
- ["test-old (any)"],
- (str(x) for x in package.obsoletes)
- )
- self.assertTrue(package.mandatory)
- self.assertFalse(packages[1].mandatory)
-
- def test_get_packages_if_group_not_gzipped(self):
- streams = []
- for conv, fname in zip(
- (lambda x: six.BytesIO(x.read()),
- lambda x: six.BytesIO(x.read()),
- get_compressed),
- (REPOMD2, GROUPS_DB, PRIMARY_DB)
- ):
- with open(fname, "rb") as s:
- streams.append(conv(s))
-
- packages = []
- self.connection.open_stream.side_effect = streams
- self.driver.get_packages(
- self.connection,
- gen_repository("test", url="http://host/centos/os/x86_64/"),
- packages.append
- )
- self.connection.open_stream.assert_any_call(
- "http://host/centos/os/x86_64/repodata/groups.xml"
- )
- self.assertEqual(2, len(packages))
- package = packages[0]
- self.assertTrue(package.mandatory)
-
- @mock.patch("packetary.drivers.rpm_driver.shutil")
- def test_rebuild_repository(self, shutil):
- self.createrepo.MDError = ValueError
- self.createrepo.MetaDataGenerator().doFinalMove.side_effect = [
- None, self.createrepo.MDError()
- ]
- repo = gen_repository("test", url="file:///repo/os/x86_64")
- self.createrepo.MetaDataConfig().outputdir = "/repo/os/x86_64"
- self.createrepo.MetaDataConfig().tempdir = "tmp"
-
- self.driver.rebuild_repository(repo, set())
-
- self.assertEqual(
- "/repo/os/x86_64",
- self.createrepo.MetaDataConfig().directory
- )
- self.assertTrue(self.createrepo.MetaDataConfig().update)
- self.createrepo.MetaDataGenerator()\
- .doPkgMetadata.assert_called_once_with()
- self.createrepo.MetaDataGenerator()\
- .doRepoMetadata.assert_called_once_with()
- self.createrepo.MetaDataGenerator()\
- .doFinalMove.assert_called_once_with()
-
- with self.assertRaises(RuntimeError):
- self.driver.rebuild_repository(repo, set())
- shutil.rmtree.assert_called_once_with(
- "/repo/os/x86_64/tmp", ignore_errors=True
- )
-
- @mock.patch("packetary.drivers.rpm_driver.utils")
- def test_fork_repository(self, utils):
- repo = gen_repository("os", url="http://localhost/os/x86_64/")
- utils.localize_repo_url = localize_repo_url
- new_repo = self.driver.fork_repository(
- self.connection,
- repo,
- "/repo"
- )
-
- utils.ensure_dir_exist.assert_called_once_with("/repo/os/x86_64/")
- self.assertEqual(repo.name, new_repo.name)
- self.assertEqual(repo.architecture, new_repo.architecture)
- self.assertEqual("/repo/os/x86_64/", new_repo.url)
- self.createrepo.MetaDataGenerator()\
- .doFinalMove.assert_called_once_with()
diff --git a/packetary/tests/test_streams.py b/packetary/tests/test_streams.py
deleted file mode 100644
index 68b4a8f..0000000
--- a/packetary/tests/test_streams.py
+++ /dev/null
@@ -1,99 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-import six
-
-from packetary.library import streams
-from packetary.tests import base
-from packetary.tests.stubs.helpers import get_compressed
-
-
-class TestBufferedStream(base.TestCase):
- def setUp(self):
- super(TestBufferedStream, self).setUp()
- self.stream = streams.StreamWrapper(
- six.BytesIO(b"line1\nline2\nline3\n")
- )
-
- def test_read(self):
- self.stream.CHUNK_SIZE = 10
- chunk = self.stream.read(5)
- self.assertEqual(b"line1", chunk)
- self.assertEqual(b"\nline", self.stream.unread_tail)
- chunk = self.stream.read(1024)
- self.assertEqual(b"\nline2\nline3\n", chunk)
- self.assertEqual(b"", self.stream.unread_tail)
-
- def test_readline(self):
- self.stream.CHUNK_SIZE = 12
- chunk = self.stream.readline()
- self.assertEqual(b"line1\n", chunk)
- self.assertEqual(b"line2\n", self.stream.unread_tail)
- lines = list(self.stream.readlines())
- self.assertEqual([b"line2\n", b"line3\n"], lines)
- self.assertEqual(b"", self.stream.unread_tail)
-
- def test_readlines(self):
- self.stream.CHUNK_SIZE = 12
- lines = list(self.stream.readlines())
- self.assertEqual(
- [b"line1\n", b"line2\n", b"line3\n"],
- lines)
-
-
-class TestGzipDecompress(base.TestCase):
- @classmethod
- def setUpClass(cls):
- cls.compressed = get_compressed(six.BytesIO(b"line1\nline2\nline3\n"))
-
- def setUp(self):
- super(TestGzipDecompress, self).setUp()
- self.compressed.reset()
- self.stream = streams.GzipDecompress(self.compressed)
-
- def test_read(self):
- chunk = self.stream.read(5)
- self.assertEqual(b"line1", chunk)
- self.assertEqual(b"\nline2\nline3\n", self.stream.unread_tail)
- chunk = self.stream.read(1024)
- self.assertEqual(b"\nline2\nline3\n", chunk)
- self.assertEqual(b"", self.stream.unread_tail)
-
- def test_readline(self):
- self.stream.CHUNK_SIZE = 12
- chunk = self.stream.readline()
- self.assertEqual(b"line1\n", chunk)
- self.assertEqual(b"line2\nl", self.stream.unread_tail)
- lines = list(self.stream.readlines())
- self.assertEqual([b"line2\n", b"line3\n"], lines)
- self.assertEqual(b"", self.stream.unread_tail)
-
- def test_readlines(self):
- self.stream.CHUNK_SIZE = 12
- lines = list(self.stream.readlines())
- self.assertEqual(
- [b"line1\n", b"line2\n", b"line3\n"],
- lines)
-
- def test_handle_case_if_not_enough_data_to_decompress(self):
- self.stream.CHUNK_SIZE = 1
- chunk = self.stream.read()
- self.assertEqual(
- b"line1\nline2\nline3\n",
- chunk
- )
diff --git a/requirements.txt b/requirements.txt
deleted file mode 100644
index a369282..0000000
--- a/requirements.txt
+++ /dev/null
@@ -1,14 +0,0 @@
-# The order of packages is significant, because pip processes them in the order
-# of appearance. Changing the order has an impact on the overall integration
-# process, which may cause wedges in the gate later.
-
-pbr>=0.8
-Babel>=1.3
-cliff>=1.7.0
-eventlet>=0.15
-bintrees>=2.0.2
-chardet>=2.0.1
-stevedore>=1.1.0
-six>=1.5.2
-python-debian>=0.1.21
-lxml>=3.2
diff --git a/setup.cfg b/setup.cfg
deleted file mode 100644
index 8c0a808..0000000
--- a/setup.cfg
+++ /dev/null
@@ -1,67 +0,0 @@
-[metadata]
-name = packetary
-version = 10.0.0
-summary = The chain of tools to manage package`s lifecycle.
-description-file =
- README.rst
-author = Mirantis Inc.
-author_email = product@mirantis.com
-url = http://mirantis.com
-home-page = http://mirantis.com
-classifier =
- Development Status :: 4 - Beta
- Environment :: OpenStack
- Intended Audience :: Information Technology
- Intended Audience :: System Administrators
- License :: OSI Approved :: GNU General Public License v2 (GPLv2)
- Operating System :: POSIX :: Linux
- Programming Language :: Python
- Programming Language :: Python :: 2
- Programming Language :: Python :: 2.7
- Programming Language :: Python :: 3
- Programming Language :: Python :: 3.4
- Topic :: Utilities
-
-[files]
-packages =
- packetary
-
-[entry_points]
-console_scripts =
- packetary=packetary.cli.app:main
-
-packetary.drivers =
- deb=packetary.drivers.deb_driver:DebRepositoryDriver
- rpm=packetary.drivers.rpm_driver:RpmRepositoryDriver
-
-packetary =
- clone=packetary.cli.commands.clone:CloneCommand
- packages=packetary.cli.commands.packages:ListOfPackages
- unresolved=packetary.cli.commands.unresolved:ListOfUnresolved
-
-[build_sphinx]
-source-dir = doc/source
-build-dir = doc/build
-all_files = 1
-
-[upload_sphinx]
-upload-dir = doc/build/html
-
-[compile_catalog]
-directory = packetary/locale
-domain = packetary
-
-[update_catalog]
-domain = packetary
-output_dir = packetary/locale
-input_file = packetary/locale/packetary.pot
-
-[extract_messages]
-keywords = _ gettext ngettext l_ lazy_gettext
-mapping_file = babel.cfg
-output_file = packetary/locale/packetary.pot
-
-[global]
-setup-hooks =
- pbr.hooks.setup_hook
- setup_hooks.setup_hook
diff --git a/setup.py b/setup.py
deleted file mode 100644
index ec4d54a..0000000
--- a/setup.py
+++ /dev/null
@@ -1,33 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-# THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT
-import setuptools
-
-# In python < 2.7.4, a lazy loading of package `pbr` will break
-# setuptools if some other modules registered functions in `atexit`.
-# solution from: http://bugs.python.org/issue15881#msg170215
-try:
- import multiprocessing # noqa
-except ImportError:
- pass
-
-
-setuptools.setup(
- setup_requires=['pbr'],
- pbr=True)
diff --git a/setup_hooks.py b/setup_hooks.py
deleted file mode 100644
index 94cc99e..0000000
--- a/setup_hooks.py
+++ /dev/null
@@ -1,25 +0,0 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2015 Mirantis, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-
-def setup_hook(config):
- import pbr
- import pbr.packaging
-
- # this monkey patch is to avoid appending git version to version
- pbr.packaging._get_version_from_git = lambda pre_version: pre_version
diff --git a/specs/fuel-mirror.spec b/specs/fuel-mirror.spec
deleted file mode 100644
index ef646cc..0000000
--- a/specs/fuel-mirror.spec
+++ /dev/null
@@ -1,95 +0,0 @@
-%define name fuel-mirror
-%{!?version: %define version 10.0.0}
-%{!?release: %define release 1}
-
-Name: %{name}
-Version: %{version}
-Release: %{release}
-Source0: %{name}-%{version}.tar.gz
-Summary: Utility to create RPM and DEB mirror
-URL: http://mirantis.com
-License: GPLv2
-Group: Utilities
-BuildRoot: %{_tmppath}/%{name}-%{version}-buildroot
-Prefix: %{_prefix}
-BuildRequires: git
-BuildRequires: python-setuptools
-BuildRequires: python-pbr
-BuildArch: noarch
-
-Requires: python
-Requires: python-babel >= 1.3
-Requires: python-cliff >= 1.7.0
-Requires: python-fuelclient >= 7.0.0
-Requires: python-packetary == %{version}
-Requires: python-pbr >= 0.8
-Requires: python-six >= 1.5.2
-Requires: PyYAML >= 3.10
-# Workaroud for babel bug
-Requires: pytz
-
-
-%description
-Provides two commands fuel-mirror and fuel-createmirror.
-Second one is for backward compatibility with the previous
-generation of the utility. These commands could be used
-to create local copies of MOS and upstream deb and rpm
-repositories.
-
-%package -n python-packetary
-Summary: Library that allows to build and clone deb and rpm repositories
-Group: Development/Libraries
-
-Requires: createrepo
-Requires: python
-Requires: python-babel >= 1.3
-Requires: python-bintrees >= 2.0.2
-Requires: python-chardet >= 2.0.1
-Requires: python-cliff >= 1.7.0
-Requires: python-debian >= 0.1.21
-Requires: python-eventlet >= 0.15
-Requires: python-lxml >= 1.1.23
-Requires: python-pbr >= 0.8
-Requires: python-six >= 1.5.2
-Requires: python-stevedore >= 1.1.0
-# Workaroud for babel bug
-Requires: pytz
-
-%description -n python-packetary
-Provides object model and API for dealing with deb
-and rpm repositories. One can use this framework to
-implement operations like building repository
-from a set of packages, clone repository, find package
-dependencies, mix repositories, pull out a subset of
-packages into a separate repository, etc.
-
-
-%prep
-%setup -cq -n %{name}-%{version}
-
-%build
-
-cd %{_builddir}/%{name}-%{version} && python setup.py build
-cd %{_builddir}/%{name}-%{version}/contrib/fuel_mirror && python setup.py build
-
-%install
-cd %{_builddir}/%{name}-%{version} && python setup.py install --single-version-externally-managed -O1 --root=$RPM_BUILD_ROOT --record=%{_builddir}/%{name}-%{version}/INSTALLED_FILES
-cd %{_builddir}/%{name}-%{version}/contrib/fuel_mirror && python setup.py install --single-version-externally-managed -O1 --root=$RPM_BUILD_ROOT --record=%{_builddir}/%{name}-%{version}/contrib/fuel_mirror/INSTALLED_FILES
-
-mkdir -p %{buildroot}/etc/%{name}
-mkdir -p %{buildroot}/usr/bin
-mkdir -p %{buildroot}/usr/share/%{name}
-install -m 755 %{_builddir}/%{name}-%{version}/contrib/fuel_mirror/scripts/fuel-createmirror %{buildroot}/usr/bin/fuel-createmirror
-install -m 755 %{_builddir}/%{name}-%{version}/contrib/fuel_mirror/etc/config.yaml %{buildroot}/etc/%{name}/config.yaml
-
-%clean
-rm -rf $RPM_BUILD_ROOT
-
-%files -f %{_builddir}/%{name}-%{version}/contrib/fuel_mirror/INSTALLED_FILES
-%defattr(0755,root,root)
-/usr/bin/fuel-createmirror
-%attr(0644,root,root) /etc/%{name}/config.yaml
-
-
-%files -n python-packetary -f %{_builddir}/%{name}-%{version}/INSTALLED_FILES
-%defattr(-,root,root)
diff --git a/test-requirements.txt b/test-requirements.txt
deleted file mode 100644
index 0c885d3..0000000
--- a/test-requirements.txt
+++ /dev/null
@@ -1,15 +0,0 @@
-# The order of packages is significant, because pip processes them in the order
-# of appearance. Changing the order has an impact on the overall integration
-# process, which may cause wedges in the gate later.
-
-hacking<0.11,>=0.10.0
-
-coverage>=3.6
-discover
-python-subunit>=0.0.18
-sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2
-oslosphinx>=2.5.0 # Apache-2.0
-oslotest>=1.10.0 # Apache-2.0
-testrepository>=0.0.18
-testscenarios>=0.4
-testtools>=1.4.0
diff --git a/tox.ini b/tox.ini
index 3e02f93..2717aed 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,36 +1,12 @@
[tox]
minversion = 1.6
-envlist = py34,py27,pep8
+envlist = pep8
skipsdist = True
[testenv]
-usedevelop = True
-install_command = pip install -U {opts} {packages}
-setenv =
- VIRTUAL_ENV={envdir}
-passenv = http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY
-deps = -r{toxinidir}/test-requirements.txt
-commands = python setup.py test --slowest --testr-args='{posargs:packetary}'
+usedevelop = False
+whitelist_externals = bash
+commands =
[testenv:pep8]
-commands = flake8 {posargs:packetary}
-
-[testenv:venv]
-commands = {posargs:packetary}
-
-[testenv:cover]
-commands = python setup.py test --coverage --testr-args='{posargs:packetary}'
-
-[testenv:docs]
-commands = python setup.py build_sphinx
-
-[testenv:debug]
-commands = oslo_debug_helper {posargs:packetary}
-
-[flake8]
-# E123, E125 skipped as they are invalid PEP-8.
-
-show-source = True
-ignore = E123,E125
-builtins = _
-exclude=*egg,*lib/python*,*openstack/common*,.git,.idea,.tox,.venv,build,dist,doc
+commands = bash -c "exit 0"