Commit Graph

26 Commits

Author SHA1 Message Date
Gage Hugo 0a388d5fe2 Remove mini-mirror image building
As part of the effort to retire the osh-addons repo, this change
removes the image building for mini mirror.

Change-Id: Icb1254ff3a8958adb92b30870a99583121345b8a
2022-06-02 13:24:14 -05:00
Andrii Ostapenko c30e92f201
Fix gates
Fixes get-pip url
Updates go to 1.17.6 for mini-mirror

Change-Id: Ieaf35d935381416b0acd93113ed26cf981e7461b
Signed-off-by: Andrii Ostapenko <aostapenko@microsoft.com>
2022-02-02 10:13:13 -07:00
anthony.bellino 41248cf70a mini-mirror: Use no-tty and batch mode for gpg
Change-Id: I99347b39cd0b6c4cda84a9496a2b8747088c6173
2021-11-19 17:42:43 +00:00
Stephen Taylor 608d40fd8a mini-mirror: Upate build distro to bionic
The xenial Dockerfile has been removed from the repo, but build.sh
still tries to use it by default. This change updates the build
distro to bionic.

Change-Id: Ic6fef1ea51de8bfb1d35bdbf17227086dc01bf8a
2021-11-15 13:25:27 -07:00
Andrii Ostapenko 966adc41ef Cleanup xenial based images when latter alternative is available
Also remove redundant PYTHON3 build argument since it's a default one.

Depends-On: https://review.opendev.org/813709
Change-Id: Id31f5feb5ba7837ef698b9c9bd826a8fb4b7a572
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2021-10-13 17:59:58 +00:00
Phil Sphicas 00401e1d92 Align mini-mirror Dockerfile args with build.sh
The mini-mirror build.sh script has somehow ended up out of sync with
the Dockerfiles, and is not passing the correct build-args to allow base
image overrides.

This change aligns with the standard of using BUILD_FROM (ubuntu) for
the build stage and FROM (nginx) for the final stage.

Relevant earlier changes:
0: https://review.opendev.org/714002
1: https://review.opendev.org/679315

Change-Id: I844efa669c0f11004a44f20f6725338e1d2699b6
2021-02-20 03:34:53 +00:00
Pete Birley 5046975869 Images: Update all images to allow base image over-rides
This PS updates all dockerfiles to allow base image over-rides as
required by many consumers of OSH.

Change-Id: I53daac9c1ec4707628565abc912116d8997eba16
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-08-11 04:50:26 +00:00
Gage Hugo e9b2ff0c74 Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: Ic78d29883364378cc14b11402f16d99dcec1fc96
2020-05-07 02:11:23 +00:00
Michael Beaver be9c3b6c13 Add base image overrides to mini-mirror image
This adds environment variables for the build script that can be set
to individually override the base ubuntu build image for mini-mirror
as well as the final nginx image. This does not change the deafult
functionality and allows the mini-mirror build to be more configurable
if the end user wishes to substitute in their own images

Change-Id: I49e0415354ce8953b759de92367849b8f9ece50b
2020-03-26 12:40:06 -05:00
Michael Beaver e45829dc9e Fix source publishing and publish logic
Sometime recently the logic in aptly changed so that even if the
configuration option "downloadSourcePackages" is true, the source
packages will not be published in the resulting mirror unless the
architectures list includes "source", but adding "source" to the
actual architecture list for a repo will prevent the mirror creation
from functioning as most repos don't have a "source" architecture.

This change adds a new check to look at the aptly configuration file to
determine if "downloadSourcePackages" is true, and if so then it will
add a flag to the publish command with the architectures from the
configuration plus the "source" architecture.

I also modified the publish command to bring it into one command while
still allowing it to be configurable as it was before.

This also removes the "set -x" to limit the excessive output

Change-Id: Iebf1e862cb0a48a5a5eeda6ea0e8df3537eb602b
2020-02-06 13:16:03 -06:00
Michael Beaver d503870bfe Modify mini-mirror to build from yaml
This modifies mini-mirror so that instead of parsing through .txt
documents to build the mirror, it instead has one yaml file consisting
of multiple yaml documents with a pre-defined structure that has all
of the same information. This will allow for easier modification and
visibility into the configuration of mini-mirror.

Along with that, this enables the manual setting of CodeName and Label
values of the mirror, as well as the ability to change the name of the
image that the build.sh script creates

Change-Id: I08646258492e8938b8892749172aea23e363af3d
2020-01-10 14:34:24 -06:00
Phil Sphicas 101e8f8b33 Revert "mini-mirror: Create one mirror per repo"
This reverts commit 2527c091c1.

Change-Id: Iccfd61b87596971ce44a159f81f79bcccb1bf1d3
2019-12-27 14:23:48 -08:00
Phil Sphicas 066efa5767 mini-mirror: Preserve codename and label
Ubuntu 18.04 performs strict checking of the Codename and Label when
performing an apt-get update. The snapshots published by aptly do not
set these parameters in a way that apt-get likes.

This change explicitly sets the codename and label based on the values
from the upstream repo being mirrored.

Note: this relies on a currently unmerged pull request/
https://github.com/aptly-dev/aptly/pull/892

Change-Id: I011862dcf9efbb8b1f9757c65f3d58da49a496d7
2019-12-26 12:44:11 -08:00
Phil Sphicas 6c3f46abd8 mini-mirror: Build aptly from source
This change allows aptly to be built from source instead of installed
from the aptly repo.

Change-Id: I96495bd08babadf8ab1d332e86b251bd985248be
2019-12-26 12:43:43 -08:00
Phil Sphicas 2527c091c1 mini-mirror: Create one mirror per repo
When mirroring a repo with multiple components, mini-mirror will create
a separate mirror for each component, create a snapshot for each of the
component repos, and then publish a merge of the snapshots.

This change makes it a one-to-one mapping of aptly mirrors to repos.

Change-Id: Ib12b29097c5d734982d613d3d01b057791170733
2019-12-26 12:41:28 -08:00
Egorov, Stanislav 489a3e1678 Fix build script arg and re-order Dockerfile
Simplified apt key import for minimirror

This also moves around the commands in the Dockerfiles so caching makes
a bit more sense, such as only copying over a script right before it
is used so previous steps don't need to be re-run when developing
locally.

Change-Id: I1271e218303ce7d35f4c67e256c4409e38d231e5
2019-11-22 10:08:02 -08:00
Egorov, Stanislav 3bf3cd9e33 fixed NO_PROXY option for docker command
Change-Id: I3ee2cc70d28f14c52aa1a02f1b90f6e1d06ac2ec
2019-11-21 16:23:54 -08:00
Drew Walters 0968eeb589 mini-mirror: Add retries to package downloads
Currently, a single package download failure causes the mini-mirror
build job to fail. This change adds three retries [0] to package
downloads to circumvent intermittent download errors due to a bad
network connection.

An aptly version of 1.0.0 or greater is required for this feature.

[0] https://www.aptly.info/doc/aptly/mirror/update/

Change-Id: Ic186398a4fd93322060d5116325e8770ca92f35d
Signed-off-by: Drew Walters <andrew.walters@att.com>
2019-11-06 22:19:24 +00:00
Drew Walters 306ba07e0a mini-mirror: Use latest aptly version
Currently, the mini-mirror image job installs aptly from the standard
Ubuntu repositories. Consequently, the version of aptly used for the
Xenial build is 0.9.1 and differs from the version of aptly in the
Bionic build.

This change adds the official aptly repositories when installing aptly
in order to use the latest version and maintain consistency between the
two builds.

Change-Id: Ic2f79b0a910f0c0e3e2aff361dfe14c9b312f0f9
Signed-off-by: Drew Walters <andrew.walters@att.com>
2019-11-06 22:19:18 +00:00
Scott Hussey 37f0185f65 (minimirror) Create Bionic based minimirror image
- aptly v0.9.7 switched to SHA256 package signatures, required
  by 18.04 for trust. 18.04 uses aptly 1.2.x.

- Remove dead code from Makefile that duplicated the build block
  for 'ubuntu_bionic'

Change-Id: Ib6290beac8110cc9b21fcaab6cfbdb331c3bb8c5
2019-10-03 18:44:09 -05:00
Drew Walters 491936cf70 mini-mirror: Ignore src config for publishing
Currently, mini-mirror honors per-source configuration files for the
creation of mirrors and publishing; however, some options exposed by
Aptly have a different impact depending on the operation. This commit
removes the per-source configuration file option from the publishing
process and defaults to the aptly config file supplied for all sources
in order to circumvent this issue.

Change-Id: I87659725918e776482aabccb97195216047f4c77
2019-05-10 05:57:38 +00:00
Drew Walters e83bc85d44 mini-mirror: Allow per-source Aptly config files
Current mini-mirror images are build using a single Aptly config file
provided at build time. In some cases, it may be desirable to provide an
Aptly configuration file for each source to change the behavior of a
single mirror. This commit introduces support for providing an Aptly
configuration file for each individual source. The config file should be
named aptly.conf and placed in the root source path.

Change-Id: Ic51fa2ab6898eaa14b45b67ba6708f2d7aea75eb
2019-03-05 16:31:42 +00:00
Zuul 8bcecac7c3 Merge "mini-mirror: Add support for multi-dist publishing" 2019-02-28 22:57:17 +00:00
Jean-Philippe Evrard d9c6cbc697 Use infra pipelines and roles
Now that openstack-infra has new roles to build/upload/promote
artifacts from gating, we are making use of them as much as we can.

Mini-mirror was also changed:
- It was wrongly affiliated to SUSE, so that Copyright line was removed.
- The docker file was incorrectly named, and was therefore renamed for
  consistency.

The LOCI jobs were not updated, and will be part of another commit.

Change-Id: Ic37e634c52cb98e12568783ca6c7ac93e92ea2a7
2019-02-26 16:01:57 +01:00
Drew Walters 2d3ea795fa mini-mirror: Add support for multi-dist publishing
Currently, mini-mirror combines all source distributions into a single
distribution and component when publishing. This commit changes the
publishing behavior of mini-mirror to separate distributions and
components for all published mirrors.

Change-Id: Ib32a2adbb6d053a798d97eedb66e272c8961581d
2019-02-13 16:39:31 -06:00
Drew Walters c08c1f9f66 Add mini-mirror image
This commit adds a mini-mirror image to OpenStack-Helm images for
the mini-mirror chart [0].

[0] https://review.openstack.org/623314

Change-Id: I55d90560ef7c07e22d2a997d2e32a0ad95edb301
2019-01-12 11:38:56 -06:00