Since now Packetary is a recommended tool to build/clone
rpm/deb repositories and Fuel client is to be used
to modify default repository settings.
Change-Id: Ifd4ee718e70ff7dc8b4724c2a00f2aaeb3813b7f
dpkg-checkbuilddeps doesn't check dependencies inside chroot as in many
cases host system might not have required packages. This change replaces
dpkg-checkbuilddeps with mk-build-deps in chroot
Implements: mos-xenial
Change-Id: I27a65893c1bd33e6d82c11bf3d08423562dbe4d2
Signed-off-by: Sergii Golovatiuk <sgolovatiuk@mirantis.com>
This allows to use words as mos repo version
identifiers while it is still possible to build
packages with versions defined like X.Y.Z
Change-Id: I24e88192c6e52cb3de000fcf99ef3eec03c4e8c3
Closes-Bug: #1529251
Fuel packages should be aligned to the same versioning schema
as we have in perestroika and specs [1]
[1] https://github.com/openstack/fuel-specs/blob/master/specs/ \
8.0/separate-mos-from-centos.rst
Closes-bug: #1524758
Change-Id: I1b5bd2617b6d388b2c4852d5f74ae3adf84f8991
We moved fuel-web/network_checker directory to a separate project
and now we need to build network-checker package using the
source code from this new repo.
Change-Id: I365a9ee20374be1515c577ab6c6426bdb20a53e6
The bootstrap image is generated during the master node deployment.
Note: the CentOS based image is used by default.
blueprint fuel-bootstrap-on-ubuntu
Depends-On: I2fce1a4b1f485bf509bfe2ea81291ceebf57e4ef
Depends-On: I8916208ec5a7e54b4c784861fe20a94737267f76
Depends-On: Icdca2858889b83bb86c8d62fdc6fe530d2af705c
Change-Id: I3581b4d997a82aafe9744b33bbb65a60b5dca8cd
Necessary for building fuel-agent (specifying python-pbr as a build
dependency is not enough).
blueprint fuel-bootstrap-on-ubuntu
Change-Id: I221497a90c99239ced7e521b0202e432f9e738ba
1) Re-Define release value for packages:
* Building from master branch:
${num_of_commits}.1.git${git_sha}
* Building from gerrit:
${num_of_commits}.2.gerrit${request_number}.${patchset_number}.git${git_sha}
2) BUILD_PACKAGES=1 (true by default), we build packages during make iso
process. The resulted packages should have naming schema described above (see 1).
To define the correct naming schema we created version file during source preparation:
$(BUILD_DIR)/packages/sources/$1/$2
Format of the version file is:
VERSION=$(PRODUCT_VERSION)
#ifdef gerrit
RELEASE=${num_of_commits}.2.gerrit${request_number}.${patchset_number}.git${git_sha}
#else
RELEASE=${num_of_commits}.1.git${git_sha}
#endif
DEBFULLNAME=Commit Author
DEBEMAIL=Commit Author email address
DEBMSG=Commit message
This file is parsed each time on building corresponding package and the values
passed to build uility:
* rpmbuild
* dch
3) Versioning applied to all Fuel packages:
- fuel-main
- fuel-library$(PRODUCT_VERSION)
- astute
- nailgun
- fuel-ostf
- python-fuelclient
Versioning is NOT used for late-packages and docker-images:
- fuel-docker-images
- fuel-bootstrap-image
- fuel-target-centos-images
4) FUELMAIN_GERRIT_COMMIT variable added, which only used for custom_iso job and
must be defined explixitly if we are working on some patchset from fuel-main
project, otherwise fuel-main packages will have master branch naming schema (see 1).
Implements blueprint: package-fuel-components
Change-Id: Id43a507d521bb13f06b33ef2967277098f242211
Both repocleanup.done and reprepro.done targets
- modify the local copy of MOS repo (which is a bad idea),
- depend on ubuntu/mirror.done target
As a result two instances of reprepro try to concurrently modify the same
APT repo (which fails). Make repocleanup.done depend on reprepro.done to
avoid the problem.
Closes-Bug: #1449723
Change-Id: I283f4f10f9e1b43eb8a30cb9458a3443cc614e23
1) unconditional rebuilding due to
never existent file
repocleanup.done used to depend on PHONY target (sources)
which led to unconditional rebuilding.
This didn't influence ISO builds because CI always cleans
environment.
2) fail when try to rebuild second
time without cleaning environment
repo.done used to consume some config files
created by reprepro.done target. After consuming those files
it used to remove them not touching reprepro.done itself.
Without cleaning build environment this fails the build process.
Change-Id: Ib0de2e5ea94caeb7f93f5bc8d0f06a6f806e3ca6
Closes-Bug: #1448206
This commit is a squashed version of commits that
implement packaging of fuel components. In order to
make our packaging process better I had to introduce
some new macros and do additional cleanup.
The main paradigm that we are following with this commit is
the following:
1 GIT REPO -> 1 PACKAGING SPEC -> N PACKAGES
Though it would be ideal to have only one package per git repo,
but this is left for further releases
This change is very essential to make updates process easier
as we provide only one type of artifacts that a user can fetch -
distro-specific package
This particular commit does not introduce new DEB packages
(such as packages of OCF scripts and other binaries that
we currently put onto the slave nodes with puppet instead of
installing them as packages) which will be introduced as
a part of whole packaging initiative a little bit later
1) Add prepare_git_source macro
This change is important to always
checkout the code from git working
or we can end up with garbage getting
into package source code and add
version.txt file to source archives
2) Remove nailgun-redhat-license package
This package is obsolete - we do not need
it.
3) Package fuel-library into RPM package
4) Delete old packages and misc stuff
This change is a little cleanup of python-tasklib
and other leftovers
5) Change RPM sandbox configuration
Change RPM sandbox configuration to use upstream mirrors
to be able to install nailgun build dependencies
6) Alter Nailgun spec
a) Change nailgun spec to use NPM during package
build
b) Switch nailgun to prepare_git_source
c) Package all packages of fuel-web repo using
only nailgun.spec
7) Alter RPM sandbox preparation to install build-deps
8) Package astute and mcagents using astute.spec
9) Include all built RPMs
Include all RPMS that were built inside the sandbox
10) Package fuel-ostf with git-archive
11) Build python-fuelclient using git_archive macro
12) Move specs to particular git repositories
13) Rename fuel-library and ostf to make macros easier
14)Adjust build scripts to fetch specs from particular
fuel repositories
FUEL DEB-packaging related code
15) Restore part of make system code for DEB packages building
16) Nailgun debian spec files for all nailgun-based packages
17) Clean up SANDBOX for Ubuntu generation
18) Clean up some package building scripts
19) Put PACKAGE_VERSION variable into the top config.mk
20) Add dockerctl and change fuel-library tar.tg path
21) Move deb rules to particula repos
22) Allow to rebuild deb repo with reprepro on-the-fly
24) Fix repocleanup stanza to parse source packages metadata
25) Debmirror whole repos in order not to break reprepro
26) Make docker image build depend on late RPM packages
or it may lead to race conditions
27) Clean up old deb packages with one shot instead
of parallel run
Change-Id: I15ae4d0abe51c72b28793e1a1ef30fec0f668b73
Blueprint: package-fuel-components
1) Change mirroring procedure for Ubuntu:
- USE_MIRROR=none: rsync the whole mirror from build server
- USE_MIRROR=<location>: mirror only required MOS distro
Added support for different mirroring methods.
2) Remove procedure of building Fuel deb packages
3) Remove building of deb-based IBP images
4) Remove deb-based chroots
5) Modify creation of Ubuntu diff mirror artifact
Implements: separate-mos-from-linux
Change-Id: Icd10aea6ebddb27976e88ceb930be9afac0241ad
Create a tarball of a base system + build-essential packages, and use that
for building the packages instead of bootstrapping Ubuntu from the scratch
for every package.
Change-Id: I91a1aa326fc8de51e1e5305fbe8b9dd3ae6c226e
* packages:
- build each package in a separate directory so sereral packages can be
built in parallel.
- prepare_*_source: make sure git repositories are cloned before using
them.
* image/centos, bootstrap, docker: add dependency on the locally built
nailgun, astute and co. RPMs to make sure RPMs are ready before trying
to install them.
* docker:
- start an instance of the simple http server per a container being built,
make sure those processes listen the different ports.
- fix sporadic build failures due to the HTTP daemon premature exit.
The simple_http_daemon which serves RPM repository exits in 600 seconds
by default. However a container might require more time (due to a slow
machine, a high load, etc), thus the HTTP server might exit before
the required packages have been downloaded. Set the default timeout
to infinity to prevent such an obscure error.
* repos.mk: several repositories can be cloned at the same time, so writes
to version.yaml should be serialized. To keep things simple create
version.yaml after cloning the repositories (with Fuel components).
While at it polish the rule which clones the repository.
* packages/rpm, bootstrap: depend only on CentOS repo (more tasks can
run in parallel).
blueprint support-ubuntu-trusty
Change-Id: I3a72802a7728121581e71227b013bb7aeffaf049
The code which applies initrd patch(es) produces a half-broken initrd which
is a) not compressed, b) contains a chunk of itself. Fortunately the initrd
is still able to boot, however, it's 2.5x larger than the original. This
slows down the provisioning and puts an extra load on the master node.
Closes-bug: #1397845
blueprint support-ubuntu-trusty
Change-Id: I666a4395e577b917016c2220a760a4cf340a54d2
As of now Fuel related deb packages are built using the upstream mirror.
This yeilds inconsistent dependencies, for instance, packages depend on
both the upstream and the customized python versions, and provisioning of
Ubuntu node fails as a result.
To resolve the problem setup the build chroot using the official Fuel apt
repo. While at it use the locally created apt mirror instead of downloading
packages once more, and fix a number of inconsistencies/bugs.
* sandbox.mk:
- sandbox define is modified so as to use local pre-built mirror.
* packages/deb/module.mk:
- use the helper script to create build sandbox.
* requirements-deb.txt:
- make sure tools required for building Debian packages are included
into the local apt repo.
Change-Id: I9f7b06bba94a65b2d65168cc1122257f4bec78be
Related-bug: #1390416
- (re)building Debian packages does not create RPMs mirror any more.
- Building Debian packages can be suppressed by setting the
BUILD_DEB_PACKAGES variable to zero.
* packages/rpm/module.mk:
- make sure sources are prepared before building a deb package, i.e. add
an explicit dependency.
* packages/deb/module.mk:
- building Debian packages does not require RPMs mirror, adjust
the dependencies accordingly.
- don't build packages if BUILD_DEB_PACKAGES is zero.
Related-bug: #1390416
Change-Id: I049342e7619bd3be80bc2e67c89ad77618b67b59
1) Add routines for building deb packages and adding them to local mirror.
2) Add commands for creating Ubuntu-based build sandbox.
3) Update deb specs for Fuel components.
Closes-Bug: 1373400
Change-Id: If098654d1158aca6676f9cbb3fc3100fc2e8674d
Occassionally Ubuntu fails to download its
configuration and the installation fails.
This patch enables preseed scripts to attempt
10 times to download the configuration
before failing
blueprint 100-nodes-support
Change-Id: I4dcee04d221c77bbdf82ea280ac836b73654a86e
Andreas Jaeger