Commit Graph

45 Commits

Author SHA1 Message Date
Andreas Jaeger 24120d7b66 Retire repository
Fuel (from openstack namespace) and fuel-ccp (in x namespace)
repositories are unused and ready to retire.

This change removes all content from the repository and adds the usual
README file to point out that the repository is retired following the
process from
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project

See also
http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011647.html

Depends-On: https://review.opendev.org/699362
Change-Id: Ibd0360edc1781ea748a1ef28877c215bb4c726c7
2019-12-18 09:38:32 +01:00
Sergey Reshetnyak 0f9c0b5e7c Allow to specify remote docker engine for building
Change-Id: I1ab0129e646861d35edb16a78a7e81e00d714b27
2017-02-03 08:56:25 +00:00
Ilya Kharin ca7b999bbf Inform about orphan images during build/validate
During the describing images and missing their parents the `ccp build`
command sometime is not so informative, e.g. like that:

    ERROR    ccp             'base-tools'

Unit tests are included.

Change-Id: I14f6a4cc75e49f17c3d943d508bea9c0b819da9f
2016-11-24 13:53:17 +03:00
Andrey Pavlov bffb2aa43e Fixing 'render' function behavior
Files will be rendered directly to temporary directory and
only for matched images

Change-Id: I74431d5de90f7d001edf75a3ed41181b7b9321d1
2016-11-17 11:37:56 +00:00
Jenkins 1edd9bfebe Merge "Add `ccp images list` command" 2016-11-08 20:47:03 +00:00
Yuriy Taraday dbf1b45c4a Add `ccp images list` command
It lists images that match all components or ones listed in command
line.

Change-Id: I6fa9700e562b7ca0ecafd58521ede47dc84b71d8
2016-10-26 16:00:15 +02:00
Yuriy Taraday 48b1e51430 Add 'url' section to config, pass it down to Dockerfile.j2
Change-Id: Ib43889dcec99c515b9932de381cf3ef25fbc7ff9
2016-10-26 10:13:08 +02:00
Yuriy Taraday 269f50a948 Add 'render' function to Dockerfile templates
It allows to render templates in files being copied into container.

Change-Id: I6dd3164caf2436b0d169b86f1e650147bb77ca45
2016-10-26 10:13:08 +02:00
Yuriy Taraday cdad0ef4ac Extract method for building dockerfiles dependency tree
We'll need this to validate dockerfiles separately from build.

Change-Id: Ic025a0a895ffc097fea4358d54636bea576e670d
2016-10-20 15:25:06 +03:00
Yuriy Taraday 3cfecb7700 Use TemporaryDirectory context manager in build_components
This allows to separate tempdir cleanup from build results reporting.

Also move build results reporting to happen only if we actually started
build process.

Change-Id: I13d7f3d05c86324b0154b3331470c6253a678090
2016-10-20 15:15:45 +03:00
Andrey Pavlov 5abe5b0bed Small log improvement
Changing log level of dockerfile rendering message from info
to debug because we are now rendering all dockerfiles (including
not requested) at the beginning of the build.

Change-Id: I1dabf211bdb3abe926c580af0039ea426320b2c2
2016-10-18 20:10:13 +03:00
Yuriy Taraday 2742f4bfc2 Allow to tune image spec for every image
This requires changes in all component repos to replace FROM line to

  FROM {{ image_spec('someimage') }}

Change-Id: I18281bdb41e91cd5c9160055f1617d7ee9d3b548
2016-10-18 14:09:57 +03:00
Yuriy Taraday f48ba99b79 Move dockerfile rendering to main thread before dependencies calculation
This will allow us to extract information about parent images from
rendered content instead of mathcing by regex.

Change-Id: Iedb8d1bb153297dfddce71d47b55b57fb6a0196d
2016-10-17 18:06:00 +03:00
Yuriy Taraday 804a2bfc04 Decouple Dockerfile rendering
This will allow to render Dockerfile separately from preparing work
directory and sources for building.

Change-Id: Ic12c42204f1c2e02c005939b498aa29da376c5a0
2016-10-17 14:51:34 +03:00
Yuriy Taraday 3b24a8039c Replace runtime sources verification with jsonschema validation
Change-Id: I3d17336d26ed14970c6d9f96c10b65468da12f1f
2016-10-14 16:21:17 +03:00
Yuriy Taraday 855fda0ef3 Change repositories list in config
Allow use to add arbitrary repository to the list.

Change-Id: I7895a9a66d4499aabcd5eb9bd037e5b2bc1fa83c
2016-10-11 17:16:33 +03:00
Andrey Pavlov 905b57437c Do not print push fails when push is not requested
Change-Id: I27dd7585828b49dddbec1519bb71a9ac60cfb8ef
2016-10-07 10:40:16 +03:00
Yuriy Taraday f34b2a909b Remove adhoc calls to get_global_parameters
Replace them with storing data from components centrally in global CONF
and using it all over the place.

This allows to centrally control how configs are handled, i.e. if we
decide to not use defaults anymore, we can just remove
load_component_defaults from cli module.

Change-Id: Id58bb43a7a9cbb367719ba59484bf3121ec68379
2016-09-27 19:33:55 +03:00
Yuriy Taraday ba606cb2c4 Replace oslo_log usage for getLogger only
It was useless and we're getting rid of oslo.log, so first remove it
from all over the place

Change-Id: Ia6bf6e763fee41d060f6b68ddabf68424a51f780
2016-09-14 23:32:13 +03:00
Andrey Pavlov fb0d94f51c Child images should not be built if parent build failed
Change-Id: Ief35fcf2acb23aea8305096b59b7d9f876ba4d98
2016-09-09 07:11:38 +00:00
Yuriy Taraday 11b678805e Switch main config object to be result of YAML parsing
We still use oslo.config for CLI parsing and default values, so added
copying of all values from oslo.config to YAML dict. To keep easy
access, added AttrDict class that allows to access config dict via
attribute access. Created a new fixture for tests that loads default
values from oslo.config to reuse current definition.

Change-Id: I921830890463f295f9e6a2467d6bc15504742e1b
2016-09-08 11:56:15 +03:00
Yuriy Taraday 2d025a5701 Move all CONF.import_* to config/__init__.py
We're importing all modules from CLI anyway, so we always import all
config modules, spreading this across all modules provides no benefit.

This is another step towards YAML config.

Change-Id: I38e2e8ae7596829311388498df68a048ab1cd981
2016-09-05 17:14:55 +03:00
Yuriy Taraday d9ab3b4ccc Consolidate all oslo_config.cfg.CONF imports to fuel_cpp.config
Also move config initialization to fuel_ccp.config.

This is the first step to replacing oslo.config with YAML-based config.

Change-Id: If1b1faf006b40df95a661ecd00cb744e9fd0aafa
2016-09-05 16:59:38 +03:00
Jenkins ca0c3b563a Merge "Check all tags during ready images check" 2016-09-01 12:48:33 +00:00
Andrey 0e29411f51 Small improvement to build tool logs
Change-Id: I76fda4597ef67e19cc39b888da8e23d64eb31280
2016-09-01 09:33:23 +00:00
Andrey 1b9f83e420 Check all tags during ready images check
When "ccp build" command is searching for already built images,
only one tag for each image is being considered. This behavior
leads to rebuilding of already existing images.

Change-Id: I6c9d1ad523ddac7ed7fb4fa56ce2d5f0eeccc788
Closes-bug: #1619198
2016-09-01 09:10:15 +00:00
Jenkins 15e7e60bcd Merge "Ensure git_url xor source_dir for copy_sources" 2016-08-31 09:40:36 +00:00
Jenkins 60e945bb2c Merge "Add log string about image build start" 2016-08-27 13:59:20 +00:00
Sergey Lukjanov 583e7f0a84 Ensure git_url xor source_dir for copy_sources
Change-Id: I2a466e460cfd3130eafba40cbd7849dd273c1000
2016-08-26 10:12:10 -07:00
Andrey 3e8eebbd7d Adding build from git/local dir
Adding ability to use git repositories or local folder
for components code

example:

sources:
  openstack/keystone:
    git_url: https://github.com/openstack/keystone.git
    git_ref: master
    OR
    source_dir: /tmp/keystone

Defaults can be overriden in global config.
Validation will be added in separate patch.

Change-Id: I4d91aa8632fcd55735d791300fde475696b435b5
2016-08-26 09:57:22 +00:00
Andrey Pavlov bab0c9b879 Render only matched dockerfiles
Currently we are rendering all existing dockerfiles
in repositories, but only requested for build should
be rendered.

Change-Id: Ib811ca24372fe42776e6458d2255bb82086dd70d
Closes-bug: #1615981
2016-08-24 09:34:24 +00:00
Jenkins 900a61c184 Merge "Do not push image if build failed" 2016-08-19 12:13:18 +00:00
Andrey e16306c8b3 Do not push image if build failed
Change-Id: I4e3009df8f9652de41af755cfa02c95d1cdb532c
2016-08-19 12:33:04 +03:00
Andrey 0256920149 Do not log success on image build failure
Change-Id: If6d720ac18cc5fa9ce4d6bc85619780d7235dd70
2016-08-19 08:45:49 +00:00
Proskurin Kirill 8b6f84b29c Add log string about image build start
Without it(or debug logs) user will stare at blank screen without
knowing what is happening right now.

Change-Id: I9731efae35ff2ac68a4d3552b36cbfe0b7c944f9
2016-08-16 16:25:12 +02:00
Jenkins 09e8840104 Merge "Improving build command output" 2016-08-15 08:57:17 +00:00
Andrey Pavlov 13aeca1cd3 Improving build command output
Changes in Summary:
  * all layres of image are already in registry under
    the same tag => "already in registry" group
  * at least one layer actually pushed or mounted from
    other layer that exists in registry => "push succeeded"
    group

Change-Id: I1a66157f13211047d41e3e0189472b6be20462db
2016-08-12 07:46:23 +03:00
Jenkins d1292f0924 Merge "Skip images without tags during build" 2016-08-11 16:30:02 +00:00
Proskurin Kirill 8790507d08 Skip images without tags during build
Right now, it's possible to have a set of same images with different
tags, which is ok, but sometimes we could have an image without tag:
127.0.0.1:31500/ccp/heka latest
127.0.0.1:31500/ccp/heka <none>

This is related to how docker create layers, see also:
https://review.openstack.org/#/c/352360/

Change-Id: I6a610f49f2edd3d86cbbc2df01ee6d0d4f9fb87b
2016-08-11 15:47:54 +02:00
Andrey Pavlov b7329fe188 Make docker-py request timeouts configurable
It could be useful in case of slow connection

Change-Id: If676cd5c25455a64ab59ac0aae59a26f41bb30d7
Closes-bug: #1611673
2016-08-11 10:11:09 +03:00
Andrey Pavlov 6938ec97fb Adding one more case for image push result
Message that image already exists in registry
will be printed if its layers already exist
in registry

Change-Id: Ib13588b17ef26dfc6b220f724f0bcae0ccfb557a
2016-08-11 08:58:55 +03:00
Andrey Pavlov a419207955 Fixing images build with custom tags
Custom tag was not taken into account during images
build and push.

Change-Id: Ic8ad1c3cf756b31114e75f2ecc5220dc8eb279cc
Closes-bug: #1610889
Closes-bug: #1610881
2016-08-10 09:37:12 +03:00
Andrey 49c1afee13 Raise error if component doesn't match any dockerfiles
Change-Id: Iaf373f001c09b03b3bde61e54acc7a4cc72e8a6f
Closes-bug: #1607363
2016-08-02 05:22:00 +00:00
Sergey Reshetnyak e04cc55b1b Add support of dynamic topology
This patch removes support of hardcoded node selectors.

This patch allows to specify dynamic node selection policy via
configs.

If service is not specified in topology config, service deployment skipped.

Selectors works via affinity feature in kubernetes.

Change-Id: I1f8defd90169cd08fd2ac191c3b186efae5da010
2016-07-27 17:29:29 +03:00
Sergey Reshetnyak 5cb0d7a9fe Rename microservices to fuel-ccp
Change-Id: I883f2a583fd3ff497df91c216d4fae6a7b6a6d6e
2016-07-21 13:45:24 +03:00