Commit Graph

129 Commits

Author SHA1 Message Date
Andreas Jaeger 24120d7b66 Retire repository
Fuel (from openstack namespace) and fuel-ccp (in x namespace)
repositories are unused and ready to retire.

This change removes all content from the repository and adds the usual
README file to point out that the repository is retired following the
process from
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project

See also
http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011647.html

Depends-On: https://review.opendev.org/699362
Change-Id: Ibd0360edc1781ea748a1ef28877c215bb4c726c7
2019-12-18 09:38:32 +01:00
Andrey Pavlov 08aa942502 Updates should be triggered on service/node configs change
Change-Id: Ia80685d032bffec55113639fb277b9910e10c909
2017-03-06 07:05:21 +00:00
Andrey Pavlov e8ce334a9d Store sensitive configs in secret
* secret_configs config group added
* global secret creation added

Change-Id: Ie6a9833cdf73b076e24204d47e5898dfb24de43e
2017-03-06 07:00:03 +00:00
Andrey Pavlov ccaadd5e0b Configs overrides for custom services
"configs" section added to "services" group.

services:
  some_service:
    service_def: ads
    configs:
      asd: dsa

If you are mapping some service to some another service,
configs will be inherited as well.

Change-Id: Id64c0bf816a639c0b3dee96e5a72fcf964f9f731
2017-03-02 15:41:10 +00:00
Andrey Pavlov 9942c0e978 Adding service-per-service support
New config section introduced:

services:
  keystone-db:
    service_def: mariadb
  keystone:
    service_def: keystone
    mapping:
      database: keystone-db

Defined services can be used in topology definition.
In this example keystone-db service will be created from mariadb
definition and keystone will use it instead of mariadb.

Change-Id: I274826648390b844d240b7ae545c40264f662452
2017-03-02 15:41:03 +00:00
Andrey Pavlov 998dae2d40 Changing dependencies processing
To support multiple instances of the same service we should
be able to manage their dependencies separately

Change-Id: I3d1951537e49f56ae01b69c0eeef853dcde3b8b9
2017-03-02 15:40:50 +00:00
Sergey Kraynev f42e4d6b42 Add certificate to openrc fiel for SSL deployment
Change-Id: Id03c55c46698ee0a5117c6db89156e398ce7aecc
2017-03-02 11:55:02 +00:00
Aleksandr Mogylchenko 5f20f7e6db Expose CPU and Memory limits as variables
This is useful for applications that calculate internal values based on
available memory. And it is especially useful in containers because
majority of applications can not determine available memory correctly
(they do not work with cgroups).

Change-Id: Iadd0529d166ef1dad1697215bbc181104a7b926e
2017-03-01 17:18:14 +01:00
Sergey Kraynev 696adcf1d9 Add ability for recursive merge dicts for AttrDict
This patch adds ability to merge nested dicts if they are passed as a
paramater for "_merge" method of AttrDict class.
Also corresponding tests are added.

Change-Id: I7d18f69d4dc076c178552ae886af6df2932abc27
2017-02-17 08:23:07 +00:00
Sergey Kraynev 4e21fddd47 Add support for TLS
- Update address function to use "tls" config option and set scheme to 'https'.
  Also we check, that service is in list of services, which support TLS.
- Add nginx repo to defaults, because it will be used as a proxy
  service.
- Add Runtime error with suggestion to use extra options on cleanup,
  when 'tls' is enabled.

Current patch requires patch in fuel-ccp-nginx repository.

Change-Id: I65002b7ff9cfa2faf9d5bce470334aae95334d00
Depends-On: I0660cc3ca2723bc06871b61f859adfed42c0d807
2017-02-15 18:53:38 +00:00
Andrey Pavlov 18414f5f35 Adding scheme field for probes
Change-Id: I4dbcac825f319c6c202018f59a45de9ecbb377cc
2017-02-15 18:53:09 +00:00
Jenkins 5aa10c9cb1 Merge "Enable Galera as a default DB" 2017-02-15 12:13:07 +00:00
Jenkins 15e984e6dd Merge "Secret support" 2017-02-14 10:16:37 +00:00
Proskurin Kirill b7520afcda Enable Galera as a default DB
Additionaly I have to updates all topology and docs to address
this change. And yes, I propose to deploy 3 rabbitmq too.

Change-Id: I9db9fcb7a5f78312dabc23ed2ff56cb0a9748f4d
2017-02-13 15:59:08 +00:00
Peter Razumovsky 35b7e1f5a7 Add validation for nodes configs section
Add check for nodes configs section - it should be a dict.

Change-Id: Ie0114881aca4f62cfd50ff5358808765aa2924f8
2017-02-10 11:17:49 +00:00
Jenkins 3b1cd60dc1 Merge "Extension AttrDict" 2017-02-09 11:31:14 +00:00
Mikhail 2788592cb8 Extension AttrDict
Add implementation for some function in AttrDict
in order to avoid recourse to AttrDict._dict

In tests we use dict, but in fact should use _yaml.AttrDict

Change-Id: Ie99f3b05bd65f195f2f81191bff67cbacef1b816
2017-02-09 11:57:45 +04:00
Dmitry Klenov 80447e7411 Secret support
Support of k8s secrets is introduced. To create a secret, put
an additional section 'secrets' to the definition
of the service:

secrets:
  name-for-reference:
    type: "Opaque"
    data:
      "file1": "some content"
      "file2": "another one"
    secret:
        secretName: name-in-k8s
    path: /where/to/mount

You can reference to this secret from the container definition:

daemon:
  secrets:
    - name-for-reference

The referenced secret must be defined in the 'secrets' section.

Change-Id: Iaaede4ccb94c99d70f3ecad040d5ab6c41428c5e
Partial-Bug: #1651392
Partial-Bug: #1651394
2017-02-02 09:43:22 +00:00
Jenkins 5059474812 Merge "Implement error handling by ccp fetch" 2017-01-31 21:18:02 +00:00
Jenkins ca44b54cb7 Merge "Add configs map for nodes" 2017-01-31 14:33:23 +00:00
Anastasiya 0fd23478ad Implement error handling by ccp fetch
Change-Id: I315893243a7682243b81ac379615d4d18338fefd
Closes-Bug: #1608981
2017-01-31 14:06:04 +04:00
Jenkins 9fddedad13 Merge "Remove invalid key from Ingress metadata" 2017-01-30 11:46:40 +00:00
Jenkins 489a10c66e Merge "Serialize workflows right before configmap creation" 2017-01-30 11:45:02 +00:00
Sergey Reshetnyak 64100ecfe2 Add force flag to fetch command
Force flag allows to cleanup directory before fetching repositories

Change-Id: I71f6522da8f0969387c24c82c121ce81d752ee90
2017-01-30 11:39:04 +03:00
Andrey Pavlov aa110810db Remove invalid key from Ingress metadata
Change-Id: I29f0d2e35f1f1a593799d64d0da12e00d4c69cb7
2017-01-30 11:05:00 +03:00
Andrey Pavlov b8602e8412 Serialize workflows right before configmap creation
Change-Id: Ib2daeb9b4e17573e94d8fa2661eea84e7bf49093
2017-01-29 19:01:44 +00:00
Andrey Pavlov e7d4a26193 Execute Jobs only on nodes affected by deployment
* nodes on which Pods controlled by Jobs will be scheduled
  will be determined according to topology definition
* changes in order of nodes/roles in topology will not trigger
  update anymore

Change-Id: Ia41b50ff2b214791bec17577eb6e59fc94d0f2c2
2017-01-26 19:10:41 +00:00
Andrey Pavlov 9efa68be50 Remove namespace from ingress domain
Change-Id: Ic1ed1352e4bdaeceff48763be04cf92f68934ed8
2017-01-26 01:32:51 +03:00
Peter Razumovsky 341d3582e0 Add configs map for nodes
Now node definition can contain configs map,
which will add new configs especially for this node
or override current globals for the node.

Change-Id: I4de6a0fad94d5f83ca486c952d80d1c87c880c0e
Related-bug: #1653077
2017-01-23 13:11:01 +00:00
Sergey Reshetnyak 966ae060d0 Allow to specify k8s credentials in config
Change-Id: I483f43fbdb2c5e91edc2e0592530da01765c24b5
2017-01-16 16:36:51 +03:00
Jenkins 3a6a575d10 Merge "Make imagePullPolicy configurable" 2016-12-28 01:51:21 +00:00
Andrey Pavlov 025b1d69af Adding CCP_POD_NAME env variable
This variable will be consumed by entrypoint script
to provide pod_name jinja variable
Also dls version was bumped to 0.4.0

Change-Id: I8fc2f026e2f79355c5b80429e3ab7ee854c30639
2016-12-23 12:35:17 +03:00
Jenkins 5356ee1c65 Merge "Fixing gethostbyname filter during files render" 2016-12-21 16:39:26 +00:00
Krzysztof Szukiełojć cca092e303 Adding Support for Annotations in DSL
Full Annotations support for pod and service is needed for setting
extra options in kubernates for users.

Change-Id: Icbde776e5e8b44cfabe752fb43cab2ed9978ffe5
2016-12-21 12:50:06 +01:00
Anonymous 25045442d5 Fixing gethostbyname filter during files render
* adding stub for it on ccp side
* passing address func to render context

Change-Id: Ie32b3622d562106acfccb930c3e96a689c08a7fb
2016-12-14 13:47:04 +00:00
Kirill Bespalov 29133f2787 Introduce cross-repository config templating
Make possible sharing and using of common parts of configs
like keystone, db, messaging, etc as jinja templates (e.g.
via macros) located at 'exports' directories of related
repositories.

Example of usage:
-------------------------------------------------
share rabbitmq configuration as macros:
-------------------------------------------------
  # file fuel-ccp-rabbitmq/exports/messaging.j2
  {% macro oslo_config() -%}
    [DEFAULT]
    transport_url=rabbit://{{ rabbitmq.user }}
    [oslo_messaging_rabbit]
    rabbit_ha_queues = true
  {%- endmacro %}
-------------------------------------------------
use it in nova.conf.j2:
-------------------------------------------------
  # file fuel-ccp-nova/service/files/nova.conf.j2
  [upgrade_levels]
  compute = auto

  {{ messaging.oslo_config() }} <-----------

  [wsgi]
  api_paste_config = /etc/nova/api-paste.ini
 -------------------------------------------------

During 'ccp deploy' the following occurs:

- loading templates files from /exports/ dirs of avaliable repositories.
- push files to k8s as ConfigMap with name 'exports'.
- adding a container volume '/etc/ccp/macros' with the ConfigMap content
- implicitly adding jinja imports of these templates files to all config
  files from /fuel-ccp-xxx/service/* to make possible macros usage.

Change-Id: I4858d62a9713e90c09300f75e01e06a31d3ac0ae
Depends-On: I429656b7eaf6312ee2d27ccaf0cb8802a234e871
2016-12-13 17:02:23 +00:00
Yuriy Taraday 6f25340851 Make imagePullPolicy configurable
'Always' is not the best option as it adds round-trip to registry for
every node even if image has already been pulled. Keeping default to be
the same as in Kubernetes would make it conmfortable to use 'latest' tag
during development and proper versioned tag for production-like
environments.

Change-Id: Ifa628e00959bae062bb5bb7e95e071af8babee32
2016-12-03 22:38:14 +03:00
Oleg Bondarev f4afa36d90 Add gethostbyname filter to get ip address from host name
This is needed for ODL integration, where we need to set
a manager for OVS in OVS containers in the form of
tcp:<odl_ip_address>:<tcp_port>

Change-Id: I0a2919458770929b922148ce70100b0d71a16d14
2016-12-01 11:41:04 +03:00
Jenkins b3f64b119d Merge "Adding cluster_domain config option" 2016-11-29 08:57:10 +00:00
Ilya Kharin ca7b999bbf Inform about orphan images during build/validate
During the describing images and missing their parents the `ccp build`
command sometime is not so informative, e.g. like that:

    ERROR    ccp             'base-tools'

Unit tests are included.

Change-Id: I14f6a4cc75e49f17c3d943d508bea9c0b819da9f
2016-11-24 13:53:17 +03:00
Andrey Pavlov facf0f6282 Adding ability to specify custom config files
Example:

    files:
      keystone-conf: /tmp/keystone

In that case source file will be taken not from
`content` path, but from path defined in files config.

Change-Id: If2e71887adca9148f98b555ef8d6033211fe6375
2016-11-23 12:01:34 +00:00
Andrey Pavlov 3f797388bf Adding cluster_domain config option
This option will be used in `address` function and will be
added to internal urls, so that default address will be
changed: service.ccp -> service.ccp.cluster.local

Change-Id: Iaa7e1172120506e284a0ae7b3c4539af411c8535
2016-11-22 21:36:00 +00:00
Andrey Pavlov 2ffb9d57d0 Adding dsl version validation
* service version validation added. It will be executed before
  deployment and as part of validate command for service
  definitions.
* dsl_version field now is required field
* initial dsl_version = "0.1.0" added
* doc describing dsl versioning added

Depends-On: I0abe9781300c794d690b0822fc2a857db3ca8ea5
Depends-On: Ib755efe784a1c4bc04fb49827baa268513dd27e1
Depends-On: I62520035507ca7c5ac97a0e34062fa9eac26e2f6
Depends-On: I9afb7143dc876b839297c5bf5c03b156e0daa8c2
Depends-On: I5a05d2538a3315ce2e652219a6cd5aead9c7d617
Depends-On: I4a0c00b40b2af4f213daf7e43419dd46951fc4bc
Depends-On: Id988118f78fcdfe0599abe27959590f56f3617d3
Depends-On: I965814b147179435ef2cf94abc362df866495f8f
Depends-On: Ic9e4e79e848ab944e5910b57cc5cd524f1f97ae0
Depends-On: Ic8d9c334ac57a13066db38562423523bd23b671f
Depends-On: I581b587d36e03d60c5fb04e7dbf0c184c992b526
Depends-On: I4d4b9f363138b85198222ac2e770930ce7bd6ab5
Depends-On: I1728ee34eef02668cb54b1f54291a66d609dd6c4
Depends-On: I4d4b9f363138b85198222ac2e770930ce7bd6ab5
Change-Id: I220d99e76220c124d782c97de51112869c727148
2016-11-22 17:30:12 +03:00
Andrey Pavlov bffb2aa43e Fixing 'render' function behavior
Files will be rendered directly to temporary directory and
only for matched images

Change-Id: I74431d5de90f7d001edf75a3ed41181b7b9321d1
2016-11-17 11:37:56 +00:00
Jenkins 2ec87d8f6f Merge "Enable updates for Ingress resources" 2016-11-16 16:37:21 +00:00
Jenkins 8efea372c5 Merge "Fixing unit tests of cli" 2016-11-16 16:35:32 +00:00
Proskurin Kirill 143dd5eecc Force imagePullPolicy to Always
Right now it's defaults to Always if :latest tag is
specified, or IfNotPresent otherwise. Which very confusing,
especialy if you switch from :latest to some tag and wonder
why your images are not updated.

Change-Id: I2cdc0817c2b4e8c95993b5eae44ffff26acd7662
2016-11-15 18:21:33 +00:00
Andrey Pavlov 7c5a556abf Fixing unit tests of cli
Fixing py27 failures in case of ccp repositories clonned to
`ccp-repos` directory

Change-Id: Iad8d4e77bf1d4f887b5f5a362905eb8af0d37f44
2016-11-15 17:12:08 +03:00
Andrey Pavlov 0e8610510b Enable updates for Ingress resources
Ingress resources can be updated as well

Change-Id: Ibf281e9ac52708dba2cb9479e787b9f616db7629
2016-11-15 16:28:20 +03:00
Yuriy Taraday a06f0240ef Collect all objects to be deployed/updated before sending them to Kubernetes
For upgrade scenario we'll need to process them after initial sync jobs,
so we need to collect them and make a decision on the whole list.

We don't touch cofigmaps here because handling their changes detection
will require some more work.

Change-Id: Ic727ab08e502d9b9db9e9bf3044e64f992039ef8
2016-11-12 16:57:17 +03:00