Commit Graph

2713 Commits

Author SHA1 Message Date
OpenDev Sysadmins 9eab30d68e OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:52:01 +00:00
Andreas Jaeger a4c49fab02 Remove openstack-doc-tools
openstack-doc-test has been removed from openstack-doc-tools in release
1.0.0. Remove it's usage from this project.

Also, remove doc-test.conf file, it was used by openstack-doc-tools.

Remove checkniceness and checksyntax targets, they are unused and not
needed. Merge doc8 check into pep8 and doc environments (note that
OpenStack CI does not use doc environment, so add it to pep8 as well) as
is custom for lint like jobs. Fix requirements to follow best practices.

Add doc8 to test-requirements to document its usage properly, add sphinx
there as well.

Change-Id: I97b9df8272c4f425ec516dfb84c29d2263c316f8
2016-08-23 19:14:30 +02:00
Jenkins 08f29c530a Merge "--stop-on-error flag added to watching reports" 2016-05-13 09:08:28 +00:00
Maciej Kwiek 97815db455 --stop-on-error flag added to watching reports
This flag works only when -w option is used, it causes the report
command to end prematurely with exit code 1 and display all tasks status
when it encounters NEW error in orchestration task.

Change-Id: I7998bb1e1e8da1c76a69aa066af6460eb2fcee1c
2016-05-12 16:51:23 +02:00
Dmitry Shulyak 0d8c951b21 Refresh transaction after each orchestration report tick
It was possible to fetch stale data from database with high isolation
levels, to avoid such issues we will simply restart transaction
after each report interval

Change-Id: I64a9843aa64adf4c710a9f593bdeaa2f5b3c5fce
2016-05-12 12:45:01 +03:00
Jenkins 5008a8ccfa Merge "Implement traversal based on number of childs" 2016-05-05 23:51:41 +00:00
Jenkins 0b50415257 Merge "Add helper to preserve atomicity for database group operations" 2016-05-05 23:39:03 +00:00
Jenkins 6673a06c63 Merge "Refactoring of graph.py and usage of it in scheduler" 2016-05-05 23:34:19 +00:00
Jenkins dfd140ab54 Merge "Rework staging to avoid unnecessary children log items creation" 2016-05-05 23:31:26 +00:00
Jenkins 2f3fab16a0 Merge "Minimal solar VM packer template" 2016-05-05 14:21:03 +00:00
Jedrzej Nowak 7325a9371d Minimal solar VM packer template
Alpine linux as a base, vbox and libvirt

Change-Id: Iaa37b1ae02b4b0d22fc01fbeff5c6ce96e0b2699
2016-05-05 15:30:41 +02:00
Dmitry Shulyak 7edc781d66 Add helper to preserve atomicity for database group operations
All solar cli actions will be atomic, for example if user is going
to create resources from composer file - all or none will be added to database

If for some reason this behaviour is undesirable for particular command
developer can overwrite it by using default click command:

  @group.command(cls=click.Command)

For those who are using solar as a library - decorator and context managers
are available in following module:

  from solar.dblayer.utils import atomic

  @atomic
  def setup():

Change-Id: I8491d90f17c25edc85f18bc7bd7e16c32c3f4561
2016-05-05 11:56:07 +03:00
Jenkins fb86cb1b1d Merge "Refactor error wrappers in CLI." 2016-05-05 08:43:55 +00:00
Maciej Kwiek 71900a69cb Refactor error wrappers in CLI.
Replaced EGroup with BaseGroup which has default error_wrapper method.
Added BaseGroup to all click groups using proxy empty classes.
Added inheritance from SolarError to several custom exception classes.

Change-Id: I4afa2f23ef4486c3a1565c04419f4b8dff21705a
2016-05-05 10:25:05 +02:00
Jenkins 5b9d856ff8 Merge "Use master branch of solar-resources" 2016-04-29 13:16:36 +00:00
Jenkins 0a29d63404 Merge "Possibility to disconnect single input in resource" 2016-04-28 15:01:13 +00:00
Łukasz Oleś d5e5e5b421 Use master branch of solar-resources
Change-Id: I05ffef536fabb7ddf360930725501652e12b4454
2016-04-28 16:16:10 +02:00
Jedrzej Nowak 3abc7ded95 Possibility to disconnect single input in resource
CLI and resource.py, it was always possible to do so in dblayer

Change-Id: I0263a062036cb78741c081c5419141d7b8cf5db7
2016-04-27 15:45:49 +02:00
Jenkins bc039d31b4 Merge "Allow to use parent input more than once in CR" 2016-04-27 12:29:48 +00:00
Jenkins 48457772b8 Merge "Compare names." 2016-04-27 12:21:50 +00:00
Łukasz Oleś 51869a2057 Allow to use parent input more than once in CR
Change-Id: I740ec54e62c2deb4607ca357f0b7e3dbf7cb575c
2016-04-27 14:16:27 +02:00
Michal Rostecki e10422e812 Add *egg/ directories to .gitignore
Pip created *egg directory when using it with -e option.

TrivialFix

Change-Id: I21d4dba90ea90f8799a317637aa143338c59d106
2016-04-27 12:21:19 +02:00
Łukasz Oleś 7eb89f49cc Compare names.
Names are unique

Change-Id: I000f2052440372d475383be46550485a4ca75233
2016-04-27 11:06:18 +02:00
Michal Rostecki ae138f91e2 Don't install vagrant-vbguest when using other providers
Also, allow to define another provider-specific plugins
when needed.

Change-Id: Ie40b5c52ea6aea19c150def65785356b0db3c54e
Closes-Bug: 1574627
2016-04-25 14:53:18 +02:00
Dmitry Shulyak a2f9e37fe8 Rework staging to avoid unnecessary children log items creation
Procedure in staged_log, that was adding log items based on connections,
was reworked to avoid creation of LogItem instances before filtering
those children that weren't changed (with empty diff or connections_diff)

That problem was leading to unpredictable behaviour during updates,
removal and relevant discard/revert scenarios.

Change-Id: I65adb8262fdbe10299c02c54db9d19fb255bceea
2016-04-22 17:17:21 +03:00
Łukasz Oleś 8c66af58be Return paths to renedered files
Change-Id: Ic25637c35b2aa4c55a51fd34c016ab539e779438
2016-04-20 14:18:41 +02:00
Dmitry Shulyak 5ef0be35ca Execute update action in case resource was already created
In case if resource was already created (commit operation was
triggered) we will implicitly stage 'update' action.

It was default solar behaviour which was change in one of the
patches related to mentioned blueprint.

related to blueprint refactor-process-of-staging-changes

Change-Id: I861dfca4f6a68cb8b1d9914d6f6a082ed9e865cf
2016-04-05 13:51:21 +03:00
Dmitry Shulyak e2cfa869d8 Implement traversal based on number of childs
By using childs weights for scheduling we can unlock
concurrent and decrease total time of execution.

As an example consider next variant:
Tasks A and B can't run concurrently because of node-limit.
Tasks A and C have logical dependency, and thus not concurrent.
Tasks B and C will be executed on different nodes, and doesnt
have any logical dependency.
As A and B doesnt have parents we may schedule any of this task
and logically execution will be correct, but in case if we will choose
B total time of execution will be - B + A + C, BUT
if we will select A - total time of execution may be reduced,
and will take in total - A + max(B, C).

Change-Id: I52a6c20e8c3d729ed20da822f45cbad90e51f2df
Closes-Bug: 1554105
2016-03-23 11:08:51 +00:00
Dmitry Shulyak 16072bce2d Refactoring of graph.py and usage of it in scheduler
Current patch addresses several problems -
1. A lot of forced updates on every tick of scheduler are leading
to increased cpu consumption of solar-worker
2. In order to represent solar dbmodel Task using networkx interface
a lot of Task properties are duplicated and are copied each time
when graph object is created

Solving 2nd problem allows us to move update logic to scheduler,
and this will guarantee that we will update no more than reported task
+ childs of that task on each scheduler tick.

Closes-Bug: 1560059
Change-Id: I3ee368ff03b7e24e783e4a367d51e9a84b28a4d9
2016-03-23 13:08:14 +02:00
Dmitry Shulyak 8f1ca9708a Use WeakValueDictionary instead of WeakSet in DBModelProxy
Current implementation of DBModelProxy doesnt allow to use
origin hash function of Model class.

In order to avoid this problem we will store references to
Model instances in WeakValueDictionary instead of WeakSet.

Change-Id: If92af140c9aaad3a46b24872dae16969b1090df8
Closes-Bug: 1560369
2016-03-22 10:44:49 +02:00
Jenkins 29d3306491 Merge "Fix typo in Resource._pre_from_dict_check" 2016-03-21 11:47:35 +00:00
Dmitry Shulyak 175215ec41 Fix typo in Resource._pre_from_dict_check
cls.bucket.get expects key to be a string,
but instead of it may receive dict where one of the
keys will be *key*.
This problem can be found on riak backend.

Change-Id: I824889447ea229ac7005df31156afe79dc78f42b
2016-03-21 13:25:28 +02:00
Jenkins ee01ddce3b Merge "Rework staging procedure to support both implicit and explicit stages" 2016-03-21 11:20:21 +00:00
Jenkins 1a33a7306d Merge "Fix repo.iter_contents after recent changes in Repository module" 2016-03-18 21:11:05 +00:00
Jedrzej Nowak 3a3845ac96 Fix repo.iter_contents after recent changes in Repository module
Change-Id: If933005d2334fca55780eec736ffa86762a916a8
2016-03-18 18:06:52 +01:00
Dmitry Shulyak d1b30e4be0 Rework staging procedure to support both implicit and explicit stages
Current change addresses several problems -
- It was impossible to re-stage already commited resources.
For example re-run openstack actions without doing any artificial
updates in solar inputs.
- Also there was no way to execute actions that are not related
to state in solar (run/update/remove). An example would be -
restart of services.
- And following changes addresses isolation problem in staging
procedure. By design solar is isolated using tags semantics,
but previous implemention of *process* was building a graph
unconditionally for all staged resources. It was reworked and now
we can support partial processing of resources, based on tags.

Implicit staging will be done when resource is update/created/removed.
Additionally actions can be staged using solar ch stage command,
to support this additional flags were added:
--action, -a - action that should be staged
--tag, -t - tags to select group of resources
--name, -n - resource that will be staged
Only one from name or tag will be used, if user will provide both - name
will be of higher priority

Reverts and discard are working as previously for
creation/update/removal of resources, but Exception will be raised if
revert will be attempted for custom action, such as *restart*.

Processing staged items can be achieved with -
solar ch process -t tag1 -t tag2

History and staged log items will be stored in different buckets.

Custom siblings resolved for LogItem will ensure that there is only
one resource action is staged.

implements blueprint refactor-process-of-staging-changes

Change-Id: I9e634803a38d80213b87518cd2c8fdc022237aa0
2016-03-18 12:16:02 +02:00
Jenkins 26fcbeb0a5 Merge "Add vbox/qemu packer build for centos7" 2016-03-17 10:38:51 +00:00
Bogdan Dobrelya 542a359af2 Add vbox/qemu packer build for centos7
Rename the cloud_archive task to uca/rdo tasks for ubuntu/centos
Rename the worker_upstrart task to the worker
Rework the OSTYPE env var to "when: ansible_os_family"

TODO
* PXE (PREPROVISIONED=false) to be working for centos as well
* pygraphviz for centos7, see also https://bugs.launchpad.net/fuel/+bug/1510884
* fix ubuntu only compatible examples, for example riak cluster deploy

Closes-bug: 1553988

Change-Id: Iab130c3c79b3db99deb14ea50572c073532f4694
Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
2016-03-16 11:41:15 +01:00
Jenkins 9d55399f8c Merge "Improve version parsing in Repository module" 2016-03-16 09:38:25 +00:00
Jedrzej Nowak 3455a731dd Improve version parsing in Repository module
When wrong version is given in meta.yaml error is raised

Change-Id: I10361fce51321c40d4447203f7420e7b7d82f05c
Closes-bug: #1550368
2016-03-16 09:59:01 +01:00
Jenkins 4c5875f985 Merge "Implement better and more solar specific pool for PG" 2016-03-15 16:26:13 +00:00
Jenkins 7952a8c577 Merge "Separate vagrantfile for the docker provider" 2016-03-08 08:20:23 +00:00
Jenkins da9ff35edd Merge "Add centos7 docker support" 2016-03-08 08:20:20 +00:00
Jenkins 5134b1d4d5 Merge "Add packer build for docker container" 2016-03-08 08:20:14 +00:00
Jenkins 72c6adde38 Merge "Added naive method to check if object already exists" 2016-03-08 08:20:09 +00:00
Jenkins 0220411933 Merge "Import repo to temp dir" 2016-03-08 08:15:07 +00:00
Jedrzej Nowak 8add0260f2 Implement better and more solar specific pool for PG
Supported parameters
- pool_size how big is the pool
- pool_overflow how many overflow connections should be allowed

Change-Id: Iba92eb94754ef7314bc3d4bf0e413e7d61e027f8
2016-03-07 17:10:25 +01:00
Maciej Kwiek c281237a8f Reset times in tasks on graph reset
It will stop pollution of subsequent orchestrations with time values
from previous runs.

Change-Id: I2b0e495f84768aee3545f33e388c9bfb20d76fa4
Closes-bug: 1554058
2016-03-07 16:30:08 +01:00
Jenkins cf1fd8d697 Merge "Added systemd solar-worker.service" 2016-03-07 10:04:13 +00:00
Jedrzej Nowak ef683d7a6b Added naive method to check if object already exists
Change-Id: Iefc1e3d2d6062fbe9375187472fd867db8d60c8c
2016-03-07 10:40:18 +01:00