Commit Graph

12774 Commits

Author SHA1 Message Date
Andreas Jaeger e61cc055ab Retire repository
Fuel (from openstack namespace) and fuel-ccp (in x namespace)
repositories are unused and ready to retire.

This change removes all content from the repository and adds the usual
README file to point out that the repository is retired following the
process from
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project

See also
http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011647.html

Depends-On: https://review.opendev.org/699362
Change-Id: I10917e42829b459c41ca46514faa72fc46abf7be
2019-12-18 09:52:23 +01:00
OpenDev Sysadmins 768ac74a42 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:32:46 +00:00
Dmitry Guryanov 392f3859ff Calculation of interface types from templates fixed
In case of adding port on bond 'ether' type was set
for interface. This was root cause of the issue -
bond was processed as nic.

Closes-Bug: #1602817
Change-Id: I6a5136ce9ce5398aa6d55c795857769a7a41f7b0
2017-05-10 09:09:45 +03:00
Jenkins 9482abc11d Merge "Excluding number of nodes in stop operation notification" 2017-04-26 13:23:38 +00:00
Georgy Kibardin b8e2606daf Override keys() with a function, not an attribute
By mistake keys function has been overriden by a value.

Change-Id: I713bddcfb66c4835d477972ece7e0983d263377f
Closes-Bug: #1666913
2017-04-10 13:44:47 +03:00
Vasyl Saienko 9dabe06369 Add Ironic multitenancy support
This patch adds support to deploy Ironic with separate Neutron network
for provisioning baremetal instances.

  * Add NetworkDeploymentSerializer100 as Ironic multitenancy is supported
    from Newton, and we will backport this till stable/newton.
  * Update network scheme generation to create 'vlan' baremetal network,
    assign IPs to Ironic conductors from this network, make them
    accessible from baremetal servers.
  * Add new checkbox at 'Openstack Settings/Additional components' tab
    which allows to define if separate provisioning network should be used
    during deployment. This is a trigger to switch ironic deployment to
    multitenancy case. If not selected old behaviour is kept, 'flat'
    network is used. The checkbox is shown only when Ironic component is
    enabled.

Change-Id: I861a8b3b046202526d6a856c9f2dca2cfaddc887
Related-Bug: #1588380
2017-03-31 10:21:34 +03:00
Vladimir Sharshov (warpc) 3fb3b83f4a Excluding number of nodes in stop operation notification
Nailgun use block of nodes in stop operation to reset
such nodes in discovery state. Also Nailgun used such
data to calculate count of nodes for notifications.
But Astute will not send info about nodes in case
of task deployment.

This patch exclude count of nodes in stop notification
to prevent misslining message about successful operation
for 0 nodes

Change-Id: I32da2ccce11b22378f58759703fc4a56e31fd993
Closes-Bug: #1672964
2017-03-28 15:51:45 +00:00
Jenkins 738066cdca Merge "Fix typo in packages list for distributed serialization" 2017-03-28 13:57:21 +00:00
Jenkins aa6f282343 Merge "Reorder configs_query by time" 2017-03-28 13:29:33 +00:00
Vladimir Kuklin 962f009fce Fix typo in packages list for distributed serialization
Change-Id: Iacc987275064609f5c1f3c731073baa8a345e7d3
Related-bug: #1676197
2017-03-27 19:13:26 +03:00
Vladimir Kozhukalov 502d152f41 Add python-distributed package as nailgun dependency
Change-Id: I852ad5491b2225c8a211f7216d1a17b2b6c53487
Closes-Bug: #1676300
2017-03-27 08:17:12 +00:00
Jenkins 1c3ef48a2e Merge "Distributed serialization implementation" 2017-03-24 19:51:59 +00:00
Alexander Kislitsky 054ed4cf99 Packages for distributed serialization added to provision
We are going to use provisioned cluster nodes as workers for
distributed task serialization. Package python-distributed
provides dask-worker for nailgun code execution. Other packages
are nailgun requirements.

Change-Id: I95b7682c64fe2eedb26fc80046909974cc792c91
Implements: blueprint distributed-serialization
2017-03-24 17:17:04 +00:00
Alexander Kislitsky 4fa861fa8a Distributed serialization implementation
Distributed serialization is implemented with python distributed
library. We have scheduler for jobs management and workers for
jobs processing. Scheduler is started on the master node as well
as set of workers on it. Also workers are started on all nodes.
In the cluster settings we can select the type of serialization
and nodes statuses that allows serialization on it. By default
nodes with status 'ready' are excluded from the workers list.

For data serialization we are using only nodes from the cluster
where serialization is performing.

Before the computation fresh nailgun code is sent to the workers
as zip file and it will be imported for job execution. So we always
have fresh nailgun code on the workers.

In one job we are processing chunks of tasks on the workers. This
approach significantly boosts performance. The tasks chunk size
is defined as settings.LCM_DS_TASKS_PER_JOB parameter.

For limiting memory consumption on the master node we use parameter
settings.LCM_DS_NODE_LOAD_COEFF for calculation max number of jobs
in the processing queue.

Synthetic tests of distributed serialization for 500 nodes with
nubmer of ifaces >= 5 performed on 40 cores (4 different machines)
took 6-7 minutes on average.

Change-Id: Id8ff8fada2f1ab036775fc01c78d91befdda9ea2
Implements: blueprint distributed-serialization
2017-03-24 15:15:30 +00:00
Jenkins 36b90eacdc Merge "Use tags instead of roles" 2017-03-24 11:48:45 +00:00
Jenkins 531e275d34 Merge "add removed handlers" 2017-03-23 13:18:50 +00:00
Jenkins 2ec2a93d53 Merge "Moving task cache update into task creation transaction" 2017-03-23 09:46:24 +00:00
Igor Gajsin cb8529a84a add removed handlers
There were urls and handlers for vmware in the nailgun, that were
removed completely in
review.openstack.org/#/c/428402/15/nailgun/nailgun/api/v1/urls.py

The better approach is remain the urls and make special stub-handlers
instead of real ones.

Change-Id: I50bf740ec726c9cc57ff63d49aff718e812e6feb
Closes-Bug: #1668258
2017-03-23 12:26:38 +03:00
Georgy Kibardin 2caa56fa6e Moving task cache update into task creation transaction
Doing this we avoid inclusion of task cache update statement into the next
transaction which may cause different problems such as deadlocks.
(update happens inside make_astute_message() function)

Change-Id: I865b98beb621bee089cf79f1304498fd3637d64f
Closes-Bug: #1618852
2017-03-23 05:15:48 +00:00
Jenkins 7aa506469b Merge "Use assertIsNone instead of assertEqual(None, obj)" 2017-03-22 17:23:16 +00:00
Jenkins 0c4e4c0bf1 Merge "Introduce proper packaging for fuel-nailgun deb package" 2017-03-21 13:53:19 +00:00
Jenkins 8253c39562 Merge "text_list attributes could be empty" 2017-03-21 13:36:05 +00:00
Georgy Kibardin fba3bc3445 Use tags instead of roles
Now the deployment behaviour is driven by tags rather than role names, i.e. role
name cannot be relied upon.

Change-Id: Icfabeeb0b7fb6a9d697a09c3cf1fa020bbd4c323
Closes-Bug: #1669743
2017-03-21 13:45:07 +03:00
Jenkins c45713ead3 Merge "add tests for alembic migration" 2017-03-20 15:13:13 +00:00
Vladimir Kuklin 2f522172ce Introduce proper packaging for fuel-nailgun deb package
Closes-bug: #1665311
Related blueprint: distributed-serialization

Change-Id: I0c11d18ed456c253971a312f9fb5a41220549be5
2017-03-20 17:29:14 +03:00
Igor Gajsin 8e12d76f86 Reorder configs_query by time
There is the possibility to change OpenStack config after deployment.
Changes could be applied per role. And in the case of multiple roles
on a node, many changes could pretend to be applied. I.e. if we have
a config for the role 'compute', a config for the role  'cinder' and
a node with role 'cinder+compute' we have to choose one of them.

Previous decision was 'sort it in the lexicographical order', so it
applies 'cinder' then --- 'compute'. It is contra intuitive. The best
option is applying to a node the last related config, to new config
overlaps old ones.

Change-Id: I7db388ca3baeb351adc9fdb70c55b0be50fafe48
Closes-bug: #1671521
2017-03-20 16:51:39 +03:00
Jenkins 847e9a81b9 Merge "Serialize mtu for dpdk interface with 'i40e' driver" 2017-03-17 09:44:55 +00:00
Vladimir Kozhukalov b232c9f9e7 Fix requests requirements
Change-Id: Ib694ea4eb0fd08b2b8532ebe00b36b49155e91a1
Closes-Bug: #1672658
(cherry picked from commit ed41575d09)
2017-03-15 07:58:22 +00:00
Igor Gajsin a8ac320d8f add tests for alembic migration
There was the commit that removes vmware. It has alembic migrations,
but doesn't have any tests for that. This commit fixes that.

Change-Id: I66090b0a0d7bfbd8e2365ec027fabfefc9d612da
Closes-Bug: #1668249
2017-03-13 14:02:23 +03:00
Mikhail 9c51c8e22b Serialize mtu for dpdk interface with 'i40e' driver
On NIC: XL710 with driver 'i40e'  MTU does not take into
account 4 bytes VLAN tag, so we should increase it manually

DocImpact
Change-Id: I3d95db9ec6fae4d8cd397c429d785dbdf1502b21
Partial-Bug: #1587310
Co-Authored-By: Fedor Zhadaev <fzhadaev@mirantis.com>
2017-03-06 12:01:47 +00:00
Julia Aranovich c0384fac05 Make Ceilometer and MongoDB experimental features
Since Fuel 10, Ceilometer and MongoDB services become experimental features.

* Ceilometer and MongoDB settings are shown on Settings tab in Fuel UI
  only if "experimental" feature group is enabled
* MongoDB role is available if "experimental" feature group is enabled only
* Ceilometer option is removed from the cluster creation wizard

Implements: blueprint remove-ceilometer

Change-Id: I6df3b47c14cafb1544dfe034cd9a2c0ad14205be
2017-03-01 09:24:14 +00:00
Jenkins d72287f19e Merge "Fixed switch to deploying state for nodes which will not be deployed" 2017-02-28 21:33:49 +00:00
Julia Aranovich 9fc3e312dc Make Murano an experimental feature
Since Fuel 10, Murano service becomes an experimental feature.

* Murano settings are shown on Settings tab in Fuel UI
  only if "experimental" feature group is enabled
* Murano option is removed from the cluster creation wizard

Implements: blueprint make-murano-experimental

Change-Id: I4dd0853138c045b8d7e8f6ff940c09250763a56b
2017-02-22 14:12:31 +00:00
Jenkins 3448243ba3 Merge "remove vmware" 2017-02-21 13:06:39 +00:00
Igor Gajsin 634b66df45 remove vmware
VMware not supported since Fuel 10.

Implements: blueprint remove-vmware

Change-Id: Ia26a97d13bb9c3d2e838d639efdababbb15f86da
2017-02-14 15:52:27 +00:00
Dmitry Sutyagin 03aaca2dee Add limit, offset, order in collection GET
Allow limiting the number of objects returned via GET
by providing "limit"
Example: api/notifications?limit=5

Allow offseting (skipping N first records) via "offset"
Example: api/notifications?offset=100

Allow ordering of objects by providing "order_by"
Example: api/notifications?order_by=-id

Add helper functions/classes to:
- get HTTP parameters (limit, offset, order_by)
- get scoped collection query by applying 4 operations
  filter, order, offset, limit
- set Conent-Range header if scope limits are present

Make default NailgunCollection's GET utilize scoped query
This makes default (parent) GET of child handlers support paging
and ordering (overriden GET methods will not get this functionality
automatically)
NailgunCollection.GET is also an example of how to implement
this new functionality.

Helper functions/classes can be utilized in child handler methods
to implement filters / ordering / paging

Related-Bug: 1657348
Change-Id: I7760465f70b3f69791e7a0c558a26e8ba55c934a
2017-02-14 08:11:23 +00:00
Jenkins c77aa8923a Merge "Limit the minimal RAM amount for OVS+DPDK to 1024MB" 2017-02-13 12:25:55 +00:00
Jenkins 6c29488b0c Merge "Set nodes' statuses to 'error' when their nodegroup is deleted" 2017-02-09 11:17:07 +00:00
Jenkins da7f9b12a2 Merge "Handler for changing notifications statuses added" 2017-02-08 14:32:33 +00:00
Bulat Gaifullin dc8fd092d3 Fixed switch to deploying state for nodes which will not be deployed
Nodes can be exluded from deployment, if there is no tasks to run on this nodes.
Such nodes should not be switched to deployment state.

Change-Id: I4cd23769b7643aae7b149ba30e5b0e91a3021563
2017-02-08 13:20:22 +00:00
Andreas Jaeger e2c008418d Prepare for using standard python tests
Add simple script to setup mysql and postgresql databases, this script
can be run by users during testing and will be run by CI systems for
specific setup before running unit tests. This is exactly what is
currently done by OpenStack CI in project-config.

This allows to change in project-config the python-db jobs to
python-jobs since python-jobs will call this script initially.

See also
http://lists.openstack.org/pipermail/openstack-dev/2016-November/107784.html

Change-Id: Ib72322030d7dc6979380f74379893084610982a1
2017-02-07 19:45:38 +00:00
Vladimir Kuklin cd2ee13830 Set nodes' statuses to 'error' when their nodegroup is deleted
According to the bug below and the spec, we did not implement
one multirack feature aspect.

https://specs.openstack.org/openstack/fuel-specs/specs/8.0/multi-rack-static.html#notifications-impact

Now we add resetting node to error to node group deletion callback and
send a notification.

Change-Id: I6b2bae5601ba7dbca620bb3861e95b0e554f8699
Closes-bug: #1644630
2017-02-07 17:44:06 +00:00
Jenkins 3d6d48a233 Merge "Moving task cache update into task creation transaction" 2017-02-07 17:11:22 +00:00
Anastasiya d2a5333e99 Limit the minimal RAM amount for OVS+DPDK to 1024MB
* min value was set in consts
* appropriate validator was added
* tests for validator were changed
* test for serializer were changed

Change-Id: Ib8ccb0658bd401ce492257f855013d1d7e0f2dac
Closes-Bug: #1653081
2017-02-07 16:42:01 +00:00
Alexander Kislitsky 71fb921dbe Handler for changing notifications statuses added
As workaround for decreasing loading in the UI on unread
notifications fetching we can mark all notifications as read.
For such purposes we add NotificationsMarkAllHandler

Change-Id: I2e6a0daaf8712ab3064df728a8fb463ef805aa06
Partial-Bug: #1657348
2017-02-07 16:39:05 +00:00
Georgy Kibardin 5eb41d7bc1 Stick to jsonschema prior 2.6.0
Since they have broken compatibility in error messages.

Closes-Bug: #1662529

Change-Id: I8c7449297a488ea3fe90cda58e5dba0d8d4d68ff
2017-02-07 14:13:55 +00:00
Georgy Kibardin ff04eb8394 Moving task cache update into task creation transaction
Doing this we avoid inclusion of task cache update statement into the next
transaction which may cause different problems such as deadlock.
In this particular case we've got the following deadlock:
1. DeleteIBPImagesTask makes UPDATE tasks SET cache....
2. Response handler in receiver makes SELECT clusters FOR UPDATE
3. The code following DeleteIBPImagesTask makes SELECT clusters FOR UPDATE
4. Response handler performs SELECTS tasks FOR UPDATE

Change-Id: Ic8e5f2386364421b0667f920499e031f587f726e
Closes-Bug: #1653083
2017-01-31 12:31:59 +00:00
Jenkins 542b8558f2 Merge "Handler for counting notifications statuses added" 2017-01-31 11:36:39 +00:00
Alexander Kislitsky d4bf85957e Handler for counting notifications statuses added
For calculation of notifications statuses we made requests in the UI
and fetch all notifications data and process them on the UI side.
We want to replace a polling of the whole notification collection by
a polling of unread notifications number. This dramatically decrease
Fuel UI load in case of a big amount of notifications.

Change-Id: I8f83d4e2d7f58beaf06c489b2264ccb69f9927ce
Partial-Bug: #1657348
2017-01-30 18:03:15 +03:00
Alexander Kislitsky e46b57befc Message format on node(-s) deletion error changed
In case of already running task we are returning JSON instead
of plain text.

Change-Id: Ic3998b8e534b2fa67813faf033335e849abe55af
Closes-Bug: #1657350
2017-01-26 20:41:56 +03:00