Commit Graph

320 Commits

Author SHA1 Message Date
Chi Lo a97d54e8ef Optimize IMS service to send request to region(s)
Instead of broadcasting service request to all regions, IMS service
will now send request to only affected regions.  Un-affected regions
will not receive request to perform a no-op.

Change-Id: I4008739676ce8ccd71f72bca57fb33dd05f001bc
2021-01-05 15:39:13 -08:00
Chi Lo 8f13fc47ed Modified Makefile to force building image with lastest Ubuntu base image
Change-Id: I3d1deaadfe67daf6650e60cfb8cb95503e658a9f
2020-12-11 07:10:23 -08:00
Chi Lo aa15e29fb3 Refactor catching exception at services controller level
Change-Id: I17ba7b9f1142b9bc52e3facc0813cf962ecf87c0
2020-12-10 10:25:45 -08:00
Chi Lo 6b91d346a4 Optimize FMS service to send request to region(s)
Instead of broadcasting service request to all regions, FMS service
will now send request to only affected regions.  Un-affected regions
will not receive request to perform a no-op.

This patch also catches exceptions from flavor logic which has
been missed.

Change-Id: Id285ba98b8a53f11cada05a7fb380f6facf69594
2020-12-08 15:34:38 -08:00
Chi Lo 6889e77462 Optimize CMS service to send request to region(s)
Instead of broadcasting service request to all regions, CMS service
will now send request to only affected regions.  Un-affected regions
will not receive request to perform a no-op.

This patch also catches exceptions from customer logic which has
been missed.

Change-Id: Id86886fcb80329582baa596c1faa1d4126b0764d
2020-12-03 08:43:42 -08:00
Chi Lo 30ccde094b Format string and error log correction
Change-Id: I54829ae69d7f44b6aa46c3c2aabecfb380103fee
2020-11-30 13:06:52 -08:00
jh629g 987cc8c770 Update Error Statement WRT recent rebase
Continuing to improve error message
catching with regard to the recent
overhaul of error base code.

Change-Id: I1635647ad7edc5ebb3021c421a82cc81ee976256
2020-11-19 16:02:46 +00:00
jh629g baecd4b567 Update Error Types, Log Heat Errors
Some error types were updated to recommendations
which better fit their use cases, providing
better information. Heat errors are now
logged when seen in rds status checking,
to make debugging of heat errors such
as stack validation more readily apparent.

Change-Id: Ifd11edaead65bf2fcd6d828e397bf41ba0816b63
2020-11-05 20:28:51 +00:00
jh629g 64bd786367 Updating Ranger Error Bases
Ranger errors are split out
unecessarily in the code base
and need to be refactored for
python 3.6 best practice

Change-Id: I06b1e2679ff2f0d7cadf7eab4ab0a7cc61e138ca
2020-11-03 01:38:10 +00:00
Chi Lo 1f67e02206 Refactor RDS resource status retrieval
Change-Id: I25fba0f687c42d3b5782fd1434a0da6a3816dcf0
2020-10-26 22:42:50 -07:00
jh629g d2c95ac8c8 Update FMS service to provide values in error message
Ram, vcpus, disk, and ephemeral storage values
provided in error messages will allow team
to better debug output when rare out of bounds
defect occurs. This will allow us to determine
if the problem is the number calculated for
testing or the ranger formula in the code that
is responsible for this defect.

Change-Id: I8aedefa757b48319f90ac97c1dfd3495218ad4d2
2020-10-16 21:54:11 +00:00
Chi Lo 921179d5f6 Disable Groups controller for Keystone Group feature
This patchset disabled Groups controller and removed related
API/CLI unit test cases.

Change-Id: I6bbd3615c85cc9db98e57fd6fa9ccfeb80c572c9
2020-10-12 11:54:06 -07:00
Chi Lo b249c3a675 Refactor status wait time for Ranger-agent response
Ranger-agent may take a longer time to process IMS resource request from Ranger.
This ps allows Ranger to wait for Ranger-agent response an extended period of time
compared to other service requests such as FMS or CMS.

Change-Id: Ie4ab9944d9636b62b76b87b093a31724177e767e
2020-09-30 13:41:08 -07:00
Chi Lo 84c23a4781 Associate tenants as memeber list to shared image
Change-Id: I297c9a9ec77a64b07b3ad6a8c59121c7381bfd97
2020-09-24 17:05:04 +00:00
Gage Hugo 68a500b3de Pep8 and Ranger Image Build process update
Updated pep8 requirements and fixed resulting
codes to fix broken pep8 job.

Updated docker image build process to use roles
to ensure docker rather than using manual docker
install process to fix broken ranger image job.

Co-Authored-By: Jeremy Houser <jeremyhouser@protonmail.com>
Co-Authored-By: Chi Lo <cl566n@att.com>

Change-Id: I28df0a27e4b354dd53c17fbb1a9468cb7ff5bc16
2020-09-24 15:26:26 +00:00
Chi Lo 8366b10bef Add customer's domain attribute in CMS API call
When CMS API call is made for either create and update customer, a

A new optional customer domain field is introduced for CMS API call
to create or update customer. The customer domain value will be used
by Rds to set the customer and users domain value when generating the
heat template.  In the event customer domain is not provided by the
CMS API call, a default value will be retrieved from the config file.

Change-Id: I58dbae760d56b756b72c5d9eaf7c0b61cf193847
2020-09-01 09:15:04 -07:00
Zuul 88b95a0dcb Merge "RMS direct access to database for region resources" 2020-08-20 18:24:29 +00:00
Chi Lo 330c0ee6a9 RMS direct access to database for region resources
RMS can access database to verify if customer, flavor, and image
recources exist for a particular region.  This avoids the need
to invoke an api call to CMS, FMS, and IMS services while these
services also making api call to RMS itself.

Change-Id: Ic64e848b03a8fd8f078162f59a148b2dfb3322df
2020-08-20 16:56:16 +00:00
jh629g 94170b688e Update CMS for cross domain role assignment
When a user with a domain different than
the domain of the user authorizing heat
to create the resources that ranger defines
is added to a customer template, heat returns
stack validation as the user cannot be found
in the domain of the authorizing user.
Updated to check if use match auth user and
if not, use ranger conf domain as previous

Change-Id: I1406b61c695a5d9a3d94e732b95c2b683c94852b
2020-08-19 17:59:57 +00:00
Chi Lo c72a3759b6 Specify public keystone endpoint when creating client
Change-Id: Ib5bab9e7b21937e70f4f90a65b3b518705378b74
2020-08-12 14:42:11 +00:00
jh629g 23b9d62dc5 Persist User Domain for role validation in Heat
Heat must have the domain of the user
in order to validate the user for role
assignment. Regardless of if roles are being
changed or assigned, heat always requires
and validates this data when heat receives
a project template. This work persists the user
domain when calls are made to CMS so that
RDS can add this data to project heat templates.

Change-Id: I5a8e72241e68dac730c3522d820a17d926fa3be8
2020-08-03 14:49:15 +00:00
Andrii Ostapenko 058c534796
Ensure pip is installed on zuul executor
Also add mock to test requirements

Change-Id: I016238e927f3c012fcc2b1510a1671003b839da8
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-13 21:56:35 -05:00
jh629g 7eef9932fd Update Ranger Auth Process for Update Region action
Changed ranger rms authentication to use
auth of site in request rather than site
of the ranger region when authorizing
updating the site

Change-Id: I85899e319b47eb70b1a5569894097a21b57e92c0
2020-06-08 18:06:40 -05:00
Haider, Nafiz (nh532m) efb8397228 Update activate/deactivate to single Active boolean property for image
Change-Id: Ie5e2d387c1aff7ee78c65bd5cbff18e81fa26c72
2020-05-18 21:27:11 +00:00
st6218 f9ce6832f2 Refactor uuid handling for Ranger service resources
This commit provides the following uuid management:

(1) Remove uuid record from database after the uuid owned by
a Ranger service (cms, fms, ims) resource is deleted.

(2) Remove stale uuid record from database when such
record is inadvertently created during unsuccessful Ranger
service resource creation.

(3) Add authentication for uuidgen service delete uuid API.

Change-Id: Ifebd68851ff6c0996e7b19d331f4dd99682bd7d2
2020-05-08 11:31:58 -07:00
hosingh000 36bacfb2cf Include attributes of Image
Change-Id: Ic602f4584646faaa3a1fbc7cd2663b46f3c60ba0
2020-04-27 14:04:07 +00:00
jh629g 42f1c52620 Collapse policy.json files
RMS has three rules which were not included into
the top level policy.json. These rules have been
moved to the top level in order to fix 403 against
uuid server. RMS specific policy.json was not
pointed at by any code, so with this fix
the file has been removed.

Change-Id: I8bf507a6336b8b07885e3e11490f1324bf29c5d4
2020-04-22 13:23:36 -05:00
st6218 c0b491797e Minor fix to create private flavor tenant
Change-Id: Ib4f4933e6f3a8ee296d58610cb29d97599632326
2020-04-17 13:46:59 -07:00
Zuul 93d0818457 Merge "Additional changes to private flavors" 2020-04-16 21:16:45 +00:00
st6218 385b4b7723 Additional changes to private flavors
add_tenants logic:
a) allow add tenant only if flavor already assigned to region(s)

b) each tenant in the request will be validated against the
   regions assigned to flavor - at least ONE tenant must
   pass validation:
   i. if NO tenant in tenant list is associated with any of the
      regions assigned to the flavor, Ranger will reject the
      request entirely and user will be prompted to submit new
      request with valid tenants.
  ii. only the tenants in tenant list that pass validation will
      be kept in flavor tenant list;  those that failed
      validation will be DROPPED from the tenant list.

delete_region logic (only for flavors with tenants):
a) if a tenant is associated only with the deleted region, the
   delete_region logic will delete the tenant from fms table
   and the tenant is dropped from the tenant list.

   However, if the tenant is associated with other regions still
   assigned to the flavor, the tenant stays in the list.

Change-Id: I31935477733c8597741cf7c7c57350ab1e2b4452
2020-04-16 13:38:41 -07:00
Zuul b01a5e8384 Merge "Shorten length of event details during image creation" 2020-04-16 19:15:41 +00:00
jh629g ebd3e448d6 Shorten length of event details during image creation
Fixed image create event_details to not be in excess
of allotted database size parameters.

Change-Id: Id8771f4be777d98ef3e058ed20a552d1398087c9
2020-04-16 16:57:43 +00:00
hosingh000 efd73bbd37 Change to use rms url
Change-Id: Ia505e68918f98a16253455d08c5db391510e3e0e
2020-04-15 13:04:15 -05:00
Chi Lo 14173e3111 Added psutil to requirements.txt file
Added psutil module for health-probe.

Change-Id: I5d88eef0e389826af682b4583b0c6c20687224a7
2020-04-14 12:37:36 -07:00
hosingh000 5d0335faaa Pass service url through configuration
Change-Id: I77334e2604481540954ea751ff2e46f46aa91aae
2020-04-06 22:14:47 +00:00
Chi Lo 4dbd0e4058 Allow i2 option to be independent option for flavor creation
Change-Id: I5d0f27544fa40d43d4b416d1d8e11d107ec44176
2020-03-20 13:35:37 -07:00
st6218 0d9596317e Update user domain for customer heat template
Change-Id: I1a03e58f96af57fb6737c7c2eedd519bafbefb6d
2020-03-17 12:59:56 -07:00
jh629g 09fb39e961 Change ranger policy.json
changed ranger policy.json to allow
use of expected project when running
tempest tests against ranger

Change-Id: I1319404899a68140443517a818497ac0856746f5
2020-03-06 10:32:26 -06:00
jh629g 3f76cb63b9 Fix Transactions table breaking when image url too long
When an image is created with a url that is in excess
of 240 characters, the transacations table in ranger
audit database throws error "sqlalchemy.exc.DataError:
(MySQLdb._exceptions.DataError) (1406, "Data too long
for column 'event_details' at row 1")". By removing
this url from the log, we prevent this error while
maintaining other important data. Also restored ranger
to allowed projects in ranger policy.json

Change-Id: I4af73beed9bd40054f8d1e7281bc6f9216fdd243
2020-03-05 15:26:59 -06:00
jh629g a5aacaa045 Add update statement for updating old sites
Added update statement which will update
old ranger deployment databases to add
ranger region domain name

Change-Id: If0020b8d3e9c5ff2cabfa5d8d6f9656806f2bc6c
2020-02-25 12:55:43 -06:00
Zuul 308b9af12d Merge "Removed RDS service call (cont'd)" 2020-02-24 21:45:51 +00:00
Chi Lo 2cba5155c7 Removed RDS service call (cont'd)
Change-Id: I0c5bebd53faca231f58b0280ef1423ef0d5d50fb
2020-02-24 12:08:22 -08:00
jh629g 93c873bc96 Update RDS to create resources as specified in Ranger Region
Ranger region now includes a field which declares
which openstack domain a resource should be deployed
in. This change updates rds to create resources in
that domain via the heat template

Change-Id: I43144eb75a34661fae15399b9d32842d65327621
2020-02-21 18:07:49 +00:00
Zuul d026199784 Merge "Removed RDS service call" 2020-02-21 17:45:34 +00:00
Chi Lo 305483dcc2 Removed RDS service call
This patch removes RDS service call to retreive resource status by
CMS, FMS, and IMS Ranger services.  These services retrieve the
status by accessing the resource status database directly.

Change-Id: I7695eb5dca697f3326fa2d19aa45681e4b5fc2ce
2020-02-20 18:03:12 -08:00
Chi Lo c06ecba646 Purged unused records from uuids table
This patch also removed transaction id hooks when calling
RootController default get.  As a result, uuids record will
not be created.

Change-Id: I2ca0be02256ec62c8f9d48266867cc44121c1c4b
2020-02-19 14:15:41 -08:00
Zuul c7c6cd725a Merge "Revert "Fix Audit Trail event_details db size"" 2020-02-18 16:45:45 +00:00
Jeremy Houser 1cbfa02d6a Revert "Fix Audit Trail event_details db size"
This reverts commit e5b04669c8

Change-Id: Ic5aec4848db35bb8a257742981615414b8777dcf
2020-02-18 16:19:10 +00:00
jh629g 5eb91bd5d5 Add Domain name to ranger region
Refactor ranger to allow domain
for resource deployment to be
defined in create region api call,
rather than only being retrieved
from ranger configuration.

Change-Id: I1b481c49d4402438c8aa91a8ebd8b6dbe797997b
2020-02-13 19:14:32 +00:00
Zuul 7ca26d96a5 Merge "Delete region logic and orm script changes" 2020-02-10 16:34:28 +00:00