Commit Graph

15 Commits

Author SHA1 Message Date
Scott Hussey 0f39a55942 Refactor build to include Go
Adding the baclient code to Drydock requires a refactor
of the build automation to support multiple languages
and multiple artifacts included in a single Docker image

NOTE: the go source here is a placeholder 'hello world' sample

Change-Id: I1b4883f018b33b3d4fcd7cbcb6cba660fcdc93de
2018-09-04 13:13:21 -05:00
Scott Hussey e2b3e8ee8e Database tunability
Add additional tunables to how SQLalchemy creates connection pools

Also start including static doc assets to they can be published

Change-Id: I268dc265a6b6cf1a200b235a5f99e65e89a95637
2018-08-21 14:09:36 -05:00
Scott Hussey fda50be35e [411429] maasdriver support of packages
- Support pkg_list bootactions in the MAAS driver by using
  cloud-init user_data on deployment
- Add site definition caching to ease load on Deckhand

Change-Id: I2c8c7dfdd23992fae42fa32edab308f801d05867
2018-06-12 09:58:02 -05:00
Scott Hussey cc77125953 [411387] Schema update to support repo
- Support one or more repo specifications for a site
- Add object model for repository
- Add testing for repository parsing

Update freeze job with make target

- Update the requirements freeze job
  to have a mark target that rebuilds the tox
  virtualenv each run
- Update Dockerfile to create a valid /etc/protocols file

Change-Id: I9d09b7dd7226827995e23756ff968b36eaa4d16c
2018-06-07 23:08:20 +00:00
Scott Hussey dbad75775b Libvirt OOB driver
- Create a driver to support OOB actions via libvirt API
- Update Makefile with external dependency target
- Update Makefile and tooling to support new chart pipeline
   - Add 'drydock' make target for chart building
   - Add step to install helm binary

Change-Id: I8a3984d8fd70f99a82a954b7a869eab8e30145b4
2018-05-25 11:13:53 -05:00
Scott Hussey 94d351b364 [Fix] Migrate to pymongo BSON library
* bson is incompatible w/ pip 10, migrate to pymongo bson
* Update tox to run tests w/ locked requirements file as this
  is what production images are built with

Change-Id: Iacf3301ddc224f3525102b0204f349b733608138
2018-04-19 09:30:18 -05:00
Aaron Sheffield 8939447669 Gets a dictionary of device aliases to logicalnames for the site_design.
Change-Id: I9c5ec6d4dd3ac0114fc6d866a7dd8033b7807b15
2018-02-26 10:09:12 -05:00
Scott Hussey f4dba218ac Implement bootaction API
- Implement boot action rendering and API
- Reorganize DB integration tests and add a tox -e postgres entrypoint
- Add boot action unit tests
- Add node filter unit test
- Add boot action context creation to deployment workflow
- Fix regression bug in MaaS Machines model
- Downgrade to Python 3.5 due to CICD limitations

Change-Id: I6c8f100cbe209f9b1c6c6ff1285365d89343ae2a
2017-11-20 16:29:54 -06:00
Scott Hussey d12ef71f9f Refactor orchestrator
Refactor orchestrator to break large
monolithic functions into small functions
per action.

- Update orchestrator to match new statemgmt API
- Pull most code out of __init__.py files
- Create action classes for Orchestrator actions
- Create action classes for Driver actions
- Orchestrator consumes tasks from database queue
- Additional encapsulation of task functionality into Task class
- Create shared integration test fixtures
- Fix Sphinx entrypoint so package install works
- Disable bootdata API until BootAction implementation
- Bring codebase into PEP8 compliance
- Update documentation reflect code changes
- Mark SQL #nosec for bandit

Change-Id: Id9a7bdedcdd5bbf07aeabbdb52db0f0b71f1e4a4
2017-10-26 15:00:39 -05:00
Scott Hussey e042811c76 DRYD47 - Task persistance and refactor
Refactor the task model and add database persistence for tasks.

- Document task schema
- Use Alembic for database creation
- Use SQLalchemy for database abstraction
- Update Task status and result enumerations

Change-Id: I247b88f293144a0bdf891958e19711d975c729ba
2017-10-26 14:54:25 -05:00
Scott Hussey 27d54b3c46 Add client access to enquiry API
Add drydock client access to the enquiry API
endpoint and CLI commands to access this endpoit.
Use PrettyTable to output the data.

Add a tox job for creating the frozen dependency list

Change-Id: Ie1724052eb9ae9500e6b0df8f0c78e25ae0617f4
2017-10-18 16:18:07 -05:00
Scott Hussey e892df58dc Fix issues failing CI pipeline
- Run codebase through YAPF for formatting
- Add tox configuration for yapf and pep8
- Fix some non-YAPF pep8 failures
- Enhance verify_site for better MaaS-integration testing
- Create initial basic functional test

Change-Id: Ie5b5275d7795693a6551764362aee916b99b3e56
2017-08-24 10:18:11 -05:00
Scott Hussey 4ae627be44 Add config generation to tox.ini
Move sample config to etc/drydock
Update docs to generate a config with tox

Update configuration for Keystone

- Add config generation to tox.ini
- Fix default in bootdata config
- Add keystone dependencies
- Add config generator config
- Move sample config to a skeleton etc/drydock tree

Use PasteDeploy for WSGI integration

Using keystonemiddleware outside of a PasteDeploy
pipeline is deprecated. Move Drydock to use PasteDeploy
and integrate with keystonemiddleware

Update Falcon context object

Add keystone identity fields to context object
Clean up context marker field

Fix AuthMiddleware for keystone

Update falcon middleware to harvest headers injected
by keystonemiddleware

Fix context middleware

Update context middleware to enforce
a UUID-formatted external context marker

Lock keystonemiddleware version

Lock keystonemiddleware version to the Newton release

Sample drydock.conf with keystone

This drydock.conf file is known to integrate successfully
with Keystone via keystonemiddleware and the password plugin

Add .dockerignore

Stop adding .tox environment to docker images

Integrate with oslo.policy

Add oslo.policy 1.9.0 to requirements (Newton release)
Add tox job to generate sample policy.yaml
Create DrydockPolicy as facade for RBAC

Inject policy engine into API init

Create a DrydockPolicy instance and inject it into
the Drydock API resources.

Remove per-resource authorization

Update Drydock context and auth middleware

Update Drydock context to use keystone IDs instead of names as required
by oslo.policy
Update AuthMiddleware to capture headers when request provides
a service token

Add RBAC for /designs API

Add RBAC enforcement for GET and POST of
/api/v1.0/designs endpoint

Refactor check_policy

Refactor check_policy into the base class

Enforce RBAC for /designs/id endpoint

Enforce RBAC on /designs/id/parts endpoint

Enforce RBAC on /designs/id/parts/kind

Enforce RBAC on /designs/id/parts/kinds/

Enforce RBAC on /tasks/ endpoints

Create unit tests

- New unit tests for DrydockPolicy
- New unit tests for AuthMiddleware w/ Keystone integration

Address impacting keystonemiddleware bug

Use v4.9.1 to address https://bugs.launchpad.net/keystonemiddleware/+bug/1653646

Add oslo_config fixtures for unit testing

API base class fixes

Fix an import error in API resource base class

More graceful error handling in drydock_client

Create shared function for checking API response status codes

Create client errors for auth

Create specific Exceptions for Unauthorized
and Forbidden responses

Ignore generated sample configs

Lock iso8601 version

oslo.versionedobjects appears to be impcompatible with
iso8601 0.1.12 on Python 3.2+

Update docs for Keystone

Note Keystone as a external depdendency and
add notes on correctly configuring Drydock for
Keystone integration

Add keystoneauth1 to list_opts

Explicitly pull keystoneauth password plugin
options when generating a config template

Update reference config for keystone

Update the reference config template
for Keystone integration

Add keystoneauth1 to requirements

Need to directly include keystoneauth1 so that
oslo_config options can be pulled from it

Update config doc for keystoneauth1

Use the keystoneauth1 generated configuration options
for the configuration  docs

Remove auth options

Force dependence on Keystone as the only authentication
backend

Clean up imports

Fix how falcon modules are imported

Default to empty role list

Move param extraction

Enforce RBAC before starting to parse parameters

Implement DocumentedRuleDefault

Use DocumentedRuleDefault for policy defaults at request
of @tlam. Requires v 1.21.1 of oslo_policy, which is tied
to the Pike openstack release.

Change sample output filenames

Update filenames to follow Openstack convention

Fix tests to use hex formatted IDs

Openstack resource IDs are not hyphenated, so update
unit tests to reflect this

Fix formating and whitespace

Refactor a few small items for code review

Update keystone integration to be more
robust with Newton codebase

Centralize policy_engine reference to
support a decorator-based model

RBAC enforcement decorator

Add units tests for decorator-based
RBAC and the tasks API

Minor refactoring and format changes

Change-Id: I35f90b0c88ec577fda1077814f5eac5c0ffb41e9
2017-08-21 14:35:56 -05:00
Scott Hussey 8a46aa5761 Add click dependency
Add click 6.7 to the requirements-direct.txt file
2017-07-21 13:47:17 -05:00
Scott Hussey 4efe40d184 Update Dockerfile to better pattern
Add requirements layer
Add entrypoint.sh script to support future CLI
Change Dockerfile to use ENTRYPOINT
2017-06-28 11:26:53 -05:00