Retire mors

This hasn't been used in a couple of years and appears to be dead.

Change-Id: I183dc04a8bca3a1b954de1fe436c15af308dd278
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This commit is contained in:
Stephen Finucane 2019-05-03 15:34:26 -06:00
parent 84a8238118
commit f028929402
48 changed files with 12 additions and 2112 deletions

5
.gitreview Normal file
View File

@ -0,0 +1,5 @@
[gerrit]
host=review.opendev.org
port=29418
project=x/mors.git
defaultbranch=master

View File

@ -1,6 +0,0 @@
REVIEWBOARD_URL = 'https://rbcommons.com/s/platform9/'
REPOSITORY = 'pf9-mors'
GUESS_DESCRIPTION = True
GUESS_SUMMARY = True
TARGET_GROUPS = 'platform9'
TRACKING_BRANCH = 'origin/atherton'

View File

@ -1,17 +0,0 @@
If you would like to contribute to the development of OpenStack, you must
follow the steps in this page:
http://docs.openstack.org/infra/manual/developers.html
If you already have a good understanding of how the system works and your
OpenStack accounts are set up, you can skip to the development workflow
section of this documentation to learn how changes to OpenStack should be
submitted for review via the Gerrit tool:
http://docs.openstack.org/infra/manual/developers.html#development-workflow
Pull requests submitted through GitHub will be ignored.
Bugs should be filed on Launchpad, not GitHub:
https://bugs.launchpad.net/mors

View File

@ -1,13 +0,0 @@
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@ -1,55 +0,0 @@
# Mors - OpenStack Lease Manager
![Mors](https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRIzc5fgaiZfJnbym_ZEx4CsZJ7qIiYjcrxth5hi80Q0IhfnxOg)
https://en.wikipedia.org/wiki/Mors_(mythology)
is a simple lease manager for OpenStack objects like Instances.
Mors is a useful tool for OpenStack based cloud used for dev, test or lab setups.
Typical usage in these scenarios include automatically or manual creation of Instances for demo, test or experiments.
In most cases these Instances are forgotten and never deleted eating up valuable resources.
Mors is a simple service that helps enforce a policy per Tenant or Instance and automatically delete Instances after
a specified duration.
## Details
Mors works by specification of lease policy in a hierarchical fashion, first at a Tenant level and further at
individual Instance level.
### Tenant Lease Policy
Mors lease policy can be enabled or disabled at Tenant level. If Mors policy is disabled (default for each tenant)
no lease policies apply to the instances within that tenant.
At Tenant level, policy is specified in terms of _duration_ . Once Mors policy is enabled, any Instance will be deleted
after `instance.created_time + tenant.lease duration = instance_expiration`
#### Roles
Tenant leases can be viewed by user with 'member' role and modified by users with 'admin' role
### Instance Lease Policy
By default Instance leases are governed by the policies at Instance's Tenant level. As mentioned earlier:
`instance.created_time + tenant.lease duration = instance_expiration`
A member of tenant can change the Instance expiry at any time, but it can never be later than now + tenant.lease duration
`max instance lease <= now + tenant.lease duration`
A user can always come back at a later point of time and renew the release again.
#### Roles
Instance leases can be modified by both 'member' and 'admin' roles.
## Build & Installation
Support subdirectory contains Makefile to build a RPM, apart from python 2.7, virtualenv it needs [fpm](https://github.com/jordansissel/fpm), _fpm_
is a simple package build utility that can build both RPM and deb packages. RPM itself is a thin wrapper on top of the virtualenv.
Configuration files are expected to be in /etc/pf9 directory. These are usual OpenStack style config files:
* pf9-mors.ini: configure the nova section with the user/password that can be used by mors to perform delete operations on nova instances.
The user needs to be an administrator.
* pf9-mors-api-paste.ini: configure the keystone middleware with keystone auth tokens.
The packages comes with an init script that works on RHEL 7 compatible systems

View File

@ -1,79 +1,9 @@
Mors - OpenStack Lease Manager
==============================
This project is no longer maintained.
|Mors| https://en.wikipedia.org/wiki/Mors\_(mythology) is a simple lease
manager for OpenStack objects like Instances.
The contents of this repository are still available in the Git
source code management system. To see the contents of this
repository before it reached its end of life, please check out the
previous commit with "git checkout HEAD^1".
Mors is a useful tool for OpenStack based cloud used for dev, test or
lab setups. Typical usage in these scenarios include automatically or
manual creation of Instances for demo, test or experiments. In most
cases these Instances are forgotten and never deleted eating up valuable
resources.
Mors is a simple service that helps enforce a policy per Tenant or
Instance and automatically delete Instances after a specified duration.
Details
-------
Mors works by specification of lease policy in a hierarchical fashion,
first at a Tenant level and further at individual Instance level.
Tenant Lease Policy
~~~~~~~~~~~~~~~~~~~
Mors lease policy can be enabled or disabled at Tenant level. If Mors
policy is disabled (default for each tenant) no lease policies apply to
the instances within that tenant.
At Tenant level, policy is specified in terms of *duration* . Once Mors
policy is enabled, any Instance will be deleted after
``instance.created_time + tenant.lease duration = instance_expiration``
Roles
^^^^^
Tenant leases can be viewed by user with member role and modified by
users with admin role
Instance Lease Policy
~~~~~~~~~~~~~~~~~~~~~
By default Instance leases are governed by the policies at Instances
Tenant level. As mentioned earlier:
``instance.created_time + tenant.lease duration = instance_expiration``
A member of tenant can change the Instance expiry at any time, but it
can never be later than now + tenant.lease duration
``max instance lease <= now + tenant.lease duration``
A user can always come back at a later point of time and renew the
release again.
Roles
^^^^^
Instance leases can be modified by both member and admin roles.
Build & Installation
--------------------
Support subdirectory contains Makefile to build a RPM, apart from python
2.7, virtualenv it needs `fpm`_, *fpm* is a simple package build utility
that can build both RPM and deb packages. RPM itself is a thin wrapper
on top of the virtualenv.
Configuration files are expected to be in /etc/pf9 directory. These are
usual OpenStack style config files: \* pf9-mors.ini: configure the nova
section with the user/password that can be used by mors to perform
delete operations on nova instances. The user needs to be an
administrator. \* pf9-mors-api-paste.ini: configure the keystone
middleware with keystone auth tokens.
The packages comes with an init script that works on RHEL 7 compatible
systems
.. _fpm: https://github.com/jordansissel/fpm
.. |Mors| image:: https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRIzc5fgaiZfJnbym_ZEx4CsZJ7qIiYjcrxth5hi80Q0IhfnxOg
For any further questions, please email
openstack-discuss@lists.openstack.org.

View File

@ -1,6 +0,0 @@
#!/bin/sh
set -e
source ../pf9-version/pf9-version.rc
export ROOT_DIR=`pwd`
cd $ROOT_DIR/support/
make all

View File

@ -1,75 +0,0 @@
# -*- coding: utf-8 -*-
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import sys
sys.path.insert(0, os.path.abspath('../..'))
# -- General configuration ----------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = [
'sphinx.ext.autodoc',
#'sphinx.ext.intersphinx',
'oslosphinx'
]
# autodoc generation is a bit aggressive and a nuisance when doing heavy
# text edit cycles.
# execute "export SPHINX_DEBUG=1" in your terminal to disable
# The suffix of source filenames.
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'mors'
copyright = u'2016, OpenStack Foundation'
# If true, '()' will be appended to :func: etc. cross-reference text.
add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
add_module_names = True
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# -- Options for HTML output --------------------------------------------------
# The theme to use for HTML and HTML Help pages. Major themes that come with
# Sphinx are currently 'default' and 'sphinxdoc'.
# html_theme_path = ["."]
# html_theme = '_theme'
# html_static_path = ['static']
# Output file base name for HTML help builder.
htmlhelp_basename = '%sdoc' % project
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass
# [howto/manual]).
latex_documents = [
('index',
'%s.tex' % project,
u'%s Documentation' % project,
u'OpenStack Foundation', 'manual'),
]
# Example configuration for intersphinx: refer to the Python standard library.
#intersphinx_mapping = {'http://docs.python.org/': None}

View File

@ -1,4 +0,0 @@
============
Contributing
============
.. include:: ../../CONTRIBUTING.rst

View File

@ -1,25 +0,0 @@
.. mors documentation master file, created by
sphinx-quickstart on Tue Jul 9 22:26:36 2013.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to mors's documentation!
========================================================
Contents:
.. toctree::
:maxdepth: 2
readme
installation
usage
contributing
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

View File

@ -1,38 +0,0 @@
============
Installation
============
From PyPi
~~~~~~~~~~~~~~~~~~~~~
At the command line::
$ pip install mors
Or, if you have virtualenvwrapper installed::
$ mkvirtualenv mors
$ pip install mors
RPM BUILD
~~~~~~~~~~~~~~~~~~~~~
Mors comes with an RPM installation and associated init.d scripts. Run the makefile under 'support' directory
and it will produce RPM under the build directory.
Support subdirectory contains Makefile to build a RPM, apart from python
2.7, virtualenv it needs `fpm`_, *fpm* is a simple package build utility
that can build both RPM and deb packages. RPM itself is a thin wrapper
on top of the virtualenv.
Configuration files are expected to be in /etc/pf9 directory. These are
usual OpenStack style config files: \* pf9-mors.ini: configure the nova
section with the user/password that can be used by mors to perform
delete operations on nova instances. The user needs to be an
administrator. \* pf9-mors-api-paste.ini: configure the keystone
middleware with keystone auth tokens.
The packages comes with an init script that works on RHEL 7 compatible
systems
.. _fpm: https://github.com/jordansissel/fpm

View File

@ -1 +0,0 @@
.. include:: ../../README.rst

View File

@ -1,5 +0,0 @@
========
Usage
========
Refer to README for details on the policies and how the API is structured.

View File

@ -1,100 +0,0 @@
#!/bin/sh
#
# chkconfig: - 98 02
# description: Platform9 Lease Manager (code -named Mors)
### BEGIN INIT INFO
# Provides: pf9-mors
# Required-Start: $remote_fs $network $syslog
# Required-Stop: $remote_fs $syslog
# Default-Stop: 0 1 6
# Short-Description: Mors Server
# Description: Platform9 Lease Manager (code -named Mors)
### END INIT INFO
. /etc/rc.d/init.d/functions
name=pf9-mors
prog=pf9-mors
bindir=/opt/pf9/$name/bin
python=$bindir/python
exec="$python $bindir/pf9_mors.py"
pidfile="/var/run/$name.pid"
log_stdout="/var/log/pf9/$name-out.log"
[ -e /etc/sysconfig/$prog ] && . /etc/sysconfig/$prog
lockfile=/var/lock/subsys/$prog
start() {
[ -x $python ] || exit 5
echo -n $"Starting $prog: "
daemon --pidfile $pidfile "$exec >> $log_stdout 2>&1 & echo \$! > $pidfile"
retval=$?
echo
[ $retval -eq 0 ] && touch $lockfile
return $retval
}
stop() {
echo -n $"Stopping $prog: "
killproc -p $pidfile $prog
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
stop
start
}
reload() {
restart
}
force_reload() {
restart
}
rh_status() {
status -p $pidfile $prog
}
rh_status_q() {
rh_status >/dev/null 2>&1
}
case "$1" in
start)
rh_status_q && exit 0
$1
;;
stop)
rh_status_q || exit 0
$1
;;
restart)
$1
;;
reload)
rh_status_q || exit 7
$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
restart
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload}"
exit 2
esac
exit $?

View File

@ -1,4 +0,0 @@
location /mors/ {
proxy_pass http://127.0.0.1:8989/;
include /etc/nginx/conf.d/pf9/cors.conf;
}

View File

@ -1,14 +0,0 @@
[app:myService]
paste.app_factory = mors.mors_wsgi:app_factory
[pipeline:main]
pipeline = authtoken myService
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
admin_token = RzvUwrEgiOaQFTXV
auth_uri = http://127.0.0.1:8080/keystone
identity_uri = http://127.0.0.1:8080/keystone_admin

View File

@ -1,17 +0,0 @@
[DEFAULT]
db_conn=
context_factory=
lease_handler=
listen_port=8989
sleep_seconds=60
paste-ini=/etc/pf9/pf9-mors-api-paste.ini
log_file=/var/log/pf9/pf9-mors.log
repo=/opt/pf9/pf9-mors/lib/python2.7/site-packages/mors_repo
[nova]
user_name=
password=
version=2
auth_url=
region_name=

View File

@ -1,15 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""

View File

@ -1,82 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
from flask import request, jsonify
import functools, os
def get_context():
return Context(request.headers['X-User-Id'],
request.headers['X-User'],
request.headers['X-Roles'],
request.headers['X-Tenant-Id'])
def error_handler(func):
from sqlalchemy.exc import IntegrityError
import traceback,sys
@functools.wraps(func)
def inner(*args, **kwargs):
try:
return func(*args, **kwargs)
except ValueError as exc:
traceback.print_exc(file=sys.stdout)
return jsonify({'error': 'Invalid input'}), 422, {'ContentType': 'application/json'}
except IntegrityError as exc:
traceback.print_exc(file=sys.stdout)
return jsonify({'error': 'Already exists'}), 409, {'ContentType': 'application/json'}
return inner
def enforce(required=[]):
"""
Generates a decorator that checks permissions before calling the
contained pecan handler function.
:param list[str] required: Roles require to run function.
"""
def _enforce(fun):
@functools.wraps(fun)
def newfun(self, *args, **kwargs):
if not (required):
return fun(*args, **kwargs)
else:
roles_hdr = request.headers('X-Roles')
if roles_hdr:
roles = roles_hdr.split(',')
else:
roles = []
if set(roles) & set(required):
return fun( *args, **kwargs)
else:
return jsonify({'error': 'Unauthorized'}), 403, {'ContentType': 'application/json'}
return newfun
return _enforce
class Context:
def __init__(self, user_id, user_name, roles_str, tenant_id):
self.user_id = user_id
self.user_name = user_name
self.roles = roles_str.split(',')
self.tenant_id = tenant_id

View File

@ -1,212 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
from datetime import datetime, timedelta
from leasehandler import get_lease_handler
from persistence import DbPersistence
from eventlet.greenthread import spawn_after
import logging
from leasehandler.constants import SUCCESS_OK, ERR_UNKNOWN, ERR_NOT_FOUND
logger = logging.getLogger(__name__)
def get_tenant_lease_data(data):
"""
Simple function to transform tenant database proxy object into an externally
consumable dictionary.
:param data: database row object
"""
return {'vm_lease_policy': {'tenant_uuid': data['tenant_uuid'],
'expiry_mins': data['expiry_mins'],
'created_at': data['created_at'],
'created_by': data['created_by'],
'updated_at': data['updated_at'],
'updated_by': data['updated_by']}}
def get_vm_lease_data(data):
"""
Simple function to transform instance database proxy object into an externally
consumable dictionary.
:param data: database row object
"""
return {'instance_uuid': data['instance_uuid'],
'tenant_uuid': data['tenant_uuid'],
'expiry': data['expiry'],
'created_at': data['created_at'],
'created_by': data['created_by'],
'updated_at': data['updated_at'],
'updated_by': data['updated_by']}
class LeaseManager:
"""
Lease Manager is the main class for mors dealing with CRUD operations for the REST API
as well as the actual deletion of the Instances. Instance deletion and discovery is achieved
through an object 'leasehandler'.
"""
def __init__(self, conf):
self.domain_mgr = DbPersistence(conf.get("DEFAULT", "db_conn"))
self.lease_handler = get_lease_handler(conf)
self.sleep_seconds = conf.getint("DEFAULT", "sleep_seconds")
def add_tenant_lease(self, context, tenant_obj):
logger.info("Adding tenant lease %s", tenant_obj)
self.domain_mgr.add_tenant_lease(
tenant_obj['tenant_uuid'],
tenant_obj['expiry_mins'],
context.user_id,
datetime.utcnow())
def update_tenant_lease(self, context, tenant_obj):
logger.info("Update tenant lease %s", tenant_obj)
self.domain_mgr.update_tenant_lease(
tenant_obj['tenant_uuid'],
tenant_obj['expiry_mins'],
context.user_id,
datetime.utcnow())
def delete_tenant_lease(self, context, tenant_id):
logger.info("Delete tenant lease %s", tenant_id)
return self.domain_mgr.delete_tenant_lease(tenant_id)
def get_tenant_leases(self, context):
logger.debug("Getting all tenant lease")
all_tenants = self.domain_mgr.get_all_tenant_leases()
all_tenants = map(lambda x: get_tenant_lease_data(x), all_tenants)
logger.debug("Getting all tenant lease %s", all_tenants)
return all_tenants
def get_tenant_lease(self, context, tenant_id):
data = self.domain_mgr.get_tenant_lease(tenant_id)
logger.debug("Getting tenant lease %s", data)
if data:
return get_tenant_lease_data(data)
return {}
def get_tenant_and_associated_instance_leases(self, context, tenant_uuid):
logger.debug("Getting tenant and instances leases %s", tenant_uuid)
return {
'tenant_lease': self.get_tenant_lease(context, tenant_uuid),
'all_vms':
map(lambda x: get_vm_lease_data(x), self.domain_mgr.get_instance_leases_by_tenant(tenant_uuid))
}
def check_instance_lease_violation(self, instance_lease, tenant_lease):
violation = False
expiry = instance_lease['expiry']
if (datetime.utcnow() + timedelta(seconds=tenant_lease['expiry_mins']*60)) < expiry:
violation = True
return violation
def get_instance_lease(self, context, instance_id):
data = self.domain_mgr.get_instance_lease(instance_id)
if data:
data = get_vm_lease_data(data)
logger.debug("Get instance lease %s %s", instance_id, data)
return data
def add_instance_lease(self, context, tenant_uuid, instance_lease_obj):
logger.info("Add instance lease %s", instance_lease_obj)
tenant_lease = self.domain_mgr.get_tenant_lease(tenant_uuid)
if not self.check_instance_lease_violation(instance_lease_obj, tenant_lease):
self.domain_mgr.add_instance_lease(instance_lease_obj['instance_uuid'],
tenant_uuid,
instance_lease_obj['expiry'],
context.user_id,
datetime.utcnow())
else:
raise ValueError("Instance lease exceeds tenant lease")
def update_instance_lease(self, context, tenant_uuid, instance_lease_obj):
logger.info("Update instance lease %s", instance_lease_obj)
tenant_lease = self.domain_mgr.get_tenant_lease(tenant_uuid)
if not self.check_instance_lease_violation(instance_lease_obj, tenant_lease):
self.domain_mgr.update_instance_lease(instance_lease_obj['instance_uuid'],
tenant_uuid,
instance_lease_obj['expiry'],
context.user_id,
datetime.utcnow())
else:
raise ValueError("Instance lease exceeds tenant lease")
def delete_instance_lease(self, context, instance_uuid):
logger.info("Delete instance lease %s", instance_uuid)
self.domain_mgr.delete_instance_leases([instance_uuid])
def start(self):
spawn_after(self.sleep_seconds, self.run)
# Could have used a generator here, would save memory but wonder if it is a good idea given the error conditions
# This is a simple implementation which goes and deletes VMs one by one
def _get_vms_to_delete_for_tenant(self, tenant_uuid, expiry_mins):
vms_to_delete = []
vm_ids_to_delete = set()
do_not_delete = set()
now = datetime.utcnow()
add_seconds = timedelta(seconds=expiry_mins*60)
instance_leases = self.get_tenant_and_associated_instance_leases(None, tenant_uuid)['all_vms']
for i_lease in instance_leases:
if now > i_lease['expiry']:
logger.info("Explicit lease for %s queueing for deletion", i_lease['instance_uuid'])
vms_to_delete.append(i_lease)
vm_ids_to_delete.add(i_lease['instance_uuid'])
else:
do_not_delete.add(i_lease['instance_uuid'])
logger.debug("Ignoring vm, vm not expired yet %s", i_lease['instance_uuid'])
tenant_vms = self.lease_handler.get_all_vms(tenant_uuid)
for vm in tenant_vms:
expiry_date = vm['created_at'] + add_seconds
if now > expiry_date and vm['instance_uuid'] not in vm_ids_to_delete \
and vm['instance_uuid'] not in do_not_delete:
logger.info("Instance %s queued up for deletion creation date %s", vm['instance_uuid'],
vm['created_at'])
vms_to_delete.append(vm)
else:
logger.debug("Ignoring vm, vm not expired yet or already deleted %s, %s", vm['instance_uuid'],
vm['created_at'])
return vms_to_delete
def _delete_vms_for_tenant(self, t_lease):
tenant_vms_to_delete = self._get_vms_to_delete_for_tenant(t_lease['tenant_uuid'], t_lease['expiry_mins'])
# Keep it simple and delete them serially
result = self.lease_handler.delete_vms(tenant_vms_to_delete)
remove_from_db = []
for vm_result in result.items():
# If either the VM has been successfully deleted or has already been deleted
# Remove from our database
if vm_result[1] == SUCCESS_OK or vm_result[1] == ERR_NOT_FOUND:
remove_from_db.append(vm_result[0])
if len(remove_from_db) > 0:
logger.info("Removing vms %s from db", remove_from_db)
self.domain_mgr.delete_instance_leases(remove_from_db)
def run(self):
# Delete the cleanup
tenant_leases = self.domain_mgr.get_all_tenant_leases()
for t_lease in tenant_leases:
self._delete_vms_for_tenant(t_lease)
# Sleep again for sleep_seconds
spawn_after(self.sleep_seconds, self.run)

View File

@ -1,25 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
from nova_lease_handler import NovaLeaseHandler
from fake_lease_handler import FakeLeaseHandler
import constants
def get_lease_handler(conf):
if conf.get("DEFAULT", "lease_handler") == "test":
return FakeLeaseHandler(conf)
else:
return NovaLeaseHandler(conf)

View File

@ -1,19 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
SUCCESS_OK = 0
ERR_NOT_FOUND = 1
ERR_UNKNOWN = 2

View File

@ -1,52 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
import constants
import logging
from datetime import datetime
# @TODO: Need to move this to a test folder
class FakeLeaseHandler:
# Singleton tenants
tenants = {}
def __init__(self,conf):
self.logger = logging.getLogger("test-lease-handler")
pass
def add_tenant_data(self, tenant_id, instances):
FakeLeaseHandler.tenants[tenant_id] = instances
print FakeLeaseHandler.tenants
def get_tenant_data(self, tenant_id):
return FakeLeaseHandler.tenants[tenant_id]
def get_all_vms(self, tenant_uuid):
return FakeLeaseHandler.tenants[tenant_uuid]
def delete_vm(self, tenant_uuid, vm_id):
vms = FakeLeaseHandler.tenants[tenant_uuid]
new_vm_data = filter(lambda x: x['instance_uuid'] != vm_id, vms)
FakeLeaseHandler.tenants[tenant_uuid] = new_vm_data
def delete_vms(self, vms):
result = {}
for vm in vms:
self.logger.info("Deleting VM vm %s", vm)
self.delete_vm(vm['tenant_uuid'], vm['instance_uuid'])
result[vm['instance_uuid']] = constants.SUCCESS_OK
return result

View File

@ -1,87 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
from novaclient import client
import logging
import novaclient
from datetime import datetime
from constants import SUCCESS_OK, ERR_NOT_FOUND, ERR_UNKNOWN
logger = logging.getLogger(__name__)
DATE_FORMAT = "%Y-%m-%dT%H:%M:%SZ"
def get_vm_data(data):
return {'instance_uuid': data.id,
'tenant_uuid': data.tenant_id,
'created_at': datetime.strptime(data.created, DATE_FORMAT)}
class NovaLeaseHandler:
def __init__(self, conf):
self.conf = conf
self.nova_client = client.Client(self.conf.get("nova", "version"),
username=self.conf.get("nova", "user_name"),
region_name=self.conf.get("nova", "region_name"),
tenant_id=self.conf.get("nova", "tenant_uuid"),
api_key=self.conf.get("nova", "password"),
auth_url=self.conf.get("nova", "auth_url"),
insecure=True, # Insecure to handle test systems
connection_pool=False)
def _get_nova_client(self):
return self.nova_client
def get_all_vms(self, tenant_uuid):
"""
Get all vms for a given tenant
:param tenant_uuid:
:return: an iteratble that returns a set of vms (each vm has a UUID and a created_at field)
"""
try:
with self._get_nova_client() as nova:
vms = nova.servers.list(search_opts={'all_tenants':1, 'tenant_id':tenant_uuid})
return map(lambda x: get_vm_data(x), vms)
except Exception as e:
logger.exception("Error getting list of vms for tenant %s", tenant_uuid)
return []
def _delete_vm(self, nova, vm_uuid):
try:
logger.info("Deleting VM %s", vm_uuid)
nova.servers.delete(vm_uuid)
return SUCCESS_OK
except novaclient.exceptions.NotFound:
return ERR_NOT_FOUND
except Exception as e:
logger.exception("Error deleting vm %s", vm_uuid)
return ERR_UNKNOWN
def delete_vms(self, vms):
"""
Delete a VM on a given tenant
:param tenant_uuid:
:param vm_uuid:
:return: dictionary of vm_id to result
"""
result = {}
try:
with self._get_nova_client() as nova:
for vm in vms:
result[vm['instance_uuid']] = self._delete_vm(nova, vm['instance_uuid'])
return result
except Exception as e:
logger.exception("Error deleting vm %s", vms)
return result

View File

@ -1,49 +0,0 @@
#!/opt/pf9/pf9-mors/bin/python
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
import argparse, logging
import ConfigParser
from migrate.versioning.api import upgrade, create, version_control
from migrate.exceptions import DatabaseAlreadyControlledError
def _get_arg_parser():
parser = argparse.ArgumentParser(description="Lease Manager for VirtualMachines")
parser.add_argument('--config-file', dest='config_file', default='/etc/pf9/pf9-mors.ini')
parser.add_argument('--command', dest='command', default='db_sync')
return parser.parse_args()
def _version_control(conf):
try:
version_control(conf.get("DEFAULT", "db_conn"), conf.get("DEFAULT", "repo"))
except DatabaseAlreadyControlledError as e:
print e
# Ignore the already controlled error
if __name__ == '__main__':
parser = _get_arg_parser()
conf = ConfigParser.ConfigParser()
conf.readfp(open(parser.config_file))
if 'db_sync' == parser.command:
_version_control(conf)
upgrade(conf.get("DEFAULT", "db_conn"), conf.get("DEFAULT", "repo"))
exit(0)
else:
print 'Unknown command'
exit(1)

View File

@ -1,141 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
from flask import Flask, request, jsonify
from lease_manager import LeaseManager
from context_util import enforce, get_context, error_handler
from flask.json import JSONEncoder
from datetime import datetime
DATE_FORMAT = "%Y-%m-%dT%H:%M:%SZ"
APP_NAME = "MORS"
class CustomJSONEncoder(JSONEncoder):
def default(self, obj):
try:
if isinstance(obj, datetime):
return obj.strftime(DATE_FORMAT)
iterable = iter(obj)
except TypeError:
pass
else:
return list(iterable)
return JSONEncoder.default(self, obj)
app = Flask(__name__)
app.debug = True
app.json_encoder = CustomJSONEncoder
lease_manager = None
@enforce(required=['admin'])
@app.route("/v1/tenant/", methods=['GET'], strict_slashes=False)
@error_handler
def get_all_tenants():
all_tenants = lease_manager.get_tenant_leases(get_context())
if all_tenants:
return jsonify({"all_tenants":all_tenants})
else:
return jsonify({}), 200, {'ContentType': 'application/json'}
@enforce(required=['_member_'])
@app.route("/v1/tenant/<tenant_id>", methods=['GET'])
@error_handler
def get_tenant(tenant_id):
tenant_lease = lease_manager.get_tenant_lease(get_context(), tenant_id)
if not tenant_lease:
return jsonify({'success': False}), 404, {'ContentType': 'application/json'}
return jsonify(tenant_lease)
@enforce(required=['admin'])
@app.route("/v1/tenant/<tenant_id>", methods=['PUT', 'POST'])
@error_handler
def add_update_tenant(tenant_id):
tenant_lease = request.get_json()["vm_lease_policy"]
if request.method == "POST":
lease_manager.add_tenant_lease(get_context(), tenant_lease)
else:
lease_manager.update_tenant_lease(get_context(), tenant_lease)
return jsonify({'success': True}), 200, {'ContentType': 'application/json'}
@enforce(required=['admin'])
@app.route("/v1/tenant/<tenant_id>", methods=['DELETE'])
@error_handler
def delete_tenant_lease(tenant_id):
lease_manager.delete_tenant_lease(get_context(), tenant_id)
return jsonify({'success': True}), 200, {'ContentType': 'application/json'}
#-- Instance - tenant related
@enforce(required=['_member_'])
@app.route("/v1/tenant/<tenant_id>/instances/", methods=['GET'], strict_slashes=False)
@error_handler
def get_tenant_and_instances(tenant_id):
instances = lease_manager.get_tenant_and_associated_instance_leases(get_context(), tenant_id)
if not instances:
return jsonify({'success': False}), 404, {'ContentType': 'application/json'}
return jsonify(instances)
# --- Instance related ---
@enforce(required=['_member_'])
@app.route("/v1/tenant/<tenant_id>/instance/<instance_id>", methods=['GET'])
@error_handler
def get_vm_lease(tenant_id, instance_id):
lease_info = lease_manager.get_instance_lease(get_context(), instance_id)
if lease_info:
return jsonify(lease_info), 200, {'ContentType': 'application/json'}
else:
return jsonify({'error': 'Not found'}), 404, {'ContentType': 'application/json'}
@enforce(required=['_member_'])
@app.route("/v1/tenant/<tenant_id>/instance/<instance_id>", methods=['DELETE'])
@error_handler
def delete_vm_lease(tenant_id, instance_id):
lease_manager.delete_instance_lease(get_context(), instance_id)
return jsonify({'success': True}), 200, {'ContentType': 'application/json'}
@enforce(required=['_member_'])
@app.route("/v1/tenant/<tenant_id>/instance/<instance_id>", methods=['PUT', 'POST'])
@error_handler
def add_update_vm_lease(tenant_id, instance_id):
lease_obj = request.get_json()
# ds = '2012-03-01T10:00:00Z' # or any date sting of differing formats.
date = datetime.strptime(lease_obj['expiry'], DATE_FORMAT)
lease_obj['expiry'] = date
if request.method == "POST":
lease_manager.add_instance_lease(get_context(), tenant_id, lease_obj)
else:
lease_manager.update_instance_lease(get_context(), tenant_id, lease_obj)
return jsonify({'success': True}), 200, {'ContentType': 'application/json'}
def start_server(conf):
global lease_manager
lease_manager = LeaseManager(conf)
lease_manager.start()
def shutdown_server():
func = request.environ.get('werkzeug.server.shutdown')
if func is None:
raise RuntimeError('Not running with the Werkzeug Server')
func()
def app_factory(global_config, **local_conf):
return app

View File

@ -1,128 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
import functools
import logging
from sqlalchemy import Table, MetaData
from sqlalchemy import create_engine
from sqlalchemy.pool import QueuePool
logger = logging.getLogger(__name__)
def db_connect(transaction=False):
"""
Generates a decorator that get connection from a pool and returns
it to the pool when the internal function is done
:param transaction: (boolean) should this function create and end transaction.
"""
def _db_connect(fun):
if hasattr(fun, '__name__'):
fun.__name__ = 'method_decorator(%s)' % fun.__name__
else:
fun.__name__ = 'method_decorator(%s)' % fun.__class__.__name__
@functools.wraps(fun)
def newfun(self, *args, **kwargs):
trans = None
conn = self.engine.connect()
if transaction:
trans = conn.begin()
try:
ret = fun(self, conn, *args, **kwargs)
if transaction:
trans.commit()
return ret
except Exception:
if transaction:
trans.rollback()
logger.exception("Error during transaction ")
raise
finally:
conn.close()
return newfun
return _db_connect
class DbPersistence:
def __init__(self, db_conn_string):
self.engine = create_engine(db_conn_string, poolclass=QueuePool)
self.metadata = MetaData(bind=self.engine)
self.tenant_lease = Table('tenant_lease', self.metadata, autoload=True)
self.instance_lease = Table('instance_lease', self.metadata, autoload=True)
@db_connect(transaction=False)
def get_all_tenant_leases(self, conn):
return conn.execute(self.tenant_lease.select()).fetchall()
@db_connect(transaction=False)
def get_tenant_lease(self, conn, tenant_uuid):
return conn.execute(self.tenant_lease.select(self.tenant_lease.c.tenant_uuid == tenant_uuid)).first()
@db_connect(transaction=True)
def add_tenant_lease(self, conn, tenant_uuid, expiry_mins, created_by, created_at):
logger.debug("Adding tenant lease %s %d %s %s", tenant_uuid, expiry_mins, str(created_at), created_by)
conn.execute(self.tenant_lease.insert(), tenant_uuid=tenant_uuid, expiry_mins=expiry_mins,
created_at=created_at, created_by=created_by)
@db_connect(transaction=True)
def update_tenant_lease(self, conn, tenant_uuid, expiry_mins, updated_by, updated_at):
logger.debug("Updating tenant lease %s %d %s %s", tenant_uuid, expiry_mins, str(updated_at), updated_by)
conn.execute(self.tenant_lease.update().where(
self.tenant_lease.c.tenant_uuid == tenant_uuid).
values(expiry_mins=expiry_mins,
updated_at=updated_at, updated_by=updated_by))
@db_connect(transaction=True)
def delete_tenant_lease(self, conn, tenant_uuid):
# Should we just soft delete ?
logger.debug("Deleting tenant lease %s", tenant_uuid)
conn.execute(self.tenant_lease.delete().where(self.tenant_lease.c.tenant_uuid == tenant_uuid))
conn.execute(self.instance_lease.delete().where(self.instance_lease.c.tenant_uuid == tenant_uuid))
@db_connect(transaction=False)
def get_instance_leases_by_tenant(self, conn, tenant_uuid):
return conn.execute(self.instance_lease.select(
self.instance_lease.c.tenant_uuid == tenant_uuid)).fetchall()
@db_connect(transaction=False)
def get_instance_lease(self, conn, instance_uuid):
return conn.execute(self.instance_lease.select((
self.instance_lease.c.instance_uuid == instance_uuid))).first()
@db_connect(transaction=True)
def add_instance_lease(self, conn, instance_uuid, tenant_uuid, expiry, created_by, created_at):
logger.debug("Adding instance lease %s %s %s %s", instance_uuid, tenant_uuid, expiry, created_by)
conn.execute(self.instance_lease.insert(), instance_uuid=instance_uuid, tenant_uuid=tenant_uuid,
expiry=expiry,
created_at=created_at, created_by=created_by)
@db_connect(transaction=True)
def update_instance_lease(self, conn, instance_uuid, tenant_uuid, expiry, updated_by, updated_at):
logger.debug("Updating instance lease %s %s %s %s", instance_uuid, tenant_uuid, expiry, updated_by)
conn.execute(self.instance_lease.update().where(
self.instance_lease.c.instance_uuid == instance_uuid).values
(tenant_uuid=tenant_uuid, expiry=expiry,
updated_at=updated_at, updated_by=updated_by))
@db_connect(transaction=True)
def delete_instance_leases(self, conn, instance_uuids):
# Delete 10 at a time, should we soft delete
logger.debug("Deleting instance leases %s", str(instance_uuids))
conn.execute(self.instance_lease.delete().where(self.instance_lease.c.instance_uuid.in_(instance_uuids)))

View File

@ -1,61 +0,0 @@
#!/opt/pf9/pf9-mors/bin/python
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
import eventlet
eventlet.monkey_patch()
from eventlet import wsgi
from paste.deploy import loadapp
import argparse, logging
import logging.handlers
import ConfigParser, os
from mors import mors_wsgi
def _get_arg_parser():
parser = argparse.ArgumentParser(description="Lease Manager for VirtualMachines")
parser.add_argument('--config-file', dest='config_file', default='/etc/pf9/pf9-mors.ini')
parser.add_argument('--paste-ini', dest='paste_file')
return parser.parse_args()
def _configure_logging(conf):
log_filename = conf.get("DEFAULT", "log_file")
logging.basicConfig(filename=log_filename,
level=logging.DEBUG,
format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
datefmt='%m-%d %H:%M')
handler = logging.handlers.RotatingFileHandler(
log_filename, maxBytes=1024 * 1024 * 5, backupCount=5)
logging.root.addHandler(handler)
def start_server(conf, paste_ini):
_configure_logging(conf)
paste_file = None
if paste_ini:
paste_file = paste_ini
else:
paste_file = conf.get("DEFAULT", "paste-ini")
wsgi_app = loadapp('config:%s' % paste_file, 'main')
mors_wsgi.start_server(conf)
wsgi.server(eventlet.listen(('', conf.getint("DEFAULT", "listen_port"))), wsgi_app)
if __name__ == '__main__':
parser = _get_arg_parser()
conf = ConfigParser.ConfigParser()
conf.readfp(open(parser.config_file))
start_server(conf, parser.paste_file)

View File

@ -1,4 +0,0 @@
This is a database migration repository.
More information at
http://code.google.com/p/sqlalchemy-migrate/

View File

@ -1,15 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""

Binary file not shown.

View File

@ -1,20 +0,0 @@
#!/usr/bin/env python
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
from migrate.versioning.shell import main
if __name__ == '__main__':
main(debug='False')

View File

@ -1,25 +0,0 @@
[db_settings]
# Used to identify which repository this database is versioned under.
# You can use the name of your project.
repository_id=Mors
# The name of the database table used to track the schema version.
# This name shouldn't already be used by your project.
# If this is changed once a database is under version control, you'll need to
# change the table name in each database too.
version_table=migrate_version
# When committing a change script, Migrate will attempt to generate the
# sql for all supported databases; normally, if one of them fails - probably
# because you don't have that database installed - it is ignored and the
# commit continues, perhaps ending successfully.
# Databases in this list MUST compile successfully during a commit, or the
# entire commit will fail. List the databases your application will actually
# be using to ensure your updates to that database work properly.
# This must be a list; example: ['postgres','sqlite']
required_dbs=[]
# When creating new change scripts, Migrate will stamp the new script with
# a version number. By default this is latest_version + 1. You can set this
# to 'true' to tell Migrate to use the UTC timestamp instead.
use_timestamp_numbering=False

View File

@ -1,36 +0,0 @@
# Copyright Platform9 Systems Inc. 2016
from sqlalchemy import Table, Column, Integer, String, MetaData, DateTime
meta = MetaData()
tenant_lease = Table(
'tenant_lease', meta,
Column('tenant_uuid', String(40), primary_key=True),
Column('expiry_days', Integer),
Column('created_at', DateTime),
Column('updated_at', DateTime),
Column('created_by', String(40)),
Column('updated_by', String(40))
)
vm_lease = Table(
'instance_lease', meta,
Column('instance_uuid', String(40), primary_key=True),
Column('tenant_uuid', String(40)),
Column('expiry', DateTime),
Column('created_at', DateTime),
Column('updated_at', DateTime),
Column('created_by', String(40)),
Column('updated_by', String(40))
)
def upgrade(migrate_engine):
meta.bind = migrate_engine
tenant_lease.create()
vm_lease.create()
def downgrade(migrate_engine):
meta.bind = migrate_engine
vm_lease.drop()
tenant_lease.drop()

View File

@ -1,33 +0,0 @@
# Copyright Platform9 Systems Inc. 2016
from sqlalchemy import MetaData, String, Table
def upgrade(migrate_engine):
meta = MetaData(bind=migrate_engine)
table = Table('tenant_lease', meta, autoload=True)
tenant_leases = list(table.select().execute())
for tenant_lease in tenant_leases:
old_val = tenant_lease['expiry_days']
new_val = old_val*24*60
table.update().where(
table.c.expiry_days == old_val).values(
expiry_days=str(new_val)).execute()
col = getattr(table.c, 'expiry_days')
col.alter(name='expiry_mins')
def downgrade(migrate_engine):
meta = MetaData(bind=migrate_engine)
table = Table('tenant_lease', meta, autoload=True)
tenant_leases = list(table.select().execute())
for tenant_lease in tenant_leases:
old_val = tenant_lease['expiry_mins']
new_val = old_val/(24*60)
table.update().where(
table.c.expiry_mins == old_val).values(
expiry_mins=str(new_val)).execute()
col = getattr(table.c, 'expiry_mins')
col.alter(name='expiry_days')

View File

@ -1,29 +0,0 @@
[metadata]
name = mors
summary = OpenStack project for lease management of Instance and other OpenStack objects
description-file =
README.rst
author = OpenStack
author-email = openstack-dev@lists.openstack.org
home-page = http://www.openstack.org/
classifier =
Environment :: OpenStack
Intended Audience :: Information Technology
Intended Audience :: System Administrators
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: 2.7
[files]
packages =
mors
[build_sphinx]
source-dir = doc/source
build-dir = doc/build
all_files = 1
[upload_sphinx]
upload-dir = doc/build/html

View File

@ -1,51 +0,0 @@
#!/usr/bin/env python
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
from setuptools import setup
setup(name='mors',
version='0.1',
description='Platform9 Mors (lease manager)',
author='Platform9',
author_email='opensource@platform9.com',
url='https://github.com/platform9/pf9-mors',
packages=['mors',
'mors/leasehandler',
'mors_repo',
'mors_repo/versions'],
install_requires=[
'pbr==0.11.0',
'pytz==2015.7',
'keystoneauth1==2.3.0',
'oslo.i18n==3.4.0',
'oslo.serialization==2.4.0',
'oslo.utils==3.7.0',
'keystonemiddleware==4.3.0',
'Paste==1.7.5.1',
'PasteDeploy==1.5.2',
'pip==1.5.2',
'python-novaclient==3.2.0',
'flask==0.10.0',
'SQLAlchemy==0.9.8',
'sqlalchemy-migrate==0.9.5',
'PyMySQL',
'eventlet==0.18.4',
'requests==2.13',
'nose',
'proboscis'
],
scripts=['mors/pf9_mors.py', 'mors/mors_manage.py']
)

View File

@ -1,65 +0,0 @@
#! vim noexpandtab
# Copyright (C) 2016 Platform 9 Systems, Inc.
TOP_DIR := $(abspath ../)
SRC_DIR := $(TOP_DIR)
BUILD_DIR := $(TOP_DIR)/build
NPM := npm
APP_NAME :=pf9-mors
APP_DESC :="Platform9 mors (lease manager)"
APP_BUILD_DIR := $(BUILD_DIR)
PF9_VERSION ?=2.0.0
BUILD_NUMBER ?= 0
GIT_HASH := $(shell git rev-parse --short HEAD)
FULL_VERSION := $(PF9_VERSION)-$(BUILD_NUMBER)
APP_DESC :="Platform9 mors(lease manager) git hash $(GIT_HASH)"
APP_RPM_DIR := $(APP_BUILD_DIR)/rpmbuild
APP_RPM_STAGE_DIR := $(APP_BUILD_DIR)/stage
APP_RPM_VENV := $(APP_RPM_STAGE_DIR)/opt/pf9/$(APP_NAME)
APP_ARCHITECTURE := noarch
APP_RPM := $(APP_RPM_DIR)/$(APP_NAME)-$(FULL_VERSION).noarch.rpm
APP_SPEC_FILE := $(APP_BUILD_DIR)/$(APP_NAME)-rpm.spec
############################################################
${APP_RPM_DIR}:
mkdir -p $@
${APP_RPM_STAGE_DIR}:
mkdir -p $@
${APP_RPM_VENV}:
mkdir -p $@
virtualenv $@
$@/bin/pip install ${SRC_DIR}
stage: $(APP_RPM_DIR) $(APP_RPM_STAGE_DIR) $(APP_RPM_VENV)
cp -r $(SRC_DIR)/etc/ $(APP_RPM_STAGE_DIR)/
cp $(SRC_DIR)/mors_repo/migrate.cfg $(APP_RPM_VENV)/lib/python2.7/site-packages/mors_repo/
${APP_RPM}: stage
echo "RPM build "
fpm -t rpm \
-s dir \
-n $(APP_NAME) \
--description $(APP_DESC) \
--version $(PF9_VERSION) \
--iteration $(BUILD_NUMBER) \
--provides $(APP_NAME) \
--license "Commercial" \
--architecture $(APP_ARCHITECTURE) \
--url "http://www.platform9.net" \
--vendor Platform9 \
-p $@ \
-C $(APP_RPM_STAGE_DIR) . && \
$(SRC_DIR)/support/sign_packages.sh ${APP_RPM}
clean:
rm -rf $(BUILD_DIR)
all: clean $(APP_RPM)

View File

@ -1,9 +0,0 @@
#!/usr/bin/expect
set timeout 15
spawn bash -c "rpm --resign $argv"
match_max 100000
expect -exact "Enter pass phrase: "
send -- "\r"
expect "Pass phrase is good."
expect eof

View File

@ -1,5 +0,0 @@
#!/bin/sh
if [ "x${SIGN_PACKAGES}" = "x1" ]; then
expect $(dirname $0)/mors.expect $@
fi

19
test.sh
View File

@ -1,19 +0,0 @@
#!/usr/bin/env bash
# Copyright (c) Platform9 systems. All rights reserved
output_dir=./build
log_filter=-paramiko.transport
setup_venv() {
virtualenv ${output_dir}/venv
source ${output_dir}/venv/bin/activate
pip install -e .
}
run_tests() {
python ./test/run_tests.py --verbose --with-xunit --xunit-file=${output_dir}/test_output.xml \
--logging-clear-handlers ${exclude} ${nocapture} --logging-filter=${log_filter} ${module} \
--logging-format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s'
}
setup_venv
run_tests

View File

@ -1,14 +0,0 @@
[app:myService]
paste.app_factory = mors.mors_wsgi:app_factory
[pipeline:main]
pipeline = myService
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
admin_token = RzvUwrEgiOaQFTXV
auth_uri = http://127.0.0.1:8080/keystone
identity_uri = http://127.0.0.1:8080/keystone_admin

View File

@ -1,16 +0,0 @@
[DEFAULT]
db_conn=sqlite+pysqlite:///test/test.db
context_factory=test
lease_handler=test
listen_port=8989
sleep_seconds=3
paste-ini=test/api-paste.ini
log_file=build/test.log
[nova]
user_name=rparikh@platform9.net
password=asdsdsadf
version=2
auth_url=https://pf9.platform9.net/v2/keystone
region_name=RegionOne

View File

@ -1,26 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
def run_tests():
from proboscis import TestProgram
import test_api, test_persistence
# Run Proboscis and exit.
TestProgram().run_and_exit()
if __name__ == '__main__':
print "Run tests"
run_tests()

View File

@ -1,276 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
import ConfigParser
import os
import sys;
import eventlet
import requests
from migrate.versioning.api import upgrade, version_control
print sys.path
from mors.pf9_mors import start_server
from mors.mors_wsgi import DATE_FORMAT
import logging, sys
from datetime import datetime, timedelta
from proboscis.asserts import assert_equal
from proboscis import test
import shutil
from mors.leasehandler.fake_lease_handler import FakeLeaseHandler
try:
import http.client as http_client
except ImportError:
# Python 2
import httplib as http_client
http_client.HTTPConnection.debuglevel = 1
root = logging.getLogger()
root.setLevel(logging.DEBUG)
ch = logging.StreamHandler(sys.stdout)
ch.setLevel(logging.DEBUG)
root.addHandler(ch)
logger = logging.getLogger(__name__)
eventlet.monkey_patch()
conf = None
headers = {
'X-User-Id': 'asdfsd-asdf-sdadf',
'X-User': 'roopak@pf9.com',
'X-roles': 'admin,_member_',
'X-Tenant-Id': 'poioio-oio-oioo'
}
tenant_id1 = "tenantid-1"
tenant_id2 = "tenantid-2"
instance_id1 = "instanceid-1-t-1"
instance_id2 = "instanceid-2-t-1"
instance_id3 = "instanceid-3-t-2"
expiry_mins1 = 4
port = 8080
def _setup_lease_handler():
fakeLeaseHandler = FakeLeaseHandler(conf)
now = datetime.now()
dt = timedelta(days=3)
creation_time = now - dt
t1_vms = [{'instance_uuid': 'instance-123-t1', 'tenant_uuid': tenant_id1, 'created_at': creation_time},
{'instance_uuid': 'instance-456-t1', 'tenant_uuid': tenant_id1, 'created_at': now},
{'instance_uuid': instance_id1, 'tenant_uuid': tenant_id1, 'created_at': now},
{'instance_uuid': instance_id2, 'tenant_uuid': tenant_id1, 'created_at': now}]
fakeLeaseHandler.add_tenant_data(tenant_id1, t1_vms)
t2_vms = [{'instance_uuid': 'instance-123-t2', 'tenant_uuid': tenant_id2, 'created_at': creation_time},
{'instance_uuid': 'instance-456-t2', 'tenant_uuid': tenant_id2, 'created_at': now},
{'instance_uuid': instance_id3, 'tenant_uuid': tenant_id2, 'created_at': now}]
fakeLeaseHandler.add_tenant_data(tenant_id2, t2_vms)
@test
def initialize():
global conf
global port
if os.path.exists("./sqlite+pysqlite:"):
shutil.rmtree("./sqlite+pysqlite:")
if os.path.exists("./test/test.db"):
os.remove("./test/test.db")
conf = ConfigParser.ConfigParser()
conf.readfp(open("test/pf9-mors.ini"))
#create(conf.get("DEFAULT", "db_conn"), "./mors_repo")
version_control(conf.get("DEFAULT", "db_conn"), "./mors_repo")
upgrade(conf.get("DEFAULT", "db_conn"), "./mors_repo")
port = conf.get("DEFAULT", "listen_port")
_setup_lease_handler()
api_paste_file = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'api-paste.ini')
eventlet.greenthread.spawn(start_server, conf, api_paste_file)
eventlet.greenthread.sleep(5)
@test(depends_on=[initialize])
def test_create_tenant():
r = requests.post('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1,
json={"vm_lease_policy": {"tenant_uuid": tenant_id1, "expiry_mins": expiry_mins1}},
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_create_tenant])
def test_update_tenant():
r = requests.put('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1,
json={"vm_lease_policy": {"tenant_uuid": tenant_id1, "expiry_mins": 3}}, headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_update_tenant])
def test_get_all_tenants():
r = requests.get('http://127.0.0.1:' + port + '/v1/tenant/',
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_get_all_tenants])
def test_get_tenant():
r = requests.get('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1, headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_get_tenant])
def test_create_tenant_neg():
# Try creating again and it should result in error
r = requests.post('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1,
json={"vm_lease_policy": {"tenant_uuid": tenant_id1, "expiry_mins": expiry_mins1}},
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 409)
@test(depends_on=[test_create_tenant_neg])
def test_create_instance():
# Now test the instance manipulation
expiry = datetime.utcnow()
expiry_str = datetime.strftime(expiry, DATE_FORMAT)
r = requests.post('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1 + '/instance/' + instance_id1,
json={"instance_uuid": instance_id1, "expiry": expiry_str},
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_create_instance])
def test_get_instance():
r = requests.get('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1 + '/instance/' + instance_id1,
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_get_instance])
def test_update_instance():
expiry = datetime.utcnow()
expiry_str = datetime.strftime(expiry, DATE_FORMAT)
r = requests.put('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1 + '/instance/' + instance_id1,
json={"instance_uuid": instance_id1, "expiry": expiry_str},
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_update_instance])
def test_update_instance_violation():
expiry = datetime.utcnow() + timedelta(days=4)
expiry_str = datetime.strftime(expiry, DATE_FORMAT)
r = requests.put('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1 + '/instance/' + instance_id1,
json={"instance_uuid": instance_id1, "expiry": expiry_str},
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 422)
@test(depends_on=[test_update_instance])
def test_get_instance2():
r = requests.get('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1 + '/instance/' + instance_id1,
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_get_instance2])
def test_deleted_instance():
eventlet.greenthread.sleep(50)
# The instance lease should be deleted by now
r = requests.get('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1 + '/instance/' + instance_id1,
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 404)
@test(depends_on=[test_deleted_instance])
def test_create_instance2():
# Now test the instance manipulation
expiry = datetime.utcnow()
expiry_str = datetime.strftime(expiry, DATE_FORMAT)
r = requests.post('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1 + '/instance/' + instance_id2,
json={"instance_uuid": instance_id2, "expiry": expiry_str},
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_create_instance2])
def test_delete_instance_lease():
r = requests.delete('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id1 + '/instance/' + instance_id2,
json={"tenant_uuid": tenant_id1, "instance_uuid": instance_id2},
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_deleted_instance])
def test_create_tenant2():
r = requests.post('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id2,
json={"vm_lease_policy": {"tenant_uuid": tenant_id2, "expiry_mins": expiry_mins1}},
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_create_tenant2])
def test_create_instance3():
# Now test the instance manipulation
expiry = datetime.utcnow()
expiry_str = datetime.strftime(expiry, DATE_FORMAT)
r = requests.post('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id2 + '/instance/' + instance_id3,
json={"tenant_uuid": tenant_id2, "instance_uuid": instance_id3, "expiry": expiry_str},
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_create_instance3])
def test_delete_tenant2():
r = requests.delete('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id2,
json={"tenant_uuid": tenant_id2}, headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)
@test(depends_on=[test_delete_tenant2])
def test_get_tenant2():
r = requests.get('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id2, headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 404)
@test(depends_on=[test_delete_tenant2])
def test_get_instance3():
r = requests.get('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id2 + '/instance/' + instance_id3,
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 404)
@test(depends_on=[test_get_instance3])
def test_get_all_instances_for_tenant2():
r = requests.get('http://127.0.0.1:' + port + '/v1/tenant/' + tenant_id2 + '/instances/',
headers=headers)
logger.debug(r.text)
assert_equal(r.status_code, 200)

View File

@ -1,132 +0,0 @@
"""
Copyright 2016 Platform9 Systems Inc.(http://www.platform9.com)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
from proboscis import test
from mors.persistence import DbPersistence
import uuid
import logging
from migrate.versioning.api import upgrade,create,version_control
from datetime import datetime, timedelta
import os
import shutil
logger = logging.getLogger(__name__)
db_persistence = None
TEST_DB="test/test_db11"
@test
def setup_module():
if os.path.exists(TEST_DB):
os.remove(TEST_DB)
global db_persistence
DB_URL = "sqlite:///"+TEST_DB
if os.path.exists(DB_URL):
shutil.rmtree(DB_URL)
create(DB_URL, "./mors_repo")
version_control(DB_URL, "./mors_repo")
upgrade(DB_URL,"./mors_repo")
db_persistence = DbPersistence(DB_URL)
return db_persistence
def teardown_module():
os.unlink(TEST_DB)
def _verify_tenant_lease(tenants):
for tenant in tenants:
t_lease = db_persistence.get_tenant_lease(tenant)
assert (t_lease.tenant_uuid == tenant)
assert (t_lease.created_by == tenants[tenant]["user"])
assert (t_lease.created_at == tenants[tenant]["tenant_created_date"])
if "updated_by" in tenants[tenant].keys():
assert (t_lease.updated_by == tenants[tenant]["updated_by"])
if "updated_at" in tenants[tenant].keys():
assert (t_lease.updated_at == tenants[tenant]["updated_at"])
def _verify_instance_lease(instances):
for instance_uuid in instances:
instance = instances[instance_uuid]
i_lease = db_persistence.get_instance_lease(instance['instance_uuid'])
assert (i_lease.instance_uuid == instance['instance_uuid'])
assert (i_lease.tenant_uuid == instance['tenant_uuid'])
assert (i_lease.expiry == instance['expiry'])
assert (i_lease.created_by == instance["created_by"])
if "updated_by" in instance.keys():
assert (i_lease.updated_by == instance["updated_by"])
if "updated_at" in instance.keys():
assert (i_lease.updated_at == instance["updated_at"])
@test(depends_on=[setup_module])
def test_apis():
tenants = {"tenant-1": { "user": "a@xyz.com",
"expiry_mins": 3,
"tenant_created_date": datetime.utcnow()},
"tenant-2": { "user": "c@xyz.com",
"expiry_mins": 1,
"tenant_created_date": datetime.utcnow()}}
for tenant in tenants:
db_persistence.add_tenant_lease(tenant, tenants[tenant]["expiry_mins"],
tenants[tenant]["user"],
tenants[tenant]["tenant_created_date"])
_verify_tenant_lease(tenants)
# Now try update
tenants["tenant-1"]["updated_by"] = "b@xyz.com"
tenants["tenant-2"]["updated_by"] = "d@xyz.com"
tenant_updated_date = datetime.utcnow()
for tenant in tenants:
tenants[tenant]["updated_at"] = tenant_updated_date
tenant_values = tenants[tenant]
db_persistence.update_tenant_lease(tenant, tenant_values["expiry_mins"],
tenant_values["updated_by"],
tenant_values["updated_at"])
_verify_tenant_lease(tenants)
now = datetime.utcnow()
# Instance lease now
instances = {"instance-1": {"instance_uuid": "instance-1",
"tenant_uuid": "tenant-1",
"expiry": now,
"created_at": now,
"created_by": "d@xyz.com"},
"instance-2": {"instance_uuid": "instance-2",
"tenant_uuid": "tenant-2",
"expiry": now + timedelta(seconds=60),
"created_at": now + timedelta(seconds=60),
"created_by": "e@xyz.com"}}
tenant_id = tenants.keys()[0]
now = datetime.utcnow()
for instance_uuid in instances:
instance = instances[instance_uuid]
db_persistence.add_instance_lease(instance['instance_uuid'], instance['tenant_uuid'],
instance['expiry'], instance["created_by"],
instance["created_at"])
_verify_instance_lease(instances)
newtime1 = datetime.utcnow() + timedelta(seconds=240)
newtime2 = datetime.utcnow() + timedelta(seconds=120)
instances['instance-1']['expiry'] = newtime1
instances['instance-1']['updated_at'] = newtime1
instances['instance-2']['expiry'] = newtime2
instances['instance-2']['updated_at'] = newtime2
for instance_uuid in instances:
instance = instances[instance_uuid]
db_persistence.update_instance_lease(instance['instance_uuid'], instance['tenant_uuid'],
instance['expiry'], instance["created_by"],
instance["updated_at"])
_verify_instance_lease(instances)

View File

@ -1,9 +0,0 @@
[tox]
envlist = py27
[testenv:py27]
deps =
nose
proboscis
commands =
python test/run_tests.py