Merge pull request #11 from denismakogon/project-picasso

New project name: Picasso
This commit is contained in:
Derek Schultz 2016-12-07 11:22:46 -07:00 committed by GitHub
commit a5d0bb734b
44 changed files with 90 additions and 105 deletions

View File

@ -11,5 +11,5 @@ RUN pip3 install -r /code/requirements.txt
RUN python3 /code/setup.py install
ENTRYPOINT ["python3", "/code/service/laos_api.py"]
ENTRYPOINT ["python3", "/code/service/picasso_api.py"]
EXPOSE 10001

View File

@ -1,6 +1,6 @@
LAOS_HOST=0.0.0.0
LAOS_PORT=10001
LAOS_DB=mysql+pymysql://root:ubuntu@192.168.0.120/functions
PICASSO_HOST=0.0.0.0
PICASSO_PORT=10001
PICASSO_DB=mysql+pymysql://root:ubuntu@192.168.0.120/functions
KEYSTONE_ENDPOINT=http://192.168.0.120:5000/v3
FUNCTIONS_URL=http://192.168.0.120:8080/v1
LAOS_LOG_LEVEL=INFO
PICASSO_LOG_LEVEL=INFO

View File

@ -1,10 +1,10 @@
Project LaOS aka Lambdas-on-OpenStack
=====================================
Picasso: Functions-as-a-Service (FaaS) on OpenStack
===================================================
Mission
-------
Provide capabilities to run software in "serverless" way.
Picasso provides an API abstraction layer for Functions-as-a-Service (FaaS) on OpenStack.
Serverless
----------
@ -50,7 +50,7 @@ Quick-start guide
-----------------
Install DevStack with [IronFunctions enabled](https://github.com/iron-io/functions-devstack-plugin/blob/master/README.rst).
Pull down [Project LaOS sources](https://github.com/iron-io/project-laos).
Pull down [Picasso sources](https://github.com/iron-io/project-picasso).
Create Python3.5 virtualenv:
@ -61,7 +61,7 @@ Install dependencies:
$ pip install -r requirements.txt -r test-requirements.txt
Install LaOS itself:
Install Picasso itself:
$ pip install -e .
@ -76,7 +76,7 @@ Migrations
Once all dependencies are installed it is necessary to run database migrations.
Before that it is necessary to set env variable:
export LAOS_MIGRATIONS_DB=mysql+pymysql://root:root@localhost/functions
export PICASSO_MIGRATIONS_DB=mysql+pymysql://root:root@localhost/functions
In this section please specify connection URI to your own MySQL database.
Once the file is saved, just use alembic to apply the migrations:
@ -86,35 +86,35 @@ Once the file is saved, just use alembic to apply the migrations:
Starting a server
-----------------
Once it is finished you will have a console script `laos-api`:
Once it is finished you will have a console script `picasso-api`:
$ laos-api --help
$ picasso-api --help
Usage: laos-api [OPTIONS]
Usage: picasso-api [OPTIONS]
Starts an Project Laos API service
Starts Picasso API service
Options:
--host TEXT API service bind host.
--port INTEGER API service bind port.
--db-uri TEXT LaOS persistence storage URI.
--db-uri TEXT Picasso persistence storage URI.
--keystone-endpoint TEXT OpenStack Identity service endpoint.
--functions-url TEXT IronFunctions API URL
--log-level TEXT Logging file
--log-file TEXT Log file path
--help Show this message and exit.
Minimum required options to start LaOS API service:
Minimum required options to start Picasso API service:
--db-uri mysql://root:root@192.168.0.112/functions
--keystone-endpoint http://192.168.0.112:5000/v3
--functions-url http://192.168.0.112:8080/v1
--log-level INFO
Creating and running LaOS inside Docker container
Creating and running Picasso inside Docker container
-------------------------------------------------
As part of regular Python distribution, LaOS also has its own Docker container to run.
As part of regular Python distribution, Picasso also has its own Docker container to run.
There are two options:
* run from sources
@ -123,13 +123,13 @@ There are two options:
In order to build container from sources run following commands:
export DOCKER_HOST=tcp://<docker-host>:<docker-port>
docker build -t laos-api -f Dockerfile .
docker build -t picasso-api -f Dockerfile .
After that it is required to create correct version of [Dockerfile.env](Dockerfile.env.example).
It container all required options to start LaOS API service properly.
It container all required options to start Picasso API service properly.
Once it is done run following commands:
docker run -d -p 10001:10001 --env-file Dockerfile.env laos-api
docker run -d -p 10001:10001 --env-file Dockerfile.env picasso-api
Navigate to your web browser to check if service is running:
@ -144,7 +144,7 @@ Examining API
In [examples](examples/) folder you can find a script that examines available API endpoints, but this script relays on:
* `LAOS_API_URL` - Project LaOS API endpoint
* `PICASSO_API_URL` - Picasso API endpoint
* `OS_AUTH_URL` - OpenStack Auth URL
* `OS_PROJECT_ID` - it can be found in OpenStack Dashboard or in CLI
* `OS_USERNAME` - OpenStack project-aligned username
@ -161,10 +161,10 @@ Please note, that given values are project-specific, so they can't be reused.
API docs
--------
As part of LaOS ReST API it is possible to discover API doc using Swagger Doc.
As part of Picasso ReST API it is possible to discover API doc using Swagger Doc.
Once server is launched you can navigate to:
http://<laos-host>:<laos-port>/api
http://<picasso-host>:<picasso-port>/api
to see recent API docs

View File

@ -23,7 +23,7 @@ target_metadata = None
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.
config.set_main_option("sqlalchemy.url", os.getenv("LAOS_MIGRATIONS_DB"))
config.set_main_option("sqlalchemy.url", os.getenv("PICASSO_MIGRATIONS_DB"))
def run_migrations_offline():

View File

@ -72,12 +72,10 @@ class AppRouteV1Controller(controller.ServiceController):
path=fn_route.path)).pop()
setattr(fn_route, "is_public", stored_route.public)
api_url = "{}://{}".format(request.scheme, request.host)
log.info("Listing app {} routes for project: {}."
.format(app, project_id))
return web.json_response(data={
"routes": app_view.AppRouteView(api_url,
project_id,
"routes": app_view.AppRouteView(project_id,
app,
fn_app_routes).view(),
"message": "Successfully loaded app routes",
@ -175,11 +173,10 @@ class AppRouteV1Controller(controller.ServiceController):
log.info("Creating new route in app {} "
"for project: {} with data {}"
.format(app, project_id, str(data)))
api_url = "{}://{}".format(request.scheme, request.host)
setattr(new_fn_route, "is_public", stored_route.public)
view = app_view.AppRouteView(
api_url, project_id, app, [new_fn_route]).view_one()
project_id, app, [new_fn_route]).view_one()
return web.json_response(data={
"route": view,
@ -233,8 +230,6 @@ class AppRouteV1Controller(controller.ServiceController):
log.info("Requesting route {} in app {} for project: {}"
.format(path, app, project_id))
api_url = "{}://{}".format(request.scheme, request.host)
stored_route = (await app_model.Routes.find_by(
app_name=app,
project_id=project_id,
@ -243,8 +238,7 @@ class AppRouteV1Controller(controller.ServiceController):
setattr(route, "is_public", stored_route.public)
return web.json_response(data={
"route": app_view.AppRouteView(api_url,
project_id,
"route": app_view.AppRouteView(project_id,
app,
[route]).view_one(),
"message": "App route successfully loaded"
@ -297,8 +291,6 @@ class AppRouteV1Controller(controller.ServiceController):
}
}, status=getattr(ex, "status", 500))
api_url = "{}://{}".format(request.scheme, request.host)
stored_route = (await app_model.Routes.find_by(
app_name=app,
project_id=project_id,
@ -307,8 +299,7 @@ class AppRouteV1Controller(controller.ServiceController):
setattr(route, "is_public", stored_route.public)
return web.json_response(data={
"route": app_view.AppRouteView(api_url,
project_id,
"route": app_view.AppRouteView(project_id,
app,
[route]).view_one(),

View File

@ -33,9 +33,8 @@ class AppView(object):
class AppRouteView(object):
def __init__(self, api_url, project_id, app_name, fn_app_routes):
def __init__(self, project_id, app_name, fn_app_routes):
self.routes = fn_app_routes
self.api_url = api_url
self.project_id = project_id
self.app_name = app_name
@ -45,15 +44,8 @@ class AppRouteView(object):
def view(self):
view = []
for route in self.routes:
if not route.is_public:
path = ("{}/v1/r/{}/{}{}".format(
self.api_url, self.project_id,
self.app_name, route.path))
else:
path = ("{}/r/{}{}".format(
self.api_url, self.app_name, route.path))
one = {
"path": path,
"path": route.path,
"type": route.type,
"image": route.image,
"is_public": route.is_public,
@ -65,6 +57,7 @@ class AppRouteView(object):
if hasattr(route, "timeout"):
one.update(timeout=route.timeout)
if hasattr(route, "max_concurrency"):
one.update(timeout=route.max_concurrency)
one.update(max_concurrency=
route.max_concurrency)
view.append(one)
return view

View File

@ -21,7 +21,7 @@ from . import utils
def common_logger_setup(
level=logging.DEBUG,
filename='/tmp/laos-api.log',
filename='/tmp/picasso-api.log',
log_formatter='[%(asctime)s] - '
'%(name)s - '
'%(levelname)s - '
@ -40,7 +40,7 @@ def common_logger_setup(
def setup_logging(name,
filename='/tmp/laos-api-{}.log'.format(
filename='/tmp/picasso-api-{}.log'.format(
datetime.datetime.now()),
level=logging.DEBUG,
log_to_console=False,
@ -68,7 +68,7 @@ class Singleton(type):
class UnifiedLogger(object, metaclass=utils.Singleton):
def __init__(self,
filename='/tmp/laos-api-{}.log'.format(
filename='/tmp/picasso-api-{}.log'.format(
datetime.datetime.now()),
level=logging.DEBUG, log_to_console=False):
self.filename = filename

View File

@ -18,12 +18,13 @@ from urllib import parse
def split_db_uri(db_uri):
"""
Splits DB URI into consumable parts like:
- username
- password
- hostname
- port
- protocol schema
- path
- username
- password
- hostname
- port
- protocol schema
- path
:param db_uri:
:return:
"""

View File

@ -19,7 +19,7 @@ import uvloop
from ...common import logger as log
class LaosTestsBase(object):
class PicassoTestsBase(object):
def get_loop_and_logger(self, test_type):
self.route_data = {
@ -36,7 +36,7 @@ class LaosTestsBase(object):
logger = log.UnifiedLogger(
log_to_console=False,
filename=("/tmp/laos-{}-tests-run-{}.log"
filename=("/tmp/picasso-{}-tests-run-{}.log"
.format(test_type, datetime.datetime.now())),
level="DEBUG").setup_logger(__package__)
return testloop, logger

View File

@ -93,10 +93,10 @@ class RoutesV1(object):
ignore_project_id=True, **data)
class ProjectBoundLaosTestClient(test_utils.TestClient):
class ProjectBoundTestClient(test_utils.TestClient):
def __init__(self, app_or_server, project_id, **kwargs):
super(ProjectBoundLaosTestClient, self).__init__(app_or_server)
super(ProjectBoundTestClient, self).__init__(app_or_server)
self.project_id = project_id
self.headers = {
"Content-Type": "application/json"

View File

@ -31,7 +31,7 @@ from ..common import client
from ..fakes import functions_api
class LaosFunctionalTestsBase(base.LaosTestsBase, testtools.TestCase):
class FunctionalTestsBase(base.PicassoTestsBase, testtools.TestCase):
def setUp(self):
self.testloop, logger = self.get_loop_and_logger("functional")
@ -74,12 +74,12 @@ class LaosFunctionalTestsBase(base.LaosTestsBase, testtools.TestCase):
)
self.project_id = str(uuid.uuid4()).replace("-", "")
self.test_client = client.ProjectBoundLaosTestClient(
self.test_client = client.ProjectBoundTestClient(
self.testapp, self.project_id)
self.testloop.run_until_complete(self.test_client.start_server())
super(LaosFunctionalTestsBase, self).setUp()
super(FunctionalTestsBase, self).setUp()
def tearDown(self):
self.testloop.run_until_complete(self.test_client.close())
super(LaosFunctionalTestsBase, self).tearDown()
super(FunctionalTestsBase, self).tearDown()

View File

@ -16,7 +16,7 @@ from ..common import apps as apps_suite
from ..functional import base
class TestApps(base.LaosFunctionalTestsBase, apps_suite.AppsTestSuite):
class TestApps(base.FunctionalTestsBase, apps_suite.AppsTestSuite):
def test_list_apps(self):
super(TestApps, self).list_apps()

View File

@ -16,7 +16,7 @@ from ..common import routes as routes_suite
from ..functional import base
class TestAppRoutes(base.LaosFunctionalTestsBase,
class TestAppRoutes(base.FunctionalTestsBase,
routes_suite.AppRoutesTestSuite):
def test_list_routes_from_unknown_app(self):

View File

@ -24,10 +24,10 @@ from ..common import base
from ..common import client
from service import laos_api
from service import picasso_api
class LaosIntegrationTestsBase(base.LaosTestsBase, testtools.TestCase):
class PicassoIntegrationTestsBase(base.PicassoTestsBase, testtools.TestCase):
async def authenticate(self, os_user, os_pass, os_project, url):
auth_request_data = {
@ -99,7 +99,7 @@ class LaosIntegrationTestsBase(base.LaosTestsBase, testtools.TestCase):
event_loop=self.testloop,
)
self.test_app = laos_api.API(
self.test_app = picasso_api.API(
loop=self.testloop, logger=logger, debug=True)
os_token, project_id = self.testloop.run_until_complete(
@ -107,14 +107,14 @@ class LaosIntegrationTestsBase(base.LaosTestsBase, testtools.TestCase):
os_user, os_pass,
os_project, os_auth_url))
self.test_client = client.ProjectBoundLaosTestClient(
self.test_client = client.ProjectBoundTestClient(
self.test_app.root, project_id, headers={
"X-Auth-Token": os_token
})
self.testloop.run_until_complete(
self.test_client.start_server())
super(LaosIntegrationTestsBase, self).setUp()
super(PicassoIntegrationTestsBase, self).setUp()
def tearDown(self):
self.testloop.run_until_complete(self.test_client.close())
super(LaosIntegrationTestsBase, self).tearDown()
super(PicassoIntegrationTestsBase, self).tearDown()

View File

@ -16,7 +16,7 @@ from ..common import apps as apps_suite
from ..integration import base
class TestIntegrationApps(base.LaosIntegrationTestsBase,
class TestIntegrationApps(base.PicassoIntegrationTestsBase,
apps_suite.AppsTestSuite):
def test_list_apps(self):

View File

@ -16,7 +16,7 @@ from ..common import routes as routes_suite
from ..functional import base
class TestIntegrationAppRoutes(base.LaosFunctionalTestsBase,
class TestIntegrationAppRoutes(base.FunctionalTestsBase,
routes_suite.AppRoutesTestSuite):
def test_list_routes_from_unknown_app(self):

View File

@ -4,13 +4,13 @@ set +x
set +e
set +i
docker build -t laos-api -f Dockerfile . 2>&1
docker build -t picasso-api -f Dockerfile . 2>&1
docker_h=$(echo ${DOCKER_HOST} | tr "://" " " |awk '{print $2}')
echo -e "Docker IP address ${docker_h}"
echo -e "OpenStack Identity service URL ${OS_AUTH_URL}"
echo -e "IronFunctions URL ${FUNCTIONS_API_URL}"
echo -e "Persistent storage URI ${TEST_DB_URI}"
docker run -d -p ${docker_h}:10002:10001 --env LAOS_HOST=0.0.0.0 --env LAOS_PORT=10001 --env LAOS_DB=${TEST_DB_URI} --env KEYSTONE_ENDPOINT=${OS_AUTH_URL} --env FUNCTIONS_URL=${FUNCTIONS_API_URL} --env LAOS_LOG_LEVEL=INFO laos-api
docker run -d -p ${docker_h}:10002:10001 --env PICASSO_HOST=0.0.0.0 --env PICASSO_PORT=10001 --env PICASSO_DB=${TEST_DB_URI} --env KEYSTONE_ENDPOINT=${OS_AUTH_URL} --env FUNCTIONS_URL=${FUNCTIONS_API_URL} --env PICASSO_LOG_LEVEL=INFO picasso-api
sleep 2
docker ps
echo -e "Service running on ${docker_h}:10002"

View File

@ -4,7 +4,7 @@ set +x
set +e
function get_current_coverage {
local prev_stat_raw=`pytest --tb=long --capture=sys --cov=laos --capture=fd laos/tests/${1:-functional} | grep TOTAL | awk '{print $4}'`
local prev_stat_raw=`pytest --tb=long --capture=sys --cov=picasso --capture=fd picasso/tests/${1:-functional} | grep TOTAL | awk '{print $4}'`
echo ${prev_stat_raw:0:2}
}

View File

@ -21,16 +21,16 @@ import uvloop
from aioservice.http import service
from laos.api.controllers import apps
from laos.api.controllers import routes
from laos.api.controllers import runnable
from laos.api.controllers import tasks
from picasso.api.controllers import apps
from picasso.api.controllers import routes
from picasso.api.controllers import runnable
from picasso.api.controllers import tasks
from laos.api.middleware import content_type
from laos.api.middleware import keystone
from picasso.api.middleware import content_type
from picasso.api.middleware import keystone
from laos.common import config
from laos.common import logger as log
from picasso.common import config
from picasso.common import logger as log
from urllib import parse
@ -74,17 +74,17 @@ class API(service.HTTPService):
)
@click.command(name='laos-api')
@click.command(name='picasso-api')
@click.option('--host',
default=os.getenv("LAOS_HOST", '0.0.0.0'),
default=os.getenv("PICASSO_HOST", '0.0.0.0'),
help='API service host.')
@click.option('--port', default=int(os.getenv("LAOS_PORT", 10001)),
@click.option('--port', default=int(os.getenv("PICASSO_PORT", 10001)),
help='API service port.')
@click.option('--db-uri',
default=os.getenv(
"LAOS_DB",
"PICASSO_DB",
'mysql://root:root@localhost/functions'),
help='LaOS persistence storage URI.')
help='Picasso persistence storage URI.')
@click.option('--keystone-endpoint',
default=os.getenv("KEYSTONE_ENDPOINT",
'http://localhost:5000/v3'),
@ -94,9 +94,9 @@ class API(service.HTTPService):
"FUNCTIONS_URL", 'http://localhost:8080/v1'),
help='Functions API host')
@click.option('--log-level',
default=os.getenv("LAOS_LOG_LEVEL", 'INFO'),
default=os.getenv("PICASSO_LOG_LEVEL", 'INFO'),
help='Logging file')
@click.option('--log-file', default=os.getenv("LAOS_LOG_FILE"),
@click.option('--log-file', default=os.getenv("PICASSO_LOG_FILE"),
help='Log file path')
@click.option('--debug', default=False, is_flag=True)
def server(host, port, db_uri,
@ -107,7 +107,7 @@ def server(host, port, db_uri,
debug,
):
"""
Starts an Project Laos API service
Starts Picasso API service
"""
logger = log.UnifiedLogger(
log_to_console=True if not log_file else False,
@ -141,9 +141,9 @@ def server(host, port, db_uri,
logger=logger, debug=debug
).apply_swagger(
swagger_url="/api",
description="Laos API service docs",
description="Picasso API service docs",
api_version="v1.0.0",
title="Laos API",
title="Picasso API",
).initialize()

View File

@ -20,11 +20,12 @@ def read(fname):
return open(os.path.join(os.path.dirname(__file__), fname)).read()
setuptools.setup(
name='laos',
name='picasso',
version='0.0.1',
description='Project LaOS (Lambdas-on-OpenStack',
description=('Picasso provides abstractions for '
'Functions-as-a-Service on OpenStack'),
long_description=read('README.md'),
url='laos.readthedocs.org',
url='picasso.readthedocs.org',
author='Denis Makogon',
author_email='denis@iron.io',
packages=setuptools.find_packages(),
@ -68,8 +69,7 @@ setuptools.setup(
zip_safe=True,
entry_points={
'console_scripts': [
'laos-api = laos.service.laos_api:server',
'picasso-api = service.picasso_api:server',
]
},
)

View File

@ -32,10 +32,10 @@ commands = flake8
commands = {posargs}
[testenv:py35-integration]
commands = pytest --tb=long --capture=sys --cov=laos --capture=fd {toxinidir}/laos/tests/integration
commands = pytest --tb=long --capture=sys --cov=picasso --capture=fd {toxinidir}/picasso/tests/integration
[testenv:py35-functional]
commands = pytest --tb=long --capture=sys --cov=laos --capture=fd {toxinidir}/laos/tests/functional
commands = pytest --tb=long --capture=sys --cov=picasso --capture=fd {toxinidir}/picasso/tests/functional
[testenv:py35-functional-regression]
commands = {toxinidir}/scripts/test_regression.sh functional {posargs}