x
/
picasso
Code Issues Proposed changes

Merge pull request #10 from denismakogon/update-to-latest-iron-functions-api 

Massive update
This commit is contained in:
Derek Schultz 2016-12-06 10:16:11 -07:00 committed by GitHub
parent fee06d71a5 a6a8405510
commit fdc9fd3935
31 changed files with 528 additions and 219 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Dockerfile
View File

@ -11,5 +11,5 @@ RUN pip3 install -r /code/requirements.txt
RUN python3 /code/setup.py install
ENTRYPOINT ["python3", "/code/laos/service/laos_api.py"]
ENTRYPOINT ["python3", "/code/service/laos_api.py"]
EXPOSE 10001

2
Dockerfile.env.example
View File

@ -1,6 +1,6 @@
LAOS_HOST=0.0.0.0
LAOS_PORT=10001
LAOS_DB=mysql://root:ubuntu@192.168.0.120/functions
LAOS_DB=mysql+pymysql://root:ubuntu@192.168.0.120/functions
KEYSTONE_ENDPOINT=http://192.168.0.120:5000/v3
FUNCTIONS_URL=http://192.168.0.120:8080/v1
LAOS_LOG_LEVEL=INFO

105
README.md
View File

@ -168,111 +168,6 @@ Once server is launched you can navigate to:
to see recent API docs
Testing (general information)
-----------------------------
In order to run tests you need to install `Tox`:
$ pip install tox
Also, you will need a running MySQL instance with the database migrations applied from the previous step.
Tests are dependent on pre-created MySQL database for persistence.
Please set env var
$ export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
Testing: PEP8
-------------
In order to run `PEP8` style checks run following command:
$ tox -e pep8
Testing: Functional
-------------------
In order to run `functional` tests run following command:
$ tox -e py35-functional
Pros:
* lightweight (controllers and DB models testing)
* no OpenStack required
* no IronFunctions required
Cons:
* MySQL server required
* OpenStack authentication is not tested
* IronFunctions API stubbed with fake implementation
Testing: Integration
--------------------
Integration tests are dependent on following env variables:
* TEST_DB_URI - similar to functional tests, database endpoint
* FUNCTIONS_API_URL - IronFunctions API URL (default value - `http://localhost:8080/v1`)
* OS_AUTH_URL - OpenStack Identity endpoint
* OS_PROJECT_NAME - OpenStack user-specific project name
* OS_USERNAME - OpenStack user name
* OS_PASSWORD - OpenStack user user password
To run tests use following command:
export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
export FUNCTIONS_API_URL=<functions-api-protocol>://<functions-host>:<functions-port>/<functions-api-version>
export OS_AUTH_URL=<identity-api-protocol>://<identity-host>:<identity-port>/<identity-api-version>
export OS_PROJECT_NAME=<project-name>
export OS_USERNAME=<project-name>
export OS_PASSWORD=<project-name>
tox -epy35-integration
Testing: Coverage regression
----------------------------
In order to build quality software it is necessary to keep test coverage at its highest point.
So, as part of `Tox` testing new check was added - functional test coverage regression.
In order to run it use following command:
$ tox -e py35-functional-regression
3rd party bugs to resolve
-------------------------
IronFunctions:
* https://github.com/iron-io/functions/issues/298
* https://github.com/iron-io/functions/issues/296
* https://github.com/iron-io/functions/issues/275
* https://github.com/iron-io/functions/issues/274
TODOs
-----
Swagger doc:
* Make swagger doc more explicit on HTTP POST/UPDATE body content
* HTTP headers requests
IronFunctions:
* Support app deletion in IronFunctions
* Support tasks listing/showing
Laos:
* Tests: integration, functional, units
* Better logging coverage
Python Functions client:
* Support logging instance passing in [function-python](https://github.com/iron-io/functions_python)
* python-laosclient (ReST API client and CLI tool)
* App writing examples
Contacts
--------

79
laos/api/controllers/apps.py
View File

@ -17,9 +17,9 @@ from aiohttp import web
from aioservice.http import controller
from aioservice.http import requests
from laos.api.views import app as app_view
from laos.common import config
from laos.models import app as app_model
from ...common import config
from ...models import app as app_model
from ..views import app as app_view
class AppV1Controller(controller.ServiceController):
@ -159,22 +159,58 @@ class AppV1Controller(controller.ServiceController):
},
status=200
)
# TODO(denismakogon): disabled until iron-io/functions/pull/259
#
# @requests.api_action(method='PUT', route='{project_id}/apps/{app}')
# async def update(self, request, **kwargs):
# log = config.Config.config_instance().logger
# project_id = request.match_info.get('project_id')
# app = request.match_info.get('app')
# data = await request.json()
# log.info("Updating an app {} for project: {} with data {}"
# .format(app, project_id, str(data)))
# return web.json_response(
# data={
# "app": {}
# },
# status=200
# )
@requests.api_action(method='PUT', route='{project_id}/apps/{app}')
async def update(self, request, **kwargs):
"""
---
description: Updating project-scoped app
tags:
- Apps
produces:
- application/json
responses:
"200":
description: successful operation. Return "app" JSON
"401":
description: Not authorized.
"404":
description: App not found
"""
project_id = request.match_info.get('project_id')
app_name = request.match_info.get('app')
data = await request.json()
if not (await app_model.Apps.exists(app_name, project_id)):
return web.json_response(data={
"error": {
"message": "App {0} not found".format(app_name),
}
}, status=404)
c = config.Config.config_instance()
fnclient = c.functions_client
try:
fn_app = await fnclient.apps.update(
app_name, loop=c.event_loop, **data)
except Exception as ex:
return web.json_response(data={
"error": {
"message": getattr(ex, "reason", str(ex)),
}
}, status=getattr(ex, "status", 500))
stored_app = (await app_model.Apps.find_by(
project_id=project_id, name=app_name)).pop()
c.logger.info("Updating app {} for project: {} with data {}"
.format(app_name, project_id, str(data)))
return web.json_response(
data={
"app": app_view.AppView(stored_app, fn_app).view(),
"message": "App successfully updated"
},
status=200
)
@requests.api_action(method='DELETE', route='{project_id}/apps/{app}')
async def delete(self, request, **kwargs):
@ -217,9 +253,8 @@ class AppV1Controller(controller.ServiceController):
await app_model.Apps.delete(
project_id=project_id, name=app)
# TODO(denismakogon): enable DELETE to IronFunctions when once
# https://github.com/iron-io/functions/issues/274 implemented
# fn_app = await fnclient.apps.delete(app, loop=c.event_loop)
await fnclient.apps.delete(app, loop=c.event_loop)
return web.json_response(
data={
"message": "App successfully deleted",

77
laos/api/controllers/routes.py
View File

@ -19,9 +19,9 @@ from aiohttp import web
from aioservice.http import controller
from aioservice.http import requests
from laos.api.views import app as app_view
from laos.common import config
from laos.models import app as app_model
from ...common import config
from ...models import app as app_model
from ..views import app as app_view
class AppRouteV1Controller(controller.ServiceController):
@ -78,6 +78,7 @@ class AppRouteV1Controller(controller.ServiceController):
return web.json_response(data={
"routes": app_view.AppRouteView(api_url,
project_id,
app,
fn_app_routes).view(),
"message": "Successfully loaded app routes",
}, status=200)
@ -166,7 +167,7 @@ class AppRouteV1Controller(controller.ServiceController):
**data, loop=c.event_loop))
stored_route = await app_model.Routes(
app_name=new_fn_route.appname,
app_name=app,
project_id=project_id,
path=new_fn_route.path,
is_public=is_public).save()
@ -178,7 +179,7 @@ class AppRouteV1Controller(controller.ServiceController):
setattr(new_fn_route, "is_public", stored_route.public)
view = app_view.AppRouteView(
api_url, project_id, [new_fn_route]).view_one()
api_url, project_id, app, [new_fn_route]).view_one()
return web.json_response(data={
"route": view,
@ -244,10 +245,76 @@ class AppRouteV1Controller(controller.ServiceController):
return web.json_response(data={
"route": app_view.AppRouteView(api_url,
project_id,
app,
[route]).view_one(),
"message": "App route successfully loaded"
}, status=200)
@requests.api_action(
method='PUT', route='{project_id}/apps/{app}/routes/{route}')
async def update(self, request, **kwargs):
"""
---
description: Updating project-scoped app route
tags:
- Routes
produces:
- application/json
responses:
"200":
description: Successful operation. Return empty JSON
"401":
description: Not authorized.
"404":
description: App does not exist
"404":
description: App route does not exist
"""
c = config.Config.config_instance()
log, fnclient = c.logger, c.functions_client
project_id = request.match_info.get('project_id')
app = request.match_info.get('app')
path = request.match_info.get('route')
data = await request.json()
log.info("Deleting route {} in app {} for project: {}"
.format(path, app, project_id))
if not (await app_model.Apps.exists(app, project_id)):
return web.json_response(data={
"error": {
"message": "App {0} not found".format(app),
}
}, status=404)
try:
fn_app = await fnclient.apps.show(app, loop=c.event_loop)
await fn_app.routes.show("/{}".format(path), loop=c.event_loop)
route = await fn_app.routes.update(
"/{}".format(path), loop=c.event_loop, **data)
except Exception as ex:
return web.json_response(data={
"error": {
"message": getattr(ex, "reason", str(ex)),
}
}, status=getattr(ex, "status", 500))
api_url = "{}://{}".format(request.scheme, request.host)
stored_route = (await app_model.Routes.find_by(
app_name=app,
project_id=project_id,
path=route.path)).pop()
setattr(route, "is_public", stored_route.public)
return web.json_response(data={
"route": app_view.AppRouteView(api_url,
project_id,
app,
[route]).view_one(),
"message": "Route successfully updated",
}, status=200)
@requests.api_action(
method='DELETE', route='{project_id}/apps/{app}/routes/{route}')
async def delete(self, request, **kwargs):

2
laos/api/controllers/runnable.py
View File

@ -17,7 +17,7 @@ from aiohttp import web
from aioservice.http import controller
from aioservice.http import requests
from laos.common import config
from ...common import config
class RunnableMixin(object):

4
laos/api/controllers/tasks.py
View File

@ -14,12 +14,12 @@
from aiohttp import web
# from laos.models import app as app_model
# from ...models import app as app_model
from aioservice.http import controller
from aioservice.http import requests
# from laos.common import config
# from ...common import config
# TODO(denismakogon): disabled until

2
laos/api/middleware/keystone.py
View File

@ -18,7 +18,7 @@ from keystoneclient import client
from aiohttp import web
from laos.common import config
from ...common import config
async def auth_through_token(app: web.Application, handler):

21
laos/api/views/app.py
View File

@ -33,10 +33,11 @@ class AppView(object):
class AppRouteView(object):
def __init__(self, api_url, project_id, fn_app_routes):
def __init__(self, api_url, project_id, app_name, fn_app_routes):
self.routes = fn_app_routes
self.api_url = api_url
self.project_id = project_id
self.app_name = app_name
def view_one(self):
return self.view().pop()
@ -47,15 +48,23 @@ class AppRouteView(object):
if not route.is_public:
path = ("{}/v1/r/{}/{}{}".format(
self.api_url, self.project_id,
route.appname, route.path))
self.app_name, route.path))
else:
path = ("{}/r/{}{}".format(
self.api_url, route.appname, route.path))
view.append({
self.api_url, self.app_name, route.path))
one = {
"path": path,
"type": route.type,
"memory": route.memory,
"image": route.image,
"is_public": route.is_public,
})
}
# temporary solution for
# https://github.com/iron-io/functions/issues/382
if hasattr(route, "memory"):
one.update(memory=route.memory)
if hasattr(route, "timeout"):
one.update(timeout=route.timeout)
if hasattr(route, "max_concurrency"):
one.update(timeout=route.max_concurrency)
view.append(one)
return view

2
laos/common/config.py
View File

@ -16,7 +16,7 @@
import aiomysql
import asyncio
from laos.common import utils
from . import utils
from functionsclient.v1 import client

2
laos/common/logger.py
View File

@ -16,7 +16,7 @@ import datetime
import logging
import sys
from laos.common import utils
from . import utils
def common_logger_setup(

2
laos/common/persistence.py
View File

@ -15,7 +15,7 @@
import datetime
import uuid
from laos.common import config
from . import config
class BaseDatabaseModel(object):

2
laos/models/app.py
View File

@ -12,7 +12,7 @@
# License for the specific language governing permissions and limitations
# under the License.
from laos.common import persistence
from ..common import persistence
class Apps(persistence.BaseDatabaseModel):

20
laos/tests/common/apps.py
View File

@ -29,8 +29,9 @@ class AppsTestSuite(object):
self.assertIn("error", json)
def create_and_delete(self):
app = "create_and_delete"
create_json, create_status = self.testloop.run_until_complete(
self.test_client.apps.create("testapp"))
self.test_client.apps.create(app))
delete_json, delete_status = self.testloop.run_until_complete(
self.test_client.apps.delete(create_json["app"]["name"]))
@ -42,7 +43,7 @@ class AppsTestSuite(object):
self.assertEqual(200, delete_status)
def attempt_to_double_create(self):
app = "testapp"
app = "attempt_to_double_create"
create_json, _ = self.testloop.run_until_complete(
self.test_client.apps.create(app))
err, status = self.testloop.run_until_complete(
@ -60,7 +61,7 @@ class AppsTestSuite(object):
self.assertIn("error", json)
def delete_with_routes(self):
app_name = "testapp"
app_name = "delete_with_routes"
app, _ = self.testloop.run_until_complete(
self.test_client.apps.create(app_name))
self.testloop.run_until_complete(
@ -82,3 +83,16 @@ class AppsTestSuite(object):
self.assertIn("message", attempt["error"])
self.assertIn("with routes", attempt["error"]["message"])
self.assertEqual(200, status_2)
def update_app(self):
app_name = "update_app"
app, _ = self.testloop.run_until_complete(
self.test_client.apps.create(app_name))
_, update_status = self.testloop.run_until_complete(
self.test_client.apps.update(
app["app"]["name"], config={}
)
)
self.testloop.run_until_complete(
self.test_client.apps.delete(app["app"]["name"]))
self.assertEqual(200, update_status)

4
laos/tests/common/base.py
View File

@ -16,14 +16,14 @@ import asyncio
import datetime
import uvloop
from laos.common import logger as log
from ...common import logger as log
class LaosTestsBase(object):
def get_loop_and_logger(self, test_type):
self.route_data = {
"type": "sync",
"type": "async",
"path": "/hello-sync-private",
"image": "iron/hello",
"is_public": "false"

40
laos/tests/common/client.py
View File

@ -44,6 +44,10 @@ class AppsV1(object):
async def delete(self, app_name):
return await self.client.remove(self.app_path, app_name)
async def update(self, app_name, **data):
return await self.client.update(
self.app_path, app_name, **data)
class RoutesV1(object):
@ -51,6 +55,10 @@ class RoutesV1(object):
routes_path = "/v1/{}/apps/{}/routes"
# /v1/{project_id}/apps/{app}/routes{}
route_path = routes_path + "{}"
# /v1/r/{project_id}/{app}{route}
private_execution_path = "/v1/r/{}/{}{}"
# /r/{app}{route}
public_execution_path = "/r/{}{}"
def __init__(self, test_client):
self.client = test_client
@ -71,6 +79,19 @@ class RoutesV1(object):
return await self.client.remove(
self.route_path, app_name, path)
async def update(self, app_name, path, **data):
return await self.client.update(
self.route_path, app_name, path, **data)
async def execute_private(self, app_name, path, **data):
return await self.client.execute(
self.private_execution_path, app_name, path, **data)
async def execute_public(self, app_name, path, **data):
return await self.client.execute(
self.public_execution_path, app_name, path,
ignore_project_id=True, **data)
class ProjectBoundLaosTestClient(test_utils.TestClient):
@ -113,3 +134,22 @@ class ProjectBoundLaosTestClient(test_utils.TestClient):
headers=self.headers)
json = await resp.json()
return json, resp.status
async def update(self, route_path, resource_name, *parts, **data):
resp = await self.put(
route_path.format(self.project_id, resource_name, *parts),
data=jsonlib.dumps(data),
headers=self.headers,)
json = await resp.json()
return json, resp.status
async def execute(self, route, resource_name, *parts,
ignore_project_id=False, **data):
if not ignore_project_id:
route = route.format(self.project_id, resource_name, *parts)
else:
route = route.format(resource_name, *parts)
resp = await self.post(
route, data=jsonlib.dumps(data), headers=self.headers)
json = await resp.json()
return json, resp.status

119
laos/tests/common/routes.py
View File

@ -12,9 +12,40 @@
# License for the specific language governing permissions and limitations
# under the License.
import contextlib
import json as jsonlib
@contextlib.contextmanager
def setup_execute(self, app_name):
app, _ = self.testloop.run_until_complete(
self.test_client.apps.create(app_name)
)
new_app_name = app["app"]["name"]
route, _ = self.testloop.run_until_complete(
self.test_client.routes.create(
new_app_name, **self.route_data)
)
self.testloop.run_until_complete(
self.test_client.routes.update(
new_app_name, self.route_data["path"], **{
"type": "sync"
}
)
)
try:
yield new_app_name
except Exception as ex:
print(ex)
finally:
self.testloop.run_until_complete(
self.test_client.routes.delete(
new_app_name, self.route_data["path"])
)
self.testloop.run_until_complete(
self.test_client.apps.delete(new_app_name))
class AppRoutesTestSuite(object):
def list_routes_from_unknown_app(self):
@ -26,8 +57,9 @@ class AppRoutesTestSuite(object):
self.assertIn("not found", json["error"]["message"])
def list_routes_from_existing_app(self):
app = "list_routes_from_existing_app"
create_json, _ = self.testloop.run_until_complete(
self.test_client.apps.create("testapp"))
self.test_client.apps.create(app))
json, status = self.testloop.run_until_complete(
self.test_client.routes.list(create_json["app"]["name"])
)
@ -38,9 +70,10 @@ class AppRoutesTestSuite(object):
self.assertIn("message", json)
def show_unknown_route_from_existing_app(self):
app = "show_unknown_route_from_existing_app"
path = "/unknown_path"
create_json, _ = self.testloop.run_until_complete(
self.test_client.apps.create("testapp"))
self.test_client.apps.create(app))
json, status = self.testloop.run_until_complete(
self.test_client.routes.show(
create_json["app"]["name"], path)
@ -53,8 +86,9 @@ class AppRoutesTestSuite(object):
self.assertIn("not found", json["error"]["message"])
def delete_unknown_route_from_existing_app(self):
app = "delete_unknown_route_from_existing_app"
create_json, _ = self.testloop.run_until_complete(
self.test_client.apps.create("testapp"))
self.test_client.apps.create(app))
json, status = self.testloop.run_until_complete(
self.test_client.routes.delete(
create_json["app"]["name"], "/unknown_path")
@ -67,18 +101,23 @@ class AppRoutesTestSuite(object):
self.assertIn("not found", json["error"]["message"])
def create_and_delete_route(self):
app, _ = self.testloop.run_until_complete(
self.test_client.apps.create("testapp"))
app_name = "create_and_delete_route"
created, _ = self.testloop.run_until_complete(
self.test_client.apps.create(app_name))
new_app_name = created["app"]["name"]
route, create_status = self.testloop.run_until_complete(
self.test_client.routes.create(
app["app"]["name"], **self.route_data)
new_app_name, **self.route_data)
)
route_deleted, delete_status = self.testloop.run_until_complete(
self.test_client.routes.delete(
app["app"]["name"], self.route_data["path"])
new_app_name, self.route_data["path"])
)
self.testloop.run_until_complete(
self.test_client.apps.delete(app["app"]["name"]))
self.test_client.apps.delete(new_app_name))
print(route)
after_post = route["route"]
for k in self.route_data:
if k == "path":
@ -94,25 +133,77 @@ class AppRoutesTestSuite(object):
self.assertIn("message", route_deleted)
def double_create_route(self):
app, _ = self.testloop.run_until_complete(
self.test_client.apps.create("testapp"))
app = "double_create_route"
created_app, _ = self.testloop.run_until_complete(
self.test_client.apps.create(app))
new_app_name = created_app["app"]["name"]
self.testloop.run_until_complete(
self.test_client.routes.create(
app["app"]["name"], **self.route_data)
new_app_name, **self.route_data)
)
json, double_create_status = self.testloop.run_until_complete(
self.test_client.routes.create(
app["app"]["name"], **self.route_data)
new_app_name, **self.route_data)
)
self.testloop.run_until_complete(
self.test_client.routes.delete(
app["app"]["name"], self.route_data["path"])
new_app_name, self.route_data["path"])
)
self.testloop.run_until_complete(
self.test_client.apps.delete(app["app"]["name"]))
self.test_client.apps.delete(new_app_name))
self.assertEqual(409, double_create_status)
self.assertIn("error", json)
self.assertIn("message", json["error"])
self.assertIn("already exist", json["error"]["message"])
def update_route(self):
app = "update_route"
created, _ = self.testloop.run_until_complete(
self.test_client.apps.create(app)
)
new_app_name = created["app"]["name"]
route, _ = self.testloop.run_until_complete(
self.test_client.routes.create(
new_app_name, **self.route_data)
)
print(route)
updated, update_status = self.testloop.run_until_complete(
self.test_client.routes.update(
new_app_name, self.route_data["path"], **{
"type": "sync"
}
)
)
print(updated)
self.testloop.run_until_complete(
self.test_client.routes.delete(
new_app_name, self.route_data["path"])
)
self.testloop.run_until_complete(
self.test_client.apps.delete(new_app_name))
self.assertEqual(200, update_status)
self.assertNotIn(route["route"]["type"], updated["route"]["type"])
def execute_private(self):
with setup_execute(self, "execute_private") as app_name:
result, status = self.testloop.run_until_complete(
self.test_client.routes.execute_private(
app_name, self.route_data["path"]
)
)
self.assertIsNotNone(result)
self.assertEqual(200, status)
def execute_public(self):
with setup_execute(self, "execute_public") as app_name:
result, status = self.testloop.run_until_complete(
self.test_client.routes.execute_public(
app_name, self.route_data["path"]
)
)
self.assertIsNotNone(result)
self.assertEqual(200, status)

29
laos/tests/fakes/functions_api.py
View File

@ -69,18 +69,27 @@ class FakeRoutes(object):
)
)
async def execute(self, path, loop=None):
async def execute(self, path, loop=None, **data):
app_routes = APP_ROUTES[self.app_name]
if path not in [route.path for route in app_routes]:
raise client.FunctionsAPIException(
"App {} route {} not found.".format(
self.app_name, path), 404)
else:
route = await self.show(path)
route = await self.show(path, loop=loop)
return "Hello world!" if route.type == "sync" else {
"call_id": uuid.uuid4().hex
}
async def update(self, route_path, loop=None, **data):
route = await self.show(route_path, loop=loop)
if "path" in data:
del data['path']
route.__kwargs__.update(data)
for k, v in route.__kwargs__.items():
setattr(route, k, v)
return route
class FakeApps(object):
@ -104,17 +113,29 @@ class FakeApps(object):
async def show(self, app_name, loop=None):
if app_name not in APPS:
raise client.FunctionsAPIException(
"App {} already exist.".format(app_name), 404)
"App {} not found.".format(app_name), 404)
else:
return APPS.get(app_name)
async def delete(self, app_name, loop=None):
if app_name not in APPS:
raise client.FunctionsAPIException(
"App {} already exist.".format(app_name), 404)
"App {} not found.".format(app_name), 404)
else:
if APP_ROUTES[app_name]:
raise client.FunctionsAPIException(
"Cannot remove apps with routes", 403)
del APPS[app_name]
async def update(self, app_name, loop=None, **data):
app = await self.show(app_name, loop=loop)
if 'name' in data:
del data['name']
app.__kwargs__.update(data)
for k, v in app.__kwargs__.items():
setattr(app, k, v)
return app
class FunctionsAPIV1(object):

30
laos/tests/functional/base.py
View File

@ -12,24 +12,23 @@
# License for the specific language governing permissions and limitations
# under the License.
import collections
import os
import testtools
import uuid
from aioservice.http import service
from laos.api.controllers import apps
from laos.api.controllers import routes
from laos.api.controllers import runnable
from laos.api.controllers import tasks
from laos.api.middleware import content_type
from ...api.controllers import apps
from ...api.controllers import routes
from ...api.controllers import runnable
from ...api.controllers import tasks
from ...api.middleware import content_type
from laos.common import config
from ...common import config
from laos.tests.common import base
from laos.tests.common import client
from laos.tests.fakes import functions_api
from ..common import base
from ..common import client
from ..fakes import functions_api
class LaosFunctionalTestsBase(base.LaosTestsBase, testtools.TestCase):
@ -78,20 +77,9 @@ class LaosFunctionalTestsBase(base.LaosTestsBase, testtools.TestCase):
self.test_client = client.ProjectBoundLaosTestClient(
self.testapp, self.project_id)
self.route_data = {
"type": "sync",
"path": "/hello-sync-private",
"image": "iron/hello",
"is_public": "false"
}
self.testloop.run_until_complete(self.test_client.start_server())
super(LaosFunctionalTestsBase, self).setUp()
def tearDown(self):
functions_api.APPS = {}
functions_api.ROUTES = collections.defaultdict(list)
# ^ temporary solution,
# until https://github.com/iron-io/functions/issues/274 fixed
self.testloop.run_until_complete(self.test_client.close())
super(LaosFunctionalTestsBase, self).tearDown()

7
laos/tests/functional/test_apps.py
View File

@ -12,8 +12,8 @@
# License for the specific language governing permissions and limitations
# under the License.
from laos.tests.common import apps as apps_suite
from laos.tests.functional import base
from ..common import apps as apps_suite
from ..functional import base
class TestApps(base.LaosFunctionalTestsBase, apps_suite.AppsTestSuite):
@ -35,3 +35,6 @@ class TestApps(base.LaosFunctionalTestsBase, apps_suite.AppsTestSuite):
def test_delete_with_routes(self):
super(TestApps, self).delete_with_routes()
def test_update_app(self):
super(TestApps, self).update_app()

13
laos/tests/functional/test_routes.py
View File

@ -12,8 +12,8 @@
# License for the specific language governing permissions and limitations
# under the License.
from laos.tests.common import routes as routes_suite
from laos.tests.functional import base
from ..common import routes as routes_suite
from ..functional import base
class TestAppRoutes(base.LaosFunctionalTestsBase,
@ -40,3 +40,12 @@ class TestAppRoutes(base.LaosFunctionalTestsBase,
def test_double_create_route(self):
super(TestAppRoutes, self).double_create_route()
def test_update_route(self):
super(TestAppRoutes, self).update_route()
def test_private_execution(self):
super(TestAppRoutes, self).execute_private()
def test_public_execution(self):
super(TestAppRoutes, self).execute_private()

13
laos/tests/integration/base.py
View File

@ -12,18 +12,19 @@
# License for the specific language governing permissions and limitations
# under the License.
import aiohttp
import json
import os
from urllib import parse
import aiohttp
import testtools
from laos.common import config
from laos.service import laos_api
from ...common import config
from ..common import base
from ..common import client
from laos.tests.common import base
from laos.tests.common import client
from urllib import parse
from service import laos_api
class LaosIntegrationTestsBase(base.LaosTestsBase, testtools.TestCase):

7
laos/tests/integration/test_apps.py
View File

@ -12,8 +12,8 @@
# License for the specific language governing permissions and limitations
# under the License.
from laos.tests.common import apps as apps_suite
from laos.tests.integration import base
from ..common import apps as apps_suite
from ..integration import base
class TestIntegrationApps(base.LaosIntegrationTestsBase,
@ -36,3 +36,6 @@ class TestIntegrationApps(base.LaosIntegrationTestsBase,
def test_delete_with_routes(self):
super(TestIntegrationApps, self).delete_with_routes()
def test_update_app(self):
super(TestIntegrationApps, self).update_app()

17
laos/tests/integration/test_routes.py
View File

@ -12,8 +12,8 @@
# License for the specific language governing permissions and limitations
# under the License.
from laos.tests.common import routes as routes_suite
from laos.tests.functional import base
from ..common import routes as routes_suite
from ..functional import base
class TestIntegrationAppRoutes(base.LaosFunctionalTestsBase,
@ -45,6 +45,13 @@ class TestIntegrationAppRoutes(base.LaosFunctionalTestsBase,
).create_and_delete_route()
def test_double_create_route(self):
super(
TestIntegrationAppRoutes, self
).double_create_route()
super(TestIntegrationAppRoutes, self).double_create_route()
def test_update_route(self):
super(TestIntegrationAppRoutes, self).update_route()
def test_private_execution(self):
super(TestIntegrationAppRoutes, self).execute_private()
def test_public_execution(self):
super(TestIntegrationAppRoutes, self).execute_private()

4
requirements.txt
View File

@ -3,13 +3,13 @@
# process, which may cause wedges in the gate later.
uvloop==0.6.0 # Apache-2.0
aioservice==0.0.1 # Apache-2.0
aioservice==0.0.2 # Apache-2.0
aiomysql==0.0.9 # Apache-2.0
alembic==0.8.8 # MIT
click==6.6 # Apache-2.0
# IronFunctions
python-functionsclient==0.0.1
python-functionsclient==0.0.2
# OpenStack
keystoneauth1==2.15.0 # Apache-2.0

18
scripts/docker_full.sh Executable file
View File

@ -0,0 +1,18 @@
#!/usr/bin/env bash
set +x
set +e
set +i
docker build -t laos-api -f Dockerfile . 2>&1
docker_h=$(echo ${DOCKER_HOST} | tr "://" " " |awk '{print $2}')
echo -e "Docker IP address ${docker_h}"
echo -e "OpenStack Identity service URL ${OS_AUTH_URL}"
echo -e "IronFunctions URL ${FUNCTIONS_API_URL}"
echo -e "Persistent storage URI ${TEST_DB_URI}"
docker run -d -p ${docker_h}:10002:10001 --env LAOS_HOST=0.0.0.0 --env LAOS_PORT=10001 --env LAOS_DB=${TEST_DB_URI} --env KEYSTONE_ENDPOINT=${OS_AUTH_URL} --env FUNCTIONS_URL=${FUNCTIONS_API_URL} --env LAOS_LOG_LEVEL=INFO laos-api
sleep 2
docker ps
echo -e "Service running on ${docker_h}:10002"
curl -X GET http://${docker_h}:10002/api/swagger.json | python -mjson.tool
docker stop -t 1 $(docker ps | grep "${docker_h}:10002" | awk '{print $1}')

0
laos/service/__init__.py → service/__init__.py
View File

0
laos/service/laos_api.py → service/laos_api.py
View File

6
setup.py
View File

@ -30,11 +30,11 @@ setuptools.setup(
packages=setuptools.find_packages(),
install_requires=[
"uvloop==0.6.0",
"aioservice==0.0.1",
"aioservice==0.0.2",
"aiomysql==0.0.9",
"alembic==0.8.8",
"click==6.6",
"python-functionsclient==0.0.1",
"python-functionsclient==0.0.2",
"keystoneauth1==2.15.0",
"python-keystoneclient==3.6.0",
"aiohttp-swagger==1.0.2",
@ -46,8 +46,6 @@ setuptools.setup(
'Intended Audience :: System Administrators',
'Intended Audience :: Developers',
'Environment :: No Input/Output (Daemon)',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Topic :: Software Development :: '
'Libraries :: Python Modules',

107
testing.md Normal file
View File

@ -0,0 +1,107 @@
Testing
-------
In order to run tests you need to install `Tox`:
$ pip install tox
Also, you will need a running MySQL instance with the database migrations applied from the previous step.
Tests are dependent on pre-created MySQL database for persistence.
Please set env var
$ export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
PEP8 style checks
-----------------
In order to run `PEP8` style checks run following command:
$ tox -e pep8
Functional testing
------------------
In order to run `functional` tests run following command:
$ tox -e py35-functional
Pros:
* lightweight (controllers and DB models testing)
* no OpenStack required
* no IronFunctions required
Cons:
* MySQL server required
* OpenStack authentication is not tested
* IronFunctions API stubbed with fake implementation
Integration integrations
------------------------
Integration tests are dependent on following env variables:
* TEST_DB_URI - similar to functional tests, database endpoint
* FUNCTIONS_API_URL - IronFunctions API URL (default value - `http://localhost:8080/v1`)
* OS_AUTH_URL - OpenStack Identity endpoint
* OS_PROJECT_NAME - OpenStack user-specific project name
* OS_USERNAME - OpenStack user name
* OS_PASSWORD - OpenStack user user password
To run tests use following command:
export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
export FUNCTIONS_API_URL=<functions-api-protocol>://<functions-host>:<functions-port>/<functions-api-version>
export OS_AUTH_URL=<identity-api-protocol>://<identity-host>:<identity-port>/<identity-api-version>
export OS_PROJECT_NAME=<project-name>
export OS_USERNAME=<project-name>
export OS_PASSWORD=<project-name>
tox -epy35-integration
Testing: Docker-build
---------------------
This type of testing allows to ensure if code can be build inside docker container with no problems.
In order to run this check use following commands::
export DOCKER_HOST=tcp://<docker-host>:<docker-port>>
export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
export FUNCTIONS_API_URL=<functions-api-protocol>://<functions-host>:<functions-port>/<functions-api-version>
export OS_AUTH_URL=<identity-api-protocol>://<identity-host>:<identity-port>/<identity-api-version>
tox -e docker-build
During this check Tox:
* builds an image
* deletes all artifacts (Python3.5 image and recently built image)
Testing Docker-full
-------------------
This type of testing allows to ensure if code code can be build and run successfully inside docker container with no problems.
In order to run this check use following commands::
export DOCKER_HOST=tcp://<docker-host>:<docker-port>>
export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
export FUNCTIONS_API_URL=<functions-api-protocol>://<functions-host>:<functions-port>/<functions-api-version>
export OS_AUTH_URL=<identity-api-protocol>://<identity-host>:<identity-port>/<identity-api-version>
tox -e docker-full
During this check following operations are performed::
* build container from source code
* run container with exposed ports
* request Swagger API doc to see if API is responsive
* tear-down running container
Coverage regression testing
---------------------------
In order to build quality software it is necessary to keep test coverage at its highest point.
So, as part of `Tox` testing new check was added - functional test coverage regression.
In order to run it use following command:
$ tox -e py35-functional-regression

11
tox.ini
View File

@ -1,7 +1,7 @@
# Project LaOS
[tox]
envlist = py35-functional,py35-functional-regression,py35-integration,py35-integration-regression,pep8
envlist = py35-functional,py35-functional-regression,py35-integration,py35-integration-regression,pep8,docker-build
minversion = 1.6
skipsdist = True
@ -14,16 +14,16 @@ passenv =
OS_PASSWORD
OS_USERNAME
OS_PROJECT_NAME
DOCKER_HOST
setenv = VIRTUAL_ENV={envdir}
usedevelop = True
install_command = pip install -U {opts} {packages}
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
commands = find . -type f -name "*.pyc" -delete
rm -f .testrepository/times.dbm
python setup.py testr --testr-args='{posargs}'
whitelist_externals = find
rm
docker
[testenv:pep8]
commands = flake8
@ -48,7 +48,10 @@ commands =
rm -rf doc/html doc/build
python setup.py build_sphinx
[testenv:docker-full]
commands = {toxinidir}/scripts/docker_full.sh
[flake8]
ignore = H202,H404,H405,H501
ignore = H202,H304,H404,H405,H501
show-source = True
exclude=.venv,.git,.tox,dist,doc,*lib/python*,*egg,build,migrations