Merge pull request #37 from iron-io/doc-updates

doc updates
This commit is contained in:
Derek Schultz 2016-12-16 10:55:11 -07:00 committed by GitHub
commit 45af81accb
5 changed files with 121 additions and 261 deletions

View File

@ -1,4 +1,6 @@
# Picasso: Functions-as-a-Service (FaaS) on OpenStack
# Picasso
Functions-as-a-Service (FaaS) on OpenStack
## Mission
@ -37,13 +39,13 @@ then the benefits are different, but related.
* Scaling is simply adding more IronFunctions nodes
### System requirements
## System requirements
* Operating system: Linux/MacOS
* Python version: 3.5 or greater
* Database: MySQL 5.7 or greater
### Quick-start guide
## Quick-start guide
* Install DevStack with [IronFunctions enabled](https://github.com/iron-io/picasso/blob/master/devstack/README.rst)
* Clone the [Picasso source](https://github.com/iron-io/picasso)
@ -53,7 +55,7 @@ Create a Python3.5 virtualenv
$ virtualenv -p python3.5 .venv
$ source .venv/bin/activate
Install dependencies
Install Picasso dependencies
$ pip install -r requirements.txt -r test-requirements.txt
@ -72,7 +74,7 @@ set the following environment variable:
export PICASSO_MIGRATIONS_DB=mysql+pymysql://root:root@localhost/functions
Use `alembic` to apply the migrations:
Then use `alembic` to apply the migrations
$ alembic upgrade head
@ -103,22 +105,21 @@ The following are the minimum required options to start the Picasso API service:
### Building and Running Picasso in Docker
From the Picasso repo, build a Docker image:
Install [Docker engine](https://docs.docker.com/engine/installation/) if you haven't already.
From the Picasso repo, build a Docker image using the following commands:
export DOCKER_HOST=tcp://<docker-host>:<docker-port>
docker build -t picasso-api -f Dockerfile .
To start the container, pass in the required env vars, by
To start the container, pass in the required env vars with `--env-file` or by entering all required
values in `-e <KEY>=<VALUE>` format to the `docker run` command.
`--env-file` example [Dockerfile.env](Dockerfile.env.example)
Example [Dockerfile.env](Dockerfile.env.example)
docker run -d -p 10001:10001 --env-file Dockerfile.env picasso-api
docker run -d -p 10001:10001 --env-file Dockerfile.env picasso-api
or by entering all values in `-e <KEY>=<VALUE>` format.
Once the container is started, check if the service in running:
In your web browser navigate to:
Once the container is started, check if the service in running. In your web browser navigate to:
<docker-host>:10001/api
@ -126,32 +127,16 @@ or using the CLI:
curl -X GET http://<docker-host>:10001/api/swagger.json | python -mjson.tool
### Examining the API
In [examples](examples/) folder you can find a script that examines available API endpoints.
Note that this script depends on the following env vars:
* `PICASSO_API_URL` - Picasso API endpoint
* `OS_AUTH_URL` - OpenStack Auth URL
* `OS_PROJECT_ID` - it can be found in OpenStack Dashboard or in CLI
* `OS_USERNAME` - OpenStack project-aligned username
* `OS_PASSWORD` - OpenStack project-aligned user password
* `OS_DOMAIN` - OpenStack project domain name
* `OS_PROJECT_NAME` - OpenStack project name
To run the script:
OS_AUTH_URL=http://192.168.0.112:5000/v3 OS_PROJECT_ID=8fb76785313a4500ac5367eb44a31677 OS_USERNAME=admin OS_PASSWORD=root OS_DOMAIN=default OS_PROJECT_NAME=admin ./examples/hello-lambda.sh
Please note that values provided are project-specific, so they can't be reused.
### API docs
API docs are discoverable via Swagger. Just launch the Picasso API and browse to:
http://<picasso-host>:<picasso-port>/api
### Testing Picasso
See [Testing.md](TESTING.md)
### Support
Join us on [Slack](https://open-iron.herokuapp.com/)!
Join us on [Slack](https://open-iron.slack.com/)!

93
TESTING.md Normal file
View File

@ -0,0 +1,93 @@
# Testing
## Requirements
* Install `Tox`
$ pip install tox
* MySQL instance with database migrations applied (refer to quick start guide)
$ export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
### PEP8 style checks
$ tox -e pep8
### Functional testing
$ tox -e py35-functional
Pros:
* lightweight (controllers and DB models testing)
* no OpenStack required
* no IronFunctions required
Cons:
* MySQL server required
* OpenStack authentication is not tested
* IronFunctions API stubbed with fake implementation
### Integration tests
The following env variables are required:
* TEST_DB_URI - similar to functional tests, database endpoint
* FUNCTIONS_API_URL - IronFunctions API URL (default value - `http://localhost:8080/v1`)
* OS_AUTH_URL - OpenStack Identity endpoint
* OS_PROJECT_NAME - OpenStack user-specific project name
* OS_USERNAME - OpenStack user name
* OS_PASSWORD - OpenStack user user password
```bash
export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
export FUNCTIONS_API_URL=<functions-api-protocol>://<functions-host>:<functions-port>/<functions-api-version>
export OS_AUTH_URL=<identity-api-protocol>://<identity-host>:<identity-port>/<identity-api-version>
export OS_PROJECT_NAME=<project-name>
export OS_USERNAME=<project-name>
export OS_PASSWORD=<project-name>
tox -epy35-integration
```
### Testing: Docker-build
The following operations are performed:
* builds an image
* deletes all artifacts (Python3.5 image and recently built image)
```bash
export DOCKER_HOST=tcp://<docker-host>:<docker-port>>
export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
export FUNCTIONS_API_URL=<functions-api-protocol>://<functions-host>:<functions-port>/<functions-api-version>
export OS_AUTH_URL=<identity-api-protocol>://<identity-host>:<identity-port>/<identity-api-version>
tox -e docker-build
```
### Testing Docker-full
The following operations are performed:
* build container from source code
* run container with exposed ports
* request Swagger API doc to see if API is responsive
* destroy running container
```bash
export DOCKER_HOST=tcp://<docker-host>:<docker-port>>
export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
export FUNCTIONS_API_URL=<functions-api-protocol>://<functions-host>:<functions-port>/<functions-api-version>
export OS_AUTH_URL=<identity-api-protocol>://<identity-host>:<identity-port>/<identity-api-version>
tox -e docker-full
```
### Coverage regression testing
$ tox -e py35-functional-regression
### Static code analysis with Bandit
$ tox -e bandit

View File

@ -1,28 +1,21 @@
Enabling Picasso (Functions-as-a-Service) in DevStack
=====================================================
# Enabling Picasso (Functions-as-a-Service) in DevStack
Installing Glide
================
## Install Glide
Note that your machine should have Glide installed.
It is required to install Glide on the system in which you plan to run DevStack on, as
the Functions service is written in Go and we must fetch dependencies during install.
See more info at https://github.com/Masterminds/glide
Download DevStack
=================
.. sourcecode:: bash
## Download DevStack
export DEVSTACK_DIR=~/devstack
git clone git://git.openstack.org/openstack-dev/devstack.git $DEVSTACK_DIR
Enable the FaaS plugin
======================
## Enable the Functions plugin
Enable the plugin by adding the following section to ``$DEVSTACK_DIR/local.conf``
.. sourcecode:: bash
[[local|localrc]]
enable_plugin picasso git@github.com:iron-io/picasso.git
@ -40,7 +33,7 @@ Enable the plugin by adding the following section to ``$DEVSTACK_DIR/local.conf`
PICASSO_CLIENT_DIR=${PICASSO_CLIENT_DIR:-${DEST}/python-picassoclient}
PICASSO_CLIENT_BRANCH=${PICASSO_CLIENT_BRANCH:-master}
# IronFunctions parameters
# Functions parameters
FUNCTIONS_REPO=${FUNCTIONS_REPO:-git@github.com:iron-io/functions.git}
FUNCTIONS_BRANCH=${FUNCTIONS_BRANCH:-master}
FUNCTIONS_PORT=${FUNCTIONS_PORT:-10501}
@ -50,10 +43,7 @@ Enable the plugin by adding the following section to ``$DEVSTACK_DIR/local.conf`
DOCKERD_OPTS=${DOCKERD_OPTS:---dns 8.8.8.8 --dns 8.8.4.4 --storage-driver=overlay2 -H fd://}
Run the DevStack utility
========================
.. sourcecode:: bash
## Run the DevStack utility
cd $DEVSTACK_DIR
./stack.sh

View File

@ -1,96 +0,0 @@
#!/usr/bin/env bash
set +x
set +e
export LAOS_API_URL=${LAOS_API_URL:-http://localhost:10001}
export OS_AUTH_URL=${OS_AUTH_URL:-http://localhost:5000/v3}
export OS_USERNAME=${OS_USERNAME:-admin}
export OS_PASSOWRD=${OS_PASSWORD:-root}
export OS_DOMAIN=${OS_DOMAIN:-default}
export OS_PROJECT_ID=${OS_PROJECT_ID:-"dummy_project_id"}
rm -fr examples/token_request.json
echo -e "{
\"auth\": {
\"identity\": {
\"methods\": [\"password\"],
\"password\": {
\"user\": {
\"name\": \"${OS_USERNAME:-admin}\",
\"domain\": { \"id\": \"${OS_DOMAIN:-default}\" },
\"password\": \"${OS_PASSWORD:-root}\"
}
}
},
\"scope\": {
\"project\": {
\"name\": \"${OS_PROJECT_NAME:-admin}\",
\"domain\": {\"id\": \"${OS_DOMAIN:-default}\" }
}
}
}
}" >> examples/token_request.json
export OS_TOKEN=`curl -si -d @examples/token_request.json -H "Content-type: application/json" ${OS_AUTH_URL}/auth/tokens | awk '/X-Subject-Token/ {print $2}'`
echo -e "Listing apps\n"
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Creating app\n"
curl -X POST -d '{"app":{"name": "testapp"}}' ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Listing apps\n"
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Showing app info\n"
export raw_app_info=`curl localhost:10001/v1/${OS_PROJECT_ID}/apps -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool | grep name | awk '{print $2}'`
export app_name=${raw_app_info:1:30}
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name} -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Listing app routes\n"
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name}/routes -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Creating app sync private route\n"
curl -X POST -d '{"route":{"type": "sync", "path": "/hello-sync-private", "image": "iron/hello", "is_public": "false" }}' ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${APP_NAME}/routes -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Creating app sync public route\n"
curl -X POST -d '{"route":{"type": "sync", "path": "/hello-sync-public", "image": "iron/hello", "is_public": "true" }}' ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${APP_NAME}/routes -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Listing app routes\n"
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name}/routes -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Show app private route\n"
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name}/routes/hello-sync-private -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Show app public route\n"
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name}/routes/hello-sync-public -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Running app sync private route\n"
curl -X POST -d '{"name": "Johnny"}' ${LAOS_API_URL}/v1/r/${OS_PROJECT_ID}/${app_name}/hello-sync-private -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Running app sync public route\n"
curl -X POST -d '{"name": "Johnny"}' ${LAOS_API_URL}/r/${app_name}/hello-sync-public -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Creating app async route\n"
curl -X POST -d '{"route":{"type": "async", "path": "/hello-async-private", "image": "iron/hello", "is_public": "false"}}' ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name}/routes -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Running app async route\n"
curl -X POST -d '{"name": "Johnny"}' ${LAOS_API_URL}/v1/r/${OS_PROJECT_ID}/${app_name}/hello-async-private -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Deleting app route\n"
curl -X DELETE ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name}/routes/hello-sync-public -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
curl -X DELETE ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name}/routes/hello-sync-private -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
curl -X DELETE ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name}/routes/hello-async-private -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Listing app routes\n"
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name}/routes -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Deleting app\n"
curl -X DELETE ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps/${app_name} -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool
echo -e "Listing apps\n"
curl ${LAOS_API_URL}/v1/${OS_PROJECT_ID}/apps -H "X-Auth-Token:${OS_TOKEN}" -H "Content-Type: application/json" | python3 -mjson.tool

View File

@ -1,112 +0,0 @@
Testing
-------
In order to run tests you need to install `Tox`:
$ pip install tox
Also, you will need a running MySQL instance with the database migrations applied from the previous step.
Tests are dependent on pre-created MySQL database for persistence.
Please set env var
$ export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
PEP8 style checks
-----------------
In order to run `PEP8` style checks run following command:
$ tox -e pep8
Functional testing
------------------
In order to run `functional` tests run following command:
$ tox -e py35-functional
Pros:
* lightweight (controllers and DB models testing)
* no OpenStack required
* no IronFunctions required
Cons:
* MySQL server required
* OpenStack authentication is not tested
* IronFunctions API stubbed with fake implementation
Integration integrations
------------------------
Integration tests are dependent on following env variables:
* TEST_DB_URI - similar to functional tests, database endpoint
* FUNCTIONS_API_URL - IronFunctions API URL (default value - `http://localhost:8080/v1`)
* OS_AUTH_URL - OpenStack Identity endpoint
* OS_PROJECT_NAME - OpenStack user-specific project name
* OS_USERNAME - OpenStack user name
* OS_PASSWORD - OpenStack user user password
To run tests use following command:
export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
export FUNCTIONS_API_URL=<functions-api-protocol>://<functions-host>:<functions-port>/<functions-api-version>
export OS_AUTH_URL=<identity-api-protocol>://<identity-host>:<identity-port>/<identity-api-version>
export OS_PROJECT_NAME=<project-name>
export OS_USERNAME=<project-name>
export OS_PASSWORD=<project-name>
tox -epy35-integration
Testing: Docker-build
---------------------
This type of testing allows to ensure if code can be build inside docker container with no problems.
In order to run this check use following commands::
export DOCKER_HOST=tcp://<docker-host>:<docker-port>>
export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
export FUNCTIONS_API_URL=<functions-api-protocol>://<functions-host>:<functions-port>/<functions-api-version>
export OS_AUTH_URL=<identity-api-protocol>://<identity-host>:<identity-port>/<identity-api-version>
tox -e docker-build
During this check Tox:
* builds an image
* deletes all artifacts (Python3.5 image and recently built image)
Testing Docker-full
-------------------
This type of testing allows to ensure if code code can be build and run successfully inside docker container with no problems.
In order to run this check use following commands::
export DOCKER_HOST=tcp://<docker-host>:<docker-port>>
export TEST_DB_URI=mysql://<your-user>:<your-user-password>@<mysql-host>:<mysql-port>/<functions-db>
export FUNCTIONS_API_URL=<functions-api-protocol>://<functions-host>:<functions-port>/<functions-api-version>
export OS_AUTH_URL=<identity-api-protocol>://<identity-host>:<identity-port>/<identity-api-version>
tox -e docker-full
During this check following operations are performed::
* build container from source code
* run container with exposed ports
* request Swagger API doc to see if API is responsive
* tear-down running container
Coverage regression testing
---------------------------
In order to build quality software it is necessary to keep test coverage at its highest point.
So, as part of `Tox` testing new check was added - functional test coverage regression.
In order to run it use following command:
$ tox -e py35-functional-regression
Static code analysis with Bandit
================================
$ tox -e bandit