diff --git a/README.rst b/README.rst index d88dae9..da8de78 100644 --- a/README.rst +++ b/README.rst @@ -1,155 +1,2 @@ First App Application for OpenStack (faafo) =========================================== - -Workflow --------- - -.. image:: images/diagram.png - -FIXME(berendt): Add new API service and webinterface to the workflow description. - -* The producer generates a random number of tasks with random parameters and a UUID as identifier. -* The producer pushes the generated tasks into the exchange :code:`tasks`. -* The producer inserts a new record for each task into the database (including all parameters and the UUID). -* The producer sleeps for a random number of seconds and will generate more tasks after awakening. -* All messages in the :code:`tasks` exchange will be routed into the :code:`tasks` queue. -* The worker waits for new messages in the :code:`tasks` queue. -* After receiving a message the worker generates an image based on the received parameters and writes the result into a local file (identified by the UUID). -* After writing an image the worker pushes the result (the checksum of the generated image and the duration identified by the UUID) into the exchange :code:`results`. -* All messages in the :code:`results` exchange will be routed into the :code:`results` queue. -* The tracker waits for new messages in the :code:`results` queue. -* After receiving a message the tracker updates the duration and checksum value of the corresponding database record (identified by the UUID). - -Frameworks ----------- - -* http://flask.pocoo.org/ -* http://python-requests.org -* http://www.sqlalchemy.org/ -* https://github.com/celery/kombu -* https://pillow.readthedocs.org/ - -Example image -------------- - -.. image:: images/example.png - -Vagrant environment -------------------- - -The `Vagrant `_ environment and the `Ansible `_ -playbook is used only for local tests and development of the application. - -The installation of Vagrant is described at https://docs.vagrantup.com/v2/installation/index.html. - -The Vagrant plugin `vagrant-hostmanager `_ is required. - -.. code:: - - $ vagrant plugin install vagrant-hostmanager - -To speedup the provisioning you can install the Vagrant plugin `vagrant-cachier `_. - -.. code:: - - $ vagrant plugin install vagrant-cachier - -Bootstrap the Vagrant environment. - -.. code:: - - $ vagrant up - -The RabbitMQ server and the MySQL server are running on the machine :code:`service.` - -There is a machine for each service of the tutorial application: - -* :code:`api` - :code:`vagrant ssh api` - :code:`sh run_api.sh` -* :code:`producer` - :code:`vagrant ssh producer` - :code:`sh run_producer.sh` -* :code:`tracker` - :code:`vagrant ssh tracker` - :code:`sh run_tracker.sh` -* :code:`worker` - :code:`vagrant ssh worker` - :code:`sh run_worker.sh` - -RabbitMQ server -~~~~~~~~~~~~~~~ - -The webinterface of the RabbitMQ server is reachable on TCP port :code:`15672`. The login is -possible with the user :code:`guest` and the password :code:`secretsecret`. - -MySQL server -~~~~~~~~~~~~ - -The password of the user :code:`root` is :code:`secretsecret`. The password of the user :code:`tutorial` -for the database :code:`tutorial` is also :code:`secretsecret`. - -Virtual environment -------------------- - -Create a new virtual environment, install all required dependencies and -the application itself. - -.. code:: - - $ virtualenv .venv - $ source .venv/bin/activate - $ pip install -r requirements.txt - $ python setup.py install - -Now open a new screen or tmux session. Aftwards run the api, worker, producer, and -tracker services in the foreground, each service in a separate window. - -.. code:: - - $ source .venv/bin/activate; faafo-api - $ source .venv/bin/activate; faafo-worker - $ source .venv/bin/activate; faafo-tracker - $ source .venv/bin/activate; faafo-producer - -Example outputs ---------------- - -API Service -~~~~~~~~~~~ - -FIXME(berendt): add output of the API service - -Producer service -~~~~~~~~~~~~~~~~ - -FIXME(berendt): update output (introduction of oslo.logging) - -.. code:: - - 2015-02-12 22:21:42,870 generating 2 task(s) - 2015-02-12 22:21:42,876 generated task: {'width': 728, 'yb': 2.6351683415972076, 'uuid': UUID('66d5f67e-d26d-42fb-9d88-3c3830b4187a'), 'iterations': 395, 'xb': 1.6486035545865234, 'xa': -1.2576814065507933, 'ya': -2.8587178863035616, 'height': 876} - 2015-02-12 22:21:42,897 generated task: {'width': 944, 'yb': 2.981696583462036, 'uuid': UUID('6f873111-8bc2-4d73-9a36-ed49915699c8'), 'iterations': 201, 'xb': 3.530775320058914, 'xa': -3.3511031734533794, 'ya': -0.921920674639712, 'height': 962} - 2015-02-12 22:21:42,927 sleeping for 2.680171 seconds - -Tracker service -~~~~~~~~~~~~~~~ - -FIXME(berendt): update output (introduction of oslo.logging) - -.. code:: - - 2015-02-12 22:20:26,630 processing result be42a131-e4aa-4db5-80d1-1956784f4b81 - 2015-02-12 22:20:26,630 elapsed time 5.749099 seconds - 2015-02-12 22:20:26,631 checksum 7ba5bf955a94f1aa02e5f442869b8db88a5915b7c2fb91ffba74708b8d799c2a - -Worker service -~~~~~~~~~~~~~~ - -FIXME(berendt): update output (introduction of oslo.logging) - -.. code:: - - 2015-02-12 22:20:59,258 processing task 20a00e9e-baec-4045-bc57-2cb9d8d1aa61 - 2015-02-12 22:21:01,506 task 20a00e9e-baec-4045-bc57-2cb9d8d1aa61 processed in 2.246601 seconds - 2015-02-12 22:21:01,553 saved result of task 20a00e9e-baec-4045-bc57-2cb9d8d1aa61 to file /home/vagrant/20a00e9e-baec-4045-bc57-2cb9d8d1aa61.png - 2015-02-12 22:21:01,554 pushed result: {'duration': 2.246600866317749, 'checksum': 'faa0f00a72fac53e02c3eb392c5da8365139e509899e269227e5c27047af6c1f', 'uuid': UUID('20a00e9e-baec-4045-bc57-2cb9d8d1aa61')} - -References ----------- - -* http://en.wikipedia.org/wiki/Julia_set -* http://en.wikipedia.org/wiki/Mandelbrot_set -* http://code.activestate.com/recipes/577120-julia-fractals/ diff --git a/doc/source/conf.py b/doc/source/conf.py new file mode 100644 index 0000000..d78328c --- /dev/null +++ b/doc/source/conf.py @@ -0,0 +1,17 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or +# implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +copyright = u'2015, OpenStack contributors' +master_doc = 'index' +project = u'First App Application for OpenStack' +source_suffix = '.rst' diff --git a/doc/source/development.rst b/doc/source/development.rst new file mode 100644 index 0000000..8d0a769 --- /dev/null +++ b/doc/source/development.rst @@ -0,0 +1,72 @@ +Development +=========== + +Vagrant environment +------------------- + +The `Vagrant `_ environment and the `Ansible `_ +playbook is used only for local tests and development of the application. + +The installation of Vagrant is described at https://docs.vagrantup.com/v2/installation/index.html. + +The Vagrant plugin `vagrant-hostmanager `_ is required. + +.. code:: + + $ vagrant plugin install vagrant-hostmanager + +To speedup the provisioning you can install the Vagrant plugin `vagrant-cachier `_. + +.. code:: + + $ vagrant plugin install vagrant-cachier + +Bootstrap the Vagrant environment. + +.. code:: + + $ vagrant up + +The RabbitMQ server and the MySQL server are running on the machine :code:`service.` + +There is a machine for each service of the tutorial application: + +* :code:`api` - :code:`vagrant ssh api` - :code:`sh run_api.sh` +* :code:`producer` - :code:`vagrant ssh producer` - :code:`sh run_producer.sh` +* :code:`tracker` - :code:`vagrant ssh tracker` - :code:`sh run_tracker.sh` +* :code:`worker` - :code:`vagrant ssh worker` - :code:`sh run_worker.sh` + +RabbitMQ server +~~~~~~~~~~~~~~~ + +The webinterface of the RabbitMQ server is reachable on TCP port :code:`15672`. The login is +possible with the user :code:`guest` and the password :code:`secretsecret`. + +MySQL server +~~~~~~~~~~~~ + +The password of the user :code:`root` is :code:`secretsecret`. The password of the user :code:`tutorial` +for the database :code:`tutorial` is also :code:`secretsecret`. + +Virtual environment +------------------- + +Create a new virtual environment, install all required dependencies and +the application itself. + +.. code:: + + $ virtualenv .venv + $ source .venv/bin/activate + $ pip install -r requirements.txt + $ python setup.py install + +Now open a new screen or tmux session. Aftwards run the api, worker, producer, and +tracker services in the foreground, each service in a separate window. + +.. code:: + + $ source .venv/bin/activate; faafo-api + $ source .venv/bin/activate; faafo-worker + $ source .venv/bin/activate; faafo-tracker + $ source .venv/bin/activate; faafo-producer diff --git a/images/diagram.dot b/doc/source/images/diagram.dot similarity index 100% rename from images/diagram.dot rename to doc/source/images/diagram.dot diff --git a/images/diagram.png b/doc/source/images/diagram.png similarity index 100% rename from images/diagram.png rename to doc/source/images/diagram.png diff --git a/images/dot2png.sh b/doc/source/images/dot2png.sh similarity index 100% rename from images/dot2png.sh rename to doc/source/images/dot2png.sh diff --git a/images/example.png b/doc/source/images/example.png similarity index 100% rename from images/example.png rename to doc/source/images/example.png diff --git a/doc/source/implementation.rst b/doc/source/implementation.rst new file mode 100644 index 0000000..d95013d --- /dev/null +++ b/doc/source/implementation.rst @@ -0,0 +1,11 @@ +Implementation +============== + +Frameworks +---------- + +* http://flask.pocoo.org/ +* http://python-requests.org +* http://www.sqlalchemy.org/ +* https://github.com/celery/kombu +* https://pillow.readthedocs.org/ diff --git a/doc/source/index.rst b/doc/source/index.rst new file mode 100644 index 0000000..f05dd43 --- /dev/null +++ b/doc/source/index.rst @@ -0,0 +1,13 @@ +First App Application for OpenStack +=================================== + +Contents: + +.. toctree:: + :maxdepth: 2 + + workflow + implementation + usage + development + references diff --git a/doc/source/references.rst b/doc/source/references.rst new file mode 100644 index 0000000..6748135 --- /dev/null +++ b/doc/source/references.rst @@ -0,0 +1,6 @@ +References +========== + +* http://en.wikipedia.org/wiki/Julia_set +* http://en.wikipedia.org/wiki/Mandelbrot_set +* http://code.activestate.com/recipes/577120-julia-fractals/ diff --git a/doc/source/usage.rst b/doc/source/usage.rst new file mode 100644 index 0000000..86916cf --- /dev/null +++ b/doc/source/usage.rst @@ -0,0 +1,51 @@ +Usage +===== + +Example image +------------- + +.. image:: images/example.png + + +Example outputs +--------------- + +API Service +~~~~~~~~~~~ + +FIXME(berendt): add output of the API service + +Producer service +~~~~~~~~~~~~~~~~ + +FIXME(berendt): update output (introduction of oslo.logging) + +.. code:: + + 2015-02-12 22:21:42,870 generating 2 task(s) + 2015-02-12 22:21:42,876 generated task: {'width': 728, 'yb': 2.6351683415972076, 'uuid': UUID('66d5f67e-d26d-42fb-9d88-3c3830b4187a'), 'iterations': 395, 'xb': 1.6486035545865234, 'xa': -1.2576814065507933, 'ya': -2.8587178863035616, 'height': 876} + 2015-02-12 22:21:42,897 generated task: {'width': 944, 'yb': 2.981696583462036, 'uuid': UUID('6f873111-8bc2-4d73-9a36-ed49915699c8'), 'iterations': 201, 'xb': 3.530775320058914, 'xa': -3.3511031734533794, 'ya': -0.921920674639712, 'height': 962} + 2015-02-12 22:21:42,927 sleeping for 2.680171 seconds + +Tracker service +~~~~~~~~~~~~~~~ + +FIXME(berendt): update output (introduction of oslo.logging) + +.. code:: + + 2015-02-12 22:20:26,630 processing result be42a131-e4aa-4db5-80d1-1956784f4b81 + 2015-02-12 22:20:26,630 elapsed time 5.749099 seconds + 2015-02-12 22:20:26,631 checksum 7ba5bf955a94f1aa02e5f442869b8db88a5915b7c2fb91ffba74708b8d799c2a + +Worker service +~~~~~~~~~~~~~~ + +FIXME(berendt): update output (introduction of oslo.logging) + +.. code:: + + 2015-02-12 22:20:59,258 processing task 20a00e9e-baec-4045-bc57-2cb9d8d1aa61 + 2015-02-12 22:21:01,506 task 20a00e9e-baec-4045-bc57-2cb9d8d1aa61 processed in 2.246601 seconds + 2015-02-12 22:21:01,553 saved result of task 20a00e9e-baec-4045-bc57-2cb9d8d1aa61 to file /home/vagrant/20a00e9e-baec-4045-bc57-2cb9d8d1aa61.png + 2015-02-12 22:21:01,554 pushed result: {'duration': 2.246600866317749, 'checksum': 'faa0f00a72fac53e02c3eb392c5da8365139e509899e269227e5c27047af6c1f', 'uuid': UUID('20a00e9e-baec-4045-bc57-2cb9d8d1aa61')} diff --git a/doc/source/workflow.rst b/doc/source/workflow.rst new file mode 100644 index 0000000..eafbd74 --- /dev/null +++ b/doc/source/workflow.rst @@ -0,0 +1,18 @@ +Workflow +-------- + +.. image:: images/diagram.png + +FIXME(berendt): Add new API service and webinterface to the workflow description. + +* The producer generates a random number of tasks with random parameters and a UUID as identifier. +* The producer pushes the generated tasks into the exchange :code:`tasks`. +* The producer inserts a new record for each task into the database (including all parameters and the UUID). +* The producer sleeps for a random number of seconds and will generate more tasks after awakening. +* All messages in the :code:`tasks` exchange will be routed into the :code:`tasks` queue. +* The worker waits for new messages in the :code:`tasks` queue. +* After receiving a message the worker generates an image based on the received parameters and writes the result into a local file (identified by the UUID). +* After writing an image the worker pushes the result (the checksum of the generated image and the duration identified by the UUID) into the exchange :code:`results`. +* All messages in the :code:`results` exchange will be routed into the :code:`results` queue. +* The tracker waits for new messages in the :code:`results` queue. +* After receiving a message the tracker updates the duration and checksum value of the corresponding database record (identified by the UUID). diff --git a/setup.cfg b/setup.cfg index 8c53753..5c0dfdc 100644 --- a/setup.cfg +++ b/setup.cfg @@ -31,5 +31,16 @@ console_scripts = faafo-worker = faafo.worker:main faafo-api = faafo.api:main +[build_sphinx] +source-dir = doc/source +build-dir = doc/build +all_files = 1 + +[upload_sphinx] +upload-dir = doc/build/html + [wheel] universal = 1 + +[pbr] +warnerrors = true diff --git a/test-requirements.txt b/test-requirements.txt index 0a5b877..5e68dbb 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -1 +1,2 @@ hacking +sphinx>=1.1.2,!=1.2.0,!=1.3b1,<1.3 diff --git a/tox.ini b/tox.ini index 47212ec..28f1c4d 100644 --- a/tox.ini +++ b/tox.ini @@ -12,6 +12,9 @@ install_command = pip install {opts} {packages} [testenv:venv] commands = {posargs} +[testenv:docs] +commands = python setup.py build_sphinx + [testenv:pep8] commands = flake8 {posargs}