diff --git a/doc/source/debugging.rst b/doc/source/debugging.rst new file mode 100644 index 00000000..c30ed7b3 --- /dev/null +++ b/doc/source/debugging.rst @@ -0,0 +1,131 @@ +.. debugging: + +================================== +Debugging microservice/application +================================== + +This part of the documentation contains some practice recommendations, which +can be used for debugging some issues in service code. + +Problem description +=================== + +Workable service is perfect, but sometimes user may be in situation, when +application does not work as expected or fails with some unknown status. +Obviously if service can not provide clear traceback or logs, there is no +another option except debug this service. Let's take a look on some useful +how to do it and use heat-engine service as example. + +How to debug +============ + +#. Create a local copy of the source code related project. + + :: + + cd /tmp + git clone http://github.com/openstack/heat + +#. Do all necessary changes, i.e. add breakpoint and etc., in this source code. + +#. Update global configuration file by using local source for heat service. + + :: + + sources: + openstack/heat: + source_dir: /tmp/heat + +#. Build new image and re-deploy heat service: + + :: + + ccp build -c heat-engine + ccp deploy -c heat-engine + +#. Login in container and enjoy debugging. + +.. NOTE:: This approach is really pure for understanding, but has one issue. + If you want to change code again you need to repeat all operations + from building image again. + +Another way to debug +==================== + +The idea is to run new process with necessary changes in code (breakpoints) +inside already created container for current service. Execute +follow commands to run bash in container with heat-engine service: + +:: + + kubectl get pods | grep heat-engine + kubectl exec -it bash + +So now bash is run in container. There are two new issues here: + +#. It's not possible to change service source files, because we are logged + as ``heat`` user. + +#. If heat-engine process be killed, Kubernetes detect it and re-create + container. + +Both issues can be solved by changing Docker image and Service definition. + +- First of all change user ``heat`` used in container to ``root``. It should be + done in file: ``fuel-ccp-heat/docker/heat-engine/Dockerfile.j2``. + +.. NOTE:: There is also alternative way to obtain root access is container: + + #. find node where pod with heat-engine was run + + :: + + kubectl get pods -o wide | grep heat-engine + #. ssh to this node with heat-engine pod + #. find id of container with heat-engine and run bash as root user + + :: + + docker ps | grep heat-engine + docker exec -it -u root bash + +- The next step is to change run command in service definition: + ``fuel-ccp-heat/service/heat-engine.yaml``. Find key word ``command``, + comment it and write follow code: + + :: + + command: sleep 1h + + It will allow to run container for 1 hour without real heat-engine process, + it's usually enough for leisurely debugging. + + +To ship new container to the CCP is necessary to build new image and then +re-deploy it: + +:: + + ccp build -c heat-engine + ccp deploy -c heat-engine + +When re-deploy is finished, run bash in new container again. +The source code is placed in follow directory: + +:: + + /var/lib/microservices/venv/lib/python2.7/site-packages/ + +Change it by adding necessary breakpoints. + +.. NOTE:: Text editor ``vim`` can work incorrect from container. For fixing it + try to execute command: export TERM=xterm + +The last step is to run updated service code. Execute command, which was +commented in service definition file, in the current example it's: + +:: + + heat-engine --config-file /etc/heat/heat.conf + +Now patched service is active and can be used for debugging. diff --git a/doc/source/index.rst b/doc/source/index.rst index 60562c12..205d76fd 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -49,6 +49,7 @@ Developer docs app_def_guide docker dsl + debugging Design docs -----------