x
/
iotronic-lightning-rod
Code Issues Proposed changes

Refactored Webservice Manager: 

- webservices exposed via subdomanins
- added monitoring wstun tunnels (zombie processes managed)
- "proxies" module moved
- nginx management improved
- requirements updated

Added RPCs in Device Manager

Added "serializers" to Autobahn "Component"
Change-Id: Ie5f780c4cdcf854fd4c8af2d4ef6c3c52f68da10
This commit is contained in:
Nicola Peditto 2018-11-23 11:02:53 +01:00
parent e75cce2c3b
commit 9bf78aa2da
38 changed files with 1289 additions and 1622 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored
View File

@ -12,5 +12,6 @@ ChangeLog
*.md
.eggs
dist
STUFF/
iotronic_lightningrod/modules/test.py
iotronic_lightningrod/modules/vfs_*
iotronic_lightningrod/modules/vfs_*

2
README.rst
View File

@ -18,5 +18,3 @@ Installation guides
* `Raspberry Pi 3 <https://github.com/MDSLab/iotronic-lightning-rod-agent/blob/master/doc/installation/raspberry_pi_3.rst>`_.
* `Ubuntu 16.04 <https://github.com/MDSLab/iotronic-lightning-rod-agent/blob/master/doc/installation/ubuntu1604.rst>`_.
* `Arduino YUN <https://github.com/MDSLab/iotronic-lightning-rod-agent/blob/master/doc/installation/arduino_yun.rst>`_.

0
iotronic_lightningrod/plugins/__init__.py → __init__.py
View File

4
doc/installation/arduino_yun.rst
View File

@ -1,5 +1,5 @@
IoTronic Lightning-rod installation guide for Arduino YUN
=========================================================
[DEPRECATED] IoTronic Lightning-rod installation guide for Arduino YUN
======================================================================
We tested this procedure on a Arduino YUN board with OpenWRT LininoIO image.

86
doc/installation/raspberry_pi_3.rst
View File

@ -1,100 +1,42 @@
IoTronic Lightning-rod installation guide for Raspberry Pi 3
============================================================
We tested this procedure on a Raspberry Pi 3 board.
We tested this procedure on a Raspberry Pi 3 board (Raspbian).
Install from source code
------------------------
Install requirements
~~~~~~~~~~~~~~~~~~~~
::
pip install oslo.config oslo.log asyncio autobahn httplib2 psutil six
Set up environment:
~~~~~~~~~~~~~~~~~~~
::
mkdir -p /var/lib/iotronic
mkdir /var/lib/iotronic/plugins
mkdir /var/log/iotronic/
mkdir /etc/iotronic
Install Lightning-rod
~~~~~~~~~~~~~~~~~~~~~
Get source code
'''''''''''''''
::
cd /var/lib/iotronic
git clone git://github.com/MDSLab/iotronic-lightning-rod-agent.git
mv iotronic-lightning-rod-agent/ iotronic-lightning-rod/
pip3 install iotronic-lightningrod
Deployment
''''''''''
::
lr_install
cd iotronic-lightning-rod/
cp etc/iotronic/iotronic.conf /etc/iotronic/
cp settings.example.json /var/lib/iotronic/settings.json
cp plugins.example.json /var/lib/iotronic/plugins.json
cp services.example.json /var/lib/iotronic/services.json
cp etc/systemd/system/s4t-lightning-rod.service /etc/systemd/system/lightning-rod.service
chmod +x /etc/systemd/system/lightning-rod.service
systemctl daemon-reload
- Edit configuration file:
- nano /var/lib/iotronic/settings.json
::
{
"iotronic": {
"board": {
"token": "<REGISTRATION-TOKEN>"
},
"wamp": {
"registration-agent": {
"url": "ws://<WAMP-SERVER>:<WAMP-PORT>/",
"realm": "<IOTRONIC-REALM>"
}
}
}
}
- setup logrotate:
- nano /etc/logrotate.d/lightning-rod.log
::
/var/log/iotronic/lightning-rod.log {
weekly
rotate = 3
compress
su root root
maxsize 5M
}
Building
''''''''
Iotronic setup
''''''''''''''
::
lr_configure
cd /var/lib/iotronic/iotronic-lightning-rod/
python setup.py install
Arguments required:
<REGISTRATION-TOKEN> : token released by IoTronic registration procedure
<WAMP-REG-AGENT-URL> : IoTronic Crossbar server URL
e.g.
::
lr_configure 000001 ws(s)://<IOTRONIC-CROSSBAR-IP>:<IOTRONIC-CROSSBAR-PORT>/
Execution:
~~~~~~~~~~
::
systemctl restart lightning-rod.service
systemctl start lightning-rod.service
tail -f /var/log/iotronic/lightning-rod.log

85
doc/installation/ubuntu1604.rst
View File

@ -4,98 +4,39 @@ IoTronic Lightning-rod installation guide for Ubuntu 16.04
We tested this procedure on a Ubuntu 16.04 (also within a LXD
container). Everything needs to be run as root.
Install from source code via Git
--------------------------------
Install requirements
~~~~~~~~~~~~~~~~~~~~
::
pip install oslo.config oslo.log asyncio autobahn httplib2 psutil six
Set up environment:
~~~~~~~~~~~~~~~~~~~
::
mkdir -p /var/lib/iotronic
mkdir /var/lib/iotronic/plugins
mkdir /var/log/iotronic/
mkdir /etc/iotronic
Install Lightning-rod
~~~~~~~~~~~~~~~~~~~~~
Get source code
'''''''''''''''
::
cd /var/lib/iotronic
git clone git://github.com/MDSLab/iotronic-lightning-rod-agent.git
mv iotronic-lightning-rod-agent/ iotronic-lightning-rod/
pip3 install iotronic-lightningrod
Deployment
''''''''''
::
lr_install
cd iotronic-lightning-rod/
cp etc/iotronic/iotronic.conf /etc/iotronic/
cp settings.example.json /var/lib/iotronic/settings.json
cp plugins.example.json /var/lib/iotronic/plugins.json
cp services.example.json /var/lib/iotronic/services.json
cp etc/systemd/system/s4t-lightning-rod.service /etc/systemd/system/lightning-rod.service
chmod +x /etc/systemd/system/lightning-rod.service
systemctl daemon-reload
- Edit configuration file:
- nano /var/lib/iotronic/settings.json
::
{
"iotronic": {
"board": {
"token": "<REGISTRATION-TOKEN>"
},
"wamp": {
"registration-agent": {
"url": "ws://<WAMP-SERVER>:<WAMP-PORT>/",
"realm": "<IOTRONIC-REALM>"
}
}
}
}
- setup logrotate:
- nano /etc/logrotate.d/lightning-rod.log
::
/var/log/iotronic/lightning-rod.log {
weekly
rotate = 3
compress
su root root
maxsize 5M
}
Building
''''''''
Iotronic setup
''''''''''''''
::
lr_configure
cd /var/lib/iotronic/iotronic-lightning-rod/
python setup.py install
Arguments required:
<REGISTRATION-TOKEN> : token released by IoTronic registration procedure
<WAMP-REG-AGENT-URL> : IoTronic Crossbar server URL
e.g.
::
lr_configure 000001 ws(s)://<IOTRONIC-CROSSBAR-IP>:<IOTRONIC-CROSSBAR-PORT>/
Execution:
~~~~~~~~~~
::
systemctl restart lightning-rod.service
systemctl start lightning-rod.service
tail -f /var/log/iotronic/lightning-rod.log

2
etc/iotronic/iotronic.conf
View File

@ -4,3 +4,5 @@ skip_cert_verify = True
debug = True
proxy = nginx
log_file = /var/log/iotronic/lightning-rod.log
alive_timer = 600
rpc_alive_timer = 3

1
etc/logrotate.d/lightning-rod.log
View File

@ -1,6 +1,5 @@
/var/log/iotronic/lightning-rod.log /var/log/wstun/wstun.log{
copytruncate
create
missingok
weekly
rotate = 3

57
iotronic_lightningrod/common/exception.py
View File

@ -15,12 +15,13 @@
__author__ = "Nicola Peditto <n.peditto@gmail.com>"
import os
import signal
from oslo_log import log as logging
LOG = logging.getLogger(__name__)
from iotronic_lightningrod.common import utils
def manageTimeout(error_message, action):
try:
@ -29,8 +30,36 @@ def manageTimeout(error_message, action):
except TimeoutError as err:
details = err.args[0]
LOG.warning("Board connection call timeout: " + str(details))
os._exit(1)
if (action == "ws_alive"):
LOG.warning("Iotronic RPC-ALIVE timeout details: " + str(details))
try:
utils.destroyWampSocket()
except Exception as e:
LOG.warning("Iotronic RPC-ALIVE timeout error: " + str(e))
else:
LOG.warning("Board connection call timeout: " + str(details))
utils.LR_restart()
"""
def manageTimeoutALIVE(error_message, action):
try:
raise TimeoutError(error_message, action)
except TimeoutError as err:
details = err.args[0]
LOG.warning("Iotronic RPC-ALIVE timeout details: " + str(details))
try:
utils.destroyWampSocket()
except Exception as e:
LOG.warning("Iotronic RPC-ALIVE timeout error: " + str(e))
"""
class NginxError(Exception):
@ -67,15 +96,31 @@ class timeout(object):
class timeoutRPC(object):
def __init__(self, seconds=1, error_message='Timeout', action=None):
def __init__(self, seconds=1, error_message='Timeout-RPC', action=None):
self.seconds = seconds
self.error_message = error_message
self.action = action
def handle_timeout(self, signum, frame):
manageTimeout(self.error_message, self.action)
def __enter__(self):
signal.signal(signal.SIGALRM, self.handle_timeout)
signal.alarm(self.seconds)
def __exit__(self, type, value, traceback):
signal.alarm(0)
class timeoutALIVE(object):
def __init__(self, seconds=1, error_message='Timeout-Alive', action=None):
self.seconds = seconds
self.error_message = error_message
self.action = action
def handle_timeout(self, signum, frame):
manageTimeout(self.error_message, self.action)
# LOG.warning("RPC timeout: " + str(self.error_message))
# os._exit(1)
def __enter__(self):
signal.signal(signal.SIGALRM, self.handle_timeout)

85
iotronic_lightningrod/common/utils.py Normal file
View File

@ -0,0 +1,85 @@
# Copyright 2018 MDSLAB - University of Messina
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
__author__ = "Nicola Peditto <n.peditto@gmail.com>"
from oslo_log import log as logging
LOG = logging.getLogger(__name__)
import os
import pkg_resources
import psutil
import subprocess
import sys
def LR_restart():
try:
LOG.warning("Lightning-rod RESTARTING...")
python = sys.executable
os.execl(python, python, *sys.argv)
except Exception as err:
LOG.error("Lightning-rod restarting error" + str(err))
def checkIotronicConf(lr_CONF):
try:
if(lr_CONF.log_file == None):
LOG.warning("'log_file' is not specified!")
return False
else:
print("View logs in " + lr_CONF.log_file)
return True
except Exception as err:
print(err)
return False
def destroyWampSocket():
LR_PID = os.getpid()
try:
process = subprocess.Popen(
["gdb", "-p", str(LR_PID)],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE
)
proc = psutil.Process()
print("WAMP RECOVERY: " + str(proc.connections()[0]))
ws_fd = proc.connections()[0].fd
first = b"call shutdown("
fd = str(ws_fd).encode('ascii')
last = b"u,0)\nquit\ny"
commands = b"%s%s%s" % (first, fd, last)
process.communicate(input=commands)[0]
msg = "Websocket-Zombie closed! Restoring..."
LOG.warning(msg)
print(msg)
except Exception as e:
LOG.warning("RPC-ALIVE - destroyWampSocket error: " + str(e))
def get_version(package):
package = package.lower()
return next((p.version for p in pkg_resources.working_set if
p.project_name.lower() == package), "No version")

1135
iotronic_lightningrod/lightningrod.py
View File

File diff suppressed because it is too large Load Diff

73
iotronic_lightningrod/modules/device_manager.py
View File

@ -19,6 +19,9 @@ import importlib as imp
import inspect
import os
import subprocess
import sys
import threading
import time
from datetime import datetime
@ -27,6 +30,7 @@ from iotronic_lightningrod.lightningrod import RPC_devices
from iotronic_lightningrod.lightningrod import SESSION
from iotronic_lightningrod.modules import Module
from iotronic_lightningrod.modules import utils
import iotronic_lightningrod.wampmessage as WM
from oslo_log import log as logging
@ -85,7 +89,10 @@ class DeviceManager(Module.Module):
if (meth[0] != "__init__") & (meth[0] != "finalize"):
# LOG.info(" - " + str(meth[0]))
rpc_addr = u'iotronic.' + board.uuid + '.' + meth[0]
# rpc_addr = u'iotronic.' + board.uuid + '.' + meth[0]
rpc_addr = u'iotronic.' + str(board.session_id) + '.' + \
board.uuid + '.' + meth[0]
# LOG.debug(" --> " + str(rpc_addr))
SESSION.register(meth[1], rpc_addr)
@ -94,23 +101,50 @@ class DeviceManager(Module.Module):
async def DevicePing(self):
rpc_name = utils.getFuncName()
LOG.info("RPC " + rpc_name + " CALLED")
return datetime.now().strftime('%Y-%m-%dT%H:%M:%S.%f')
message = datetime.now().strftime('%Y-%m-%dT%H:%M:%S.%f')
w_msg = WM.WampSuccess(message)
return w_msg.serialize()
async def DeviceReboot(self):
rpc_name = utils.getFuncName()
LOG.info("RPC " + rpc_name + " CALLED")
command = "reboot"
subprocess.call(command, shell=True)
def delayBoardReboot():
time.sleep(3)
subprocess.call("reboot", shell=True)
return datetime.now().strftime('%Y-%m-%dT%H:%M:%S.%f')
threading.Thread(target=delayBoardReboot).start()
message = "Rebooting board in few seconds @" + \
str(datetime.now().strftime('%Y-%m-%dT%H:%M:%S.%f'))
w_msg = WM.WampSuccess(message)
return w_msg.serialize()
async def DeviceRestartLR(self):
rpc_name = utils.getFuncName()
LOG.info("RPC " + rpc_name + " CALLED")
def delayLRrestarting():
time.sleep(2)
python = sys.executable
os.execl(python, python, *sys.argv)
threading.Thread(target=delayLRrestarting).start()
message = "Restarting LR in 5 seconds (" + \
datetime.now().strftime('%Y-%m-%dT%H:%M:%S.%f') + ")..."
w_msg = WM.WampSuccess(message)
return w_msg.serialize()
async def DeviceHostname(self):
rpc_name = utils.getFuncName()
LOG.info("RPC " + rpc_name + " CALLED")
command = "hostname"
# subprocess.call(command, shell=True)
out = subprocess.Popen(
command,
@ -119,23 +153,28 @@ class DeviceManager(Module.Module):
)
output = out.communicate()[0].decode('utf-8').strip()
print(output)
return str(output) + "@" + \
message = str(output) + "@" + \
str(datetime.now().strftime('%Y-%m-%dT%H:%M:%S.%f'))
w_msg = WM.WampSuccess(message)
"""
async def DeviceWampDisconnect(self):
return w_msg.serialize()
async def DeviceNetConfig(self):
rpc_name = utils.getFuncName()
LOG.info("RPC " + rpc_name + " CALLED")
import threading, time
command = "ifconfig"
def delayDisconnection():
time.sleep(5)
SESSION.disconnect()
out = subprocess.Popen(
command,
shell=True,
stdout=subprocess.PIPE
)
threading.Thread(target=delayDisconnection).start()
output = out.communicate()[0].decode('utf-8').strip()
return "Device disconnection in 5 seconds..."
"""
message = str(output)
w_msg = WM.WampSuccess(message)
return w_msg.serialize()

13
iotronic_lightningrod/modules/network_manager.py
View File

@ -42,19 +42,6 @@ class NetworkManager(Module.Module):
def restore(self):
pass
async def test_function(self):
import random
s = random.uniform(0.5, 1.5)
await asyncio.sleep(s)
result = "DEVICE test result: TEST!"
LOG.info(result)
return result
async def add(self, x, y):
c = x + y
LOG.info("DEVICE add result: " + str(c))
return c
async def Create_VIF(self, r_tcp_port):
LOG.info("Creation of the VIF ")

3
iotronic_lightningrod/modules/plugin_manager.py
View File

@ -26,8 +26,8 @@ import time
from iotronic_lightningrod.modules import Module
from iotronic_lightningrod.modules.plugins import PluginSerializer
from iotronic_lightningrod.modules import utils
from iotronic_lightningrod.plugins import PluginSerializer
import iotronic_lightningrod.wampmessage as WM
@ -166,7 +166,6 @@ class PluginManager(Module.Module):
if os.path.exists(plugin_filename):
# task = imp.load_source("plugin", plugin_filename)
task = imp.machinery.SourceFileLoader(
"plugin",
plugin_filename

0
iotronic_lightningrod/plugins/Plugin.py → iotronic_lightningrod/modules/plugins/Plugin.py
View File

0
iotronic_lightningrod/plugins/PluginSerializer.py → iotronic_lightningrod/modules/plugins/PluginSerializer.py
View File

0
iotronic_lightningrod/proxies/__init__.py → iotronic_lightningrod/modules/plugins/__init__.py
View File

0
iotronic_lightningrod/plugins/pluginApis.py → iotronic_lightningrod/modules/plugins/pluginApis.py
View File

0
iotronic_lightningrod/plugins/plugins_examples/arduino_yun/demo.py → iotronic_lightningrod/modules/plugins/plugins_examples/arduino_yun/demo.py
View File

0
iotronic_lightningrod/plugins/plugins_examples/arduino_yun/led.json → iotronic_lightningrod/modules/plugins/plugins_examples/arduino_yun/led.json
View File

0
iotronic_lightningrod/plugins/plugins_examples/arduino_yun/led.py → iotronic_lightningrod/modules/plugins/plugins_examples/arduino_yun/led.py
View File

0
iotronic_lightningrod/plugins/plugins_examples/generics/echo.json → iotronic_lightningrod/modules/plugins/plugins_examples/generics/echo.json
View File

4
iotronic_lightningrod/plugins/plugins_examples/generics/echo.py → iotronic_lightningrod/modules/plugins/plugins_examples/generics/echo.py
View File

@ -13,7 +13,7 @@
# License for the specific language governing permissions and limitations
# under the License.
from iotronic_lightningrod.plugins import Plugin
from iotronic_lightningrod.modules.plugins import Plugin
from oslo_log import log as logging
LOG = logging.getLogger(__name__)
@ -29,4 +29,4 @@ class Worker(Plugin.Plugin):
def run(self):
LOG.info("Input parameters: " + str(self.params['name']))
LOG.info("Plugin " + self.name + " process completed!")
self.q_result.put("ECHO RESULT: "+str(self.params['name']))
self.q_result.put("ECHO RESULT: " + str(self.params['name']))

0
iotronic_lightningrod/plugins/plugins_examples/generics/runner.json → iotronic_lightningrod/modules/plugins/plugins_examples/generics/runner.json
View File

4
iotronic_lightningrod/plugins/plugins_examples/generics/runner.py → iotronic_lightningrod/modules/plugins/plugins_examples/generics/runner.py
View File

@ -13,8 +13,8 @@
# License for the specific language governing permissions and limitations
# under the License.
from iotronic_lightningrod.plugins import Plugin
# from iotronic_lightningrod.plugins import pluginApis as API
from iotronic_lightningrod.modules.plugins import Plugin
# from iotronic_lightningrod.modules.plugins import pluginApis as API
from oslo_log import log as logging
LOG = logging.getLogger(__name__)

0
iotronic_lightningrod/proxies/Proxy.py → iotronic_lightningrod/modules/proxies/Proxy.py
View File

0
iotronic_lightningrod/modules/proxies/__init__.py Normal file
View File

0
iotronic_lightningrod/proxies/configs/default → iotronic_lightningrod/modules/proxies/configs/default
View File

0
iotronic_lightningrod/proxies/configs/iotronic → iotronic_lightningrod/modules/proxies/configs/iotronic
View File

182
iotronic_lightningrod/proxies/nginx.py → iotronic_lightningrod/modules/proxies/nginx.py
View File

@ -16,7 +16,7 @@
__author__ = "Nicola Peditto <n.peditto@gmail.com>"
from iotronic_lightningrod.proxies import Proxy
from iotronic_lightningrod.modules.proxies import Proxy
from oslo_log import log as logging
LOG = logging.getLogger(__name__)
@ -24,7 +24,6 @@ LOG = logging.getLogger(__name__)
import json
import os
import site
import subprocess
import time
@ -162,61 +161,49 @@ class ProxyManager(Proxy.Proxy):
return json.dumps(nginxMsg)
def _proxyBoardDnsSetup(self, board_dns, owner_email):
def _nginx_conf_verify(self, fp):
with open(fp, "r") as text_file:
LOG.debug(text_file.read())
def _proxyEnableWebService(self, board_dns, owner_email):
nginxMsg = {}
try:
py_dist_pack = site.getsitepackages()[0]
nginx_path = "/etc/nginx/conf.d/"
iotronic_nginx_path = "/etc/nginx/conf.d/iotronic"
iotronic_nginx_default = "/etc/nginx/conf.d/iotronic/default"
nginx_board_conf_file = nginx_path + "/" + board_dns + ".conf"
nginx_board_conf = '''server {{
listen 80;
server_name {0};
}}
'''.format(board_dns)
if not os.path.exists(iotronic_nginx_path):
os.makedirs(iotronic_nginx_path)
nginx_default = '''proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";'''
with open(iotronic_nginx_default, "w") as text_file:
text_file.write("%s" % nginx_default)
iotronic_nginx_avl_path = "/etc/nginx/sites-available/iotronic"
string = '''server {{
listen 80;
server_name {0};
include conf.d/iotronic/*;
}}'''.format(board_dns)
with open(iotronic_nginx_avl_path, "w") as text_file:
text_file.write("%s" % string)
os.system(
'ln -s '
'/etc/nginx/sites-available/iotronic '
'/etc/nginx/sites-enabled/'
)
with open(nginx_board_conf_file, "w") as text_file:
text_file.write("%s" % nginx_board_conf)
self._nginx_conf_verify(nginx_board_conf_file)
time.sleep(3)
self._proxyReload()
time.sleep(3)
command = '/usr/bin/certbot -n ' \
'--redirect --authenticator webroot ' \
'--installer nginx -w /var/www/html/ ' \
'--domain ' + board_dns + ' --agree-tos ' \
'--email ' + owner_email
command = "/usr/bin/certbot -n " \
"--redirect " \
"--authenticator webroot " \
"--installer nginx " \
"-w /var/www/html/ " \
"--domain " + board_dns + " " \
"--agree-tos " \
"--email " + owner_email
LOG.debug(command)
call(command, shell=True)
certbot_result = call(command, shell=True)
LOG.info("CERTBOT RESULT: " + str(certbot_result))
nginxMsg['result'] = "SUCCESS"
nginxMsg['message'] = "Webservice module enabled."
LOG.info("--> " + nginxMsg['message'])
except Exception as err:
nginxMsg['log'] = "NGINX DNS setup error: " + str(err)
@ -225,99 +212,142 @@ class ProxyManager(Proxy.Proxy):
return json.dumps(nginxMsg)
def _exposeWebservice(self, service_name, local_port):
def _exposeWebservice(self, board_dns, service_dns, local_port, dns_list):
nginxMsg = {}
try:
nginx_path = "/etc/nginx/conf.d/iotronic"
nginx_path = "/etc/nginx/conf.d"
if not os.path.exists(nginx_path):
os.makedirs(nginx_path)
service_path = nginx_path + "/" + service_dns + ".conf"
string = '''server {{
listen 80;
server_name {0};
fp = nginx_path + "/" + service_name
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
string = '''location /{0}/ {{
proxy_pass http://localhost:{1}/;
include conf.d/iotronic/default;
location / {{
proxy_pass http://localhost:{1};
}}
location /{0} {{
rewrite ^ $scheme://$http_host/{0}/ redirect;
}}
'''.format(service_name, local_port)
'''.format(service_dns, local_port)
with open(fp, "w") as ws_nginx_conf:
with open(service_path, "w") as ws_nginx_conf:
ws_nginx_conf.write("%s" % string)
time.sleep(3)
nginxMsg['message'] = "Webservice '" + service_name + \
"' configuration injected in NGINX."
nginxMsg['result'] = "SUCCESS"
LOG.info("--> " + nginxMsg['message'])
self._nginx_conf_verify(service_path)
self._proxyReload()
time.sleep(3)
command = "/usr/bin/certbot " \
"--expand -n " \
"--redirect " \
"--authenticator webroot " \
"--installer nginx -w /var/www/html/ " \
"--domain " + str(dns_list)
command = "/usr/bin/certbot " \
"-n " \
"--redirect " \
"--authenticator webroot " \
"--installer nginx -w /var/www/html/ " \
"--cert-name " + str(board_dns) + " " \
"--domain " + str(dns_list)
LOG.debug(command)
certbot_result = call(command, shell=True)
LOG.info("CERTBOT RESULT: " + str(certbot_result))
LOG.info("Webservices list updated:\n" +
str(self._webserviceList()))
nginxMsg['result'] = "SUCCESS"
nginxMsg['message'] = "Webservice '" + service_dns + \
"' exposed in NGINX."
LOG.info(nginxMsg['message'])
except Exception as e:
nginxMsg['message'] = "Error exposing Webservice '" + \
service_name + \
service_dns + \
"' configuration in NGINX: {}".format(e)
nginxMsg['result'] = "ERROR"
LOG.warning("--> " + nginxMsg['message'])
return json.dumps(nginxMsg)
def _disableWebservice(self, service_name):
def _disableWebservice(self, service_dns, dns_list):
"""
:param service:
:param dns_list:
:return:
"""
nginxMsg = {}
try:
nginx_path = "/etc/nginx/conf.d/iotronic"
service_path = nginx_path + "/" + service_name
nginx_path = "/etc/nginx/conf.d"
service_path = nginx_path + "/" + service_dns + ".conf"
if os.path.exists(service_path):
os.remove(service_path)
time.sleep(3)
nginxMsg['message'] = "webservice '" \
+ service_name + "' disabled."
nginxMsg['result'] = "SUCCESS"
# LOG.info("--> " + nginxMsg['message'])
time.sleep(1)
self._proxyReload()
time.sleep(3)
command = "/usr/bin/certbot " \
"--expand -n " \
"--redirect " \
"--authenticator webroot " \
"--installer nginx -w /var/www/html/ " \
"--domain " + str(dns_list)
LOG.debug(command)
certbot_result = call(command, shell=True)
LOG.info("CERTBOT RESULT: " + str(certbot_result))
LOG.info("Webservices list updated:\n" + str(
self._webserviceList()))
nginxMsg['message'] = "webservice '" \
+ service_dns + "' disabled."
nginxMsg['result'] = "SUCCESS"
LOG.info(nginxMsg['message'])
else:
nginxMsg['message'] = "webservice file " \
+ service_path + " does not exist"
nginxMsg['result'] = "ERROR"
# LOG.info("--> " + nginxMsg['message'])
except Exception as e:
nginxMsg['message'] = "Error disabling Webservice '" + \
service_name + "': {}".format(e)
service_dns + "': {}".format(e)
nginxMsg['result'] = "ERROR"
# LOG.warning("--> " + nginxMsg['message'])
return json.dumps(nginxMsg)
def _webserviceList(self):
nginx_path = "/etc/nginx/conf.d/iotronic"
nginx_path = "/etc/nginx/conf.d/"
if os.path.exists(nginx_path):
service_list = [f for f in os.listdir(nginx_path)
if os.path.isfile(os.path.join(nginx_path, f))]
service_list.remove('default')
else:
service_list = []

303
iotronic_lightningrod/modules/service_manager.py
View File

@ -15,13 +15,17 @@
__author__ = "Nicola Peditto <n.peditto@gmail.com>"
from datetime import datetime
import errno
import json
import os
import psutil
import signal
import subprocess
import time
import traceback
from datetime import datetime
from threading import Thread
from urllib.parse import urlparse
from iotronic_lightningrod.modules import Module
@ -29,6 +33,9 @@ from iotronic_lightningrod.modules import utils
import iotronic_lightningrod.wampmessage as WM
from iotronic_lightningrod import lightningrod
from oslo_config import cfg
from oslo_log import log as logging
LOG = logging.getLogger(__name__)
@ -70,6 +77,9 @@ class ServiceManager(Module.Module):
if (p.name() == "node" and "wstun" in p.cmdline()[1]):
wstun_process_list.append(p)
if len(services_conf) != 0:
print("\nWSTUN processes:")
for service_uuid in services_conf['services']:
service_name = services_conf['services'][service_uuid]['name']
@ -85,11 +95,22 @@ class ServiceManager(Module.Module):
"' already exists; killing...")
# 1. Kill wstun process (if exists)
# No zombie alert activation
lightningrod.zombie_alert = False
LOG.debug(
"[WSTUN-RESTORE] - "
"on-finalize zombie_alert: " +
str(lightningrod.zombie_alert)
)
try:
os.kill(service_pid, signal.SIGKILL)
os.kill(service_pid, signal.SIGINT)
print("OLD WSTUN KILLED: " + str(wp))
LOG.info(" --> service '" + service_name
+ "' with PID " + str(service_pid)
+ " was killed; creating new one...")
except OSError:
LOG.warning(" - WSTUN process already killed, "
"creating new one...")
@ -102,7 +123,7 @@ class ServiceManager(Module.Module):
local_port = \
services_conf['services'][service_uuid]['local_port']
wstun = self._startWstun(public_port, local_port)
wstun = self._startWstun(public_port, local_port, event="boot")
if wstun != None:
service_pid = wstun.pid
@ -123,9 +144,144 @@ class ServiceManager(Module.Module):
+ " service tunnel!"
LOG.error(" - " + message)
signal.signal(signal.SIGCHLD, self._zombie_hunter)
# Reactivate zombies monitoring
if not lightningrod.zombie_alert:
lightningrod.zombie_alert = True
else:
LOG.info(" --> No service tunnels to establish.")
signal.signal(signal.SIGCHLD, self._zombie_hunter)
def _zombie_hunter(self, signum, frame):
wstun_found = False
if (lightningrod.zombie_alert):
# print(signum); traceback.print_stack(frame)
zombie_list = []
for p in psutil.process_iter():
if len(p.cmdline()) == 0:
if ((p.name() == "node") and
(p.status() == psutil.STATUS_ZOMBIE)):
print(" - process: " + str(p))
zombie_list.append(p.pid)
if len(zombie_list) == 0:
# print(" - no action required.")
return
print("\nCheck killed process...")
print(" - Zombies found: " + str(zombie_list))
# Load services.json configuration file
services_conf = self._loadServicesConf()
for service_uuid in services_conf['services']:
service_pid = services_conf['services'][service_uuid]['pid']
if service_pid in zombie_list:
message = "WSTUN zombie process ALERT!"
print(" - " + str(message))
LOG.debug("[WSTUN-RESTORE] --> " + str(message))
wstun_found = True
print(services_conf['services'][service_uuid])
service_public_port = \
services_conf['services'][service_uuid]['public_port']
service_local_port = \
services_conf['services'][service_uuid]['local_port']
service_name = \
services_conf['services'][service_uuid]['name']
try:
wstun = self._startWstun(
service_public_port,
service_local_port,
event="zombie"
)
if wstun != None:
service_pid = wstun.pid
# UPDATE services.json file
services_conf['services'][service_uuid]['pid'] = \
service_pid
services_conf['services'][service_uuid][
'updated_at'] = \
datetime.now().strftime('%Y-%m-%dT%H:%M:%S.%f')
self._updateServiceConf(services_conf,
service_uuid,
output=True)
message = "Zombie service " + str(service_name) \
+ " restored on port " \
+ str(service_public_port) \
+ " on " + self.wstun_ip
LOG.info(" - " + message + " with PID " + str(
service_pid))
except Exception:
pass
break
if not wstun_found:
message = "Tunnel killed by LR"
print(" - " + str(message))
# LOG.debug("[WSTUN-RESTORE] --> " + str(message))
else:
print("WSTUN kill event:")
message = "Tunnel killed by LR"
print(" - " + str(message))
# LOG.debug("[WSTUN-RESTORE] --> " + str(message))
# lightningrod.zombie_alert = True
def _restoreWSTUN(self, services_conf, service_uuid):
service_name = services_conf['services'][service_uuid]['name']
service_pid = services_conf['services'][service_uuid]['pid']
LOG.info(" - " + service_name)
# Create the reverse tunnel again
public_port = \
services_conf['services'][service_uuid]['public_port']
local_port = \
services_conf['services'][service_uuid]['local_port']
wstun = self._startWstun(public_port, local_port, event="restore")
if wstun != None:
service_pid = wstun.pid
# 3. Update services.json file
services_conf['services'][service_uuid]['pid'] = \
service_pid
services_conf['services'][service_uuid]['updated_at'] = \
datetime.now().strftime('%Y-%m-%dT%H:%M:%S.%f')
self._updateServiceConf(
services_conf,
service_uuid,
output=True
)
LOG.info(" --> Cloud service '" + service_name
+ "' tunnel restored.")
else:
message = "Error spawning " + str(service_name) \
+ " service tunnel!"
LOG.error(" - " + message)
def restore(self):
LOG.info("Cloud service tunnels to restore:")
@ -136,58 +292,38 @@ class ServiceManager(Module.Module):
wstun_process_list = []
# No zombie alert activation
lightningrod.zombie_alert = False
LOG.debug("[WSTUN-RESTORE] - Restore zombie_alert: " + str(
lightningrod.zombie_alert))
# Collect all alive WSTUN proccesses
for p in psutil.process_iter():
if len(p.cmdline()) != 0:
if (p.name() == "node") and ("wstun" in p.cmdline()[1]):
if (p.name() == "node"):
if (p.status() == psutil.STATUS_ZOMBIE):
LOG.warning("WSTUN ZOMBIE: " + str(p))
wstun_process_list.append(p)
elif ("wstun" in p.cmdline()[1]):
LOG.warning("WSTUN ALIVE: " + str(p))
wstun_process_list.append(p)
psutil.Process(p.pid).kill()
LOG.warning(" --> PID " + str(p.pid) + " killed!")
LOG.debug("[WSTUN-RESTORE] - WSTUN processes to restore:\n"
+ str(wstun_process_list))
for service_uuid in services_conf['services']:
service_name = services_conf['services'][service_uuid]['name']
service_pid = services_conf['services'][service_uuid]['pid']
LOG.info(" - " + service_name)
Thread(
target=self._restoreWSTUN,
args=(services_conf, service_uuid,)
).start()
time.sleep(2)
s_alive = False
# WSTUN is still alive
if len(wstun_process_list) != 0:
for wp in wstun_process_list:
if service_pid == wp.pid:
LOG.warning(" --> the tunnel for '" + service_name
+ "' is still established.")
s_alive = True
break
if not s_alive:
# Create the reverse tunnel again
public_port = services_conf['services'][service_uuid]
['public_port']
local_port = services_conf['services'][service_uuid]
['local_port']
wstun = self._startWstun(public_port, local_port)
if wstun != None:
service_pid = wstun.pid
# 3. Update services.json file
services_conf['services'][service_uuid]['pid'] = \
service_pid
services_conf['services'][service_uuid]['updated_at'] = \
datetime.now().strftime('%Y-%m-%dT%H:%M:%S.%f')
self._updateServiceConf(services_conf,
service_uuid, output=True)
LOG.info(" --> Cloud service '" + service_name
+ "' tunnel restored.")
else:
message = "Error spawning " + str(service_name) \
+ " service tunnel!"
LOG.error(" - " + message)
# Reactivate zombies monitoring
if not lightningrod.zombie_alert:
lightningrod.zombie_alert = True
else:
LOG.info(" --> No service tunnels to restore.")
@ -211,16 +347,24 @@ class ServiceManager(Module.Module):
return services_conf
def _startWstun(self, public_port, local_port):
def _startWstun(self, public_port, local_port, event="no-set"):
opt_reverse = "-r" + str(
public_port) + ":127.0.0.1:" + str(local_port)
opt_reverse = "-r" + str(public_port) + ":127.0.0.1:" + str(local_port)
try:
wstun = subprocess.Popen(
['/usr/bin/wstun', opt_reverse, self.wstun_url],
stdout=subprocess.PIPE
)
if(event != "boot"):
print("WSTUN start event:")
cmd_print = 'WSTUN exec: /usr/bin/wstun ' \
+ opt_reverse + ' ' + self.wstun_url
print(" - " + str(cmd_print))
LOG.debug(cmd_print)
except Exception as err:
LOG.error("Error spawning WSTUN process: " + str(err))
wstun = None
@ -254,13 +398,13 @@ class ServiceManager(Module.Module):
try:
wstun = self._startWstun(public_port, local_port)
wstun = self._startWstun(public_port, local_port, event="enable")
if wstun != None:
service_pid = wstun.pid
LOG.debug(" - WSTUN stdout: " + str(wstun.stdout))
# LOG.debug(" - WSTUN stdout: " + str(wstun.stdout))
# Update services.json file
# Load services.json configuration file
@ -338,6 +482,16 @@ class ServiceManager(Module.Module):
try:
# No zombie alert activation
lightningrod.zombie_alert = False
"""
LOG.debug(
"[WSTUN-RESTORE] - disable-RPC zombie_alert: "
+ str(lightningrod.zombie_alert)
)
"""
os.kill(service_pid, signal.SIGKILL)
message = "Cloud service '" \
@ -349,6 +503,11 @@ class ServiceManager(Module.Module):
output=False)
LOG.info(" - " + message)
# Reactivate zombies monitoring
if not lightningrod.zombie_alert:
lightningrod.zombie_alert = True
w_msg = WM.WampSuccess(message)
except Exception as err:
@ -362,6 +521,10 @@ class ServiceManager(Module.Module):
self._updateServiceConf(services_conf, service_uuid,
output=False)
# Reactivate zombies monitoring
if not lightningrod.zombie_alert:
lightningrod.zombie_alert = True
w_msg = WM.WampWarning(message)
else:
@ -369,6 +532,11 @@ class ServiceManager(Module.Module):
message = "Error disabling '" + str(
service_name) + "' service tunnel: " + str(err)
LOG.error(" - " + message)
# Reactivate zombies monitoring
if not lightningrod.zombie_alert:
lightningrod.zombie_alert = True
w_msg = WM.WampError(message)
else:
@ -407,6 +575,14 @@ class ServiceManager(Module.Module):
try:
# No zombie alert activation
lightningrod.zombie_alert = False
"""
LOG.debug(
"[WSTUN-RESTORE] - restore-RPC lightningrod.zombie_alert: "
+ str(lightningrod.zombie_alert))
"""
# 1. Kill wstun process (if exists)
try:
os.kill(service_pid, signal.SIGKILL)
@ -418,7 +594,11 @@ class ServiceManager(Module.Module):
"creating new one...")
# 2. Create the reverse tunnel
wstun = self._startWstun(public_port, local_port)
wstun = self._startWstun(
public_port,
local_port,
event="kill-restore"
)
if wstun != None:
service_pid = wstun.pid
@ -437,12 +617,21 @@ class ServiceManager(Module.Module):
+ str(public_port) + " on " + self.wstun_ip
LOG.info(" - " + message + " with PID " + str(service_pid))
# Reactivate zombies monitoring
if not lightningrod.zombie_alert:
lightningrod.zombie_alert = True
w_msg = WM.WampSuccess(message)
else:
message = "Error spawning " + str(service_name) \
+ " service tunnel!"
LOG.error(" - " + message)
# Reactivate zombies monitoring
if not lightningrod.zombie_alert:
lightningrod.zombie_alert = True
w_msg = WM.WampError(message)
except Exception as err:
@ -455,7 +644,11 @@ class ServiceManager(Module.Module):
local_port = service['port']
wstun = self._startWstun(public_port, local_port)
wstun = self._startWstun(
public_port,
local_port,
event="clean-restore"
)
if wstun != None:

43
iotronic_lightningrod/modules/test.py
View File

@ -1,43 +0,0 @@
# Copyright 2017 MDSLAB - University of Messina
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
__author__ = "Nicola Peditto <n.peditto@gmail.com>"
import asyncio
from iotronic_lightningrod.modules import Module
from oslo_log import log as logging
LOG = logging.getLogger(__name__)
class Test(Module.Module):
def __init__(self, board):
super(Test, self).__init__("Test", board)
async def test_function(self):
import random
s = random.uniform(0.5, 1.5)
await asyncio.sleep(s)
result = "DEVICE test result: TEST!"
LOG.info(result)
return result
async def add(self, x, y):
c = x + y
LOG.info("DEVICE add result: " + str(c))
return c

164
iotronic_lightningrod/modules/vfs_library.py
View File

@ -1,164 +0,0 @@
# Copyright 2017 MDSLAB - University of Messina
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
__author__ = "Nicola Peditto <n.peditto@gmail.com>"
import errno
from fuse import FuseOSError
import os
# Logging conf
from oslo_log import log as logging
LOG = logging.getLogger(__name__)
class FuseLib(object):
def __init__(self, mountSource):
self.mountSource = mountSource
def _full_path(self, partial):
if partial.startswith("/"):
partial = partial[1:]
path = os.path.join(self.mountSource, partial)
print(path)
return path
# Filesystem methods
# ==================
def access(self, path, mode):
full_path = self._full_path(path)
if not os.access(full_path, mode):
raise FuseOSError(errno.EACCES)
def chmod(self, path, mode):
full_path = self._full_path(path)
return os.chmod(full_path, mode)
def chown(self, path, uid, gid):
full_path = self._full_path(path)
return os.chown(full_path, uid, gid)
def getattr(self, path, fh=None):
full_path = self._full_path(path)
st = os.lstat(full_path)
attr = dict((key, getattr(st, key))
for key in (
'st_atime',
'st_ctime',
'st_gid',
'st_mode',
'st_mtime',
'st_nlink',
'st_size',
'st_uid'
)
)
return attr
def readdir(self, path, fh):
full_path = self._full_path(path)
dirents = ['.', '..']
if os.path.isdir(full_path):
dirents.extend(os.listdir(full_path))
for r in dirents:
yield r
def readlink(self, path):
pathname = os.readlink(self._full_path(path))
if pathname.startswith("/"):
# Path name is absolute, sanitize it.
return os.path.relpath(pathname, self.mountSource)
else:
return pathname
def mknod(self, path, mode, dev):
return os.mknod(self._full_path(path), mode, dev)
def rmdir(self, path):
full_path = self._full_path(path)
return os.rmdir(full_path)
def mkdir(self, path, mode):
return os.mkdir(self._full_path(path), mode)
def statfs(self, path):
full_path = self._full_path(path)
stv = os.statvfs(full_path)
stat = dict((key, getattr(stv, key))
for key in ('f_bavail',
'f_bfree',
'f_blocks',
'f_bsize',
'f_favail',
'f_ffree',
'f_files',
'f_flag',
'f_frsize',
'f_namemax'
)
)
return stat
def unlink(self, path):
return os.unlink(self._full_path(path))
def symlink(self, name, target):
return os.symlink(name, self._full_path(target))
def rename(self, old, new):
return os.rename(self._full_path(old), self._full_path(new))
def link(self, target, name):
return os.link(self._full_path(target), self._full_path(name))
def utimens(self, path, times=None):
return os.utime(self._full_path(path), times)
# File methods
# ============
def open(self, path, flags):
full_path = self._full_path(path)
return os.open(full_path, flags)
def create(self, path, mode, fi=None):
full_path = self._full_path(path)
return os.open(full_path, os.O_WRONLY | os.O_CREAT, mode)
def read(self, path, length, offset, fh):
os.lseek(fh, offset, os.SEEK_SET)
return os.read(fh, length)
def write(self, path, buf, offset, fh):
os.lseek(fh, offset, os.SEEK_SET)
return os.write(fh, buf)
def truncate(self, path, length, fh=None):
full_path = self._full_path(path)
with open(full_path, 'r+') as f:
f.truncate(length)
def flush(self, path, fh):
return os.fsync(fh)
def release(self, path, fh):
return os.close(fh)
def fsync(self, path, fdatasync, fh):
return self.flush(path, fh)

509
iotronic_lightningrod/modules/vfs_manager.py
View File

@ -1,509 +0,0 @@
# Copyright 2017 MDSLAB - University of Messina
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from __future__ import with_statement
__author__ = "Nicola Peditto <n.peditto@gmail.com>"
import errno
import os
from subprocess import call
import threading
# Iotronic imports
from iotronic_lightningrod.modules import Module
# Fuse imports
import ctypes
import ctypes.util
from fuse import FUSE
from fuse import FuseOSError
from fuse import Operations
# Logging conf
from oslo_log import log as logging
LOG = logging.getLogger(__name__)
class VfsManager(Module.Module):
def __init__(self, board, session):
super(VfsManager, self).__init__("VFS", board)
self.session = session
self.board = board
"""
#print session
from iotronic_lightningrod.modules import vfs_library
fuse=vfs_library.FuseLib("/opt/AAA")
print fuse.getattr("/aaa.txt")
"""
libcPath = ctypes.util.find_library("c")
self.libc = ctypes.CDLL(libcPath)
def finalize(self):
pass
def restore(self):
pass
def mountLocal(self, mountSource, mountPoint):
try:
mounter = MounterLocal(mountSource, mountPoint)
mounter.start()
result = "Mounted " + mountSource + " in " + mountPoint
except Exception as msg:
result = "Mounting error:", msg
print(result)
return result
def unmountLocal(self, mountPoint):
print("Unmounting...")
try:
# errorCode = self.libc.umount(mountPoint, None)
errorCode = call(["umount", "-l", mountPoint])
result = "Unmount " + mountPoint + " result: " + str(errorCode)
except Exception as msg:
result = "Unmounting error:", msg
print(result)
return result
def mountRemote(self,
mountSource,
mountPoint,
boardRemote=None,
agentRemote=None
):
try:
mounter = MounterRemote(
mountSource,
mountPoint,
self.board,
self.session,
boardRemote,
agentRemote
)
mounter.start()
result = "Mounted " + mountSource + " in " + mountPoint
except Exception as msg:
result = "Mounting error:", msg
print(result)
return result
def unmountRemote(self, mountPoint):
print("Unmounting...")
try:
# errorCode = self.libc.umount(mountPoint, None)
errorCode = call(["umount", "-l", mountPoint])
result = "Unmount " + mountPoint + " result: " + str(errorCode)
except Exception as msg:
result = "Unmounting error:", msg
print(result)
return result
class MounterLocal(threading.Thread):
def __init__(self, mountSource, mountPoint):
threading.Thread.__init__(self)
# self.setDaemon(1)
self.setName("VFS-Mounter") # Set thread name
self.mountSource = mountSource
self.mountPoint = mountPoint
def run(self):
"""Mount FUSE FS
"""
try:
FUSE(
FuseManager(self.mountSource),
self.mountPoint,
nothreads=False,
foreground=True
)
except Exception as msg:
LOG.error("Mounting FUSE error: " + str(msg))
class MounterRemote(threading.Thread):
def __init__(
self,
mountSource,
mountPoint,
board,
session,
boardRemote,
agentRemote
):
threading.Thread.__init__(self)
# self.setDaemon(1)
self.setName("VFS-Mounter") # Set thread name
self.mountSource = mountSource
self.mountPoint = mountPoint
self.session = session
self.board = board
self.boardRemote = boardRemote
self.agentRemote = agentRemote
def run(self):
"""Mount FUSE FS.
"""
try:
FUSE(
FuseRemoteManager(
self.mountSource,
self.board.agent,
self.session,
self.boardRemote,
self.agentRemote
),
self.mountPoint,
nothreads=False,
foreground=True
)
except Exception as msg:
LOG.error("Mounting FUSE error: " + str(msg))
async def makeCall(msg=None, agent=None, session=None):
rpc_addr = str(agent) + '.stack4things.echo'
LOG.debug("VFS - I'm calling " + rpc_addr)
try:
res = await session.call(rpc_addr, msg)
LOG.info("NOTIFICATION " + str(res))
except Exception as e:
LOG.warning("NOTIFICATION error: {0}".format(e))
class FuseRemoteManager(Operations):
def __init__(self, mountSource, agent, session, boardRemote, agentRemote):
self.mountSource = mountSource
self.session = session
self.agent = agent
self.boardRemote = boardRemote
self.agentRemote = agentRemote
# makeCall("UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU",
# self.agent, self.session) # TEMPORARY
def join_path(self, partial):
if partial.startswith("/"):
partial = partial[1:]
path = os.path.join(self.mountSource, partial)
print(path)
return path
# Filesystem methods
# ==================
def access(self, path, mode):
full_path = self.join_path(path)
if not os.access(full_path, mode):
raise FuseOSError(errno.EACCES)
def chmod(self, path, mode):
full_path = self.join_path(path)
return os.chmod(full_path, mode)
def chown(self, path, uid, gid):
full_path = self.join_path(path)
return os.chown(full_path, uid, gid)
def getattr(self, path, fh=None):
full_path = self.join_path(path)
st = os.lstat(full_path)
attr = dict((key, getattr(st, key))
for key in (
'st_atime',
'st_ctime',
'st_gid',
'st_mode',
'st_mtime',
'st_nlink',
'st_size',
'st_uid'
)
)
return attr
def readdir(self, path, fh):
full_path = self.join_path(path)
dirents = ['.', '..']
if os.path.isdir(full_path):
dirents.extend(os.listdir(full_path))
for r in dirents:
yield r
def readlink(self, path):
pathname = os.readlink(self.join_path(path))
if pathname.startswith("/"):
# Path name is absolute, sanitize it.
return os.path.relpath(pathname, self.mountSource)
else:
return pathname
def mknod(self, path, mode, dev):
return os.mknod(self.join_path(path), mode, dev)
def rmdir(self, path):
full_path = self.join_path(path)
return os.rmdir(full_path)
def mkdir(self, path, mode):
return os.mkdir(self.join_path(path), mode)
def statfs(self, path):
full_path = self.join_path(path)
stv = os.statvfs(full_path)
stat = dict((key, getattr(stv, key))
for key in ('f_bavail',
'f_bfree',
'f_blocks',
'f_bsize',
'f_favail',
'f_ffree',
'f_files',
'f_flag',
'f_frsize',
'f_namemax'
)
)
return stat
def unlink(self, path):
return os.unlink(self.join_path(path))
def symlink(self, name, target):
return os.symlink(name, self.join_path(target))
def rename(self, old, new):
return os.rename(self.join_path(old), self.join_path(new))
def link(self, target, name):
return os.link(self.join_path(target), self.join_path(name))
def utimens(self, path, times=None):
return os.utime(self.join_path(path), times)
# File methods
# ============
def open(self, path, flags):
full_path = self.join_path(path)
return os.open(full_path, flags)
def create(self, path, mode, fi=None):
full_path = self.join_path(path)
return os.open(full_path, os.O_WRONLY | os.O_CREAT, mode)
def read(self, path, length, offset, fh):
os.lseek(fh, offset, os.SEEK_SET)
return os.read(fh, length)
def write(self, path, buf, offset, fh):
os.lseek(fh, offset, os.SEEK_SET)
return os.write(fh, buf)
def truncate(self, path, length, fh=None):
full_path = self.join_path(path)
with open(full_path, 'r+') as f:
f.truncate(length)
def flush(self, path, fh):
return os.fsync(fh)
def release(self, path, fh):
return os.close(fh)
def fsync(self, path, fdatasync, fh):
return self.flush(path, fh)
class FuseManager(Operations):
def __init__(self, mountSource):
self.mountSource = mountSource
def join_path(self, partial):
if partial.startswith("/"):
partial = partial[1:]
path = os.path.join(self.mountSource, partial)
print(path)
return path
# Filesystem methods
# ==================
def access(self, path, mode):
full_path = self.join_path(path)
if not os.access(full_path, mode):
raise FuseOSError(errno.EACCES)
def chmod(self, path, mode):
full_path = self.join_path(path)
return os.chmod(full_path, mode)
def chown(self, path, uid, gid):
full_path = self.join_path(path)
return os.chown(full_path, uid, gid)
def getattr(self, path, fh=None):
full_path = self.join_path(path)
st = os.lstat(full_path)
attr = dict((key, getattr(st, key))
for key in (
'st_atime',
'st_ctime',
'st_gid',
'st_mode',
'st_mtime',
'st_nlink',
'st_size',
'st_uid'
)
)
return attr
def readdir(self, path, fh):
full_path = self.join_path(path)
dirents = ['.', '..']
if os.path.isdir(full_path):
dirents.extend(os.listdir(full_path))
for r in dirents:
yield r
def readlink(self, path):
pathname = os.readlink(self.join_path(path))
if pathname.startswith("/"):
# Path name is absolute, sanitize it.
return os.path.relpath(pathname, self.mountSource)
else:
return pathname
def mknod(self, path, mode, dev):
return os.mknod(self.join_path(path), mode, dev)
def rmdir(self, path):
full_path = self.join_path(path)
return os.rmdir(full_path)
def mkdir(self, path, mode):
return os.mkdir(self.join_path(path), mode)
def statfs(self, path):
full_path = self.join_path(path)
stv = os.statvfs(full_path)
stat = dict((key, getattr(stv, key))
for key in ('f_bavail',
'f_bfree',
'f_blocks',
'f_bsize',
'f_favail',
'f_ffree',
'f_files',
'f_flag',
'f_frsize',
'f_namemax'
)
)
return stat
def unlink(self, path):
return os.unlink(self.join_path(path))
def symlink(self, name, target):
return os.symlink(name, self.join_path(target))
def rename(self, old, new):
return os.rename(self.join_path(old), self.join_path(new))
def link(self, target, name):
return os.link(self.join_path(target), self.join_path(name))
def utimens(self, path, times=None):
return os.utime(self.join_path(path), times)
# File methods
# ============
def open(self, path, flags):
full_path = self.join_path(path)
return os.open(full_path, flags)
def create(self, path, mode, fi=None):
full_path = self.join_path(path)
return os.open(full_path, os.O_WRONLY | os.O_CREAT, mode)
def read(self, path, length, offset, fh):
os.lseek(fh, offset, os.SEEK_SET)
return os.read(fh, length)
def write(self, path, buf, offset, fh):
os.lseek(fh, offset, os.SEEK_SET)
return os.write(fh, buf)
def truncate(self, path, length, fh=None):
full_path = self.join_path(path)
with open(full_path, 'r+') as f:
f.truncate(length)
def flush(self, path, fh):
return os.fsync(fh)
def release(self, path, fh):
return os.close(fh)
def fsync(self, path, fdatasync, fh):
return self.flush(path, fh)

144
iotronic_lightningrod/modules/webservice_manager.py
View File

@ -15,7 +15,12 @@
__author__ = "Nicola Peditto <n.peditto@gmail.com>"
from iotronic_lightningrod.config import package_path
from iotronic_lightningrod.lightningrod import RPC_proxies
from iotronic_lightningrod.lightningrod import SESSION
from iotronic_lightningrod.modules import Module
from iotronic_lightningrod.modules import utils
import iotronic_lightningrod.wampmessage as WM
from oslo_config import cfg
from oslo_log import log as logging
@ -26,13 +31,9 @@ CONF = cfg.CONF
import importlib as imp
import inspect
import json
import OpenSSL.crypto
import os
from iotronic_lightningrod.config import package_path
from iotronic_lightningrod.lightningrod import RPC_proxies
from iotronic_lightningrod.lightningrod import SESSION
from iotronic_lightningrod.modules import utils
import iotronic_lightningrod.wampmessage as WM
import time
class WebServiceManager(Module.Module):
@ -42,51 +43,102 @@ class WebServiceManager(Module.Module):
LOG.info(" - Proxy used: " + CONF.proxy.upper())
proxy_type = CONF.proxy
path = package_path + "/proxies/" + proxy_type + ".py"
try:
proxy_type = CONF.proxy
path = package_path + "/modules/proxies/" + proxy_type + ".py"
if os.path.exists(path):
if os.path.exists(path):
proxy_module = imp.import_module("iotronic_lightningrod.proxies."
+ proxy_type)
LOG.info(" --> " + proxy_type.upper() + " module imported!")
proxy_module = imp.import_module(
"iotronic_lightningrod.modules.proxies." + proxy_type
)
proxy = proxy_module.ProxyManager()
LOG.info(" --> " + proxy_type.upper() + " module imported!")
proxy_meth_list = inspect.getmembers(
proxy,
predicate=inspect.ismethod
)
proxy = proxy_module.ProxyManager()
RPC_proxies[proxy_type] = proxy_meth_list
proxy_meth_list = inspect.getmembers(
proxy,
predicate=inspect.ismethod
)
board.proxy = proxy
RPC_proxies[proxy_type] = proxy_meth_list
self._proxyWampRegister(proxy_meth_list, board)
board.proxy = proxy
else:
LOG.warning("Proxy '" + proxy_type + "' not supported!")
self._proxyWampRegister(proxy_meth_list, board)
else:
LOG.warning("Proxy '" + proxy_type + "' not supported!")
except Exception as err:
LOG.error("Error init WebServiceManager: " + str(err))
def finalize(self):
proxy_status = json.loads(self.board.proxy._proxyInfo())
LOG.info("--> Proxy " + self.board.proxy.type.upper()
+ " status:\n Active: " + str(proxy_status['status'])
+ "\n Info: " + str(proxy_status['log']))
try:
LOG.info("Webservice exposed on device:")
active_webservice_list = self.board.proxy._webserviceList()
if len(active_webservice_list) != 0:
for ws in active_webservice_list:
LOG.info("-> " + ws)
else:
LOG.info("-> NO WebService!")
proxy_status = json.loads(self.board.proxy._proxyInfo())
LOG.info("--> Proxy " + self.board.proxy.type.upper()
+ " status:\n Active: " + str(proxy_status['status'])
+ "\n Info: " + str(proxy_status['log']))
LOG.info("WebService Manager initialized!")
LOG.info("Webservice exposed on device:")
active_webservice_list = self.board.proxy._webserviceList()
if len(active_webservice_list) != 0:
for ws in active_webservice_list:
ws = ws.replace('.conf', '')
LOG.info("-> " + ws)
else:
LOG.info("-> NO WebService!")
LOG.info("Certificates on device:")
active_certs_list = self._certsList()
if len(active_certs_list) != 0:
for certificate in active_certs_list:
LOG.info("-> " + certificate)
c = open('/etc/letsencrypt/live/'
+ certificate + '/cert.pem').read()
cert = OpenSSL.crypto.load_certificate(
OpenSSL.crypto.FILETYPE_PEM, c)
LOG.info("--> Subject: " + str(cert.get_subject()))
LOG.info("--> ISSUER Organization:" +
str(cert.get_issuer()))
LOG.info("--> Expire date: " + str(
cert.get_notAfter().decode("utf-8")))
LOG.info("--> Expired: " + str(cert.has_expired()))
else:
LOG.info("-> NO certificates!")
# Safe apply changes on proxy
self.board.proxy._proxyReload()
time.sleep(3)
LOG.info("WebService Manager initialized!")
except Exception as err:
LOG.error("Error finalize init WebServiceManager: " + str(err))
def restore(self):
LOG.info("WebService Manager restored.")
def _certsList(self):
letsencrypt_path = "/etc/letsencrypt/live/"
if os.path.exists(letsencrypt_path):
certs_list = [
f for f in os.listdir(letsencrypt_path)
if os.path.isdir(os.path.join(letsencrypt_path, f))
]
else:
certs_list = []
return certs_list
def _proxyWampRegister(self, proxy_meth_list, board):
LOG.info(" - " + str(board.proxy.type).upper()
@ -96,38 +148,41 @@ class WebServiceManager(Module.Module):
if (meth[0] != "__init__") & (meth[0] != "finalize") \
& (meth[0] != "restore"):
# LOG.info(" - " + str(meth[0]))
rpc_addr = u'iotronic.' + board.uuid + '.' + meth[0]
rpc_addr = u'iotronic.' + str(board.session_id) + '.' + \
board.uuid + '.' + meth[0]
# LOG.debug(" --> " + str(rpc_addr))
if not meth[0].startswith('_'):
SESSION.register(meth[1], rpc_addr)
LOG.info(" --> " + str(meth[0]))
async def ExposeWebservice(self, service_name, local_port):
async def ExposeWebservice(self, board_dns, service_dns,
local_port, dns_list):
rpc_name = utils.getFuncName()
LOG.info("RPC " + rpc_name + " CALLED")
response = self.board.proxy._exposeWebservice(service_name, local_port)
response = self.board.proxy._exposeWebservice(board_dns, service_dns,
local_port, dns_list)
response = json.loads(response)
if(response['result'] == "SUCCESS"):
message = "Webservice '" + service_name + "' successfully exposed!"
message = "Webservice '" + service_dns + "' successfully exposed!"
LOG.info("--> " + str(message))
w_msg = WM.WampSuccess(response)
else:
message = "Error exposing webservice '" + service_name + "'"
LOG.warning("--> " + str(response['message']))
w_msg = WM.WampWarning(response)
return w_msg.serialize()
async def UnexposeWebservice(self, service_name):
async def UnexposeWebservice(self, service, dns_list):
rpc_name = utils.getFuncName()
LOG.info("RPC " + rpc_name + " CALLED")
response = self.board.proxy._disableWebservice(service_name)
response = self.board.proxy._disableWebservice(service, dns_list)
response = json.loads(response)
@ -140,12 +195,15 @@ class WebServiceManager(Module.Module):
return w_msg.serialize()
async def BoardDnsCertsSetup(self, board_dns, owner_email):
async def EnableWebService(self, board_dns, owner_email):
rpc_name = utils.getFuncName()
LOG.info("RPC " + rpc_name + " CALLED")
message = self.board.proxy._proxyBoardDnsSetup(board_dns, owner_email)
message = self.board.proxy._proxyEnableWebService(
board_dns,
owner_email
)
w_msg = WM.WampSuccess(message)
return w_msg.serialize()

4
requirements.txt
View File

@ -3,9 +3,9 @@
# process, which may cause wedges in the gate later.
pbr>=2.0.0,!=2.1.0 # Apache-2.0
autobahn>=0.10.1 # MIT License
autobahn>=18.10.1 # MIT License
six>=1.10.0 # MIT
httplib2>=0.9.1 # MIT
psutil # BSD
psutil>=5.4.7 # BSD
oslo.config>=5.1.0 # Apache-2.0
oslo.log>=3.36.0 # Apache-2.0

2
scripts/lr_configure
View File

@ -19,7 +19,7 @@ import os
import sys
if len(sys.argv) < 3:
print('Arguments required: "<REGISTRATION-TOKEN> <WAMP-REG-AGENT-URL>',
print('Arguments required: <REGISTRATION-TOKEN> <WAMP-REG-AGENT-URL>',
str(sys.argv))
else:
os.system('sed -i "s|\\"code\\":.*|\\"code\\": \\"'

3
setup.cfg
View File

@ -65,9 +65,8 @@ s4t.modules =
plugin = iotronic_lightningrod.modules.plugin_manager:PluginManager
device = iotronic_lightningrod.modules.device_manager:DeviceManager
service = iotronic_lightningrod.modules.service_manager:ServiceManager
# network = iotronic_lightningrod.modules.network_manager:NetworkManager
network = iotronic_lightningrod.modules.network_manager:NetworkManager
webservice = iotronic_lightningrod.modules.webservice_manager:WebServiceManager
# vfs = iotronic_lightningrod.modules.vfs_manager:VfsManager
[options]
build_scripts =