9.0 KiB
Gear: Asynchronous Event-Driven Gearman Interface
gear
This module implements an asynchronous event-driven interface to Gearman. It provides interfaces to build a client or worker, and access to the administrative protocol. The design approach is to keep it simple, with a relatively thin abstration of the Gearman protocol itself. It should be easy to use to build a client or worker that operates either synchronously or asynchronously.
The module also provides a simple Gearman server for use as a convenience in unit tests. The server is not designed for production use under load.
Client Example
To use the client interface, instantiate a :pyClient
, and submit a
:pyJob
. For
example:
import gear
client = gear.Client()
client.addServer('gearman.example.com')
client.waitForServer() # Wait for at least one server to be connected
job = gear.Job("reverse", "test string")
client.submitJob(job)
The waitForServer() call is only necessary when running in a
synchronous context. When running asynchronously, it is probably more
desirable to omit that call and instead handle the :pyNoConnectedServersError
exception that submitJob may raise if no servers are connected at the
time.
When Gearman returns data to the client, the :pyJob
object is updated
immediately. Event handlers are called on the :pyClient
object so that
subclasses have ample facilities for reacting to events
synchronously.
Worker Example
To use the worker interface, create a :pyWorker
, register at least
one function that the worker supports, and then wait for a Job to be
dispatched to the worker.
An example of a Gearman worker:
import gear
worker = gear.Worker('reverser')
worker.addServer('gearman.example.com')
worker.registerFunction("reverse")
while True:
job = worker.getJob()
job.sendWorkComplete(job.arguments[::-1])
Server Example
You can run the Gearman server by executing the geard command. For help execute geard --help
SSL Connections
For versions of Gearman supporting SSL connections, specify the files containing the SSL private key, public certificate, and CA certificate in the addServer() call. For example:
ssl_key = '/path/to/key.pem'
ssl_cert = '/path/to/cert.pem'
ssl_ca = '/path/to/ca.pem'
client.addServer('gearman.example.com', 4730, ssl_key, ssl_cert, ssl_ca)
All three files must be specified for SSL to be used.
API Reference
The following sections document the module's public API. It is divided into sections focusing on implementing a client, a worker, using the administrative protocol, and then the classes that are common to all usages of the module.
Client Usage
The classes in this section should be all that are needed in order to implement a Gearman client.
Client Objects
gear.Client
Job Objects
gear.Job
Worker Usage
The classes in this section should be all that are needed in order to implement a Gearman worker.
Worker Objects
gear.Worker
FunctionRecord Objects
gear.FunctionRecord
WorkerJob Objects
gear.WorkerJob
Administrative Protocol
Gearman provides an administrative protocol that is multiplexed on
the same connection as the normal binary protocol for jobs. The classes
in this section are useful for working with that protocol. They need to
be used with an existing :pyConnection
object; either one obtained via a
:pyClient
or :pyWorker
, or via direct
instantiation of :pyConnection
to a Gearman server.
AdminRequest Objects
gear.AdminRequest
gear.StatusAdminRequest
gear.ShowJobsAdminRequest
gear.ShowUniqueJobsAdminRequest
gear.CancelJobAdminRequest
gear.VersionAdminRequest
Server Usage
Logging
To enable Gearman server logging you can setup a log configuration file and pass it to geard (i.e. geard --log-config=logging.config)
Example logging.config:
[loggers]
keys=root,gear
[handlers]
keys=console,debug,info
[formatters]
keys=simple
[logger_root]
level=WARNING
handlers=console
[logger_gear]
level=INFO
handlers=debug,info
qualname=gear
[handler_console]
level=WARNING
class=StreamHandler
formatter=simple
args=(sys.stdout,)
[handler_debug]
level=DEBUG
class=logging.handlers.TimedRotatingFileHandler
formatter=simple
args=('/var/log/gear/debug.log', 'midnight', 1, 30,)
[handler_info]
level=INFO
class=logging.handlers.TimedRotatingFileHandler
formatter=simple
args=('/var/log/gear/info.log', 'midnight', 1, 30,)
[formatter_simple]
format=%(asctime)s %(levelname)s %(name)s: %(message)s
datefmt=
ACL
The syntax of the optional ACL file consists of a number of sections
identified by the SSL certificate Common Name Subject, and the arguments
to the :pyACLEntry
constructor as key-value pairs:
[<subject>]
register=<regex>
invoke=<regex>
grant=<boolean>
For example:
[my_worker]
register=.*
[my_client]
invoke=.*
[my_node_manager]
grant=True
Server Objects
gear.Server
Access Control
The gear server supports authorization via access control lists. When
an :pyACL
object is
supplied to the server (or a file on the command line), gear changes
from the normal Gearman mode of allow-by-default to deny-by-default and
only clients with ACL entries will be able to perform actions such as
registering functions or submitting jobs. Authorization is based on the
SSL certificate Common Name Subject associated with the connection. An
:pyACL
object may be
modified programatically at run-time.
The administrative protocol supports modifying ACLs with the following commands:
- acl list
-
List the current acls:
acl list client register=None invoke=.* grant=True worker register=.* invoke=None grant=True .
- acl grant <verb> <subject> <pattern>
-
Grant the <verb> action for functions matching <pattern> to <subject>. Verbs can be one of
register
,invoke
, orgrant
. This requires the current connection have the grant permission. Example:acl grant register worker .* OK
- acl revoke <verb> <subject>
-
Revoke the <verb> action from <subject>. Verbs can be one of
register
,invoke
,grant
, orall
to indicate all permissions for the subject should be revoked. This requires the grant permission, except that a subject may always revoke its own permissions. Example:acl revoke register worker OK
- acl self-revoke <verb>
-
Revoke the <verb> action from connections associted with the current certificate subject. Verbs can be one of
register
,invoke
,grant
, orall
to indicate all permissions for the subject should be revoked. This is similar toacl revoke
but is a convenience method so that a subject does not need to know its own common name. A subject always has permission to revoke its own permissions. Example:acl self-revoke register OK
ACL Objects
gear.ACL
ACLEntry Objects
gear.ACLEntry
Common
These classes do not normally need to be directly instatiated to use the gear API, but they may be returned or otherwise be accessible from other classes in this module. They generally operate at a lower level, but still form part of the public API.
Connection Objects
gear.Connection
Packet Objects
gear.Packet
Exceptions
gear.ConnectionError
gear.InvalidDataError
gear.ConfigurationError
gear.NoConnectedServersError
gear.UnknownJobError
gear.InterruptedError
Constants
These constants are used by public API classes.
Normal job precedence.
Low job precedence.
High job precedence.
gear.constants
Indices and tables
genindex
modindex
search