Zookeeper Locking
This module provides a ZkLock, which should look familiar to anyone that has used Python’s threading.Lock class. In addition to normal locking behavior, revokable shared read/write locks are also supported. All of the locks can be revoked as desired. This requires the current lock holder(s) to release their lock(s).
This is implemented on the Zookeeper side by creating child lock candidate nodes under a parent node and seeing whether the node the lock created is first. It also ensures a first-come/first-serve ordering to who gets the lock after its released.
Asynchronous Lock
The ZkAsyncLock uses the Zookeeper async functions to avoid blocking while acquiring a lock, and optionally can use a callback style when the lock was acquired. It has a great amount of flexibility since it can run in the background to establish the lock while the program calling it can decide to block later and wait for the lock acquisition as desired.
If the calling program gets tired of waiting, it can delete the lock candidate node to avoid blocking any other programs waiting on the lock and handle the situation as desired.
Shared Read/Write Locks
Also known in the Zookeeper Recipes as Revocable Shared Locks with Freaking Laser Beams, ZkReadLock and ZkWriteLock locks have been implemented. A read lock can be acquired as long as no write locks are active, while a write-lock can only be acquired when there are no other read or write locks active.
Using the Lock Command Line Interface
zktools comes with a CLI to easily see current locks, details of each lock, and remove empty locks called zooky.
Usage:
$ zooky list
LOCK STATUS
fred Locked
zkLockTest Free
$ zooky show fred
LOCK HOLDER DATA INFO
write-0000000002 0 {'pzxid': 152321L, 'ctime': 1326417195365L, 'aversion': 0, 'mzxid': 152321L, 'numChildren': 0,
'ephemeralOwner': 86927055090548768L, 'version': 0, 'dataLength': 1, 'mtime': 1326417195365L,
'cversion': 0, 'modifed_ago': 16, 'created_ago': 16, 'czxid': 152321L}
The modifed_ago and created_ago fields in INFO show how many seconds ago the lock was created and modified.
Flag used to declare that revocation should occur immediately. Other lock-holders will not be given time to release their lock.
Asynchronous Zookeeper Lock
This Lock can be established asynchronously in the background.
Example non-blocking use:
lock = ZkAsyncLock(zk, '/Mylocks/resourceB')
try:
lock.acquire()
# Do some stuff that doesn't care if the lock is
# established yet, then wait for the lock to acquire
lock.wait_for_acquire()
# Do stuff with lock, after checking it was acquired
finally:
# Release and wait for release
lock.release()
lock.wait_for_release()
Example blocking use:
lock = ZkAsyncLock(zk, '/Mylocks/resourceB')
with lock:
if not lock.acquired:
# handle appropriately and return if needed!
# Won't execute until the lock is acquired
do_stuff()
# lock is released
do_more_stuff()
Warning
It’s possible when waiting for a lock, for it to run into errors during acquisition. This is why you should check to see that the lock was actually acquired before proceeding. If it was not and you’d like to know why, the errors attribute on the ZkAsyncLock will be an array indicating the errors that were encountered.
Create an Asynchronous Zookeeper Lock
| Parameters: |
|
|---|
Acquire a lock
| Parameters: | func – Function to call when the lock has been acquired. This function will be called with a single argument, the lock instance. The lock’s release() method should be called to release the lock. |
|---|---|
| Returns: | False |
Release a lock, or lock candidate node
This function can be called as long as a lock candidate node has been created. This allows a program to abandon its lock attempt if its been waiting too long, and remove itself from the lock queue.
The lock candidate node can be checked for by checking the value of ZkAsyncLock.candidate_created.
| Parameters: | func – Function to call when the candidate node has been verifiably confirmed as removed. |
|---|---|
| Returns: | False |
Zookeeper Lock
Implements a Zookeeper based lock optionally with lock revocation should locks be idle for more than a specific set of time.
Example:
from zc.zk import ZooKeeper
from zktools.locking import ZkLock
# Create a connection and a lock
conn = ZooKeeper()
my_lock = ZkLock(conn, "my_lock_name")
my_lock.acquire() # wait to acquire lock
# do something with the lock
my_lock.release() # release our lock
# Or, using the context manager
with my_lock:
# do something with the lock
Create a Zookeeper lock object
| Parameters: |
|
|---|
Acquire a lock
| Parameters: |
|
|---|---|
| Returns: | True if the lock was acquired, False otherwise |
| Return type: | bool |
Release a lock
| Returns: | True if the lock was released, or False if it is no longer valid. |
|---|---|
| Return type: | bool |
Indicate if this shared lock has been revoked
| Returns: | True if the lock has been revoked, False otherwise. |
|---|---|
| Return type: | bool |
Revoke any existing locks, gently
Unlike clear(), this asks all existing locks to release, rather than forcibly revoking them.
| Returns: | True if existing locks were present, False if there were no existing locks. |
|---|---|
| Return type: | bool |
Check with Zookeeper to see if the lock is acquired
| Returns: | Whether the lock is acquired or not |
|---|---|
| Return type: | bool |
Clear out a lock
Warning
You must be sure this is a dead lock, as clearing it will forcibly release it by deleting all lock nodes.
| Returns: | True if the lock was cleared, or False if it is no longer valid. |
|---|---|
| Return type: | bool |
Base lock implementation for subclasses
Create a Zookeeper lock object
| Parameters: |
|
|---|
Acquire a lock
Internal function used by read/write lock
| Parameters: |
|
|---|---|
| Returns: | True if the lock was acquired, False otherwise |
| Return type: | bool |
Release a lock
| Returns: | True if the lock was released, or False if it is no longer valid. |
|---|---|
| Return type: | bool |
Indicate if this shared lock has been revoked
| Returns: | True if the lock has been revoked, False otherwise. |
|---|---|
| Return type: | bool |