NAME
IPC::ConcurrencyLimit - Lock-based limits on cooperative
multi-processing
SYNOPSIS
use IPC::ConcurrencyLimit;
sub run {
my $limit = IPC::ConcurrencyLimit->new(
type => 'Flock', # that's also the default
max_procs => 10,
path => '/var/run/myapp', # an option to the locking strategy
);
my $id = $limit->get_lock;
if (not $id) {
warn "Got none of the worker locks. Exiting.";
exit(0);
}
else {
# Got one of the worker locks (ie. number $id)
do_work();
}
# lock released with $limit going out of scope here
}
run();
exit();
DESCRIPTION
This module implements a mechanism to limit the number of concurrent
processes in a cooperative multiprocessing environment. This is an
alternative to, for example, running several daemons.
Roughly speaking, a typical setup would be the following:
* Cron starts a new process every minute.
* The process attempts to get a lock as shown in synopsis.
* If it obtains a lock, it starts working and exits when the work is
done.
* If not, "max_procs" processes are already working, so it exits.
This has several distinct advantages over daemons.
* Processes do not run as long. Small memory leaks are less likely to
become a problem.
* Rolling out new code is trivial. No need to do any daemon restarting
and worrying about interrupting a unit of work.
* No complicated master/slave setups and process/thread pooling.
The implementation uses some form of locking to limit concurrency:
There's simply a limited number of locks to go around. The detailed
locking implementation is chosen using the "type" parameter to the
constructor. The base distributions ships with one locking strategy
only: IPC::ConcurrencyLimit::Lock::Flock for a file-locking based
concurrency limit.
Among the other potential strategies that are not part of this
distribution are NFS-based locking using File::SharedNFSLock or using
MySQL's "GET_LOCK". Both of these schemes would allow limiting
concurrency across multiple hosts without a special-purpose daemon.
METHODS
new
Creates a new concurrency limit. Creating the object does not lock
anything. This requires a followup call to the "get_lock()" method!
After calling "get_lock()", the lock will be held by the
"IPC::ConcurrencyLimit" object and released when either "release_lock()"
is called or the "IPC::ConcurrencyLimit" object is freed.
Takes named parameters, one of which is the "type" parameter, which
specifies the type of lock to use and defaults to "Flock".
The "max_procs" option indicates the maximum number of locks that can be
held at the same time and thus usually the maximum no. of running
processes. It defaults to 1.
All concurrency limits that refer to the same resource/limit must use
the same setting for "max_procs". If not, the behaviour is undefined.
All other named parameters will be passed as options to the lock
implementation. See IPC::ConcurrencyLimit::Lock::Flock or other
implementations.
get_lock
Creates the actual lock and if successful, returns its id (starting from
1, not 0).
Returns undef if locking was unsuccessful. Multiple calls do not stack.
They will simply return the same lock id as long as a lock is held.
release_lock
Releases the lock. Returns 1 if a lock has been released or undef if
there had been no lock to be released.
is_locked
Returns whether we have a lock.
lock_id
Returns the id of the lock or undef if there is none.
AUTHOR
Steffen Mueller, "
[email protected]"
Yves Orton
ACKNOWLEDGMENT
This module was originally developed for booking.com. With approval from
booking.com, this module was generalized and put on CPAN, for which the
authors would like to express their gratitude.
COPYRIGHT AND LICENSE
(C) 2011, 2012 Steffen Mueller. All rights reserved.
This code is available under the same license as Perl version
5.8.1 or higher.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
