Protect Scripts from Concurrent Execution

Working with plain old shell scripts often we need to protect them from double execution and we want to make portable guard command.

Rquirements

Change to scripts should be atomic and work without knowledge about script internals
Code should work with POSIX shell, so no bashisms
It should work for scripts that can run daemons without unexpected lock inheritance

Solution

Final solution is to add the following code to the beginning of the script:

[ "${FLOCKER}" != "$0" ] && { exec 9>>/tmp/my-script-group.lock ; flock 9; (exec 9<&- ; env FLOCKER="$0" "$0" "$@" ) ; exit $? ; }

This script would prevent script from double parallel execution. Let’s check how it works

Use lock around whole script code

In order to have lock around all script code with only one line change we should re-run script from this script itself. It made with the following code:

[ "${FLOCKER}" != "$0" ] && { env FLOCKER="$0" "$0" "$@" ; exit $? ; }

if [ "${FLOCKER}" != "$0" ]; then
    env FLOCKER="$0" "$0" "$@"
    exit $?
fi

This command checks that FLOCKER environment variable was not provided during the call and restart script with this variable. When script started under FLOCKER finidhed we exit with its exit code.

Use flock to lock start of the script

Now we can create lock on specific file before we started our script second time

[ "${FLOCKER}" != "$0" ] && { exec 9</tmp/my-script-group.lock ; flock 9; env FLOCKER="$0" "$0" "$@" ; exit $? ; }

if [ "${FLOCKER}" != "$0" ]; then
    exec 9</tmp/my-script-group.lock
    flock 9
    env FLOCKER="$0" "$0" "$@"
    exit $?
fi

In this example we open file "/tmp/my-script-group.lock" with file descriptor “9” and call “flock” on this file descriptor. If no other process called locked on this file then lock would be obtained and released when we’ll exit from our script. If some other process already locked on this file then “flock” command would be blocked until the moment when other process release it’s lock.

Filename to be locked on

In this example I use hard-coded filename that ends with requirement to change filename for each used script. I decided to use it this way because in my environment there was requirement to protect several scripts from concurrent execution between them. If you have only one script you can use the following trick to get lock on script file itself:

if [ -f "$0" ]; then FN="$0"; else FN=$(command -v -- "$0"); fi
exec 9<$FN
...

Note, in case of existed file it’s preferred to open file for reading instead of writing.

Protection from spawned daemons

Final solution have one more modification

[ "${FLOCKER}" != "$0" ] && { exec 9</tmp/my-script-group.lock ; flock 9; ( exec 9<&- ; env FLOCKER="$0" "$0" "$@" ) ; exit $? ; }

if [ "${FLOCKER}" != "$0" ]; then 
    exec 9>>/tmp/my-script-group.lock
    flock 9
    ( exec 9<&- ; env FLOCKER="$0" "$0" "$@" )
    exit $?
fi

This trick required to allow original script to spawn daemons.

Flock would be released when flocked file descriptor would be closed, but in POSIX systems processes spawned with execve inherits parent process file descriptors and flocks. This means that in the following example lock would be released only after 100 seconds:

$ cat lock.sh
(
flock 9 
echo lock obtained $(date +%S) entry $1
sleep 10 &
echo now we want to release lock $(date +%S) entry $1
) 9>>/tmp/my-script-group.lock
$ ./lock.sh 1 ; ./lock.sh 2
lock obtained 21 entry 1
now we want to release lock 21 entry 1
lock obtained 31 entry 2
now we want to release lock 31 entry 2

In the example first run of script finished immediately, but it spawned backgroup process for sleep command which kept the lock for additional 10 seconds. In order to solve it we should close file descriptor before starting subprocesses that can spawn daemon: