Simple shared resource in Python I want a 4-bit resource to share status between processes and to retain status after shutdown and reboot. If the system were a microprocessor, I'd use a 4-bit register in non-volatile memory. 

Process interaction with the shared resource is typically minutes to hours between interactions, however processes may attempt to access it at exactly the same moment. So there is a simple file locking mechanism. A process that finds the resource locked, will back off for a short while and then try again.  

There are only 4 or 5 processes that deal with the resource. The short while (timeout) is 15 milliseconds plus random between 0 to 9 milliseconds. This is to prevent race conditions, processes locking each other, etc. 

Processes will be daemonised, and they will use the functions below to access the pickle. 

To simulate different processes, I'm using a few command prompt windows, started at about the same time. See the code. 

All of Python's multitasking libraries, threading, Queue and DMTCP seem way too elaborate for my tiny 4-bit register. Hence this solution. Please comment.  

import cPickle as pickle
import math
from time import sleep
import sys

''' 
    There are 4 switches, A to D. Each can be On or Off.
    Switches must retain state on shutdown and/or reboot. 

    Switch state is stored in a pickle in switch_state.p
    Switch state is a 4-bit number, ie max 15 decimal (all On)
    Switch A is bit 0, switch B is bit 1, and so on. 

    Several processes may interact with switches. 
    So, also in the pickle is a boolean 'locked'.
    All processes, when switching, must load the pickle, and first check if it's locked. 
    If it is, it should wait until it becomes unlocked. 
    If it's not, it should lock it, do its thing, and then unlock it. 
'''

def new_state(statefile='switch_state.p'):
    try: 
        switch = pickle.load(open(statefile, 'rb'))
    except:    
        switch = {'state':0, 'locked': False}
        pickle.dump(switch, open(statefile, 'wb'))

# functions to turn state on and off
# 
# using bitwise operations, so no need to check current state
# i.e. if you turn on a switch that's already on, nothing will change

def turn_on(cur_state, switchID):
    ON_bits = pow(2, ord(switchID) - ord('A'))
    return cur_state | ON_bits

def turn_off(cur_state, switchID):
    OFF_bits = ~pow(2, ord(switchID) - ord('A'))
    return cur_state &amp; OFF_bits


def update_state(switchID, state, statefile='switch_state.p'):
    switch = pickle.load(open(statefile, 'rb'))

    while switch['locked']:
        # some other process is switching, wait till it finishes
        sleep(0.015 + random.random()/100)
        switch = pickle.load(open(statefile, 'rb'))

    if not switch['locked']:
        new_switch = {'state': switch['state'], 'locked': True}
        pickle.dump(new_switch, open(statefile, 'wb'))
        if state == 'On':
            newstate = turn_on(switch['state'], switchID)
        elif state == 'Off':
            newstate = turn_off(switch['state'], switchID)
        else:
            #shouldn't ever get here
            print 'unknown state'
            exit()

        new_switch = {'state': newstate, 'locked': False}
        pickle.dump(new_switch, open(statefile, 'wb'))
        report = 'switch {} is now {}, switchstate {}'
        print report.format(switchID, state, newstate)

new_state()

# test the above in three or four command shells
# python switchreg.py [switchID] [delaytime]
# switchID may be A to D
# shorter delaytimes simulate race-conditions better 

switchID = sys.argv[1]
delaytime = float(sys.argv[2])
endrange = int(30/delaytime) # so time taken up by iterations is about the same

for i in range(0, endrange):
    update_state(switchID, 'On')
    sleep(delaytime)
    update_state(switchID, 'Off')
The one major problem is the lack of synchronisation between processes,
even if you randomise the sleep interval, this is still an invitation
for race conditions (unless you can somehow proof that the randomisation
is good enough).

Using a library like
lockfile or directly
using e.g. fcntl.lock will take care of that.  If you are worried
about the performance, well you are using Python, with cPickle (good
point that).  Otherwise some IPC mechanism, maybe a shared semaphore
between processes, would suffice.

Next, I think you can leave the file simply open and rewind to the
start, the read again instead of constantly closing and opening.  I'm
also not sure, but you could investigate file (region) locking via
fcntl instead to provide the necessary synchronisation.

The turn_on/turn_off seems very complicated if you could just write
out a dict, sequence, whatever, instead of doing bit manipulation.
If, however, you really want that part, then be consistent and simply
write two bytes to the file instead of using pickle.  The mixture of
very high-level serialisation and low-level composition of four bits
doesn't seem necessary at all.

Otherwise looks fine.  Some common expressions could be shared and the
written data structure is mostly used without any abstraction (like you
did with new_state, a updated_state function to create the new
dict could be nice), but I guess for a small script that hardly
matters.