Ripple: How to monitor multiple accounts reliably and effectively? rpc = RippleNetwork(app.config['RIPPLE_SERVER'])
poll = RippleNetwork(app.config['RIPPLE_SERVER'])
last_ledger = Variables.get('ripple_last_ledger')
poll.subscribe(streams=['ledger'])

while True:
    trans = []
    try:
        rst = rpc.account_tx(account=withdraw_wallet, ledger_index_min=last_ledger, limit=1000)
        trans.extend(rst['transactions'])
        while rst.get('marker') is not None:
            rst = rpc.account_tx(account=withdraw_wallet, ledger_index_min=last_ledger, marker=rst['marker'], limit=1000)
            trans.extend(rst['transactions'])

    except RippleException as e:
        if e.resp['error'] == 'lgrIdxsInvalid':
            log.debug('ledger_index_min too new, sleeping for 10s')
            time.sleep(10)
            continue

        raise

    for rec in trans:
        tx = rec['tx']
        log.debug('Got tx %s', tx['hash'])

        if tx['TransactionType'] != 'Payment':
            continue

        if tx['Destination'] != withdraw_wallet:
            continue

        log.info('Queue tx %s', tx['hash'])
        tube.put(str(tx['hash']))

    last_ledger = rst['ledger_index_max'] + 1
    Variables.set('ripple_last_ledger', last_ledger)
    db.session.commit()
    # Wait for next(or several) ledger close here.


Code above is my current implementation, but this method does not scale to thousands of accounts.

Is there any 'best practice'?
I'm not sure a large code paste is the best way to get your example across (I'm unfamiliar with the library/API you're using for example). Assuming it's correct to summarize that you are looping, repeatedly doing account_tx RPC calls with at most a ten second delay; then that's not efficient even for a small number of accounts.

For a small-ish number of accounts, I'd recommend doing something like your loop only on restart to catchup from a last known ledger you've processed. (For example, after restarting your monitoring after an outage or disconnect). Once caught up it's better to use the subscribe RPC with the set of Ripple accounts you're interested in. The server will send you a notification anytime any transaction touches any of the accounts you list. This avoids polling and is far more efficient, especially during periods where the accounts are idle. (I'm only familar with using subscribe over a websocket, and it works very well for this even over days; I don't know what library support for this is like).

Once you go beyond a certain threshold of accounts you are trying to watch, especially if they are very active accounts, it may be easier to use the subscribe RPC to subscribe to all transactions and filter those for just the ones effecting the accounts you're interested in. Also, at that point it may become a good idea to do such queries against your own locally running rippled rather than rely on a public Ripple server.

The point at which it's best to switch from subscribing to a list of accounts versus subscribing to all transactions is unclear to me. For starters, there may be a limit to the number of accounts you can specify in a subscribe call; if so perhaps modify a locally running rippled to remove any such limit and have it do the filtering may be appropriate.