Storing many instances efficiently Theres a few aspects to what I am looking for in terms of solution.

We deal with extremely large scenes, and take huge advantage of instancing, dupligroups and particle systems

However, when you have thousands of objects (not particle systems) in a scene... blender starts to go slow.

Our initial idea was to convert the instances / dupligroups over to particle systems... that way its one object which has many instances, instead of many objects.

We have looked into trying to convert dupligroups / instances into a particle system, however... we cannot seem to alter it via the python api (https://developer.blender.org/T48022). Is there a efficient way of storing thousands of objects with locations / rotations without actually having the slowdown that normal objects would have? Is there a way of being able to create non-object instances via python?
I also work in archviz and would love to hear a solution for this,in my experience having particles does speed up the viewport a little at the expense of slower selection for objects with many particles.

Only alternative I can think of would be using perhaps duplifaces, though you would probably be limited to distributing single objects as opposed to say group instances.

There would have to be a special dummy mesh object where each face would store the location and rotation of each instance, though I am not sure it will bring any viewport performance advantage. You would also probably need a script to create said mesh from existing instances, although creating from scratch would be a little bit easier.