Python, IDPropertyGroup, isinstance
There is no such thing as bpy.types.IDPropertyGroup in bpy.types, hence this line:

if isinstance(val, bpy.types.IDPropertyGroup):
    ...


fails. Try from bpy.types import IDPropertyGroup you will get an ImportError.

&gt;&gt;&gt; from bpy.types import IDPropertyGroup
Traceback (most recent call last):
  File "&lt;blender_console&gt;", line 1, in &lt;module&gt;
ImportError: cannot import name 'IDPropertyGroup'


This is why that isinstance fails. I agree this is a little bit weird, but this changes the question a little. 


  Why isn't it there and how would I use isinstance to test complexity of an ID property. Are there alternatives?


Because ID properties are a limited collection of types, it's perhaps no less crazy to check for int, float, str, list.

import bpy

x = bpy.context.active_object

for k in x.keys():
    val = x[k]

    if isinstance(val, (int, float, str, list)):
        print('simple', val)
    else:
        print('complex', val.to_dict())


You could also check if the ID property has an attribute to_dict instead, else it's a basic type.

import bpy

x = bpy.context.active_object

for k in x.keys():
    val = x[k]
    if hasattr(val, 'to_dict'):
        val = val.to_dict()
    print(k, val)


Even use the (always present) class type returned by type(x['cycles'])

import bpy

x = bpy.context.active_object
IDPropertyGroup = type(x['cycles'])  # &lt;--- use this type's return in isinstance

for k in x.keys():
    val = x[k]
    print("k %s type %s" % (k, type(val)))
    if isinstance(val, IDPropertyGroup):
        val = val.to_dict()

    print(k, val)