How do I can grab the current image in blender game engine from a python controller? I am trying to create a python controller that executes game commands based on the image pixels of one of the camera viewports. This is for a robot simulation using computer vision.

I've tried two different approaches to no avail:

APPROACH #1: use bge.render.makeScreenshot() to save the current frame to a png file. The PROBLEM is that when I process that file later in the script, the PNG file is invalid/incomplete. I think the image file writing buffer is not flushed and closed yet. The files are there and look fine after the I exit the game engine.

APPROACH #2: try to render and grab pixel values as outlined in this thread. The PROBLEM is that image array is None.

Is there a way to fix either of these approaches or another better approach for getting an image frame from the blender game engine?

import bge

def main():
    cont = bge.logic.getCurrentController()
    own = cont.owner
    scene = bge.logic.getCurrentScene()

    if not 'init' in own:
        own['init'] = 1
        own['counter'] = 0

    own['counter'] += 1

    ######### APPROACH #1
    # get frame using makeScreenshot()
    frame_filename = "//frame" + str(own['counter']) + ".png"
    bge.render.makeScreenshot(frame_filename)

    # do image processing on frameXXX.png ...

    # ***PROBLEM: frameXXX.png is empty/invalid/incomplete

    ######### APPROACH #2
    # render a camera scene and get image pixels
    cam = scene.active_camera
    source = bge.texture.ImageRender(scene,cam)    
    source.capsize = [640,480]    
    imageArray = bge.texture.imageToArray(source, 'RGB')    
    charArray = imageArray.to_list()

    # do image proccessing on charArray ...

    # ***PROBLEM: source.image and imageArray are type 'None'
The second approach is the closest. But here's some code that may help (untested, but pulled from a working setup)

    obj = cont.owner
    camera = obj.scene.objects['NOT ACTIVE CAMERA']
    tex = bge.texture.Texture(obj, 0, 0)
    tex.source = bge.texture.ImageRender(
        camera_obj.scene,
        camera_obj,
    )
    tex.capsize(640*480)
    tex.source.background = [0, 0, 0, 0]
    source = self.tex.source
    data_array = bytes(640*480*4)
    source.refresh(data_array, 'RGBA') # Take the picture....

    print(data_array)  # Hopefully lots of data - in a 1D array.


You shouldn't recreate the data_array and texture objects each frame. Only do it on the first  frame, then just keep running the source.refresh() line.

If you run into issues, let me know and I'll make a blend file.