Photorealistic synthetic data for object segmentation Let's say that we have a very small labelled dataset for instance segmentation and there is a photorealistic physics engine available that can produce synthetic data for us. Looking on the web I haven't found any paper about a similar issue.
Has anyone tried to use solely a photorealistic synthetic dataset or mixed with real data?
If so, I would appreciate your conclusions or any relevant work that you can cite here.
I used such an approach at the Computer Vision Lab at TU Delft for my Bachelor thesis.
The goal was to analyze different aspects of the same problem: identifying LEGO bricks in a photograph of an unsorted pile of LEGO bricks. We built a small real dataset as well as a large (relatively) photorealistic synthetic dataset using Blender.
Personally, I looked at it from the explainable artificial intelligence (XAI) point of view, but some of my team mates did have objectives closer to yours (image segmentation, object detection, classification).
You can find a description which is a bit more detailed here, in section 4.1.
I can edit this answer with more details if you're interested.