Is there any transform, which "looks" at any region of a picture and then compares it with neighbor region, and if two regions are similarly colored, then unifies two regions in one and colors it with average color.

Finally, this transform will distinguish "paper" from "writing" on it.

It will color with constant value any gradient fields or dimmed pictures, while remain uncolored any area with frequently changed colors, like in writings.

Key idea is that this transform should unify any big color change if it is gradual. 

Below is my manual example

![enter image description here](https://i.stack.imgur.com/i3ify.png)

Here background is unified to gray since it is average color of gradually changing background region, while rectangle is colored with dirty green because it is it's average color. Two regions are separated with high gradient margin.

**UPDATE ABOUT SIMILARITY**

There is a notion of color difference, for example in `Lab` color space it can be defined as ![enter image description here](https://i.stack.imgur.com/bZdnh.png). So, with this value one can calculate gradient, i.e. this value per pixel step. Also one may introduce threshold, separating "big" gradient from "small" one. So, pixels with small gradient can be merged together and colored with average color.

This is one rough approach coming to mind.