How to Describe the Color of an Image Patch?
Another approach which might be appropriate in your Bayesian approach (Filter Particle) would be Mean Vector + Covariance Matrix.

Even for that you could employ your idea of weighing (Which is reasonable).

I would pay attention to the color space you're working in.
Euclidean Distance isn't working (At least not well) in RGB.

Example in CIE LAB

The mean is given by:

$$ {x}_{c} = \frac{1}{n} \sum_{i = 1}^{n} {p}_{i} $$

Where $ {p}_{i} \in \mathbb{R}^{3} $ is the vector of LAB valus of each pixel from a patch of $ n $ pixels.

The Covariance is given by:

$$ {C} = \frac{1}{n} \sum_{i = 1}^{n} {\left( {p}_{i} - {x}_{c} \right)} {\left( {p}_{i} - {x}_{c} \right)}^{T} $$