The effects of bilingualism on colour perception Peltola et al. (2012) showed that there are two types of bilinguals. 


Balanced bilinguals mix their two languages and are effected by linguistic categories from both. 
Dominant bilinguals seem to switch on the categories of the language they are using, and suppress the categories of the language not in use.


The task Peltola et al. (2012) studied was phoneme categories. The boundaries between vowels are arbitrary and set by the language of the user. The researchers focused on two sounds that are categorised as different vowels in Finnish, but are the same vowel in Swedish. When dominant bilingual participants were briefed about the study in Finnish, they distinguished the sounds quickly. When the same participants were briefed about the same experiment a week later in Swedish, their ability to distinguish the sounds slowed significantly. The balanced bilinguals were equally slow under both briefings.

A similar arbitrary language-dependent categorization exists in colour perception (Regier &amp; Kay, 2009). In fact, the effect of this arbitrary categorization is even lateralized: the Whorf hypothesis is supported in the right visual field but not the left (Gilbert et al., 2006)! So the basics of linguistic relativity seem to be cross-modal.

Do dominant bilinguals switch linguistic categories on and off in the visual domain? Is the result of Peltola et al. (2012) cross-modal?
This paper was written in 2010:

Perceptual shift in bilingualism: brain potentials reveal plasticity in pre-attentive colour perception.


  In this paper, we test whether in Greek speakers exposure to a new
  cultural environment (UK) with contrasting colour terminology from
  their native language affects early perceptual processing as indexed
  by an electrophysiological correlate of visual detection of colour
  luminance. We also report semantic mapping of native colour terms and
  colour similarity judgements. Results reveal convergence of linguistic
  descriptions, cognitive processing, and early perception of colour in
  bilinguals.


There are some important (but speculative) considerations here, in my opinion:

1) Aspects of color are defined culturally.  Each culture arbitrarily places borders to divide the regions of a continuous spectrum and we pass these divisions on to our children.

2) Different cultures may have non-overlapping hues distributed across a coarse division.  For instance, if we took the coarse color "green" and divided it up into 15 shades so that my greens are 3, 8, and 12 and my foreign friend's is 8, 10, and 15 are hues that we each strongly associate with green (because they're typical in our culture).  However, my 3 is too close to yellow for my foreign friend and his 15 is too close to blue for me (and I want to call it "aqua").

3) There's also the difficult that the colors we experience in the world aren't pure colors on a ROYGBIV spectrum.  An apparently uniform garment can often be small-grain ensembles of many different colored threads.