Image thresholding while retaining "underneath" contents
This is by no means a complete answer but I think it will get you far enough to achieve what you want. Edit: maybe not.


find main colours using kmeans
assign main colours to different types (brown line, blue line, etc). Some appear in both
clean up with morphology
segment image


What I thought might work was to pick out the most common colours and classify each pixel by the nearest colour. Then classify each pixel by type (brown line, etc), where the colour class can belong to multiple type classes. This would hopefully solve the problem of gaps in the lines - i.e. the colour where blue crosses brown would belong to both type class and fill the gap.

Not sure if it really does a better job. The issue with kmeans is that it uses random starting points so the colour classes were changing every time the script was run. I think I've fixed that now.

I wonder if there is some code out there to connect the broken lines?


% Read file
I = imread('mapdsp.jpg');

% Reshape into 2D matrix of pixel colours
ir = reshape(I, [size(I,1) * size(I,2),3]);

% Choose exact same sample points everytime
rng(0);
ir_indx = randi(size(ir,1),20,1);
ir_startpoints = ir(ir_indx,:);

% Use k-means clustering to find most common colours
[ik,c,sk,sd] = kmeans(double(ir),20,'Start',double(ir_startpoints));

% Show colours
cim = reshape(uint8(c),[1,20,3]);
image(cim); title('Colours'); pause;

% Sort colours
chsv = rgb2hsv(c);
[~,cni] = sort(chsv(:,1),1);
cn = c(cni,:);
cnim = reshape(uint8(cn),[1,20,3]);
image(cnim); title('Sorted colours'); pause;

% Get index of index 
[~,cni2] = sort(cni);

% Show pixels classified
in = reshape(cni2(ik,:), [size(I,1),size(I,2)]);
imagesc(in);  title('Pixel classification'); pause;


% Show each class
In = zeros(size(I));
for k = 1:20
    In(:,:,1) = I(:,:,1) .* uint8(in == k);
    In(:,:,2) = I(:,:,2) .* uint8(in == k);
    In(:,:,3) = I(:,:,3) .* uint8(in == k);
    image(uint8(In)); title (['Class ' num2str(k)]); pause;    
    im_dist(:,:,k) = reshape(sd(:,k), [size(I,1),size(I,2)]);
    imagesc(-im_dist(:,:,k) .* (in == k));  title('Pixel distance'); pause;
end

% 12,19 and 20 blue lines
im_blue = (in == 13) | (in == 19) | (in == 20);
imagesc(im_blue); title('Blue lines'); colormap gray; pause;
im_blue = imopen(im_blue,strel('diamond',1));
imagesc(im_blue); title('Blue lines opened'); colormap gray; pause;
for k=1:3; In(:,:,k) = I(:,:,k) .* uint8(im_blue); end;
image(uint8(In)); pause;
imagesc(bwmorph(im_blue,'skel',Inf));

% brown lines
im_brown = (in == 1) | (in == 2) | (in == 1) | ...
    (in == 7) | (in == 8) | (in == 11) | (in == 12) | (in == 13);
imagesc(im_brown); title('Brown lines'); pause;
im_brown = imopen(im_brown,strel('diamond',1));
imagesc(im_brown); title('Brown lines opened'); pause;
for k=1:3; In(:,:,k) = I(:,:,k) .* uint8(im_brown); end;
image(uint8(In)); pause;
imagesc(bwmorph(im_brown,'skel',Inf));

% 9 - black line
im_black = (in == 10);
imagesc(im_black); title('Black lines'); pause;
im_black = imopen(im_black,strel('diamond',1));
imagesc(im_black); title('Black lines opened'); pause
for k=1:3; In(:,:,k) = I(:,:,k) .* uint8(im_black); end;
image(uint8(In)); pause;
imagesc(bwmorph(im_black,'skel',Inf));

% 16,17,18 green area
im_green = (in == 16) | (in == 17) | (in == 18);
imagesc(im_green); title('Grean area'); pause;
im_green = imopen(im_green,strel('diamond',1));
imagesc(im_green); title('Green area opened'); pause;
im_green = imclose(im_green,strel('disk',7));
imagesc(im_green); title('Green area opened and closed'); pause;
for k=1:3; In(:,:,k) = I(:,:,k) .* uint8(im_green); end;
image(uint8(In)); pause;