OpenCV/C++ connect nearby contours based on distance between them
If you are not worried about the speed or exact contour of hand, below is a simple solution.

The method is like this : You take each contour and find distance to other contours. If distance is less than 50, they are nearby and you put them together. If not, they are put as different.

So checking distance to each contour is a time consuming process. Takes a few seconds. So no way you can do it real time.

Also, to join contours, I put them in a single set and drew a convex hull for that set. So the result you are getting is actually a convex hull of hand, not real hand. 

Below is my piece of code in OpenCV-Python. I haven't gone for any optimization, just wanted it to work, that's all. If it solves your problem, go for optimization.



import cv2
import numpy as np

def find_if_close(cnt1,cnt2):
    row1,row2 = cnt1.shape[0],cnt2.shape[0]
    for i in xrange(row1):
        for j in xrange(row2):
            dist = np.linalg.norm(cnt1[i]-cnt2[j])
            if abs(dist) &lt; 50 :
                return True
            elif i==row1-1 and j==row2-1:
                return False

img = cv2.imread('dspcnt.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(gray,127,255,0)
contours,hier = cv2.findContours(thresh,cv2.RETR_EXTERNAL,2)

LENGTH = len(contours)
status = np.zeros((LENGTH,1))

for i,cnt1 in enumerate(contours):
    x = i    
    if i != LENGTH-1:
        for j,cnt2 in enumerate(contours[i+1:]):
            x = x+1
            dist = find_if_close(cnt1,cnt2)
            if dist == True:
                val = min(status[i],status[x])
                status[x] = status[i] = val
            else:
                if status[x]==status[i]:
                    status[x] = i+1

unified = []
maximum = int(status.max())+1
for i in xrange(maximum):
    pos = np.where(status==i)[0]
    if pos.size != 0:
        cont = np.vstack(contours[i] for i in pos)
        hull = cv2.convexHull(cont)
        unified.append(hull)

cv2.drawContours(img,unified,-1,(0,255,0),2)
cv2.drawContours(thresh,unified,-1,255,-1)


Below are the results i got :