Improve chroma key effect in Sony Vegas
Let's first review the basics of green screen:

1) screen about 5 or more feet behind subject so it is out of focus from subject.

2) flat screen, no wrinkles.

3) flat lighting, no shadows, no hot spots.

4) subject can not wear green or wear shiny things or see through things.

Review Vegas Chroma Key: Once you engage the chroma key effect, use the eye dropper and take a sample of the green screen to set the baseline color, engage the mask and tweak the level until you get a flat black on the screen area and white everywhere else then turn the mask off. If this can not be done then something in the basics above was not done correctly.

Another idea to assist. Make sure when you shoot your subject that you have properly set the white balance on the camera so that you have the correct color baseline.

Here are some tutorials to assist:


                
            

https://vimeo.com/34365256