Script to remove .vb files from a directory
You could combine expression vs. command mode in PowerShell, process only files (!$_.PSIsContainer) and use regex instead of ChangeExtension:

gci -include *.vb -Recurse |
    ? { (!$_.PSIsContainer) -and (Test-Path ($_.FullName -replace "\.vb$", ".cs") } |
    % { rm $_ -force }