Euler Problem 7 (finding the 10001st prime) in Powershell
I'd start with 


removing redundancy
measuring the time automatically
output to screen slows down, comment it out


depending on the speed of the computer used I get results of ~74..120 seconds.
Compared with 0.450 secs here with matlalb there IS potential to get better ;-)

## Q:\Test\2018\11\08\CR_207250.ps1

$StartTime = get-date
$PrimeNo = 10001

$incNum1 = 0
$highestNum = 0
$k = 1

while($k -lt $PrimeNo){
    $incNum2 = 2
    $divisNum = 2 * $incNum2 - 1
    $incNum1++
    $nextNum = 2 * $incNum1 + 1

    $upTo = [Math]::Ceiling(($nextNum / 2))
    $break = $false
    while($divisNum -lt $upTo){
        $modRes = $nextNum % $divisNum

        if($modRes -eq 0){
            $break = $true
            break
        }
        $incNum2++
        $divisNum = 2 * $incNum2 - 1
    }
    if(!$break){
        $highestNum = $nextNum
#        "{0}:{1}" -f $k,$nextNum
        $k++
    }
}

"## PrimeNo: {0} is {1} calculated in {2} seconds" -f $PrimeNo,$highestNum,
    ((Get-Date)-$StartTime).TotalSeconds

## PrimeNo: 10001 is 104743 calculated in 73,6911884 seconds
## PrimeNo: 10001 is 104743 calculated in 118,3481173 seconds