When to use 一倍 vs 两倍
If some words that mean "is" (e.g 是/相当于 ) are used, then it's literal times. 

你的速度 是 我的两倍。

Your speed is twice as mine.

Normally you will not say "one times" in this scenario. 你的速度是我的一倍 (Your speed is one times as mine) sounds strange in both English and Chinese. You can just say "你的速度和我的一样快"/ “我们俩一样快" (You are as fast as me). 

If other "comparing" words that means "more than"(e.g. A 比 B 大/高/快/etc.) show, then it's literal times + 1. Followed one has the same meaning of previous one.

你的速度 比 我快一倍。

You are one time faster than me.