math grads, optimize one op away
Are you sure it's faster?  We have vectorized fast-reciprocal routines that can be considerably faster than true division. I would expect that true division internally does whatever is most efficient?

But also, I'm mostly interested in GPU usage, and this is elemwise, so a single op should really be faster than two ops.

But I did not measure it. Maybe I'm understanding sth wrong.
 > I would expect that true division internally does whatever is most efficient?

The division op has specific accuracy requirements (i.e. within 1 ULP from the "true" result) which multiplying by the reciprocal does not have.  Fast reciprocal is similar to the fast sqrt hack, where there's a bit manipulation, then Newton step to improve accuracy.  This can be significantly faster than division.

> But also, I'm mostly interested in GPU usage, and this is elemwise, so a single op should really be faster than two ops.
> 
> But I did not measure it. Maybe I'm understanding sth wrong.

It might depend.  I'm mainly asking if you have evidence, rather than a generic "one op is better than two".

 > I'm mainly asking if you have evidence, rather than a generic "one op is better than two".

No, I don't. My reasoning is mostly based on just that.

But I also did not really expect that `reciprocal` is not exactly the same as `1/x`, as the doc say. Maybe the docs then should be updated on `reciprocal` to clarify this?

But then also, how do you decide when to do `x / y` and when to do `x * reciprocal(y)`? I see both code being used. > But I also did not really expect that `reciprocal` is not exactly the same as `1/x`, as the doc say. Maybe the docs then should be updated on `reciprocal` to clarify this?

Reciprocal is `1/x` (well, within an ULP or two), though isn't implemented using division.  But `x * (1/y)` has different accuracy guarantees than `x/y`.
 
> But then also, how do you decide when to do `x / y` and when to do `x * reciprocal(y)`? I see both code being used.

These particular instances in your PR _were_ `x/y`, and someone changed them to `x * reciprocal(y)`.  You're changing it back.  The current change is not documented as a performance improvement (it was put in along with another change), but knowing the author, it may have been, since he's the one who wrote the fast reciprocal code.  Hence I'm asking you for evidence that it makes a difference. I see. I guess the author probably must have thought about it then. Maybe the author also had some good evidence in believing that this is faster.

I was simply seeing this and thought that `x / y` would be simpler and faster, but from everything you explained, it seems likely that my initial thought was simply wrong. 

I don't think I have the time to do any further benchmarking on this for now, so maybe we can then just close this PR.

Or maybe you ask the original author about it? I would want benchmarks now anyways.  Closing for now.  Feel free to re-open when you have time to collect benchmarks.
Instead of `x * reciprocal(y)`, do `x / y`. This saves one op. Instead of (reciprocal, mul), you only have div.