Dealing with wasted space when using SIMD on CPU for vector maths I've been getting interested in SIMD programming on CPU using SSE recently, but I'm a complete noob on the subject. I found this article describing how to make an efficient float3 type using the recent __vectorcall convention, and it made me wonder how best to deal with the wasted space inherent in using a 128-bit type to represent vectors smaller than 4.

For a typical use case, you might want to transform a bunch of positions and normals on the CPU in order to do a batched draw call. When transmitting the data over to the GPU, though, ideally you'd want this as tightly packed arrays of float3, either interleaved or in separate buffers.

It seems like there are two options here:

A - store the data tightly packed, and unpack and repack to and from SSE-friendly format when you need to manipulate it, taking the CPU performance hit, or

B - store the data loosely packed on 16-byte boundaries, SSE it directly, and take the bandwidth and VRAM hit when sending to GPU.

Which of these options would be preferred in a real-world 3D engine, or under what circumstances would you prefer one to the other?
There's a third option you've neglected. Because of how graphics APIs work, typically you have to copy the data between the CPU and GPU anyway - even if you're on a mobile SoC where they share the same physical memory. This copy is the ideal time to take the hit of changing format, since you have to touch all of the memory anyway. So the copy in user memory, owned by the CPU, can be stored padded for SIMD alignment. Then, when you need to send that data to GPU memory, instead of doing a memcpy, just read in the 4-vectors and write out 3-vectors. It's less memory bandwidth than a normal copy (you're reading just as much, but writing less) and negligible CPU work.

If you've got that fourth float doing nothing, you might as well use it and make your co-ordinates homogeneous instead of just xyz. Depending on what compute you're doing on the CPU, that might make some things easier (transforms are the obvious example, but also reducing divisions if you were renormalising your normals a lot before).

This is similar but not identical to a trick I used when implementing OpenGL texture upload. On ARM, most Neon instructions operate at memory bandwidth rate, so if you're already doing a memcpy, it's free to do a pipeline-worth of instructions at the same time. You can use those free instructions to do pixel format conversion and reordering to Z-order or U-order.