Are texture binds expensive? I am trying to get a feel for how much effort I should put into this. I am building a basic 2D engine and for my use case there are only 8 textures. Each draw call in this engine currently requires that its 1 corresponding texture be bound to '0'. My question is essentially: Is there something faster than this?

For example, what if I bound the texture for the first sprite to '0' the second to '1' etc etc. This would avoid the gpu having to "switch out" these textures. I am guessing that this wouldn't matter much however.

Is this something that is valid to be concerned about? Is it expensive for the GPU to swap out textures?
It's hardware dependent, but as it stands, you shouldn't expect a performance drop on modern dedicated GPUs. Typically they have a pipeline state cache which fits such a small number of states easily.

The only case when that cache even becomes relevant is if the workload performed in that state is insufficient to saturate the shaders, so execution of subsequent drawcalls overlaps. At that point you can eventually exceed the size of the cache if you have too many different pipeline states and effective utilization drops.

Except that when you do, you will also have trouble combining all the states into one, due to a limit on texture bindings per state and stage, so you would still end up with multiple states. Depending on the API you will have different limits, lower bound being set e.g. by OpenGL 3.x spec requiring 16 texture binding slots per shader, but realistically it doesn't even go much higher than 40-64 binding slots per shader even in high end hardware.

At that point you are better off using a texture array instead - the level of indirection associated with that doesn't cause any significant performance penalty any more, on modern architectures. You "only" have to group your textures by size and tiling at that point.

The alternative is to bake the individual textures into a a single, large texture, which does still allow a certain level of mipmapping, but only down to a level at which pixels from different source textures don't blend. Also you loose the ability to use native tiling, but have to do it manually in shader code.