Trouble with PCB design for USB3 connections
There are few things to consider:


"a male-male extension cable" is an illegal construction from USB standard viewpoint. As such, these cables cannot be officially certified in a qualified Lab, and whoever crafted that cable in a sweatshop China factory have no idea what they are doing, nor proper equipment to test anything. So the overall transmission quality of this cable is a prime suspect. This is a factor #1.
Your board is essentially a "female-female" converter, which allows you to re-configure you male-male extension cable into "male-female" extension (which is also against USB specifications BTW). On this way you have introduced one more mating connector in the link. The reason that extenders are excluded from USB specifications is that they introduce extra connectors and, as it was noted in this SE EE question, it is physically difficult to maintain good uniform differential impedance across non-coaxial connections. These imperfections can have nearly arbitrary form and cannot be corrected in USB 3.x receivers, unlike the plain cable losses. Any impedance inhomogeneity along the channel introduces reflections, which, combined together (aka "inter-symbol interference") cause horrible jitter, and USB clock-data recovery circuits can't tolerate/adapt to this. The bad thing about your combined cable is that it has two extra pairs of mating connectors, which likely breaks any expected quality of the channel even if your traces/pads are designed in the best possible way.
Some USB 3.x logic analyzers are designed using active repeaters, to sniff on the traffic and branch it for recording and analysis. As such, they can re-shape and re-time the signal, and that's why it might "work" a bit more reliably. 
USB 3.0 cables have no formal limit on how long they can be, provided that their loss characteristics do not exceed specifications, so abilities of USB receivers to "equalize" (adapt to) the losses during channel training are not exceeded. So the cable can be as long as you can make its losses low (which costs). Practice however shows that most standard A-B cables can't meet quality requirements (insertion loss, far- and near-end cross-talk limits) is they are longer than 2m. If micro connectors are used, collective wisdom of USB-IF consortium determined that the cable longer than 1m isn't worth to consider, use, and test. And again, all these considerations do not apply to extenders, since they introduce imperfection that cannot be universally modeled and therefore common correction algorithm is impossible.
Not all USB devices and hosts are made equal, particularly in the sense how well their receivers can equalize channel losses and how tolerant they are to accumulated signal jitter. A certificate of USB-IF compliance is a good start to be somewhat confident that the link will work reliably. If any of components in the link are not USB-IF certified, there is a good chance that they break some USB requirements and can't provide reliable communication, although exceptions do occur. This includes your "a USB 3.0 camera".


Therefore, it shouldn't come to any surprise that your double-extended USB link works only intermittently. You must be lucky that it works at all.

To make it work reliably, you should exclude any extenders, or, at least use natural male-female extenders, there are some good quality ones. However, you never know this unless you have expensive differential TDR (Time-Domain Reflectometer) or network analyzer to test your channel quality. If a distance is of concern, you should use "active" extenders based on new class of USB devices called "Repeaters-retimers".