How to efficiently find line-segment intersections between two sets?
The problem you describe is known as the red-blue intersection problem. Here, we have a red set of $n_1$ segments and a blue set of $n_2$ segments and we know that there are only intersections between segments of different colour.  Although this special case has been extensively studied, the $O(n\log n + k)$ algorithm, with $n=n_1+n_2$ is still optimal1 when if we have no further information.

In particular, we cannot easily combine the two approaches. To get some intuition on why adapting the sweepline algorithm in particular doesn't simply work, consider the case where $n_1=1$ and $n_2 = 1000$ and take a large blue segment such that the rectangle that has this segment as a diagonal contains all endpoints of the red segments. In this case, all red segments could be intersecting the single blue segment, so the sweepline has to consider all points in $n_2$ as event points and hence it is possible we have to spend at least $\Omega(n_2\log n_2)$ time on our sweepline algorithm.



There is one special case in which we can be faster. A planar subdivision is simply connected when each closed path over the edges within the same region of a subdivision can be topologically contracted to a point, i.e. a region cannot contain another region (similar to a simply connected space). If the two sets of segments form two simply connected planar subdivisions, we can use the algorithm by U. Finke and K. Hinrichs. to find all intersections in $O(n+k)$ time. 

1. Optimal if $n_1\approx n_2$. Of course, if $n_2 \ll\log n_1$, we can do better with the naive algorithm, but that is rarely the case.