Does it make sense to visualize data with a linear relationship using tSNE? I have used tSNE several times to visualize high dimensional data for cluster analysis, and it has always worked quite well when data falls into clusters.

However, does it make any sense to use tSNE to visualize the independent variables in a dataset which has a linear relationship $y = X \theta + b$, where $y$ is the target value and $X$ are the independent variables (b is the bias)?

As I understand tSNE, it is used more for capturing local structures in high dimensional data, so it is probably not suited for visualizing data for linear regression - is this a wrong assumption?
Your assumption is right, the results are in general misleading.

Suppose your (linearly correlated) data have missing points in some range:



Than for t-SNE the two subset of data will be two different clusters, even if they lie on the same linear distribution:



But, if you are actually interested in the fact that those two structures are separated, then t-SNE is a good choice to visualize it.

So a proper answer should be: it depends on what you need.

P.S.

Here the code used for this example:

import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from sklearn.manifold import TSNE

def function(x, y):
    return 4+3*x + 4*y

x1=np.random.rand(100)
x2=np.random.rand(100)+1.2
X=np.concatenate([x1, x2])
Y=0.5+2*X

Z=function(X, Y)

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

ax.scatter(X, Y, Z, c='r', marker='o')
plt.show()

data=[]
for x, y, z in zip (X, Y, Z):
    data.append([x, y, z])

data_embedded = TSNE(n_components=2).fit_transform(data)


plt.scatter([x for x, y in data_embedded], [y for x, y in data_embedded], color='r')
plt.show()