Does RandomForest convergence imply I can solve a problem with a NN too?
To me those are separate things since both models have a different cost function to be optimized.
On the other hand you could combine those models by constructing embeddings based on random forest splits and then using those embeddings as inputs for a neural network.
Toy example shows that there is a non-trivial configuration of a neural net that can get as good results as those obtained a by a random forest:
from sklearn.datasets import load_iris
from sklearn.neural_network import MLPClassifier
from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import RandomTreesEmbedding


X, y = load_iris(return_X_y = True)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = 42, test_size = .2)


params = dict(n_estimators=100,
    max_depth=None,
    min_samples_split=2,
    min_samples_leaf=1,
    min_weight_fraction_leaf=0.0,
    max_leaf_nodes=None,
    min_impurity_decrease=0.0,
    min_impurity_split=None,
    random_state=42,
    verbose=0)

mlp = Pipeline([(&quot;embeddings&quot;, RandomTreesEmbedding(**params)),
                (&quot;model&quot;, MLPClassifier(activation = &quot;identity&quot;,hidden_layer_sizes=(1000,), max_iter = 10000, random_state = 42))]).fit(X_train, y_train)

rf = Pipeline([(&quot;model&quot;, RandomForestClassifier(**params))]).fit(X_train, y_train)

mlp.score(X_test, y_test)
rf.score(X_test, y_test)