A simple analog neural network to implement the XOR operation
The paper has an appendix where they name the elements they used and a few other details. I'll just focus on your question:

How do I design the non-linearity and bidirectional amplifier ?

For the nonlinearity, they added two anti-parallel diodes with voltage sources for pre-polarization. For diodes, appendix C.2 gives Is=1u and n=2, so you can add a diode with this .model card:
.model D D Is=1u n=2 ; Rs=10m Cjo=10p

The extra Rs and Cjo are minor additions that can help with convergence or with possible unwanted oscillations, if needed (just uncomment them by removing the ;). Rs is mostly to have a series resistance (damping), and Cjo helps by smoothing out sharp derivatives to a sufficiently small transition. As I said, &quot;if&quot;. The sources are given as 0.3 V and -0.7 V.
The VCVS-CCCS pair is known as an &quot;ideal transformer&quot;, or a &quot;DC transformer&quot;, and it acts like an inductorless transformer: the load draws current from the VCVS, and the CCCS takes that back to the source. And &quot;DC&quot; because it will happily deliver DC.
The whole &quot;nonlinear amplifier&quot; looks like this (the values for V1 and I1 are random):

I wanted to recreate the example on page 29, but they are using iterations to reach those values and the output currents are not shown. That's not something you can just throw in values in there and hope to work, so that's a part for another day.