Analog circuits for neural networks?
Is there a possibility to develop an analog-based ASIC for neural
  networks?


This had been already implemented a long time ago. 

I guess, no one ever seriously supposes that this is impossible. There're many implementations in papers since 70th. They're still not of industry-standard quality, and with some limitations. However, they're real prototypes among them. 


  error-proneness of analog circuits shouldn't be a problem


Unfortunately, it's and it's just one of a bunch of issues. 

A general view of this field. Analog ASICs is one of the existed hardware bases for implementing neural networks along with digital ASICs, FPGAs, custom on-chip implementations, massive-parallel computers, general-purpose CPUs and GPUs. Moreover, it's not the most popular approach, currently, because of its limitations in precision and storing data overhead. Also, the analog computations aren't the mainstream, so, there's a lack of development and specialists. Analog computational elements are a kind of different world with specific problems for engineers. Hence, their cost-efficient parameters are far behind even common general purpose CPUs and they are in a narrow use. 

However, the analog ASICs have positive sides - yes, on a hardware components level, they are immensely close to the neural networks (biological) which have deeper parallelism than mainstream implementations.

A lot of references for further reading can be found here: Omondi, Amos R., eds. FPGA implementations of neural networks. Vol. 365., Springer, 2006.