How can I create my own Google duplex?
First of all, you need to realise that you will not be able to do it. Google is a multi-billion dollar company, with a large number of very bright and well-funded researchers. That tells me that it is not something a single person can do by themselves.

Then, you already have some pre-conceptions about it. You want to use a machine learning approach, using neural networks. I would start with a blank slate and think about the problem first, before considering the tools I would want to use.

If you don't know were to start, I would suggest you have a look at Eliza, which is old but still very hard to improve on. There are many open source implementations of it available, and it wouldn't be too hard to implement your own from scratch.

Eliza uses a symbolic approach (ie no neural networks), which has the big advantage that you can see what is going on, and you can actually understand how it works. Have a play around with it, create some chatbots, and see how you can make the conversations sound more human-like. Then, when you have a clearer idea about this, proceed to other frameworks such as wit.ai or Microsoft's LUIS. There you trade in control for ease of use; but I suggest you don't do this without understanding what you're actually doing first.