Using Tensorflow object detection API vs Keras
Keras provides you high level api or can say wrapper written on top of multiple backends.
These back ends have the core implementation of DNN.
List of Keras supported backends are:


Tensorflow
Theano
CNTK


**Source: Keras documentation for supported backends


  Keras hides a bit complexity of DNN implementation, but again restrict
  your freedom. In case if you write a code in Tensorflow, you have
  explicitly specifies and calculate optimizer, cost function and other
  things, but it provides you flexibility.


So for me writing in Keras just a convenience.

As far my knowledge is concern, so far we dont have any fix formula to identify number of layers sufficient for Object detection :).