Adding mongoDB document-array-element using Python Eve
Question: **Background:** (using Eve and Mongo)

I'm working in Python using the [Eve](http://python-eve.org) REST provider
library connecting and to a mongoDB to expose a number of REST endpoints from
the database. I've had good luck using Eve so far, but I've run into a problem
that might be a bit beyond what Eve can do natively.

My problem is that my mongoDb document format has a field (called "slots"),
whose value is a list/array of dictionaries/embedded-documents.

So the mongoDB document structure is:

    
    
    {
       blah1: data1,
       blah2: data2,
       ...
       slots: [
           {thing1:data1, thing2:data2},
           {thingX:dataX, thingY:dataY}
       ]
    }
    

I need to add new records (I.E. add pre-populated dictionaries) to the 'slots'
list.

If I imagine doing the insert directly via pymongo it would look like:

    
    
    mongo.connection = MongoClient()
    mongo.db = mongo.connection['myDB']
    mongo.coll = mongo.db['myCollection']
    
    ...
    
    mongo.coll.update({'_id' : document_id}, 
                      {'$push': { "slot" : {"thing1":"data1","thingX":"dataX"}  }  } )
    

The REST action/URI combo that I would like to do this action is a `POST` to
'_id/slots', e.g. URI of `/app/012345678901234567890123/slots`.

**Problem:** (inserting an element into an array in Eve)

From SO: [How to add to a list type in Python Eve without replacing old
values](http://stackoverflow.com/questions/29659942/how-to-add-to-a-list-type-
in-python-eve-without-replacing-old-values) and [eve project
issue](https://github.com/nicolaiarocci/eve/issues/229) it appears Eve doesn't
currently support operating on mongoDB embedded documents (or arrays?) unless
the entire embedded document is rewritten, and rewriting the whole array is
very undesirable in my case.

* * *

So, assuming its true Eve doesn't have a method to allow inserting of array
elements (and given I already have numerous other endpoints working well
inside of Eve)...

* * *

... I'm now looking for a way, inside of an Eve/Flask configuration with
multiple working endpoints, to intercept and change Eve's mongoDB write for
just this one endpoint.

I know (worst case) I can override the routing of Eve and to completely do the
write by hand, but then I would have manage the `_updated` and hand check &
change the documents `_etag` value, both things I would certainly prefer not
to have to write new code for.

I've looked at Eve's [Datebase event hooks](http://python-
eve.org/features.html#database-event-hooks) but I don't see a way to modify
the database commands that are executed (I can see how to change the _data_ ,
but not the _commands_).

Anyone else already solved this problem already? If not any ideas on the most
direct path to implement by hand? (hopefully reusing as much of Eve as
possible because I do want to continue using Eve for all my (already working)
endpoints)


Answer: This is an interesting question. I believe that in order to achieve your goal
you would need to perform two actions:

  1. Build and pass a custom Validator.
  2. Build and pass a custom Mongo data layer.

This might sound like too much work, but that's probably not the case.

* * *

## Custom Validator

A custom validator is going to be needed because when you perform your PATCH
request on the "push-enabled" endpoint you want to pass a document which is
syntactically different from endpoint validation schema. You are going to pass
a dict (`{"slot": {"thing1": "data1", "thingX": "dataX"}}`) whereas the
endpoint expects a list:

    
    
    'mycollection': {
        'type': 'list',
        'schema': {
            'type': 'dict',
            'schema': {
                'thing1': {'type': 'string'},
                'thingX': {'type': 'string'},
            }
        }
    }
    

If you don't customize validation you will end up with a validation error
(`list type expected`). I guess your custom validator could look something
like:

    
    
    from eve.data.mongo.validation import Validator
    from flask import request
    
    class MyValidator(Validator):
        def validate_replace(self, document, _id, original_document=None):
            if self.resource = 'mycollection' and request.method = 'PATCH':
                # you want to perform some real validation here
                return True
            return super(Validator, self).validate(document)
    

Mind you I did not try this code so it might need some adjustment here and
there.

An alternative approach would be to set up an alternative endpoint just for
PATCH requests. This endpoint would consume the same datasource and have a
dict-like schema. This would avoid the need for a custom validator and also,
you would still have normal atomic field updates (`$set`) on the standard
endpoint. Actually I think I like this approach better, as you don't lose
functionality and reduce complexity. For guidance on multiple endpoints
hitting the same datasource see [the docs](http://python-
eve.org/config.html#multiple-api-endpoints-one-datasource)

* * *

## Custom data layer

This is needed because you want to perform a `$push` instead of a `$set` when
`mycollection` is involved in a PATCH request. Something like this maybe:

    
    
    from eve.io.mongo import Mongo
    from flask import request
    
    class MyMongo(Mongo):
        def update(self, resource, id_, updates, original):
            op = '$push' if resource == 'mycollection' else '$set'
            return self._change_request(resource, id_, {op: updates}, original)
    

* * *

## Putting it all together

You then use your custom validator and data layers upon app initialisation:

    
    
    app = Eve(validator=MyValidator, data=MyMongo)
    app.run()
    

Again I did not test all of this; it's Sunday and I'm on the beach so it might
need some work but it _should_ work.

With all this being said, I am actually going to experiment with adding
support for push updates to the standard Mongo data layer. A new pair of
global/endpoint settings, like `MONGO_UPDATE_OPERATOR`/`mongo_update_operator`
are implemented on a private branch. The former defaults to `$set` so all API
endpoints still perform atomic field updates. One could decide that a certain
endpoint should perform something else, say a `$push`. Implementing validation
in a clean and elegant way is a little tricky but, assuming I find the time to
work on it, it is not unlikely that this could make it to Eve 0.6 or beyond.

Hope this helps.

