unable to run mongo-connector
Question: I have installed mongo-connector in the mongodb server.

I am executing by giving the command

    
    
    mongo-connector -m [remote mongo server IP]:[remote mongo server port] -t [elastic search server IP]:[elastic search server Port]  -d elastic_doc_manager.py
    

I also tried with this since mongo is running in the same server with the
default port.

    
    
    mongo-connector -t [elastic search server IP]:[elastic search server Port]  -d elastic_doc_manager.py
    

I am getting error

    
    
    Traceback (most recent call last):
      File "/usr/local/bin/mongo-connector", line 9, in <module>
        load_entry_point('mongo-connector==2.3.dev0', 'console_scripts', 'mongo-connector')()
    
      File "/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/util.py", line 85, in wrapped
        func(*args, **kwargs)
    
      File "/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/connector.py", line 1037, in main
        conf.parse_args()
    
      File "/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/config.py", line 118, in parse_args
        option, dict((k, values.get(k)) for k in option.cli_names))
    
      File "/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/connector.py", line 820, in apply_doc_managers
        module = import_dm_by_name(dm['docManager'])
    
      File "/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/connector.py", line 810, in import_dm_by_name
        "Could not import %s." % full_name)
    
    **mongo_connector.errors.InvalidConfiguration: Could not import mongo_connector.doc_managers.elastic_doc_manager.py.**
    

NOTE: I am using python2.7 and mongo-connector 2.3

Elastic search server is 2.2

Any suggestions ?

**[edit]** After applying `Val`'s suggestion:

> 2016-02-29 19:56:59,519 [CRITICAL] mongo_connector.oplog_manager:549 -
> Exception during collection dump
>
> Traceback (most recent call last):
>
> File "/usr/local/lib/python2.7/dist-
> packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/oplog_manager.py",
> line 501, in do_dump
>
> upsert_all(dm)
>
> File "/usr/local/lib/python2.7/dist-
> packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/oplog_manager.py",
> line 485, in upsert_all dm.bulk_upsert(docs_to_dump(namespace), mapped_ns,
> long_ts)
>
> File "/usr/local/lib/python2.7/dist-
> packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/util.py", line
> 32, in wrapped
>
> return f(*args, **kwargs)
>
> File "/usr/local/lib/python2.7/dist-
> packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/doc_managers/elastic_doc_manager.py",
> line 190, in bulk_upsert
>
> for ok, resp in responses:
>
> File "/usr/local/lib/python2.7/dist-
> packages/elasticsearch-1.9.0-py2.7.egg/elasticsearch/helpers/init.py", line
> 160, in streaming_bulk
>
> for result in _process_bulk_chunk(client, bulk_actions, raise_on_exception,
> raise_on_error, **kwargs):
>
> File "/usr/local/lib/python2.7/dist-
> packages/elasticsearch-1.9.0-py2.7.egg/elasticsearch/helpers/init.py", line
> 132, in _process_bulk_chunk
>
> raise BulkIndexError('%i document(s) failed to index.' % len(errors),
> errors)
>
> BulkIndexError: (u'2 document(s) failed to index.',..document_class=dict,
> tz_aware=False, connect=True, replicaset=u'mss'), u'local'), u'oplog.rs')
>
> 2016-02-29 19:56:59,835 [ERROR] mongo_connector.connector:302 -
> MongoConnector: OplogThread unexpectedly stopped! Shutting down

* * *

Hi Val,

I connected with another mongodb instance, which had only one database, having
one collection with 30,000+ records and I was able to execute it succesfully.
The previous mongodb collection has multiple databases (around 7), which
internally had multiple collections (around 5 to 15 per databases) and all
were having good amount of documents (ranging from 500 to 50,000) in the
collections.

Was Mongo-connector failing because of huge data residing in the mongo
database ?

I have further queries

a. Is is possible to get indexing done of only specific collections in the
mongodb, residing in different databases? I wan to index only specific
collections (not the entire database). How can I achieve this ?

b. In elasticsearch i can see duplicate indexes for one collection. First one
is with the database name (as expected), other one with the name mongodb_meta,
both of them having same data, if I am changing the collection, the update is
happening in both the collections.

c. Is it possible to configure the output index name or any other parameters
any how?


Answer: I think the only issue is that you have the `.py` extension on the doc manager
(it was needed before mongo-connector 2.0), you simply need to remove it:

    
    
    mongo-connector -m [remote mongo server IP]:[remote mongo server port] -t [elastic search server IP]:[elastic search server Port]  -d elastic_doc_manager
    

