text: Details
  - Type:
  Improvement
  - Status: Closed
  - Priority:
  Major
  - Resolution: Fixed
  - Affects Version/s: 1.3, 2.0.0
  -
  - Component/s: Replication
  - Labels:
  - Skill Level:Regular Contributors Level (Easy to Medium)
  Description
  patch to move couch replication modules in in a standalone couch_replicator application. It's also available on my github:
  Issue Links
  - is part of
  COUCHDB-1251 Factor out couch core and hook other compnonents through a module system
  - Open
  Activity
  Thanks Benoit.
  I tested the second patch, but test 002 is failing:
  fdmanana 14:00:20 ~/git/hub/couchdb4 (master)> ./test/etap/run -v src/couch_replicator/test/002-replication-compact.t
  src/couch_replicator/test/002-replication-compact.t ..
  - Current time local 2011-10-30 14:00:25
  - Using etap version "0.3.4"
  1..376
  Apache CouchDB 0.0.0 (LogLevel=info) is starting.
  Apache CouchDB has started. Time to relax.
  [info] [<0.2.0>] Apache CouchDB has started on
  ok 1 - Source database is idle before starting replication
  ok 2 - Target database is idle before starting replication
  - Test died abnormally: {'EXIT',
  {noproc,
  {gen_server,call,
  [couch_rep_sup,
  {start_child,
  {"e0ce0ffa71ae1915f953eb8d2f93e348+continuous",
  {gen_server,start_link,
  [couch_replicator,
  {rep,
  {"e0ce0ffa71ae1915f953eb8d2f93e348",
  Good work
  Thanks Filipe. Odd. I didn't reproduce that on my machine. Anyway third attempt. This one remove couch_rep_sup from couch.app and register couch_replicator_tasks_sup . All tests pass on this machine.
  All tests now pass except test/etap/001-load.t which needs some lines updated with the new module names.
  From my side, after that small change, it's ready to go into master.
  Thanks for doing this change Benoît.
  4th iteration of the patch.
  - fix 001-load.t,
  - add 01-load.t to couch_replicator
  - rename 00*.t to 0*.t in couch_replicator tests, we probably won't have more than 99 tests here.
  All tests are now passing for me.
  Just noticed that src/couch_replicator/ebin/couch_replicator.app should be added to .gitignore as wel, since it's modified after every build.
  +1 to go to master
  Thanks for doing this, Benoit, it seems like a really good idea. I have a few questions from reading the patch:
  - Why is couch_replicator.app committed to version control? The app.src should be sufficient.
  - The list of registered processes in the app.src file is incomplete. At the very least couch_replicator_manager belongs in there.
  - I have to say it: our application startup procedure is a huge mess. This patch doesn't necessarily make matters any worse in that regard, but unfortunately it doesn't improve matters either. It starts the top-level supervisor of the couch_replicator application from inside the couch application, but never actually starts the couch_replicator application. You and I both know what the Right Way looks like at this point, hopefully we'll bring Apache CouchDB up to snuff soon. The current patch might be OK, though if I'm reading it correctly application:start(couch_replicator) will crash if the couch application is already running. Yuck.
  - Are the record definitions in couch_replicator.hrl meant for use by other OTP applications? If not I think we should move the .hrl file to src/. Reserving include/ for headers needed to use the application is a good practice.
  - Nitpick - I prefer to name children in the supervision tree after the callback module. I made the mistake of calling the old child couch_replication_supervisor instead of couch_rep_sup a long time ago and it's bugged me ever since. Can we rename the new child to couch_replicator_tasks_sup instead?
  - Now that the replicator is shaped a little more like an OTP application it bugs me that couch_replicator is both the API module and a gen_server. I don't want to see it changed in this commit, but down the line I'd like to split that out so we have a different module that actually manages the various processes associated with a replication task.
  Again, this is a big step in the right direction. Thanks for getting it going.
  I've upgraded my branch with latest changes in replication. Following are my answers:
  - couch_replicator.app is commited? Shouldn't be the case now.
  - couch_replicator_manager is now registered . I'm not sure why we need to register others here. What would be the benefit.
  - About records, I'm not sure if it could be usable actually for other things. However, that is something we could change easily in another patch I think once we are sure it couldn't be used in another parts?
  - nitpick, fixed.
  - I agree, more generally we could have such things in couch_core, for cache support as well imo.
  Side points. I think we could benefit here by using a "simple_one_for_one" supervisor for replication tasks rather a "one_for_one" . So it would simplify the calll and termination handling imo. We are in a case where this kind of supervision works pretty well. In near future I really would like couchdb built as a full erlang release where couch core and others are applications booted on start. Lot of things would benefit from it.
  Ifit's OK I will commit that change later today.
  Is the link still ? I see a new commit from you but going through the checklist I see that couch_replicator.app is still being committed.
  Thanks Adam, not sure why it has been committed at first... Anyway I've committed a new change :
  which fix that issue. If this is ok i will merge the changes and commit it to the head.
  This could just be the diff being weird, but does that Makefile.am work? The indentation doesn't look correct for the targets. Could just be the diff but I thought I'd mention it just in case.
  @davisp I fixed the indentation in
  but makefile was working without.
  Looks like I should've clicked through to the file view. Apparently GitHub has decided to start representing tabs as two space indents in diffs. +1 from me then.
  LGTM Benoit
  rework previous patch and split it in two. The first one is moving parts around in the couch_replicator application folder and make sure tests pass and build work. The second is refactoring the couch_replicator application to put all the modules under the couch_replicator namespace.
  All tests pass.
  Let me know what you think about it. I would like it committed asap.
  Full patch is available in my repo :
  Thanks, Benoit. Indentation in couch_replicator.app.src is inconsistent, but otherwise everything looks good. +1.
  new version of 0002-refactor-couch_replicator.patch fixing inconsistency in indentation . spotted by randall , thanks!
  Benoît, just tried out your github branch. All tests pass for me and it looks ok, so +1
  Looking at .gitignore changes, I see the following line added and I'm curious about it.
  +.sw
  What are the .sw files?
  Is there any reason not to ignore .* everywhere in the project instead?
  For example, I often blow away my .emacs (custom couchdb dev settings) and .emacs.deskop files doing a clean.
  Do we have any .* files that are legitimately included (aside from .gitignore)?
  A quick look (find . -iname ".*") doesn't show anything outside .git (aside from .gitignore) that's actually checked in.
  I can't think of a reason not to.
  In other news, for vim users, you can use this in your vimrc to avoid the whole silly swap file issue in all projects. Found it a long time ago and have since forgotten about the dumb swap file thing completely.
  set directory=~/.backup/vim/swap
  set backupdir=~/.backup/vim
  set backup
  @paul nice trick thanks.
  I will commit the patch later in the morning then. Thanks for the review.
  committed to the master.
  fix httpc_pool tests. All tests are now working.