title: Conditional keyed join/update _and_ update a flag column for matches
body: <p>This is very similar to the <a href="https://stackoverflow.com/questions/29658627/conditional-binary-join-and-update-by-reference-using-the-data-table-package">question</a> @DavidArenburg asked about conditional keyed joins, with an additional bugbear that I can't seem to suss out.</p>
  
  <p>Basically, in addition to a conditional join, I want to define a flag saying at which step of the matching process that the match occurred; my problem is that I can only get the flag to define for <em>all</em> values, not the matched values.</p>
  
  <p>Here's what I hope is a minimal working example:</p>
  
  <pre><code>DT = data.table(
    name = c("Joe", "Joe", "Jim", "Carol", "Joe",
             "Carol", "Ann", "Ann", "Beth", "Joe", "Joe"),
    surname = c("Smith", "Smith", "Jones",
                "Clymer", "Smith", "Klein", "Cotter",
                "Cotter", "Brown", "Smith", "Smith"),
    maiden_name = c("", "", "", "", "", "Clymer",
                    "", "", "", "", ""),
    id = c(1, 1:3, rep(NA, 7)),
    year = rep(1:4, c(4, 3, 2, 2)),
    flag1 = NA, flag2 = NA, key = "year"
  )
  
  DT
  #      name surname maiden_name id year flag1 flag2
  #  1:   Joe   Smith              1    1 FALSE FALSE
  #  2:   Joe   Smith              1    1 FALSE FALSE
  #  3:   Jim   Jones              2    1 FALSE FALSE
  #  4: Carol  Clymer              3    1 FALSE FALSE
  #  5:   Joe   Smith             NA    2 FALSE FALSE
  #  6: Carol   Klein      Clymer NA    2 FALSE FALSE
  #  7:   Ann  Cotter             NA    2 FALSE FALSE
  #  8:   Ann  Cotter             NA    3 FALSE FALSE
  #  9:  Beth   Brown             NA    3 FALSE FALSE
  # 10:   Joe   Smith             NA    4 FALSE FALSE
  # 11:   Joe   Smith             NA    4 FALSE FALSE
  </code></pre>
  
  <p>My approach is, for each year, to first try and match on first name/last name from a prior year; if that fails, then try to match on first name/maiden name. I want to define <code>flag1</code> to denote an exact match and <code>flag2</code> to denote a marriage.</p>
  
  <pre><code>for (yr in 2:4) {
  
    #which ids have we hit so far?
    existing_ids = DT[.(yr), unique(id)]
  
    #find people in prior years appearing to
    #  correspond to those people
    unmatched = 
      DT[.(1:(yr - 1))][!id %in% existing_ids, .SD[.N], by = id]
    setkey(unmatched, name, surname)
  
    #merge a la Arun, define flag1
    setkey(DT, name, surname)
    DT[year == yr, c("id", "flag1") := unmatched[.SD, .(id, TRUE)]]
    setkey(DT, year)
  
    #repeat, this time keying on name/maiden_name
    existing_ids = DT[.(yr), unique(id)]
    unmatched = 
      DT[.(1:(yr - 1))][!id %in% existing_ids, .SD[.N],by=id]
    setkey(unmatched, name, surname)
  
    #now define flag2 = TRUE
    setkey(DT, name, maiden_name)
    DT[year==yr &amp; is.na(id), c("id", "flag2") := unmatched[.SD, .(id, TRUE)]]
    setkey(DT, year)
  
    #this is messy, but I'm trying to increment id
    #  for "new" individuals
    setkey(DT, name, surname, maiden_name)
    DT[year == yr &amp; is.na(id),
       id := unique(
         DT[year == yr &amp; is.na(id)], 
         by = c("name", "surname", "maiden_name")
       )[ , count := .I][.SD, count] + DT[ , max(id, na.rm = TRUE)]
       ]
  
    #re-sort by year at the end    
    setkey(DT, year)    
  }
  </code></pre>
  
  <p>I was hoping that by including the <code>TRUE</code> value in the <code>j</code> argument while I define <code>id</code>, only the matched <code>name</code>s (e.g., Joe at the first step) would have their <code>flag</code> updated to <code>TRUE</code>, but this isn't the case--they are all updated:</p>
  
  <pre><code>DT[]
  #      name surname maiden_name id year flag1 flag2
  #  1: Carol  Clymer              3    1 FALSE FALSE
  #  2:   Jim   Jones              2    1 FALSE FALSE
  #  3:   Joe   Smith              1    1 FALSE FALSE
  #  4:   Joe   Smith              1    1 FALSE FALSE
  #  5:   Ann  Cotter              4    2  TRUE  TRUE
  #  6: Carol   Klein      Clymer  3    2  TRUE  TRUE
  #  7:   Joe   Smith              1    2  TRUE FALSE
  #  8:   Ann  Cotter              4    3  TRUE FALSE
  #  9:  Beth   Brown              5    3  TRUE  TRUE
  # 10:   Joe   Smith              1    4  TRUE FALSE
  # 11:   Joe   Smith              1    4  TRUE FALSE
  </code></pre>
  
  <p>Is there any way to update only the matched rows' <code>flag</code> values? Ideal output is as follows:</p>
  
  <pre><code>DT[]
  #      name surname maiden_name id year flag1 flag2
  #  1: Carol  Clymer              3    1 FALSE FALSE
  #  2:   Jim   Jones              2    1 FALSE FALSE
  #  3:   Joe   Smith              1    1 FALSE FALSE
  #  4:   Joe   Smith              1    1 FALSE FALSE
  #  5:   Ann  Cotter              4    2 FALSE FALSE
  #  6: Carol   Klein      Clymer  3    2 FALSE  TRUE
  #  7:   Joe   Smith              1    2  TRUE FALSE
  #  8:   Ann  Cotter              4    3  TRUE FALSE
  #  9:  Beth   Brown              5    3 FALSE FALSE
  # 10:   Joe   Smith              1    4  TRUE FALSE
  # 11:   Joe   Smith              1    4  TRUE FALSE
  </code></pre>