How to determine reason for unprocessed files in Drupal to Drupal migration Finally ran my first file migration from Drupal 6 to Drupal 7 using UI.


3100 Items 
3036 Processed 
62 Unprocessed 
2498 messages


I am simply trying to figure out what those 62 files are, and why they did not process.

I looked at the "messages" link for the file migration task. They all say something like:

2701   Informational   Deleted documents from index with query id:"pyli4t/file/3782" OR sm_parent_document_id:"pyl4t/file/3782" Don't know what to do with this, or how to query for unprocessed files.

I looked at the migrate_map_43b7eadb1file table for the migration task, but don't see anything that indicates what files were not processed.

I looked at the migrate_message_43b7eadb1file table and was able to locate the two files that failed using this SQL statement: SELECT * FROM migrate_message_43b7eadb1file WHERE level != 4 But, nothing about unprocessed files.

That's all I can think to do. Any suggestions?
It looks like this poster received an answer from the Drupal-to-Drupal data migration module maintainer over at drupal.org, so for convenience I will post it here:

Source:Issue#2476077: How to determine reason for unprocessed files


  Side issue first - that informational message is coming from some
  contrib module (something search-related, perhaps) and not related to
  migration per-se. Grep your codebase for "Deleted documents from
  index" to see what it might be.
  
  So, the unprocessed count is precisely the count of source records,
  minus the number of records in the map table (so, you will never find
  an "unprocessed" record in the map table. All processed records are
  recorded in the map table with an appropriate status.
  
  The first question is, are there really unprocessed records, or is the
  source count off? The most common cause of "unprocessed" records being
  reported is a malformed source query that has more than one row with
  the same primary key (fid in this case) - this can happen when joining
  to another table that has multiple rows per "base" row. Elsewhere
  you've been speaking about extending migrate_d2d classes and modifying
  their queries - could you have done that for the file query, and
  joined to another table?
  
  The next step is to try to identify specific unprocessed files. If
  your D6 and D7 databases are on the same MySQL server and accessible
  with the same MySQL user, you can do something like:

select d6.fid 
from d6.files f
left join d7.migrate_map_myfilemigration map on f.fid=map.sourceid1
where map.sourceid1 is null

  
  If the dbs are not on the same server, you could try dumping the D6
  files table (or just the fids) and loading it into a temp table on the
  D7 server.