Paragraphs from sub_process I need to migrate a json feed with following example structure;


{  
   "data":[  
      {  
         "id":"11028",
         "first_name":"Foo",
         "last_name":"Bar",
         "functions":{  
            "group_function":{  
               "data":[  
                  {  
                     "group":"1",
                     "function":"Scrummaster",
                     "active":"1"
                  },
                  {  
                     "group":"2",
                     "function":"Developer",
                     "active":"0"
                  }
               ]
            }
         }
      },
      {  
         "id":"11029",
         "first_name":"John",
         "last_name":"Doe",
         "functions":{  
            "group_function":{  
               "data":[  
                  {  
                     "group":"1",
                     "function":"Project manager",
                     "active":"1"
                  },
                  {  
                     "group":"2",
                     "function":"Project owner",
                     "active":"1"
                  }
               ]
            }
         }
      }
   ]
}


For this, I was thinking of having a teammember node, and a teamfunction paragraph type for the functions for each team member.

If i understand migrate api correctly, i create 2 migrations for this. One for the node and one for the paragraphs.

How can I iterate over the group_function/data array within my paragraphs migration?

I tried:


source:
  plugin: url
  data_fetcher_plugin: http
  data_parser_plugin: json
  headers:
    Accept: 'application/json; charset=utf-8'
    Content-Type: 'application/json'
  urls: example.com/team.json
  item_selector: data/functions


But I get following response on a 'drush ms':
Could not retrieve source count from team_function: Passed variable is not an array or object

How would you do this?
You could also create a process plugin to create the Paragraph inline with the record you are migrating. I can't do piecemeal migrations in my case. The project has about 100,000 nodes and no data structure to 'create' Paragraphs out of. Even if I could, I don't want to have to run that dozens of times to get a full set of data.

Anyway.

Here is a real example:

&lt;?php

namespace Drupal\mymodule_migration\Plugin\migrate\process;

use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\Row;
use Drupal\paragraphs\Entity\Paragraph;

/**
 * @MigrateProcessPlugin(
 *   id = "published_author_paragraph_item",
 *   handle_multiples = TRUE
 * )
 */
class PublishedAuthorParagraphItem extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    $paragraphs = [];

    foreach ($value as $item) {
      $paragraphs[] = $this-&gt;createParagraphItem($item);
    }

    return $paragraphs;
  }

  /**
   * {@inheritdoc}
   */
  public function multiple() : bool {
    return TRUE;
  }

  /**
   * Generates a paragraph item entity.
   *
   * @param array $value
   * @return array
   * @throws \Drupal\Core\Entity\EntityStorageException
   * @throws \Drupal\migrate\MigrateException
   */
  protected function createParagraphItem(array $value) : array {
    $paragraph = Paragraph::create([
      'type' =&gt; 'paper_authors',
      'field_paper_author' =&gt; [
        'target_id'  =&gt;  $value['user_id'],
      ],
      'field_paper_published_name' =&gt; [
        'value'  =&gt;  $value['name'],
      ],
      'field_paper_disclosure' =&gt; [
        'value'  =&gt;  $value['disclosure'],
        'format' =&gt; 'basic_html',
      ],
    ]);

    $paragraph-&gt;save();

    return [
      'target_id' =&gt; $paragraph-&gt;id(),
      'target_revision_id' =&gt; $paragraph-&gt;getRevisionId(),
    ];
  }
}


In my migration definition yaml:

  _authors_shim:
    plugin: generate_authors_array
    source: author_array
  field_paper_published_authors:
    plugin: published_author_paragraph_item
    source: '@_authors_shim'


generate_authors_array is another process plugin I use to massage the data for the plugin code above. This paragraph type has a user reference, a plain text field and long text field. The passed value is an array of arrays, so this will in fact create multiple paragraph deltas for the paragraph field I am filling. It also rolls back properly.

I had tried to get this to work with sub_process at first, but could not get the paragraph field to populate at all until I went this route. I believe this is due to paragraph fields hold references to their stored entities (target_id, target_revision_id).