title: Capture requests (XHR, JS, CSS) from embedded iframes using devtool protocol
body: <p>For the context, I am developing a synthetic monitoring tool using Nodejs and puppeteer.
  For each step of a defined scenario, I capture a screenshot, a waterfall and performance metrics.</p>
  <p>My problem is on the waterfall, I previously used puppeter-har but this package is not able to capture request outside of a navigation.
  Therefore I use this piece of code to capture all interesting requests :</p>
  <pre><code>const {harFromMessages} = require('chrome-har');
  // Event types to observe for waterfall saving (probably overkill, I just set all events of Page and Network)
  const observe = [
    'Page.domContentEventFired',
    'Page.fileChooserOpened',
    'Page.frameAttached',
    'Page.frameDetached',
    'Page.frameNavigated',
    'Page.interstitialHidden',
    'Page.interstitialShown',
    'Page.javascriptDialogClosed',
    'Page.javascriptDialogOpening',
    'Page.lifecycleEvent',
    'Page.loadEventFired',
    'Page.windowOpen',
    'Page.frameClearedScheduledNavigation',
    'Page.frameScheduledNavigation',
    'Page.compilationCacheProduced',
    'Page.downloadProgress',
    'Page.downloadWillBegin',
    'Page.frameRequestedNavigation',
    'Page.frameResized',
    'Page.frameStartedLoading',
    'Page.frameStoppedLoading',
    'Page.navigatedWithinDocument',
    'Page.screencastFrame',
    'Page.screencastVisibilityChanged',
    'Network.dataReceived',
    'Network.eventSourceMessageReceived',
    'Network.loadingFailed',
    'Network.loadingFinished',
    'Network.requestServedFromCache',
    'Network.requestWillBeSent',
    'Network.responseReceived',
    'Network.webSocketClosed',
    'Network.webSocketCreated',
    'Network.webSocketFrameError',
    'Network.webSocketFrameReceived',
    'Network.webSocketFrameSent',
    'Network.webSocketHandshakeResponseReceived',
    'Network.webSocketWillSendHandshakeRequest',
    'Network.requestWillBeSentExtraInfo',
    'Network.resourceChangedPriority',
    'Network.responseReceivedExtraInfo',
    'Network.signedExchangeReceived',
    'Network.requestIntercepted'
  ];
  </code></pre>
  <p>At the start of the step :</p>
  <pre><code>// list of events for converting to HAR
    const events = [];
  
    client = await page.target().createCDPSession();
    await client.send('Page.enable');
    await client.send('Network.enable');
    observe.forEach(method =&gt; {
      client.on(method, params =&gt; {
        events.push({ method, params });
      });
    });
  </code></pre>
  <p>At the end of the step :</p>
  <pre><code>waterfall = await harFromMessages(events);
  </code></pre>
  <p>It works good for navigation events, and also for navigation inside a web application.
  However, the web application I try to monitor has iframes with the main content.
  I would like to see the iframes requests into my waterfall.</p>
  <p>So a few question :</p>
  <ul>
  <li>Why is Network.responseReceived or any other event doesn't capture this requests ?</li>
  <li>Is it possible to capture such requests ?</li>
  </ul>
  <p>So far I've red the devtool protocol documentation, nothing I could use.
  The closest to my problem I found is this question :
  <a href="https://stackoverflow.com/questions/61512722/how-can-i-receive-events-for-an-embedded-iframe-using-chrome-devtools-protocol">How can I receive events for an embedded iframe using Chrome Devtools Protocol?</a></p>
  <p>My guess is, I have to enable the Network for each iframe I may encounter.
  I didn't found any way to do this. If there is a way to do it with devtool protocol, I should have no problem to implement it with nodsjs and puppeteer.</p>
  <p>Thansk for your insights !</p>
  <p>EDIT 18/08 :</p>
  <p>After more searching on the subject, mostly Out-of-process iframes, lots of people on the internet point to that response :
  <a href="https://bugs.chromium.org/p/chromium/issues/detail?id=924937#c13" rel="nofollow noreferrer">https://bugs.chromium.org/p/chromium/issues/detail?id=924937#c13</a></p>
  <p>The answer is question states :</p>
  <p>Note that the easiest workaround is the --disable-features flag.</p>
  <blockquote>
  <p>That said, to work with out-of-process iframes over DevTools protocol,
  you need to use Target [1] domain:</p>
  <ul>
  <li>Call Target.setAutoAttach with flatten=true;</li>
  <li>You'll receive Target.attachedToTarget event with a sessionId for the iframe;</li>
  <li>Treat that session as a separate &quot;page&quot; in chrome-remote-interface. Send separate protocol messages with additional sessionId field:<br />
  {id: 3, sessionId: &quot;&quot;, method: &quot;Runtime.enable&quot;, params:
  {}}</li>
  <li>You'll get responses and events with the same &quot;sessionId&quot; field which means they are coming from that frame. For example:<br />
  {sessionId: &quot;&quot;, method: &quot;Runtime.consoleAPICalled&quot;,
  params: {...}}</li>
  </ul>
  </blockquote>
  <p>However I'm still not able to implement it.</p>
  <p>I'm trying this, mostly based on puppeteer :</p>
  <pre><code>  const events = [];
    const targets = await browser.targets();
    const nbTargets = targets.length;
    for(var i=0;i&lt;nbTargets;i++){
      console.log(targets[i].type());
      if (targets[i].type() === 'page') {
        client = await targets[i].createCDPSession();
  
        await client.send(&quot;Target.setAutoAttach&quot;, {
          autoAttach: true,
          flatten: true,
          windowOpen: true,
          waitForDebuggerOnStart: false // is set to false in pptr
        })
  
        await client.send('Page.enable');
        await client.send('Network.enable');
  
        observeTest.forEach(method =&gt; {
          client.on(method, params =&gt; {
            events.push({ method, params });
          });
        });
      }
    };
  </code></pre>
  <p>But I still don't have my expected output for the navigation in a web application inside an iframe.</p>
  <p>However I am able to capture all the requests during the step where the iframe is loaded.
  What I miss are requests that happened outside of a proper navigation.</p>
  <p>Does anyone has an idea about the integration into puppeteer of that chromium response above ? Thanks !</p>