<NME> CHANGELOG.md <BEF> # CHANGELOG - sitespeed.io ## UNRELEASED ### Fixed * Screenshot URLs in the HAR file was hardcoded to png, see [https://github.com/sitespeedio/compare/issues/11](https://github.com/sitespeedio/compare/issues/11). That made jpg image links broken in compare.sitespeed.io. * Show start/end date for collected Crux data [#3740](https://github.com/sitespeedio/sitespeed.io/pull/3740) ### Fixed * Better explanation for some PageXray metrics [#3743](https://github.com/sitespeedio/sitespeed.io/pull/3743). ## 26.0.1 - 2022-09-27 ### Fixed * Upgrading to [Browsertime 16.17.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#16170---2022-09-27) that contains a fix to collect battery temperature on Android, making it possible to send the data to Graphite again. ## 26.0.0 - 2022-09-23 Hi and welcome to 26.0.0! There's a couple of small fixes and additions and one breaking change. You are only affected by the breaking change if you used to use `--graphite.perIteration` (sending data for all iterations to Graphite) or if you used `--graphite.skipSummary` (do not send summary information). Please read the changed section if you are affected! When you upgrade to 26.0.0 you will have an easier way of sending data per run to Graphite, there will be a blog post better explaining how you can use it. ### Changed Sending metrics per run to Graphite: * The default setup did miss a lot of important performance metrics, so you needed to set them up yourself. That is fixed in this PR. * There where a lot of data sent from PageXray, third party and the coach per run. That was not smart since those metrics rarely change between runs and take a lot of space. This PR sets default so none of those metrics are sent * We introduce a limited set of run metrics from Browsertime (visual metrics and Google Web Vitals and a couple of more) that can be used when sending data per run. This will help you keep track of those metrics together with the default median/min/max values. More info coming up. It's enabled by default, disable it with `--browsertime.limitedRunData false` * Sending data per run to Graphite was broken: We sent a new key per run meaning it will take up a lot of extra space in Graphite. With this fix we send them under the run key. That way we can configure Graphite to keep data under that key that happened every 20 s (or however fast it takes to do one run) and then automatically remove the data after a week. * Graphite configuration `--graphite.perIteration` and `--graphite.skipSummary` is removed. You can now configure which data to send to Graphite by using `--graphite.messages`. By default we send _pageSummary_ (data summarised per URL) and _summary_ (data summarised per domain). If you want to send _pageSummary_ and _run_ data (data for each run) you can do that with by adding `--graphite.messages run` `--graphite.messages pageSummary`. * We removed the possibility to send VisualProgress and videoRecordingStart data to the datasource since that is something you do not need there. * We updated Grafana and the Graphite container to latest versions. The Graphite container contains _storage-schemas.conf_ configuration that is a good default: ``` [sitespeed_crux] pattern = ^sitespeed_io\.crux\. retentions = 1d:1y [sitespeed_run] pattern = ^sitespeed_io\.(.*)\.(.*)\.run\. retentions = 20s:8d [sitespeed] pattern = ^sitespeed_io\. retentions = 30m:40d ``` When you send data per run to Graphite it is stored every 20 second (do not make runs more often than that) and saved for 8 days. If you test many URLs this can still be a lot of data so use https://m30m.github.io/whisper-calculator/ to calculate how much space you need. See PR [#3721](https://github.com/sitespeedio/sitespeed.io/pull/3721). ### Added * Checkout the [pre built Raspberry Pi image](https://github.com/sitespeedio/raspberrypi) for running sitespeed.io tests on your Android phone. * Upgraded to Firefox 105 and Edge 105 in the Docker container. * Upgraded to Browsertime 16.16.0 * Include --preURL information in the latest storer info [#3729](https://github.com/sitespeedio/sitespeed.io/pull/3729). ### Fixed * Fix graphite.sendAnnotation option [#3726](https://github.com/sitespeedio/sitespeed.io/pull/3726). * Show timestamp when each run happens on the run page [#3730](https://github.com/sitespeedio/sitespeed.io/pull/3730). ## 25.11.0 - 2022-09-04 ### Added * Make it possible to configure run options for AXE (before you could only configure configuration options) [#3718](https://github.com/sitespeedio/sitespeed.io/pull/3718). Checkout [how to configure AXE](https://www.sitespeed.io/documentation/sitespeed.io/axe/#configure-axe).* Removed showing if the page is an AMP page (that battle was won a long time ago) and instead show information from the Network information API when its available [#3719](https://github.com/sitespeedio/sitespeed.io/pull/3719). ## 25.10.0 - 2022-08-31 ### Added * Updated to Chrome 105 and Firefox 104 in the Docker container [#3717](https://github.com/sitespeedio/sitespeed.io/pull/3717). ## 25.9.1 - 2022-08-30 ### Fixed * Updated to [Browsertime 16.5.1](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#16151---2022-08-30) thar makes sure long tasks are measured direct after the test finish. This fixes late long tasks that happens on slow devices caused by Browsertime collecting metrics. ## 25.9.0 - 2022-08-30 ### Added * Updated to Browsertime 16.15.0 that include your Android phones connected wifi name in the result. That is now showed in the runtime settings page. * Show relative standard deviation on the compare runs page [#3716](https://github.com/sitespeedio/sitespeed.io/pull/3716). ## 25.8.3 - 2022-08-28 ### Fixed * Ooops, it turns out 25.8.2 didn't fix the preWarm issue, it's fixed in [3715]( https://github.com/sitespeedio/sitespeed.io/pull/3715). ## 25.8.2 - 2022-08-26 ### Fixed * Updated Browsertime to 16.14.2 that fixes `--browsertime.preWarmServer` on Android and iOS. ## 25.8.1 - 2022-08-26 ### Fixed * Updated Browsertime to 16.14.1 that adds a guard for null values in rsd (this fixes the warning logs happening in 25.8.0). ## 25.8.0 - 2022-08-26 ### Added * Updated to [Browsertime 16.14.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#16140---2022-08-26). that collects relative standard deviation for metrics and the number of processes that runs on the server when you start a test. ## 25.7.3 - 2022-08-17 ### Fixed * Upgraded to [Browsertime 16.13.3](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#16133---2022-08-17) that fixes a bug when setting cookies when navigating in Chrome/Edge when using scripting. ## 25.7.2 - 2022-08-14 ### Fixed * Getting only some data from the CrUX API broke the HTML generation as reported in [#3708](https://github.com/sitespeedio/sitespeed.io/issues/3708) and fixed in [#3709](https://github.com/sitespeedio/sitespeed.io/pull/3709). * Updated to [Browsertime 16.13.2](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#16132---2022-08-14) that fixes so the lastCPULong task is not added to the HAR. ## 25.7.1 - 2022-08-10 ### Fixed * The +1 container now uses PSI plugin version 4.1.0 (before 4.0.1). * Updated to [Browsertime 16.13.1](https://github.com/sitespeedio/browsertime/releases/tag/v16.13.1) that always get the id from the Android phone when you run android tests. * The latest storer plugin sometimes missed writing Android data to the JSON. Fixed in [#3707](https://github.com/sitespeedio/sitespeed.io/pull/3707). ## 25.7.0 -2022-08-08 ### Added * Updated the Docker container to use Chrome 104 and Edge 104. * Updated to [Browsertime 16.12.0](https://github.com/sitespeedio/browsertime/releases/tag/v16.12.0) that uses Chromedriver 104. * Updated to [Browsertime 16.13.0](https://github.com/sitespeedio/browsertime/releases/tag/v16.13.0) that uses Edgedriver 104 and updated Selenium version. ## 25.6.0 - 2022-08-01 ### Added * Updated to Firefox 103 in the Docker container. * The +1 container uses Lighthouse 9.6.4. * Removed videojs and use plain video element to display the video [#3697](https://github.com/sitespeedio/sitespeed.io/pull/3697). This makes the result pages cleaner. ### Fixed * Fixed error message when you run Axe and analysisstorer at the same time [#3703](https://github.com/sitespeedio/sitespeed.io/pull/3703). ## 25.5.1 - 2022-07-16 ### Fixed * No change but when 25.5.0 was publsihed Docker had problems so the container was not pushed as reported by [Marco Fontani](https://github.com/mfontani) (thank you!). ## 25.5.0 - 2022-07-14 ### Added * New scp plugin [#3691](https://github.com/sitespeedio/sitespeed.io/pull/3691). The plugin is exprimental and documentation is coming in a couple of releases. ### Fixed * Upgraded to [Browsertime 16.1.3](https://github.com/sitespeedio/browsertime/releases/tag/v16.11.3) that fixes: * If one of the visual elements failed, all failed. Fixed in [#1818](https://github.com/sitespeedio/browsertime/pull/1818). * Use buffered long tasks instead of injecting the measurement in the page [#1817](https://github.com/sitespeedio/browsertime/pull/1817). * Fixed broken Chromedriver and Geckodriver install on Windows. * Upgraded NodeJS in the Docker container + updated to latest updated Ubuntu 20. ## 25.4.0 - 2022-07-05 ### Added * Show how many long tasks happens before LCP [#3686](https://github.com/sitespeedio/sitespeed.io/pull/3686) and show when last long task happen [#3687](https://github.com/sitespeedio/sitespeed.io/pull/3687). ### Fixed * Updated Browsertime with a fix for Geckodriver: when you run sitespeed.io on a Raspberry Pi it will pickup Geckodriver from the PATH. ## 25.3.2 - 2022-06-30 ### Fixed * Another go at fixing the preWarmServer issue [#3683](https://github.com/sitespeedio/sitespeed.io/pull/3683). ## 25.3.1 - 2022-06-29 ### Fixed * Upgraded to [Browsertime 16.11.1](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#16111---2022-06-29) that fixes the preWarmServer issue reported in [#3682](https://github.com/sitespeedio/sitespeed.io/issues/3682). ## 25.3.0 - 2022-06-28 ### Fixed * Updated to [Browsertime 16.10.1](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#16101---2022-06-26) that checks that the CDP port is not used before claiming it. ### Added * Updated to Firefox 102 and Edge 103 in the Docker container. * Updated to [Browsertime 16.11.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#16110---2022-06-28). ## 25.2.1 - 2022-06-23 ### Fixed * Bug fix for adding custom CSS from plugins, than you [Josh Duncan](https://github.com/josh-lr) for PR [#3679](https://github.com/sitespeedio/sitespeed.io/pull/3679). ## 25.2.0 - 2022-06-22 ### Added * Updated the Docker container to use Chrome 103. ## 25.1.1 - 2022-06-20 ### Fixed * Upgraded to Browsertime [16.9.1](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1691---2022-06-19) that upgraded to Throttle 4 that internally uses `ip route` instead of `route` (one less dependency). ## 25.1.0 - 2022-06-15 ### Added * Updated Browsertime to [16.9.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1690---2022-06-15) with the following changes: * New `wait.byCondition` command. Thank you [Icecold777](https://github.com/Icecold777) for PR [#1803](https://github.com/sitespeedio/browsertime/pull/1803). * Collect number of CPU longtasks before largest contentful paint [#1806](https://github.com/sitespeedio/browsertime/pull/1806). * Instead of throwing errors and exit the tests if the page hasn't finished loading after 5 minutes, we now gracefully ends the test after 2 minutes (you can change that time with `--maxLoadTime`). That makes more sense than just throwing errors [#1810](https://github.com/sitespeedio/browsertime/pull/1810). ## 25.0.0 - 2022-06-10 ### Changed `--debug` do not add verbose logging anymore, instead it uses Browsertimes debug mode. You can use [breakpoints](https://www.sitespeed.io/documentation/sitespeed.io/scripting/#breakpoint) to debug your script. You can add breakpoints to your script that will be used when you run in `--debug` mode. At each breakpoint the browser will pause. You can continue by adding `window.browsertime.pause=false;` in your developer console. Debug mode works in Chrome/Firefox/Edge when running on desktop. It do not work in Docker and on mobile. When you run in debug mode, devtools will be automatically open so you can debug your page. In debug mode, the browser will pause after each iteration. [Read the documentation](https://www.sitespeed.io/documentation/sitespeed.io/scripting/#debug) on how to debug sitespeed.io scripts. ### Added * Added Firefox 101 and Edge 102 in the Docker container. ### Fixed * Updated to Axe-core 4.2.2 [#3673](https://github.com/sitespeedio/sitespeed.io/pull/3673) ## 24.9.0 - 2022-05-25 ### Added * Updated the Docker container to use Chrome 102 [#3665](https://github.com/sitespeedio/sitespeed.io/pull/3665). ## 24.8.1 - 2022-05-21 ### Fixed * Better fix for getting the correct browser name and version when storing latest run data [#3661](https://github.com/sitespeedio/sitespeed.io/pull/3661). ### Added * Updated to [Browsertime 16.7.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1670---2022-05-20) from 16.4.0. With the latest versions we get: * Use `--browsertime.videoParams.thumbsize` to set the max size of the filmstrip thumbnails. * Interaction to next paint is collected for browsers that supports it (useful for your user journeys) * The time when the last CPU long task happens is a new metric, thank you Andy Davies for that idea! * You can now use groupAlias directly in a text file [#3655](https://github.com/sitespeedio/sitespeed.io/pull/3655). You can read how in the [updated documentation](https://www.sitespeed.io/documentation/sitespeed.io/configuration/#analyse-by-urls). * Added an option for using the full Graphite namespace when latest screenshots/video is stored for a test [#3660](https://github.com/sitespeedio/sitespeed.io/pull/3660). ### Fixed * If you set a user agent for Browsertime, also use it for the crawler [#3652](https://github.com/sitespeedio/sitespeed.io/pull/3652). * Fix missing browser info when storing latest run data [#3658](https://github.com/sitespeedio/sitespeed.io/pull/3658). ## 24.7.0 - 2022-05-11 ### Added * Updated to [Browsertime 16.4.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1640---2022-05-11) fixes a bug for LCP in the video and also output when LCP happens in the video. ### Fixed * Make sure we got CLS from Browsertime when we try to render it [#3650](https://github.com/sitespeedio/sitespeed.io/pull/3650). ## 24.6.0 - 2022-05-10 ### Added * Updated to [Browsertime 16.3.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1630---2022-05-07) that collects Largest Contentful Paint from the browser (if the browser suppoorts the LCP API). * Show total download time per domain [#3648](https://github.com/sitespeedio/sitespeed.io/pull/3648). ## 24.5.1 - 2022-05-06 ### Fixed * Reverted to Ubuntu 20 in the Docker container. Ubuntu 22 gave Firefox problems on ARM and we also seen other problems with NodeJS in that container. * Updated PageXray/Coach Core that include SSL times in the total timings per domain (that was missed before). * Added more safe check when we miss Crux data. ## 24.5.0 - 2022-05-05 ### Added * Add INP and TTFB to the CRUX metrics [#3645](https://github.com/sitespeedio/sitespeed.io/pull/3645). * Updated the Chrome USer Experience dashboard to include the new metrics. You can see the dahsboard [here](https://dashboard.sitespeed.io/d/t_bhsNGMk/chrome-user-experience-report) and download the new one from [here](https://github.com/sitespeedio/grafana-bootstrap-docker/blob/main/dashboards/graphite/ChromeUserExperienceReport.json). ### Fixed * Bumped Browsertime with a new fix release for the upcoming portable visual metrics script. ## 24.4.0 - 2022-05-04 ### Added * Updated to Edge 101 in the Docker container. Updated to latest [Browsertime 16.2.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1620---2022-05-01) with Edgedriver and Chromedriver 101. * Updated to Firefox 100 in the Docker container. ### Fixed * If `--firstParty` do not match any URL, make sure we still show first vs third party [#3643](https://github.com/sitespeedio/sitespeed.io/pull/3643). * Updated to latest NodeJs in the slim container. ## 24.3.0 - 2022-04-27 ### Added * Updated to Chrome 101 in the Docker container [#3641](https://github.com/sitespeedio/sitespeed.io/pull/3641). ## 24.2.0 - 2022-04-26 ### Added * Updated base Docker image to use Ubuntu 22.04 [#3636](https://github.com/sitespeedio/sitespeed.io/pull/3636). ### Fixed * Updated dependencies: aws-sdk, cli-color, fs-extra, influx, junit-report-builder and tape [#3640](https://github.com/sitespeedio/sitespeed.io/pull/3640) ### Tech * Use Ava for unit testing [#3637](https://github.com/sitespeedio/sitespeed.io/pull/3637) * Use local HTTP server to speed up GitHub Action tests [#3638](https://github.com/sitespeedio/sitespeed.io/pull/3638). * Updated dev dependencies [#3639](https://github.com/sitespeedio/sitespeed.io/pull/3639). ## 24.1.0 - 2022-04-22 ### Added * Update Grafana auth settings to allow api token or basic auth, thank you [Vladimir Stepanov](https://github.com/vs-odessa) for PR [#3627](https://github.com/sitespeedio/sitespeed.io/pull/3627). * Add TTFB to the metrics page [#3630](https://github.com/sitespeedio/sitespeed.io/pull/3630). * Use latest Coach core with updated PageXray and Third party web [#3629](https://github.com/sitespeedio/sitespeed.io/pull/3629). * Make it possible to disable annotations for Graphite. Set `--graphite.sendAnnotation false` to disable sending annotations [#3625](https://github.com/sitespeedio/sitespeed.io/pull/3625). * Show which run that is used in the metrics tab [#3631](https://github.com/sitespeedio/sitespeed.io/pull/3631). * Updated to [Browsertime 16.1.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1610---2022-04-20). ### Updated * Update dependencies: google-cloud/storage, aws-sdk, dayjs, yargs [#3635](https://github.com/sitespeedio/sitespeed.io/pull/3635). ## 24.0.0 - 2022-04-06 ### Changed * Upgraded to [Browsertime 16.0.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1600---2022-04-05) that changed how the DNS is flushed. You need to add `--flushDNS` to your conifguration to flush the DNS between runs. ### Added * Use Firefox 99 and Edge/Edgedriver 100 in the Dockker container. ### Fixed * Better catch if Visual Metrics fails [#3619](https://github.com/sitespeedio/sitespeed.io/pull/3619) * Fixed JUnit/alias bug [#3620](https://github.com/sitespeedio/sitespeed.io/pull/3620) ## 23.7.0 - 2022-03-31 ### Added * Updated to Chrome/Chromedriver 100 and [Browsertime 15.4.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1540---2022-03-30). * Updated the +1 container to use Lighthouse 9.5.0. * Updated to [Axe-core 4.4.1](https://github.com/sitespeedio/sitespeed.io/pull/3615). ## 23.6.1 - 2022-03-25 ### Fixed * Fixing text formatting in the JSON from latest run and make sure we display the time with UTC[#3611](https://github.com/sitespeedio/sitespeed.io/pull/3611). ## 23.6.0 - 2022-03-25 ### Added * Various fixes to add more content to the JSON stored from the latest run, making it easier to show more meta data for a run in Grafana [#3607](https://github.com/sitespeedio/sitespeed.io/pull/3607), [#3609](https://github.com/sitespeedio/sitespeed.io/pull/3609) and [#3610](https://github.com/sitespeedio/sitespeed.io/pull/3610). ## 23.5.2 - 2022-03-22 ### Fixed * Updated to latest NodeJS and Ubuntu updates in the Docker container with some security updates[#3306](https://github.com/sitespeedio/sitespeed.io/pull/3606). Also updated the slim container with latest NodeJS. * Make sure the shrinkwrap file is used when building the container [#3604](https://github.com/sitespeedio/sitespeed.io/pull/3604) and (hopefully) fix so that .dockerignore is not ignored when Github Actions build the containers. ## 23.5.0 - 2022-03-11 ### Added * The -slim Docker container now uses Firefox 98. * Updated Grafana to latest 8.4.3 in the Docker compose file. * Updated [Graphite dashboards](https://github.com/sitespeedio/grafana-bootstrap-docker/tree/main/dashboards/graphite) to use Timeseries graphs instead of old "Graph". * You can use `-o` or `--open` or `--view` to open the result page after you run sitespeed.io on Mac and Linux [#3569](https://github.com/sitespeedio/sitespeed.io/pull/3596). * Open the result with `-o` on Linux using xdg-open [#3597](https://github.com/sitespeedio/sitespeed.io/pull/3597). ## 23.4.0 - 2022-03-09 ### Added * Firefox 98 in the Docker container [#3592](https://github.com/sitespeedio/sitespeed.io/pull/3592). * Added link to each run in the side by side metric page, making it easier to go to the correct run [#3593](https://github.com/sitespeedio/sitespeed.io/pull/3593). ## 23.3.0 - 2022-03-07 ### Added * The +1 container is upgraded to use Lighthouse 9.4.0. * Updated to [Browsertime 15.3.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1530---2022-03-07). ## 23.2.0 - 2022-03-05 ### Added * Updated to Chrome and Edge 99 in the Docker container. Updated to Chromedriver and Edgedriver 99 [#3590](https://github.com/sitespeedio/sitespeed.io/pull/3590). ### Fixed * If you use alias and budget files, the outcome (result budget json and others) should use the alias of the URL instead of the URL [#3582](https://github.com/sitespeedio/sitespeed.io/pull/3582). * Ignore sustainable.setup messages when storing analysistorer [#3578](https://github.com/sitespeedio/sitespeed.io/pull/3587). ## 23.1.0 - 2022-02-24 ### Added * Updated [Browsertime](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1510---2022-02-24) that collect timings from main document. The result includes a field named mainDocumentTimings and contains blocked, dns, connect, send, wait, receive, ssl as long as you get a HAR file from the browser. * Show Browsertime version in runtime settings [#3575](https://github.com/sitespeedio/sitespeed.io/pull/3575). ### Fixed * Updated Chromedriver dependency that fixes installation on Mac M1 and some send keys issues. ## 23.0.1 - 2022-02-21 ### Fixed * Bumped to Browsertime 15.0.1 to make sure connectivity is set only once. In last release it was set in the beginning of the test and for eacch iteration. In practice that doesn't matter but it's very confusing if you check the logs. ## 23.0.0 - 2022-02-21 ### Breaking changes * In this release we updated to Browsertime 15.0.0 that drops built in support for [TSProxy](https://github.com/WPO-Foundation/tsproxy). The reason to drop TSProxy is that it only works in Python 2 and the sunset for Python 2 was January 1, 2020. If you still need TSProxy you can set it up yourself. * Browsertime also drop support for getting visual metrics from the trace log. It was added to evaluate if it could be a compliment to the video visual metrics but it was not. Removing it also decreased the number of dependencies in Browsertime. * Linux/Mac OS will flush the DNS between runs, that means that you can have some changes in DNS lookup time when you upgrade to 23.0.0. ### Fixed * Fix broken JUnit output. Thank you [rghetu](https://github.com/rghetu) for finding it [#3569](https://github.com/sitespeedio/sitespeed.io/pull/3569). * When comparing metrics side by side for different runs, mean/median and stddev was sometimes broken for some timing metrics. Fixed with [#3573](https://github.com/sitespeedio/sitespeed.io/pull/3573) ### Added * Use Browsertime 15.0.0. * Show alias name in the budget report page [#3572](https://github.com/sitespeedio/sitespeed.io/pull/3572). ## 22.1.2 - 2022-02-09 ### Fixed * Updated to Browsertime 14.21.1 that disables the new Chrome splash screen by default. ## 22.1.1 - 2022-02-08 ### Fixed * The slim container with only Firefox was broken in last release. It's now re-released with Firefox 96 and Firefox 97 coming soon. ## 22.1.0 - 2022-02-08 ### Added * Upgraded to latest Browsertime with Egdedriver 98. * Upgraded to Edge 98 and Firefox 97 in the (amd64) Docker containers. ## 22.0.0 - 2022-02-07 ### Breaking changes * If you use the Lighthouse plugin there's breaking changes: In the new version we drop support for the following: * Running multiple runs with Lighthouse. * Using scripts to login the user (or whatever you need before you run your tests) It's a couple of reasons why I remove those features: * I been looking for a maintainer of the Lighthouse plugin for +1 year and I haven't found one. For me to be able to maintain it I want the plugin to be as simple as possible. * I deeply regret merging the PR for adding multiple runs for Lighthouse. That PR goes against everything I know about measuring performance. Lighthouse is not built for getting correct performance metrics, it's built to help (Chrome) developers to get insights how they make the page "faster". Lets stick to the basics and keep it possible to get those recommendations from Lighthouse. * Maybe someday Lighthouse will have support for user journeys, lets wait until that is officially supported and then I can check if it could be used in the plugin. With the new release we also break how you configure Lighthouse. People has had problem with that since day 1. With the new version we support two new ways to configure Lighthouse: - By configuration JSON file. `--lighthouse.config config.js` - By Lightouse flags file. `--lighthouse.flags flag.json` If you don't need to configure Lightouse you can use the default settings both for desktop and mobile. If you run without any settings, the plugin will use desktop settings. If you run with `--mobile`, `--android` or `--ios` the mobile settings will be used. ### Added * Build Docker containers for both amd64 and arm64 to make containers work on Mac M1. The arm container contains Firefox and Chromium. Thank you [whichfinder](https://github.com/whichfinder) and [Radu Micu](https://github.com/radum) for the help! Fixed in PR [#3554](https://github.com/sitespeedio/sitespeed.io/pull/3554). * When plugins is loaded, there's a new extra last step where we try to load the plugin as a globally installed npm module [#3546](https://github.com/sitespeedio/sitespeed.io/pull/3546). ### Fixed * Fix so that we do not display the same 3rd party cookie multiple times [#3545](https://github.com/sitespeedio/sitespeed.io/pull/3545). * Updated Coach Core that includes the latest version of third party web and PageXray that find more fonts without mime type. ## 21.6.1 - 2022-01-24 ### Fixed * Updated to [Browsertime 14.18.1](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#14181---2022-01-24) that makes the summary metric log message use median (instead of mean) and change a log message level to debug. ## 21.6.0 - 2022-01-24 ### Added * Updated to Edge stable release in the Docker container. * Remove Crux distribution table and use pie charts instead [#3537](https://github.com/sitespeedio/sitespeed.io/pull/3537) * Add extra sleep time between Crux calls to make sure to not overload the API limit [#3536](https://github.com/sitespeedio/sitespeed.io/pull/3536). * Added extra Crux enable command line `--crux.enable` to enable Crux [#3538](https://github.com/sitespeedio/sitespeed.io/pull/3538). Its default value is `true` and you also need to supply the Crux key to run Crux. The reason for the new parameter is that you can now configure the key in your configuration JSON and set the enable to false and then you enable it with the CLI parameter when you actually need to run Crux. * Show Crux-metrics on the Summary page [#3540](https://github.com/sitespeedio/sitespeed.io/pull/3540). * Updated summary metrics tables with headings to make it easier to read [#3541](https://github.com/sitespeedio/sitespeed.io/pull/3541). * Added [Browsertime 14.17.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#14170---2022-01-23) with new Select and click.byName commands. With that Browsertime version you also need to have ffprobe installed when you run Visual Metrics but that should already be installed. * Added [Browsertime 14.18.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#14180---2022-01-24) with a fix for Firefox [#1698](https://github.com/sitespeedio/browsertime/issues/1698) ## 21.5.0 - 2022-01-14 ### Added * Upgraded to [Browsertime 14.15.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#14150---2022-01-12) that adds support for `--appendToUserAgent` for Chrome/Edge/Firefox. And then Browsertime 14.16.0 that supports Geckodriver for Raspberry Pi. ## 21.4.0 - 2022-01-12 ### Added * Updated to a new build of WebPageReplay in the Docker container * Updated the Ubuntu base image to latest version and latest NodeJS in the Docketr container. * Upgraded Browsertime [#3528](https://github.com/sitespeedio/sitespeed.io/pull/3528): * Add support for Humble as connectivity engine for mobile phone testing. Make sure to setup Humble on a Raspberry Pi 4 and the choose engine with --connectivity.engine humble and set the URL to your instance --connectivity.humble.url http://raspberrypi.local:3000. Added in #1691. * Upgraded to Chrome 97 and Edge 97 in the Docker container. * Upgraded to Chromedriver 97. ### Fixed * Updated Chromedriver library that automatically picks up the Chromedriver if it's installed on Raspberry Pi. ## 21.3.0 - 2022-01-01 ### Added * Updated to [Browsertime 14.13.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#14130---2021-12-30) with the following fixes for the user agent: * Append text to Chrome/Edge user agent using `--chrome.appendToUserAgent` * When you use Chrome/Edge and use a "emulated device" that will use the user agent that you provide using `--userAgent`. Else it will use the user agent from your emulated device setting. * You can also use Edge to run emulated mobile with the same settings as Chrome. ## 21.2.2 - 2021-12-23 ### Fixed * Fix the error introduced in 21.2.0 for "Include page summary URL in the result JSON" [#3525](https://github.com/sitespeedio/sitespeed.io/pull/3525). ## 21.2.1 - 2021-12-22 ### Fixed * It turns out that Firefox 95 doesn't work with the HAR export trigger and the workaround that worked in Firefox 94 seems to not work in 95 see https://github.com/sitespeedio/browsertime/issues/1671#issuecomment-999412035. That's why we are reverting to Firefox 94 in the Docker containers. ## 21.2.0 - 2021-12-22 ### Fixed * The catching of errors in the queue was broken and reported the error x times (x=number of plugins). Also when we had an error the result JSON was not stored. [#3522](https://github.com/sitespeedio/sitespeed.io/pull/3522). ### Added * Updated to Firefox 95 and Edge 96 in the Docker container. * Include page summary URL in the result JSON [#3523](https://github.com/sitespeedio/sitespeed.io/pull/3523). ## 21.1.0 - 2021-12-06 ### Added * Added possibility choose name for storing a result JSON [#3520](https://github.com/sitespeedio/sitespeed.io/pull/3520). We use this for a feature that will be released soon. ## 21.0.1 - 2021-12-01 ### Fixed * Updated Browsertime that logs Chrome document request failures on debug log level instead so that the log is not cluttered. ## 21.0.0 - 2021-12-01 ### Changed * Updated to [Coach Core 7.0.0](https://github.com/sitespeedio/coach-core/blob/main/CHANGELOG.md#700---2021-12-01). This will probably change your Coach score. The new Coach has the following changes: * Moved AMP advice to best practice instead of privacy [#67](https://github.com/sitespeedio/coach-core/pull/67). * Increased favicon max size advice from 5 to 10 kb [#68](https://github.com/sitespeedio/coach-core/pull/68) * Renamed the fastRender advice to avoidRenderBlocking [#73](https://github.com/sitespeedio/coach-core/pull/73) * Remove the third party async advice [#74](https://github.com/sitespeedio/coach-core/pull/74) * Updated the layout shift advice to use cumulative layout shift [#75](https://github.com/sitespeedio/coach-core/pull/75) * Changed id of the Google Tag Manager advice [#79](https://github.com/sitespeedio/coach-core/pull/79) * Updated third-party-web to 0.12.6. * Use Chrome(ium) render blocking information to know if a request is render blocking or not [#66](https://github.com/sitespeedio/coach-core/pull/66). * Report offending JavaScript assets if the JavaScript max limits kicks in [#70](https://github.com/sitespeedio/coach-core/pull/70). * New largest contentful paint advice [#76](https://github.com/sitespeedio/coach-core/pull/76). * New first contentful paint advice [#77](https://github.com/sitespeedio/coach-core/pull/77). * Added TBT in the CPU longtask advice [#80](https://github.com/sitespeedio/coach-core/pull/80). * Report content and transfer size for offending URLs [#81](https://github.com/sitespeedio/coach-core/pull/81). * Report offending assets with transfer/content size for page size limit [#82](https://github.com/sitespeedio/coach-core/pull/82). * Fix cases when JQuery is undefined. Thank you [shubham jajodia](https://github.com/jajo-shubham) for PR [#64](https://github.com/sitespeedio/coach-core/pull/64). * A better way to find offending layout shifters. Thank you [shubham jajodia](https://github.com/jajo-shubham) for PR [#65](https://github.com/sitespeedio/coach-core/pull/65). * Removed mentions aboout server push [#69](https://github.com/sitespeedio/coach-core/pull/69) * Added more information on how to debug CPU advice [#71](https://github.com/sitespeedio/coach-core/pull/71). * Updated to sustainable plugin core code to [co2 0.8.0](https://github.com/thegreenwebfoundation/co2.js/blob/main/CHANGELOG.md#080---2021-11-28). This include a biug fix to the 1byte model that will lower your co2 score [#3519](https://github.com/sitespeedio/sitespeed.io/pull/3519). ### Added * Updated to [Browsertime 14.12.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#14120---2021-11-30) that fixes the [FF 94 HAR issue](https://github.com/sitespeedio/browsertime/issues/1671). * Updated green domains from the Green Web Foundation [#3513](https://github.com/sitespeedio/sitespeed.io/pull/3513). * Updated PerfCascade that support chunks when you use Chrome(ium) [#3514](https://github.com/sitespeedio/sitespeed.io/pull/3514). * Updated to Firefox 94 in the Docker image. ### Fixed * Added missing summary boxes on start page [#3515](https://github.com/sitespeedio/sitespeed.io/pull/3515). ## 20.6.2 - 2021-11-20 ### Fixed * Updated to [Browsertime 14.10.2](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#14102---2021-11-20) that disables the Edge/Edgedriver version check. ## 20.6.1 - 2021-11-20 ### Fixed * Updated to [Browsertime 14.10.1](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#14101---2021-11-19) that disables the Chrome/Chromedriver version check. ## 20.6.0 - 2021-11-17 Note: Do your test fails with Firefox 94? See [Browsertime #1671](https://github.com/sitespeedio/browsertime/issues/1671). Until that is fixed in Firefox, it seems most use cases is fixed by adding a settle time (the browser rest for a while before we start the test). Try with `--browsertime.settleTime 10000` to add a 10 seconds wait time before the test starts. ### Added * Updated to [Browsertime 14.10.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#14100---2021-11-16) with Chromedriver 96 and Chrome 96 in the Docker container. ## 20.5.0 - 2021-11-09 ### Added * Updated to [Browsertime 14.9.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1490---2021-11-07) with updated Chrome HAR generator. ### Fixed * Ignore Influxdb and Grafana setup messages when you use the analysisstorer plugin [#3508](https://github.com/sitespeedio/sitespeed.io/pull/3508). ## 20.4.2 - 2021-11-07 ### Fixed * Updated to AXE core 4.3.5 [#3501](https://github.com/sitespeedio/sitespeed.io/pull/3501). * Use latest npm in the Docker container [#3502](https://github.com/sitespeedio/sitespeed.io/pull/3502). * Fixed the bug that caused so that you couldn't use the analyisstorer plugin together with Graphite [#3506](https://github.com/sitespeedio/sitespeed.io/pull/3506). * Updated to yargs 17.2.1 ## 20.4.1 - 2021-10-29 ### Fixed * Reverted the change with npm in the Docker container since it broke both webpagetest and the +1 container installation ## 20.4.0 - 2021-10-29 ### Added * Updated to [Browsertime 14.8.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md) with Edgeriver 95. * Updated to Edge 95 in the Docker container. * Update to use NodeJS 16 in the Docker container [#3495](https://github.com/sitespeedio/sitespeed.io/pull/3495). * On Mac OS: Use `--open` or `-o` to open the result in your default browser. Thank you [Johanna Lindh](https://github.com/zhouhana) for the idea! PR [#3493](https://github.com/sitespeedio/sitespeed.io/pull/3493). * The +1 container now had Lighthouse 8.6.0. ### Fixed * Removed npm from the Docker container when everything has been installed to fix npm security issues + decrease the size of the container. ## 20.3.1 - 2021-10-21 ### Fixed * Fix to handle integer value Android device serial, thank you [Saurav Kumar](https://github.com/svkrclg) for PR [#3490](https://github.com/sitespeedio/sitespeed.io/pull/3490). ## 20.3.0 - 2021-10-20 ### Added * Updated Browsertime that uses Chromedriver 95. * Updated Chrome 95 in the Docker container. ## 20.2.0 - 2021-10-14 ### Added * Add support for using cookies when crawling, thank you [dammg](https://github.com/dammg) for PR [#3472](https://github.com/sitespeedio/sitespeed.io/pull/3472). * [Browsertime 14.6.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1460---2021-10-13). * [Use alias in your budget file](https://www.sitespeed.io/documentation/sitespeed.io/performance-budget/#override-per-url-or-alias) [#3479](https://github.com/sitespeedio/sitespeed.io/pull/3479). * Support for using [User Timing API marks/measures in the budget file](https://www.sitespeed.io/documentation/sitespeed.io/performance-budget/#user-timing-api-metrics) [#3482](https://github.com/sitespeedio/sitespeed.io/pull/3482) and [#3483](https://github.com/sitespeedio/sitespeed.io/pull/3483). * Support for [using custom metrics from scripting in your budget](https://www.sitespeed.io/documentation/sitespeed.io/performance-budget/#metrics-from-scripting) [#3484](https://github.com/sitespeedio/sitespeed.io/pull/3484) and [#3486](https://github.com/sitespeedio/sitespeed.io/pull/3486). ### Fixed * Allow crawler to use cookies, thank you [dammg](https://github.com/dammg) for the PR [#3472](https://github.com/sitespeedio/sitespeed.io/pull/3472) with small fix [#3473](https://github.com/sitespeedio/sitespeed.io/pull/3473). ## 20.1.0 - 2021-10-05 ### Added * Use Firefox 93 in the Docker and the Docker slim container [#3471](https://github.com/sitespeedio/sitespeed.io/pull/3471). ## 20.0.0 - 2021-09-30 Please read the [20.0 blog post](https://www.sitespeed.io/sitespeed.io-20.0/)! ### Breaking changes * Before you update to the new version: If you save your metrics to Graphite, have you upgraded those metrics to the new format as introduced in sitespeed.io the **15/4-2021**? If not, please follow the [guide](https://www.sitespeed.io/documentation/sitespeed.io/graphite/#upgrade-to-use-the-test-slug-in-the-namespace) in the documentation. If you haven't done that and still upgrade to sitespeed.io 20 you need to make sure you add `--graphite.addSlugToKey false` to your test else the metrics will be reported under a new key structure. The change was done in [#3434](https://github.com/sitespeedio/sitespeed.io/pull/3434). * Set [throttle](https://github.com/sitespeedio/throttle) as default connectivity engine if you use Mac or Linux [#3433](https://github.com/sitespeedio/sitespeed.io/pull/3433). This makes it much easier to enable throttling. Our Docker container is not affected by this change. * There's a new default mobile `--mobile` for Chrome. The new default is Moto G4 (instead of iPhone 6) [#3467](https://github.com/sitespeedio/sitespeed.io/pull/3467). * When you run your tests on Safari on iOS the Coach is disabled by default [#3468](https://github.com/sitespeedio/sitespeed.io/pull/3468). ### Added * [Browsertime 14.5.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1450---2021-09-30) with CSS selector support for the mouse commands. ## 19.6.0 - 2021-09-23 ### Added * Updated to Chrome 94 in the Docker container. * Upgraded to [Browsertime 14.4.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1440---2021-09-22) that included Chromedriver 94. ## 19.5.0 - 2021-09-17 ### Added * Updated to Edge 93 in the Docker container. * Updated to [Browsertime 14.3.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1430---2021-09-16). * Updated WebPageReplay to new version [#3642](https://github.com/sitespeedio/sitespeed.io/pull/3462). ## 19.4.2 - 2021-09-08 ### Fixed * Updated the Docker container to use Firefox 92 (instead of beta 92) * Updated the base Docker container to use a newer updated version of Ubuntu 20.04. See [#3456](https://github.com/sitespeedio/sitespeed.io/issues/3456). ## 19.4.1 - 2021-09-06 ### Fixed * Updated to [Browsertime 14.2.1](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1421---2021-09-06). * Updated to AXE 4.3.3 ## 19.4.0 - 2021-09-05 ### Added * Add option to ignore robots.txt when crawling. Use `--crawler.ignoreRobotsTxt true` to ignore. Thank you [dammg](https://github.com/dammg) for PR [#3454](https://github.com/sitespeedio/sitespeed.io/pull/3454)! * Updated to [Browsertime 14.2.0](https://github.com/sitespeedio/browsertime/releases/tag/v14.2.0). ### Fixed * If generating a HTML file failed, all generation failed. This fixes that and continue with the next file [#3453](https://github.com/sitespeedio/sitespeed.io/pull/3453). ## 19.3.0 - 2021-09-01 ### Added * Upgraded to [Browsertime 14.1.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1410---2021-09-01) with Chromedriver 93. * Added Chrome 93 in the Docker container. ### Fixed * Upgraded to [Browsertime 14.0.3](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1403---2021-08-31). ## 19.2.0 - 2021-08-27 ### Added * Upgraded to AXE core 4.3.2 [#3441](https://github.com/sitespeedio/sitespeed.io/pull/3441). * Added stddev/median/mean to the metrics side by side page [#3443](https://github.com/sitespeedio/sitespeed.io/pull/3443). * Added a generic text that we miss out of many metrics for Safari at the moment [#3442](https://github.com/sitespeedio/sitespeed.io/pull/3442). * Add option to add friendly name to junit test cases. Use `--budget.friendlyName` to set that. Thank you [Vishal](https://github.com/vishallanke) for the request. Done in PR [#3448](https://github.com/sitespeedio/sitespeed.io/pull/3448). ### Fixed * Upgrade PerfCascade that catches if an HAR entry is missing content type [#3445](https://github.com/sitespeedio/sitespeed.io/pull/3445). ## 19.1.0 - 2021-08-20 ### Added * You can now see curated metrics side by side for all runs [#3439](https://github.com/sitespeedio/sitespeed.io/pull/3439). * The WebPageTest plugin is using the latest (0.5.0) version of the WebPageTest API. ### Fixed * Upgraded to [Browsertime 14.0.2](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1402---2021-08-20). ## 19.0.0 - 2021-08-13 ### Changed * Updated to [Browsertime 14.0.1](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1401---2021-08-12). The new 14 version uses Throttle 3.0 that has change if you use it on Mac OS: Updated Throttle 3.0 that do not set throttling on localhost by default on Mac OS. If you run test against a local server or use WebPageReplay on a Mac, you should add `--browsertime.connectivity.throttle.localhost` to your test and it will work as before. ### Added * Updated to Edge 92 and Edgedriver 92 in the Docker container. * Updated to Firefox 92 beta in the Docker container to fix the [devtools slowness bug](https://bugzilla.mozilla.org/show_bug.cgi?id=1712983) that was inroduced in Firefox 90. ## 18.0.1 - 2021-07-29 ### Fixed * Updated to [Browsertime 13.1.4](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1314---2021-07-28). * Added cli parameter for keeping the original video `--videoParams.keepOriginalVideo`. The functionalty already exists but it wasn't exposed in sitespeed.io [#3430](https://github.com/sitespeedio/sitespeed.io/pull/3430). * The GPSI-plugin has a fix to catch when first input delay data is missing. * Fix broken tags for InfluxDB when you get Crux data from the GPSI-plugin. [#3429](https://github.com/sitespeedio/sitespeed.io/pull/3429). Data from the plugin has a testType tag that can have the following values: googleWebVitals, score or crux. Crux data has two more tags: metric and experience. ## 18.0.0 - 2021-07-26 ### Breaking changes * Drop support for NodeJS 10. * If you use Chrome the `--chrome.timeline` is now true by default (you can remove that from your configuration). ### Added * Updated the Docker container to use Chrome 92. * Updated to [Browsertime 13.1.1](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1300---2021-07-22) (that uses Chromedriver 92). * Updated to latest PerfCascade so that the waterfall highlights which request that is the largest contentful paint (if its an image) and show render blocking info (if you use Chrome) [#3407](https://github.com/sitespeedio/sitespeed.io/pull/3407). You can checkout the [documentation](https://www.sitespeed.io/documentation/sitespeed.io/browsers/#render-blocking-information) about where to see the render blocking information. * Show render blocking info in the Page Xray section [#3246](https://github.com/sitespeedio/sitespeed.io/pull/3426). ### Fixed * Updated to Coach core 6.4.3 ## 17.10.0 - 2021-07-16 ### Added * The Docker containers now contains Firefox 89 again. It seems like there's something with 90 that increase First Visual Change, especially when running in Docker. See [https://phabricator.wikimedia.org/T286761](https://phabricator.wikimedia.org/T286761) and [https://bugzilla.mozilla.org/show_bug.cgi?id=1720843](https://bugzilla.mozilla.org/show_bug.cgi?id=1720843). ## 17.9.0 - 2021-07-16 ### Added * Updated to [Browsertime 12.11.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#12110---2021-07-15). * Updated to Firefox 90 in the browser container [#3420](https://github.com/sitespeedio/sitespeed.io/pull/3420). * Update to AXE-core 4.2.3 [#3417](https://github.com/sitespeedio/sitespeed.io/pull/3417). * Added support for Firefox memory report, turn it on with `--firefox.memoryReport` [#3416](https://github.com/sitespeedio/sitespeed.io/pull/3416) * The +1 container has been updated to use Lightouse 8.1.0. ## 17.8.3 - 2021-07-06 ### Fixed * Updated to [Coach-core 6.4.2](https://github.com/sitespeedio/coach-core/blob/v6.4.2/CHANGELOG.md#642---2021-07-05). * Updated to [Browsertime 12.10.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#12100---2021-07-05). ## 17.8.2 - 2021-06-27 ### Fixed * Keep selected tab open across runs. Thank you [Tanishq](https://github.com/amtanq) for PR [#3409](https://github.com/sitespeedio/sitespeed.io/pull/3409). * Update Docker container to use NodeJS 14.7.1. * Upgraded to [Browsertime 12.9.3](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1293---2021-06-24). * Updated [Coach-core](https://github.com/sitespeedio/coach-core/blob/main/CHANGELOG.md#641---2021-06-23): Use all headers for Wappalyzer (before only the main document was used) ## 17.8.1 - 2021-06-10 ### Fixed * Updated Browsertime to [12.9.1](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1291---2021-06-09) that fixes the problem with running Safari on iOS. However there are still one bug/problem that needs to be fixed with Safari on iOS: recording a video do not work. * The docker-compose file now uses Grafana 8.0.0 ## 17.8.0 - 2021-06-04 ### Added * New [Browsertime 12.9.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1290---2021-06-04) that updates Cumulative Layout Shift to the [new defintion](https://web.dev/evolving-cls/). ### Fixed * Guard if you try to run Safari simulator in Docker [#3405](https://github.com/sitespeedio/sitespeed.io/pull/3405) ## 17.7.0 - 2021-06-03 ### Added * New [Browsertime 12.8.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1280---2021-06-02) with Edgedriver 91 and updated to Firefox 89 and Edge 91 in the Docker container [#3404](https://github.com/sitespeedio/sitespeed.io/pull/3404). * Updated to use [Coach-core 6.4.0](https://github.com/sitespeedio/coach-core/blob/main/CHANGELOG.md#640---2021-06-02) that uses wappalyzer-core 6.6.0 and a bug fix that make sure the private assets and cache header advice only checks GET requests. * The Lighthouse-plugin was updated to 8.0.0 in the +1 container. ### Fixed * Catch if visual elements do not produce data (special case for amazon.com) [3402](https://github.com/sitespeedio/sitespeed.io/pull/3402). * Upgrade to AXE-core 4.2.1 [#3396](https://github.com/sitespeedio/sitespeed.io/pull/3396). ### Tech * Switch to sass instead of node-sass [#3396](https://github.com/sitespeedio/sitespeed.io/pull/3396). ## 17.6.0 - 3 ### Added * Updated to [Browsertime 12.7.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1270---2021-05-26) that fixes the problem with the broken CPU throttling in Chrome. * Chrome and Chromedriver 91. Edge 90 (Eddge 91 coming later this week) in the Docker container ### Fixed * Display more information about the visual element that you are measuring [#3394](https://github.com/sitespeedio/sitespeed.io/pull/3394). Checkout the [updated documentation](https://www.sitespeed.io/documentation/sitespeed.io/video/#collect-visual-elements-metrics) on how to measure when elements are displayed in the viewport. ## 17.5.0 - 2021-05-21 ### Added * Make it easier to set up budget for Google Web Vitals [#3386](https://github.com/sitespeedio/sitespeed.io/pull/3386) * Updated dashboards to in the Docker setup. ### Fixed * New Browsertime [12.6.1](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1261---2021-05-21) that fixes a bug when you use a equals sign in the value field in a cookie. * Updated dependencies with install warnings [#3387](https://github.com/sitespeedio/sitespeed.io/pull/3387), ## 17.4.0 - 2021-05-14 ### Added * Updated Browsertime and more love for showing better info from LCP and LS [#3381](https://github.com/sitespeedio/sitespeed.io/pull/3381). * Updated to Lighthouse 7.4.0 in the plus-1 Docker container. ## 17.3.1 - 2021-05-10 ### Fixed * 17.3.0 introduced a bug copying the same screenshot for multiple pages when you use the lateststorer plugin[#3376](https://github.com/sitespeedio/sitespeed.io/pull/3376). ## 17.3.0 - 2021-05-06 ### Added * Updated to Axe-core [4.2.0](https://github.com/dequelabs/axe-core/blob/develop/CHANGELOG.md#420-2021-04-23). * Show browser window size in runtime setting when using Android/iPhone [#3362](https://github.com/sitespeedio/sitespeed.io/pull/3362). * Also copy LS and LCP screenshots when using lateststorer plugin [#3371](https://github.com/sitespeedio/sitespeed.io/pull/3371). * Added colors (red/yellow/green) to Google Web Vitals summary box, using the same limits as set by Crux [#3370](https://github.com/sitespeedio/sitespeed.io/pull/3370). * Updated to latest Browsertime and remake on how to show LCP and LS. ### Fixed * [Do not show undefined when you only set RTT for throttle](https://github.com/sitespeedio/sitespeed.io/commit/5446a8e11424c8170a42533f0e40cbe28bf8a5c4). * [Fix test running without connectivity settings](https://github.com/sitespeedio/sitespeed.io/commit/3ac3c2ab6885689b6689c6f2974ae7c256be9faf). ## 17.2.0 - 2021-04-27 ### Added * Show runtime settings in the HTML [#3359](https://github.com/sitespeedio/sitespeed.io/pull/3359). This makes it easier to see what settings are used. We gonna iterate and add more settings later. * Upgraded to [Browsertime 12.2.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1220---2021-04-27). ## 17.1.1 - 2021-04-23 ### Fixed * Updated to latest Browsertime that fixes the missing dev-shm flag for Chrome making running Chrome in Docker crash for some sites see [#3357](https://github.com/sitespeedio/sitespeed.io/issues/3357). ## 17.1.0 - 2021-04-20 ### Added * Updated to Firefox 88 in the Docker containers. ## 17.0.1 - 2021-04-17 ### Fixed * Updated Browsertime to 12.0.1 that fixes the problem with Chrome/Chromedriver 90 that introduced longer time to close the browser than earlier versions, so when trying the next run, the previous browser is not closed, fixed by adding a 2 second sleep time when closing the browser. ## 17.0.0 - 2021-04-15 Woohoo we shipped 17.0.0! There are many changes and you should read through the full changelog and focus on the new best practise section and breaking changes. ### New best practices One of the new things in 17 is the support for one extra key in Graphite: the name of the test. Set a computer friendly name of your test by using `--slug`. Then use the slug in the graphite key by adding `--graphite.addSlugToKey` to your run. When you do that change, should also convert your graphite data and your dashboards. The plan is like this: * In April 2021 you can convert your data and use the slug. You need to add `--graphite.addSlugToKey true` else you will get a log warning that you miss the slug for your test. All default dashboards in sitespeed.io will use the slug, so to use them you should add that new key and convert your data. * In September 2021 `--graphite.addSlugToKey true` will be set to default, meaning if you haven't upgraded your Graphite data yet, you need to set `--graphite.addSlugToKey false` to be able to run as before. * In November 2021 the CLI functionality will disappear and you need upgrade your Graphite metrics when you upgrade sitespeed.io. You can read how to upgrade in the [documentation](https://www.sitespeed.io/documentation/sitespeed.io/graphite/#upgrade-to-use-the-test-slug-in-the-namespace). When you have updated you will add one extra parameter to your test: `--copyLatestFilesToBase`. When you have done all these step you are ready for ... using the new dashboards in Grafana with screenshot and video from last run. This is SUPER useful to be able to fast see what's going on. Checkout what it looks like [here](https://dashboard.sitespeed.io/d/000000064/page-metrics-mobile?orgId=1) and [here](https://dashboard.sitespeed.io/d/d-pdqGBGdse/wikipedia-login?orgId=1). We ship [all new dashboards](https://github.com/sitespeedio/grafana-bootstrap-docker/tree/main/dashboards/graphite) with some extra focus on Google Web Vitals (for sitespeed.io, WebPageTest, Google Page Speed Insights, Lighthouse and CRUX). We understand that these metrics is important for some users so lets focus on them. We also ship a couple new example dahsboards for how to setup user journeys. Documentation for those will come soon. There's a new version of the Coach with a new super important privacy advice: Make sure you disable Chrome's new FLoC for your site. [Read more about FLoC](https://www.eff.org/deeplinks/2021/03/googles-floc-terrible-idea). ### Breaking changes * We have changed some of the connectivity profiles, you can see the changes [here](https://github.com/sitespeedio/browsertime/pull/1160/files). This means that if you use 3g connectivity using Throttle, your tests will have a faster TTFB than before. If you wanna hold on to the old settings you can do that by adding `--browsertime.legacyConnectivityProfiles true` to your tests. * If you have a budget using layoutShift that metric has now been renamed to cumulativeLayoutShift. * Read [the full list](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#changed) of other changes in Browsertime. ### What to think about when upgrading You need to do a plan on when you want to upgrade Graphite to be able to use those new dashboards. You can upgrade to 17.0.0 and continue as before but to be able to use the new things in the dashboards you need to [upgrade your Graphite data](https://www.sitespeed.io/documentation/sitespeed.io/graphite/#upgrade-to-use-the-test-slug-in-the-namespace)). ### Other notable changes Many thanks to [Inderpartap Singh Cheema](https://github.com/inderpartap) that fixed so that you can use `--chrome.blockDomainsExcept` together with WebPageReplay in the Docker container, so you more easily can focus on the performance disregarding 3rd party marketing scripts. We have a couple of new metrics that is the delta between TTFB and First Contentful Paint, Largest Contentful paint and First visual change [#1528]((https://github.com/sitespeedio/browsertime/pull/1528)). You can use this if you have unstable TTFB and want to alert on front end metrics. The metrics are automatically sent to Graphite. Lets continue with all the changes. ### Added * Updated to [Browsertime 12.0.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1200---2021-04-15) with the following changes: * Display standard deviation instead of the home made median deviation in the cli output [#1529](https://github.com/sitespeedio/browsertime/pull/1529). * Renamed layoutShift to the more correct cumulativeLayoutShift. This will is a breaking change if you use that metric. Updates to the new and coming layout shift changes announced by Google will be implemented the coming weeks. * Updated Chrome start flags on desktop following [best practices](https://github.com/GoogleChrome/chrome-launcher/blob/master/docs/chrome-flags-for-tools.md) and removing old flags [#1507](https://github.com/sitespeedio/browsertime/pull/1507). * Updated Chrome start flags on Android following [best practices](https://github.com/GoogleChrome/chrome-launcher/blob/master/docs/chrome-flags-for-tools.md) and removing old flags [#1506](https://github.com/sitespeedio/browsertime/pull/1506). * Finally the "new" connectivity settings are default. You can see the difference in https://github.com/sitespeedio/browsertime/blob/main/lib/connectivity/trafficShapeParser.js#L5-L104. Changed in [#1540](https://github.com/sitespeedio/browsertime/pull/1540). If you wanna run with the legacy setting use `--legacyConnectivityProfiles`. See [#1160](https://github.com/sitespeedio/browsertime/pull/1160) for the original change. * The default minimum wait time for waiting if a test is finished is now 8 seconds (instead of 5) [#1542](https://github.com/sitespeedio/browsertime/pull/1542). * Fix `--chrome.blockDomainsExcept` when you are using WebPageReplay [#1532](https://github.com/sitespeedio/browsertime/pull/1532). Thank you [Inderpartap Singh Cheema](https://github.com/inderpartap) for the original fix! * Make sure gnirehtet is closed on the right device using device id [#1527](https://github.com/sitespeedio/browsertime/pull/1527) * Upgraded to GeckoDriver 0.29.1. * Updated wpr_cert.pem to a new version for WebPageReplay [#1316](https://github.com/sitespeedio/browsertime/pull/1316). * Include Google Web Vitals in the HAR file [#1535](https://github.com/sitespeedio/browsertime/pull/1535). * Added 3 seconds wait time for geckoprofiler to start see [#1538](https://github.com/sitespeedio/browsertime/issues/1538) and [#1539](https://github.com/sitespeedio/browsertime/pull/1539) * Added a ten seconds wait if getting the HAR from Firefox fails the first time [#1546](https://github.com/sitespeedio/browsertime/pull/1546) fixes [#3346](https://github.com/sitespeedio/sitespeed.io/issues/3346). * New metrics: Delta between TTFB and First Contentful Paint, Largest Contentful paint and First visual change [#1528](https://github.com/sitespeedio/browsertime/pull/1528). You can use this if you have unstable TTFB and want to alert on front end metrics. Lets see when other also implement this :) * Made it easier for people to get Google Web Vitals. We copy that data under the googleWebVitals namespace in the result JSON [#1521](https://github.com/sitespeedio/browsertime/pull/1521). * Added TTFB as a single metric [#1522](https://github.com/sitespeedio/browsertime/pull/1522). * New stop watch command [#1512](https://github.com/sitespeedio/browsertime/pull/1512). Measure time by: ```const timer = commands.stopWatch.get('my_timer'); timer.start(); // Do something // Stop the timer and add the result to the last tested URL timer.stopAndAdd(); ``` * Pre test/warm a URL with `--preWarmServer`. Do that to make sure your server has cached everything that is needed before your test [#1515](https://github.com/sitespeedio/browsertime/pull/1515) and [#1516](https://github.com/sitespeedio/browsertime/pull/1516). * Collect what HTML element change in cumulative layout shifts [#1534](https://github.com/sitespeedio/browsertime/pull/1534) * Added support for recording video on Safari iOS [#1541](https://github.com/sitespeedio/browsertime/pull/1541). * New commands: scrolling by Pixels, Lines or Pages forward, back or refresh navigations, create new tabs or windows and switch to them and new mouse events such as context click, single click, double click, click and hold, release, and movement. Thank you [Denis Palmeiro](https://github.com/dpalmeiro) for PR [#1533](https://github.com/sitespeedio/browsertime/pull/1533). * Improve proxy configuration support, thank you [Olaf Meeuwissen](https://github.com/paddy-hack) for PR [#1542](https://github.com/sitespeedio/browsertime/pull/1524). * Upgraded to Selenium 4.0.0-beta.3 [#1544](https://github.com/sitespeedio/browsertime/pull/1544). * Updated to Chrome 90 in the Docker container and Chromedriver 90 as default [#1543](https://github.com/sitespeedio/browsertime/pull/1543). * Added `--preWarmServer` that makes a request to your web server to fill the server cache before you start your tests. * Send navigation timings metrics by default to Graphite/Influx [#3316](https://github.com/sitespeedio/sitespeed.io/pull/3316). * Updated the wpt-plugin to send TBT and CLS to data storage. * Updated the Lighthouse plugin to send Goggle Web Vitals to data storage. * Updated to Lighthouse 7.3.0 in the Lighthouse plugin. * Lighthouse and GPSI plugins published to npm * Updated the GPSI plugin to send Google Web Vitals to data storage (and show it in the HTML). * Renamed layouShift to cumulativeLayoutShift * Send TTFB and Google Web Vitals by default to data storage for Browsertime (making it easier to find them in Influx). ### Fixed * Updated to [latest aws-sdk](https://github.com/sitespeedio/sitespeed.io/commit/427d69f7de5417fa0fd6e305457c61f4ab634811) , google cloud storage and axe-core 4.1.4 [#3334](https://github.com/sitespeedio/sitespeed.io/pull/3334) * Updated to latest Coach and PageXray. ## 16.10.3 - 2021-03-17 ### Fixed * Updated to Browsertime 11.6.3 since last version broke getting the netlog on desktop for Chrome. ## 16.10.2 - 2021-03-17 ### Fixed * Added missing download link in the HTML for the Chrome netlog [#3315](https://github.com/sitespeedio/sitespeed.io/pull/3315) ## 16.10.1 - 2021-03-17 ### Fixed * Updated to [Browsertime 11.6.2](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1162---2021-03-17) with new Selenium beta 4 and a fix for getting the netlog for Chrome on Android. ## 16.10.0 - 2021-03-11 ### Added * Updated to [Browsertime 11.6.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1160---2021-03-08) with Chromedriver 89, Edgedriver 89. * Updated Docker containers to use Chrome 89 and Firefox 86. ### Fixed * The Lighthouse plugin was upgraded to Lighthouse 7.2.0 and fixed broken support for alias. * Updated to AXE-core 4.1.3 [#3302](https://github.com/sitespeedio/sitespeed.io/pull/3302). * More fixes for showing correct screenshots [#3305](https://github.com/sitespeedio/sitespeed.io/pull/3305) and [#3306](https://github.com/sitespeedio/sitespeed.io/pull/3306). * Updated to Browsertime 11.6.1 that makes sure TSProxy uses Python2 when you run in a Docker container. ## 16.9.2 - 2021-02-24 ### Fixed * Another screenshot fix, to make sure we support whatever image format when we copy the image from the last run [#3290](https://github.com/sitespeedio/sitespeed.io/pull/3290). ## 16.9.1 - 2021-02-24 ### Fixed * The screenshot fix in 16.9.0 introduced an error when you copy latest images to the parent dir (upcoming feature in 17.0). Fixed in [#3288](https://github.com/sitespeedio/sitespeed.io/pull/3288). ## 16.9.0 - 2021-02-24 ### Added * Upgraded to Coach core 6.1 that finds Google reCAPTCHAs [#3284](https://github.com/sitespeedio/sitespeed.io/pull/3284). * Upgraded to Browsertime 11.5.0 that makes it easier to use gnirehtet [#3281](https://github.com/sitespeedio/sitespeed.io/pull/3281). ### Fixed * Upgraded to AXE-core 4.1.2 [#3282](https://github.com/sitespeedio/sitespeed.io/pull/3282) * Screenshots was not shown in the screenshot tab as reported in [#3286](https://github.com/sitespeedio/sitespeed.io/issues/3286) fixed in [#3278](https://github.com/sitespeedio/sitespeed.io/pull/3287). ## 16.8.1 - 2021-02-12 ### Fixed * Avoid sending slug/domain annotation names that collide. Fixed in [#3279](https://github.com/sitespeedio/sitespeed.io/pull/3279) and reported in [#3277](https://github.com/sitespeedio/sitespeed.io/issues/3277). ## 16.8.0 - 2021-02-08 ### Added * Updated Browsertime with the ability to mark a run as a failure. We gonna add more docs and try this ourselves and push it in 17.0 [#3272](https://github.com/sitespeedio/sitespeed.io/pull/3272). * Updated to latest VideoJS and changed how the video is displayed [#3268](https://github.com/sitespeedio/sitespeed.io/pull/3268). ### Fixed * The `--addSlugToKey` command introduced 16.3.0 was broken and inserted the slug at the wrong place. Fixed now and we gonna push it and the documentation on how to use it in 17.0 [#3274](https://github.com/sitespeedio/sitespeed.io/pull/3274). ## 16.7.1 - 2021-02-02 ### Fixed * Updated to Browsertime 11.3.1 that open/closes the iOS simulator at the start/end of the test. ## 16.7.0 - 2021-02-01 ### Added * Updated to Browsertime 11.3.0 with better iOS simulator support. There are still some work to do, but you can try it out with `sitespeed.io https://www.wikipedia.org -b safari --safari.useSimulator --safari.deviceUDID YOUR_DEVICE_ID --video --visualMetrics`. List your device ids with `xcrun simctl list devices`. ## 16.6.0 - 2021-01-25 ### Added * Upgraded to Browsertime 11.2.0 with support for listening on events for CDP in scripting. ## 16.5.0 - 2021-01-20 ### Added * Upgraded to Chrome 88 in the Docker container and updated Browsertime that uses Chromedriver 88. Also upgraded Browsertime to 11.1.2 that fixes a bug if Browsertime do not collect any metrics, that broke testing with WebPageReplay. ## 16.4.0 - 2021-01-20 ### Added * Added support for GCS to also upload latest screenshots/video [#3258](https://github.com/sitespeedio/sitespeed.io/pull/3258)- * Automatically create a result base URL if you upload to GCS (and do not configure one) [#3259](https://github.com/sitespeedio/sitespeed.io/pull/3259). * Include index.html when we log the link to the S3/GCS result bucket to make it easier to copy/paste [#3260](https://github.com/sitespeedio/sitespeed.io/pull/3260). ### Fixed * Updated to [Browsertime 11.0.2](https://github.com/sitespeedio/browsertime/releases/tag/v11.0.2) that hopefully fixes the problem when you use an alias in scripting and the URL change between runs. ## 16.3.2 - 2021-01-14 ### Fixed * Make sure the right connectivity name is picked up for the latest copied file. Before the fix using alias for your connectivity didn't work [#3255](https://github.com/sitespeedio/sitespeed.io/pull/3255). ## 16.3.1 - 2021-01-14 ### Fixed * Fixed broken URL in Slack message (if you used an alias) [#3254](https://github.com/sitespeedio/sitespeed.io/pull/3254) * Make sure alias from the cli, from file or from script is handled the same way [#3253](https://github.com/sitespeedio/sitespeed.io/pull/3253). * Added missing slug in annotation [#3251](https://github.com/sitespeedio/sitespeed.io/pull/3251). * Add missing browser/connectivity name in the latest files [#3249](https://github.com/sitespeedio/sitespeed.io/pull/3249). ## 16.3.0 - 2021-01-13 ### Added There's a couple of new functionality that will have documentation in a week or two. Until then do not use it :) * You can now use `--addSlugToKey` to add the `--slug` to your Graphite key. It will be appended to your _graphite.namespace.pageSummary_ [#3240](https://github.com/sitespeedio/sitespeed.io/pull/3240). * New plugin to copy screenshots [#3243](https://github.com/sitespeedio/sitespeed.io/pull/3243) and videos
{'addition_count': 1, 'commit_subject': 'screenshot fix!', 'deletion_count': 1, 'file_extension': '.md', 'lang': 'md', 'license': 'mit', 'repo_name': 'sitespeedio/sitespeed.io'}