{'fraction_non_alphanumeric': 0.029302383626606764, 'fraction_numerical': 0.01320354424060901, 'mean_word_length': 4.803302433371958, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 1, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Data-Driven Load Profiles and the Dynamics of Residential Electric PowerConsumption

The dynamics of power consumption constitutes an essential building block for planning and operating energy systems based on renewable energy supply. Whereas variations in the dynamics of renewable energy generation are reasonably well studied, a deeper understanding of short and long term variations in consumption dynamics is still missing. Here, we analyse highly resolved residential electricity consumption data of Austrian and German households and propose a generally applicable methodology for extracting both the average demand profiles and the demand fluctuations purely from time series data. The analysis reveals that demand fluctuations of individual households are skewed and consistently highly intermittent. We introduce a stochastic model to quantitatively capture such real-world fluctuations. The analysis indicates in how far the broadly used standard load profile (SLP) may be is insufficient to describe the key characteristics observed. These results offer a better understanding of demand dynamics, in particular its fluctuations, and provide general tools for disentangling mean demand and fluctuations for any given system. The insights on the demand dynamics may support planning and operating future-compliant (micro) grids in maintaining supply-demand balance.


I. Introduction
Electrical energy is an essential part of the daily life that should be generated, transmitted, stored and, finally, consumed. Generation and storage of electricity shall match the dynamic consumption of residential, industrial and other sectors at all times. To maintain the balance between the electricity generated by energy providers, and the electricity consumed by consumers, energy suppliers need to know the electricity required by all consumer sectors on a broad range of time scales, i.e. seconds to days. Estimating the typical variations in the electricity demand over the course of a day yields a load profile, which can be attained either through the definition of a methodology to extract a load profile from empirical data, the creation of a model or a combination of both.
Research aiming at developing load profiles goes back to at least the 1940s [1], however the issue of finding a precise, high resolution load profile is becoming more and more urgent due to increasing population, electrical heating systems, electrical vehicles for transportation, solar home systems as well as the increasing share of fluctuating renewable energy (RE) feed-in and the construction of distributed power grids, especially smart grids. In 1999, the first methodologically systematic German household load profile, known as the H0 Standard Load Profile (H0 SLP), was developed [2] and has since been in use without alterations in at least Germany and Austria [3].
We focus here on the residential sector consuming around 29% of all electricity in the European Union [4]. Although initially household load profiles used a temporal resolution of around one hour, newer models have a temporal resolutions up to and including one second, due to the recent availability of highly resolved data sets of electricity consumption and the usage of smart meters in selected houses (see Supplementary Note 1). These new data sets allow the grid operators to record the electricity consumption of individual houses at high temporal resolutions. Several previous studies [5][6][7] analysed high temporal resolution datasets and reported the presence of extreme and significant peaks in the datasets which have not been reported for data sets with temporal resolutions of 15 minutes to 1 hour. However, a structured approach to translate these high-resolution data sets into usable load profiles is not readily available to date. Therefore, in this work we first analyse highly resolved electricity consumption data for groups of houses in Austria and Germany. Our data-driven analysis indicates the potential for the presence of strong fluctuations and high levels of unpredictability in the distribution grids, see Section II. Then, we use the same analysis to introduce the new load profile which is consistent with high-resolution electricity consumption data.
As mentioned above, one important reason of having a high resolution load profile is the increasing share of RE feed-in. In contrast to electric energy generated from burning fossil fuels, RE feed-in is weather-dependent, intermittent, and highly variable [8][9][10]. It thus becomes harder to balance supply and demand. As the share of renewable feed-in is increasing, recent and current research focuses on gaining a deeper understanding of the electricity generated by RE as well as advanced approaches of balancing demand and supply, e.g. by load-shifting [11,12]. In contrast, the dynamics of electricity consumption is far from understood, in particular on the residential level, partially because electricity consumption data are difficult to obtain.
Here, we analyse the dynamic characteristics of the residential power consumption at a high temporal resolution and develop a generally applicable methodology to extract both the consumption trend and the consumption fluctuations. After reviewing the need for a new demand model in more technical detail (Section II), we analyse the electricity consumption data of Germany and Austrian houses measured for several weeks. We disentangle the variations in the consumption dynamics into two main factors, the average load profile (ALP) and the statistics of short-time fluctuations of the ALP. First, through the application of the empirical mode decomposition (EMD) [13], we extract the average load profile from the time series data. This extracted trend captures the demand much more accurately than the often-used H0 SLP (Section III). Next, we use superstatistics to model the fluctuations around this trend into a stochastic fluctuation profile (SFP). Combining the trend and fluctuation analysis, we successfully reproduce a synthetic high resolution load profile, yielding a full data-driven load profile (DLP). Our modelling approach can be readily generalised for use with other data sets and in order to facilitate this we are providing executable code as (Section IV). We thereby develop a load profile methodology that is applicable to existing power grids and data sets, and also provides the tools for extracting load profiles in different regions and under different boundary conditions, for instance for microgrids with high shares of on-site generation or electrical cars.  1. Systematic, asymmetric and intermittent deviations of empirical power consumption dynamics and German standard load profile (H0 SLP) (a) We compare the H0 SLP with the averaged real consumption data of a single day in winter for the NOVAREF (12 houses) and ADRES (30 houses) data sets. The H0 SLP not only fails to capture the correct daily trend, but we also observe large fluctuations of the consumption at short time scales. (b) Shows the same data sets at 15 minutes resolution to emulate the temporal resolution that was used to generate the H0 SLP. Here the failure of the H0 SLP to capture the correct daily trend is even more pronounced. (c) Shows the recorded household power consumption with and without photovoltaics as well as the electricity generated by the PV module of a single household. Note the smaller spikes visible in panels (a) and (b) because the data is averaged over 12 and 30 houses, respectively, for NOVAREF and ADRES, while in panel (c) only a single household is shown.

II. Complex demand dynamics -the necessity of new load profiles
Notwithstanding recent advances, energy suppliers still mostly use the older load profiles, such as the H0 standard load profile, which only has a 15 minute temporal resolution. In Germany, the standard load profile (H0 SLP), was introduced in 1999. Ninety percent of the residential load data used in its creation were measured in the 1970s or earlier with an hourly temporal resolution. Only 10% of the measured load data had a temporal resolution of 15 minutes. Here, we review some of the recent advances towards a modern load profile and touch on the persistent need for a generally applicable consumption framework which we provide in the next sections.
Three well-known model classes exist to describe a household load profile. Top-down or conditional demand analysis models are downward models that use the total electricity consumption estimates of multiple households as well as macro-variables to predict the household energy consumption and generate household load profiles [14,15]. Bottom-up models use micro-variables as input, such as the number of active occupants, the appliances' energy demand and usage time etc. They also often use Markov chains to generate household load profiles [16,17]. Finally, hybrid models employ a combination of the techniques used in top-down and bottom-up models to build up a Statistical Adjusted Engineering (SAE) model [18]. A detailed analysis of how these models have been applied was recently reviewed in [19]. At present, many demand side management models exist that can generate daily residential electricity load profiles, see Supplementary Note 1, [20] and [1] for details. Only a few of these models use a high temporal resolution of the order of seconds and those require a lot of micro-parameters, which still leaves us with the need for an accurate, high-resolution, easy-to-use load profile to be developed.
A focus on higher temporal resolution is necessary to fully understand modern consumption patterns and respond quickly, for instance the disturbances caused by input fluctuations or regulatory or trading anomalies [21]. Recent statistical research on power consumption (see for example [6] and [7]) demonstrates that substantial differences exist

FIG. 2.
Indicating the interaction between households energy consumption by considering the diversity factor (a) shows the diversity factor or the ratio between coincident demand, P cd , and non-coincident demand, P ncd , of 30 households belonging to the ADRES data set every 15 minutes and for 14 days. As it is clear, there are some time intervals during a day whose diversity factor reaches 0.6. (b) and (c) respectively show the energy consumption trajectory of all 30 households from 6:00 to 6:15 and 11:30 to 11:45. It is clear from the figures that the energy consumption of all houses is low around the 6:00 and then is gradually increasing together around 11:00 and, consequently the difference between P cd and P ncd is decreasing proving the obvious interaction between houses consuming the electricity.
between the statistical features of the highly resolved power consumption and consumption on a 15 minute time scale. Analysing power consumption on the short time scale of seconds to one minute reveals extreme consumption spikes, which are completely ignored in the 15 minute load profile [6]. In Fig. 1(a) and (b) a comparison of the H0 SLP with the load profiles of two residential data sets of high temporal resolution, measured in Germany and Austria is shown. The Austrian data set was recorded in 2009-2010 during the ADRES project [22]. The German data set was recorded between 2013 and 2016 during the NOVAREF project [23]. In the Supplementary Note 2 we report further analysis on data sets related to 70 households, recorded in August and September 2019 in Germany during the ENERA project.
As can be seen in Fig. 1(a) and (b), the averaged single day load profiles of both the ADRES and the NOVAREF data sets, strongly deviate from the H0 SLP. Specifically, the trend (mean) of the load profile of the measured data is not well described by the H0 SLP and the fluctuations on short times scales are significant (see Section III). Furthermore, the analysis of the statistics of the highly resolved power consumption reveals the presence of non-Gaussian, intermittent and asymmetric fluctuations, which need to be taken into account when designing demand side management and control mechanisms (see Section IV and Supplementary Note 4.) Some of the observed stochastic properties could be explained by the increased usage of new power generation and consumption devices: Rooftop photovoltaic panels (PVs) in the first case and new electronic devices such as electrical heating systems, smartphones, tablets, robot vacuum cleaners, electric vehicles etc. in the second case [24]. The usage of such devices will influence and alter the shape of the current standard load profile. As shown in Fig. 1(c), the consumption can even reach negative values if houses are directly connected to local RE resources [25]. Local fluctuations in solar and wind generation may further lead to coincident load profile spikes, both positive and negative. Therefore, the precise prediction and management of the power consumption of households on a short time scale is essential to ensure the stability of distributed grids. In addition to installing more on-site generations in the residential sector, replacing fossil-fuelled cars with electric ones, which are charged using household electricity, results in changes in the shape of the load profile of the households (see Supplementary Note 2), especially when organising consumers in micro grids [26]. Since we expect an increasing number of such electrical cars, as well as an increasing penetration of strongly fluctuating RE generating electricity, a new data driven load profile based on modern and highly resolved measurements is urgently needed.
As mentioned above, extreme consumption spikes are completely ignored in the 15 minute load profile as they are averaged out. However, these spikes are of particular importance to lower voltage distribution grids, where coincident consumption can dominate the consumption patterns locally or on a country scale, due to e.g. synchronised activity during major (e.g., sports) events [27]. A well-known simple measure to quantify the coincident electricity consumption between households and moreover to see how this coincidence depends on the number of households is the diversity factor between households [28]. To determine the diversity factor between households, we first evaluate the maximum coincident demand, P cd and non-coincident demand, P ncd , respectively in 15 minutes and one day time windows. For this purpose, we sum the maximum power demand of all houses every 15 minutes and then divide the P cd with the sum of the maximum power demand of all houses over the course of a day, i.e. P ncd . The diversity factor varies from zero to one, where zero indicates no coincident electricity consumption between households, while a diversity factor equal to one shows strong coincidence. As an example, the diversity factor of the ADRES data set, sampled every 15 minutes for 14 days, is shown Fig. 2(a). To clearly indicate the interaction between houses during a day (i) the Energy consumption trajectory of all 30 households (in grey); (ii) the maximum coincident demand (in red) and (iii) the non-coincident demand (in purple) between 6:00 to 6:15 ( Fig. 2(b)) and 11:30 to 11:45 ( Fig. 2(c)) for day 13 are shown in Fig. 2

(b) and (c).
Looking at the households trajectories, it is clear that the energy consumption of all houses is low at the beginning of the day and then it is gradually increasing until around 11:00. Thus, the value of the diversity factor, which is the ratio between coincident demand (P cd ), and the non-coincident demand (P ncd ) increases during daytime. This proves the interaction between the demand of the households during the day and is the reason why the significant spikes in average energy consumption do not disappear after averaging.
To show the relationship between the value of the diversity factor and the number of households, we calculate the diversity factor for the NOVAREF and ENERA data sets, which are composed of 12 and 70 households, respectively, in the Supplementary Note 2. Our results demonstrate that the values of the diversity factor are not zero during a day, even for the 70 ENERA houses and for some time intervals the diversity factor ranges between 0.3 to 0.4 or even larger (maximum 0.6). This indicates that regardless of the number of houses coincident demand will take place during certain time intervals (e.g lunch and dinner time) and spikes during that time will not be averaged out. This will result in a spiky load profile, such as the single day averaged ENERA load profile, see also Supplementary Note 1.
In the next sections, we introduce a generally applicable methodology for creating residential load profiles and demonstrate its applicability through comparisons with power consumption data sets measured in Germany and Austria, as well as the industry standard load profile. In the next Section (Section III) we present a methodology to extract the averaged load profile (ALP).

III. Demand trend: Mode decomposition
In this section we present the Average Load Profile predictor, a methodology that extracts the Averaged Load Profile (ALP) from high temporal resolution data sets and creates the load profile of a full week that explicitly respects differences between the different days during a week, and in particular differences between different working days, much as the H0 SLP does. Its main advantage compared to the H0 SLP is that it only requires 4 weeks of high resolution measured data and can, therefore, be applied to both small neighbourhoods and whole cities. Furthermore, its only input is the electricity consumption trajectory and therefore does not rely on numerous micro-or macroscopic parameters, as existing models do. As a result, it can easily be used to analyse both present and future residential power systems, which might include new technologies and devices. Due to the high temporal resolution of the data sets we use, the ALP predictor has a higher level of accuracy than the H0 SLP. It is, therefore, a good alternative to be used instead of the H0 SLP and could provide better and more accurate results, especially when investigating or operating microgrids (a detailed discussion on the subject can be found in Sec. V). In its present form, the ALP predictor can predict the weekly trend of the load profile of a group of houses (anywhere from 12 to 70 or more houses).
To produce the ALP predictor we extract the consumption trend from four consecutive weeks of high resolution electricity consumption data measurements. In this Section we present our results for the NOVAREF data set (temporal resolution = 2 s). The methodology we use to create the ALP predictor is the following. We split the measured data into multiple modes using the Empirical Mode Decomposition (EMD) [13], and so separate the longterm trends from the short-term fluctuations. Because the EMD extracts all the signals present in the data, the original data set can be recreated 1-for-1 without any loss of information by summing up all the modes. A detailed   The largest difference between the H0 SLP and the ALP (and so the optimal number of mode to sum) is achieved for N = 8, but the ALP clearly overperforms the H0 SLP for any N ≥ 7.) discussion of the adaptive time-frequency data analysis via the EMD can be found in Methods, Supplementary Note 3 and in a recent paper applying EMD in a stochastic context [29]. In order to predict the future daily and weekly household demand we must first train the predictor. As a first step we select 4 chronologically consecutive weeks with no datagaps of measured data from the NOVAREF data set, i.e 07.01-14.01, 14.01-21.01, 04.02-11.02, 11.2-18.04. This constitutes our training set. Then, we calculate the average of these four weekly consumption profiles and then apply the EMD method to it. Through this process 18 individual and independent modes are extracted. We then divide these modes into two categories: (a) high-frequency; and (b) low-frequency modes. The high-frequency modes contain information on the stochastic consumption fluctuations (see Section IV), while the low-frequency modes are almost free of stochastic fluctuations and are instead dominated by deterministic effects. Next, we sum the last N low frequency modes.
In order to get the best performing ALP we must determine the optimal number of modes N (N optimal ). If we sum too few modes, the predictor will fail to generate an ALP that tracks the high resolution consumption better than the H0 SLP. Whereas, if we sum too many modes it will overfit the training data set and lead to inaccurate future load predictions. To determine the N optimal and quantify the performance of the optimised ALP, we use the mean-squared error (MSE) on a validation set, consisting of 29 randomly chosen weeks of data from the NOVAREF data set (see Methods for details on MSE). We must stress here that we used 29 weeks of data because they were available to us. For the purpose of applying this method to other (possibly shorter datasets) only two additional weeks of data are needed to determine the ALP.
The results of this validation analysis reveal the optimal number of modes N optimal , which in this case is obtained at N optimal = 8, see Fig. 3. Indeed, even for values of N > 7, the generated ALP outperforms the H0 SLP in terms of demand prediction. Fig. 3 shows the results for one of the 29 weeks, and the ALP is found to predict the future daily and weekly electricity consumption of the 12 NOVAREF households much more accurately compared to the H0 SLP.
Finally, we use 3 randomly chosen weeks to test our predictor and visualise modes and trajectories. We find that the ALP outperforms the H0 SLP when the N optimal is chosen, while it performs worse for a small number of modes, see Fig. 4. Furthermore, we observe the different modes, ranging from high-frequency fluctuations (mode 1) to slowly changing trend (mode 16). We continue to evaluate the ALP performance in the next section (Section IV).
The ALP predictor has several major advantages compared to both the H0 SLP and the available demand side  management models. Firstly, it is data driven and is based on modern, high temporal resolution (2 s) measurements, whereas the H0 SLP is based mostly on hourly resolution data measured before 1999, with a small subset of 15-minute resolution data measured between 1990 and 1999 [2]. Secondly, because only four chronologically consecutive weeks of data are necessary to generate the ALP and only two additional weeks of data are necessary to determine the N optimal , it can be used to predict the electricity consumption of both small and large groups of houses and does not need the plethora of the micro-and-macro-parameters that presently available demand side management models require. Lastly, the ALP has a higher level of accuracy than the H0 SLP due the high temporal resolution of the data sets it uses. In the next Section (Section IV) a stochastic model to generate the stochastic fluctuations around the trend is presented.

IV. Demand fluctuations: Stochastic model
Having extracted the predominantly deterministic trend of the consumption, we now turn to the stochastic fluctuations around this trend. Specifically, we split the power consumption into trend P (trend) and fluctuations P (fluc.) : First, we investigate the statistical properties of these consumption fluctuations. Analysing the histograms, we notice that the fluctuations are skewed, i.e. asymmetric, and heavy tailed, so large deviations are much more likely than if they were characterised by a simple Gaussian distribution, see Fig. 5. Next, we construct a stochastic model, which describes the observed consumption fluctuation statistics by applying superstatistics methods [21,[30][31][32][33]: Dividing the full trajectory into several shorter trajectories allows us to characterise each local distribution with a simpler distribution, such as a Gaussian or an exponential distribution. In the case of consumption fluctuations, the fluctuations within each time window of approximately T ≈ 1000 seconds follow a distinct Maxwell-Boltzmann distribution, see scale parameter σ M B and offset from zero µ M B . When we now move our analysis from one local distribution to the next one, we observe a slow time dynamics of these Maxwell-Boltzmann parameters σ M B and µ M B and thereby a time dynamics of the local Maxwell-Boltzmann distribution itself. Superimposing these time-varying local distributions, we re-obtain the full aggregated statistics as approximately a q-Maxwell-Boltzmann distribution, see Fig. 5.
Mathematically, we formulate stochastic equations of motion for the fluctuations leading to local Maxwell-Boltzmann distributions as follows: We define auxiliary variables x i , with i ∈ {1, 2, ..., J}, each following a simple Ornstein-Uhlenbeck process, based on independent Wiener processes W i : with damping γ and fluctuations amplitude . Hence, the x i are identical but independently distributed Gaussian random variables with a mean 0 and standard deviation σ = √ 2γ . Then, the demand fluctuations P (fluc.) are obtained by aggregating these Gaussian distributions and applying the observed shift from zero µ M B : As is known from statistical physics [34], choosing J = 3 yields exactly Maxwell-Boltzmann distributions in the probability density p: where the shape parameter σ M B of the local Maxwell-Boltzmann distribution is identical to the standard deviation of the independent Gaussian variables σ M B = σ. We do consider cases of J = 3 in Supplementary Note 4 and find that J = 3 is the best fit to the data. Indeed, the approach of 3 independent Gaussian variables is very convenient for the computational application. Alternative modelling approaches using a mathematically simple 1-D process would require more complex dynamics. In applying the superstatistical approach, we implicitly assumed a separation of time scales. Here, we have the long time scale, on which we locally observe Maxwell-Boltzmann distributions of the power fluctuations of T ≈ 1000 seconds (Fig. 6). Furthermore, we estimate the short time scale, on which each local distribution relaxes to its equilibrium, based on the autocorrelation decay of the data as τ = 1/γ ≈ 300...400 seconds, see also Supplementary Note 4. Comparing these two time scales, we observe a clear time separation between long time scales T and short time scales τ , which differ by a factor of 20. Hence, each local Maxwell-Boltzmann distribution relaxes much faster towards its equilibrium (with rate 1/τ = γ) than the overall process changes towards a new Maxwell-Boltzmann distribution (which happens with a rate of 1/T ).
Finally, we combine the EMD-based trend of the demand with the stochastic fluctuation model, obtaining a datadriven load profile (DLP) and, then, compare it to the original NOVAREF consumption data. We notice that while the precise trajectories are not identical (by construction), the stochastic properties align very well with drastically reduced error compared to the standard H0 SLP model (see Fig. 7). In the Methods section we provide a link to the software and pseudo-code to generate these kind of trajectories for other demand regions and data sets.

V. Discussion
Summarising, we have shown that modern residential electricity load profiles strongly differ from the H0 standard load profile (H0 SLP), which is widely used in the industry. We set out to replace or supplement the existing standard load profile and obtained three main results: First, using the empirical mode decomposition (EMD) method, we developed a method to approximate the average load profile (ALP). Second, using superstatistics, we then developed a simple stochastic fluctuation profile (SFP) to describe the non-Gaussian fluctuations of the demand around the trend. Finally, combining both trend and fluctuations yields a full data-driven load profile (DLP).
All three findings are critical for the stable operation of the power system: Knowledge of the expected demand is critical for energy providers to calculate how much power is needed by each household within a given time period. Simultaneously, knowledge of how much the demand might fluctuate around this trend is also essential, to have sufficient balancing and back-up power at hand. According to the Bundesverband der Energie-und Wasserwirtschaft e. V. (BDEW) [2], the electricity load of a household can be expected to deviate between 10 % and 20 % at any given time. Our fluctuation model provides a better quantitative estimate for these fluctuations to avoid an overestimation of the demand and too much usage of expensive quickly dispatchable generation [35]. On the other hand, we have to prevent an underestimation of the demand as this could easily lead to a collapse of the system.
Another strong point of our modelling approach is its flexibility: We do not present a fixed load profile but a methodology to extract trend (ALP) and fluctuations (SFP) for any present or future power system. This can be applied to different regions or even continents. Maybe even more importantly, the extracted profiles can easily be updated based on recent developments in consumption, e.g. due to additional PV installation or adaptation of electrical cars. Similarly, new load profiles can be created for newly built micro or smart grids [36]. The H0 SLP focused mainly on the time scale of 15 minutes and reports much smaller fluctuations than our new load profiles. How can we explain this? First, we note that the H0 SLP used older data, with a temporal resolution much lower than the 1 seconds of our data sources. More relevant is however the power system perspective: Generation and demand are scheduled for fixed time intervals, such as 15 minutes and all fluctuations and deviations within each interval are taken care by control mechanisms [37]. This view also emphasises the role of the high-voltage transmission grid where some demand fluctuations, which we observe on the distribution grid, might not occur. Our temporally highly resolved model and focus on the distribution grid becomes increasingly relevant: Conventional generators and their stabilising inertia are removed from the grid and renewable generators are often directly coupled with households. Hence, the balance of supply and demand has to be present also on the distribution level and on an increasingly fast time scale. Finally, we note that not only do both households and renewable introduce fluctuations in the distribution power grid but their fluctuations share similarities. In particular, the heavy tails of consumption fluctuations and its slowly decreasing power spectrum are also observed in renewable generation [10].
Our model is also especially useful in the case of micro-grids coupled with smart meters. Micro-grids are small autonomous grids, which can function independently or semi-independently from the main power grid [38]. In the past, small autonomous grids were comprised mainly by remote, usually rural villages or households powered by fossil fuel generators. The concept has been reconfigured to help overcome the challenges of integrating intermittent renewable energy production to the main energy grid [38], [39] as well as provide electricity to populations in remote regions while replacing fossil fuels with renewable resources. Such grids however, always face the problem of matching the intermittent energy generation with the, as we have shown in Section II, also intermittent power consumption of the households that comprise them. A task that is made harder by the fact that many microgrids are composed of less than the 332 households that the H0 SLP assumes. As showed in Sections II and III, the consumption behaviour of a small number of houses deviates significantly from the H0 SLP and displays a strong stochastic component. Which technologies can be used to achieve a successful balancing between the generation and consumption [38], [40], which protection schemes [41] and control systems can be used to ensure the stable and secure operation of these grids [42], [43], [44] and even more scheduling and operation issues [45], [39], [46] are all active areas of research. Furthermore, our model could allow the administrators of an autonomous or semi-autonomous microgrid equipped with smart meters to use the collected data to create an accurate prediction of the electricity consumption of the microgrids units in the following months and take the appropriate measures to ensure a continuous stable operation, e.g. by ensuring enough balancing power is available. At present the H0 SLP, which does not differentiate between the electricity consumption of a handful of houses and a small city, makes this impossible. Our model would also provide researchers, who have access to smart meter data (or any other high temporal resolution data), with much more accurate electricity consumption predictions compared to the ones that H0 SLP allows them at the moment. These prediction could then be used as input for further consumption models.
Because our approach is entirely dependent on data with no prior assumptions being made, it could theoretically be used to predict the electricity consumption not only of households but also of small businesses or full countries. The current lack of freely available data makes this a task for the future. For example, we wish to apply our model on consumption data of other regions, such as the UK or the US and compare it to the local equivalent of the H0 SLP. Regions like Canada, where a large number of smart meters are already installed, are particularly promising. While we focused here on household demand, the presented methods should also be applicable to extract the demand trend and fluctuations of industry consumption or combinations of very different consumers. Finally, the fluctuation modelling could be extended to include longer-term correlations or match the precise increments in the observed data sets. This would move the study of consumption fluctuations closer to the well-researched state of fluctuations in renewable generation.

EMD
The Empirical mode decomposition (EMD) is a well-known method used for nonlinear, non-stationary data sets. It decomposes the data into a finite number of intrinsic mode functions based on the local properties of the data. Therefore, this method is not restricted to linear or stationary time series, as is the case with other methods, such as Fourier spectral analysis. To determine the empirical mode functions, one applies the following steps on a data set: (1) first the envelope of both local maxima and minima in a data set are defined separately. For instance all local maxima are connected by cubic spline lines for the upper envelope and then the same procedure is repeated for the lower envelope. Consequently, all data is confined between upper and lower envelopes. It is worth mentioning that in this method there is no need for the data to have zero crossings, therefore all values can be just positive or negative.
(2) Defining the upper and lower envelopes, their mean value, m 1 has to be calculated and then subtracted from the original data, i.e. X(t) − m 1 = h 1 , where X(t) is the original data and h 1 is called the first component. (3) In the next step the first component, h 1 is converted to the intrinsic mode function (IMF), i.e. h 1 should have same number of extrema and zero crossings plus the symmetry of upper and lower envelops around zero (for details see [13]). After finding the IMF, c 1 , we subtract it from the original data: X(t) − c 1 = r 1 . This c 1 is the first mode and contains the shortest period of the original data X(t). (4) Steps (1)-(3) are repeated until r n becomes a monotonic function, and it becomes impossible to define any envelop for that. If we sum up all IMFs and the residue, we reproduce again the original data, i.e.
Consequently, we can obtain all the intrinsic oscillation modes of the data set with the EMD method.

Trend extraction
In this section we present the adaptive time-frequency data analysis we used to created the ALP. It is based on a normalised version of the one step prediction mean squared error (MSE) [47] of an estimator.
In order to produce the ALP, we create a predictor of the future daily and weekly household electricity demand which we then train using four chronologically consecutive weeks of high resolution electricity consumption data measurements which have no data gaps. In this case, weeks 07.01-14.01, 14.01-21.01, 04.02-11.02, 11.2-18.04 of the NOVAREF data set. For accurate and meaningful results we must use data from the same group of houses whose demand we want to predict.
The first step in this process is calculating the average of the four weeks of data: Where E i (t) is the electricity consumption of each week and i is the number of weeks used in the calculation (i = 4).
For the next step we apply the EMD on the averaged weekly electricity demand profile E mean . In the case of the NOVAREF data 18 individual and independent modes are extracted. Finally to calculate the ALP we sum the last N low frequency modes.
where M 1+s is highest-frequency mode still to be included. In the case of the NOVAREF data, we have N = 8 and s = 10, hence all modes from i = 11 up to i = 18 are summed up to obtain the ALP.
To determine the optimal number of low modes to sum (N optimal ) and quantify the performance of the optimised ALP, we use the mean-squared error (MSE). The MSE measures the average squared difference between the estimated values and the actual value [47] and in this paper we calculate it as follows: where C(predicted, t i ) is the predicted consumption timeseries, C(actual, t i ) is the measured consumption timeseries, t i is the time in seconds, 0 < i < L and L is the length of data used to normalise the ALP. L is 7 days measured in 2 seconds increments M = 7 days * 24 hours * 60 minutes * 60 seconds 2 The MSE is calculated for both the ALP (M SE ALP ) and the H0 SLP (M SE H0SLP ) and compared (see Fig. 3) in Section III. The comparison reveals that: for the majority of modes summed. Therefore, the ALP performs better than the H0 SLP and can replace it. Regardless of the number of modes summed to create the ALP there is always a minimum which fulfils the M SE ALP < M SE H0SLP condition (see Fig. 3). This minimum gives the optimal number of low modes N (N optimal ) that must be summed to create the optimal ALP for a given set of chronologically consecutive weeks four weeks. This minimum varies with the set of weeks used, though for the NOVAREF data set N optimal = 8 for the majority of the weeks investigated. A detailed analysis of the model training, validation and testing can be found in the Supplementary Note 3.
The non-randomness of our results were also verified by calculating the fraction of the M ES ALP and the M SE H0SLP M SE f raction = M SE ALP M SE H0SLP (11) for the N ≈ N optimal , M SE f raction < 1. A more detailed analysis can be found in Supplementary Note 3.

Data availability statement
The ADRES household consumption data were provided by the TU Wien and are available for research purposes upon request https://www.ea.tuwien.ac.at/projects/adres_concept/EN/. The ENERA household consumption data that support the findings of this study were made available by the DLR Institute for Networked Energy Systems and the EWE, respectively. Restrictions apply to their availability as they were used under license for the current study, and so are not publicly available. The data are, however, available from the authors upon reasonable request and with permission of the DLR Institute for Networked Energy Systems and the EWE. The NOVAREF household consumption data that support the findings of this study are available for download under the CC-BY license together with code on how the EMD and fluctuation analyses were performed on OSF: https://osf.io/yu2dm/?view_only= 685370675ca145eb88234031158fc32c.