title: Optimizing aggregation query against wikidata
body: <p>I am running an aggregation query against wiki data. The query tries to calculate the average duration of films, grouped by their genre and the year of publication</p>
  
  <p>The multiple grouping/subqueries in the query are intended to retain an <code>n-1</code> relationship from film to the grouping criteria (year and genre) and a <code>1-1</code> relationship between a film and its duration. Reason for this is having approximately correct aggregations (n-1 relationships are familiar for OLAP and data warehousing practitioners).</p>
  
  <p>More explanation is embedded in the query. Hence I cannot drop down the groupings done in the subqueries and the if statements or the group concatenation. This query times out on <a href="https://query.wikidata.org" rel="nofollow noreferrer">Wikidata SPARQL endpoint</a>.</p>
  
  <p><strong>QUESTION</strong></p>
  
  <p>I need some suggestion for performance enhancement... <strong><em>Any optimization hints</em></strong>? In case that's not possible, anyone aware of some <strong><em>authenticated way</em></strong> (so that they know I am not playing around) to query Wikidata so that timeout can be increased, or a way to <strong><em>increase timeout generally</em></strong>?</p>
  
  <pre><code>    # Average duration of films, grouped by their genre and the year of publication       
  SELECT  
          ?genre1                    # film genre
          ?year1                     # film year of publication
          (AVG(?duration1) AS ?avg)   # film average duration
  
  WHERE
          {      
              # Calculating the average duration for each single film.
              # As there are films with multiple duration, these durations are 
              # averagred by grouping aggregating durations by film.
              # Hence, a single duration for each film is projected out from the subquery.
              {
                select ?film (avg(?duration) as ?duration1)  
                where{
                  ?film   &lt;http://www.wikidata.org/prop/direct/P2047&gt;   ?duration .    
                }group by ?film
              }
  
              # Here the grouping criteria (genre and year) are calculated.
              # The criteria is grouped by film, so that in case multiple 
              # genre/multiple year exist for a single film, all of them are
              # group concated into a single value.
              # Also in case of a lack of a value of year or genre for some
              # specific film, a dummy value "OtherYear"/"OtherGenre" is generated.
              {
                select ?film (
                                  IF
                                  (
                                      group_concat(distinct ?year ; separator="-- ") != "", 
                                      # In case multiple year exist for a single film, all of them are group concated into a single value.
                                      group_concat(distinct ?year ; separator="-- "), 
                                     # In case of a lack of a value of year for some specific film, a dummy value "OtherYear" is generated.
                                      "OtherYear"                                        
                                  )
                                  as ?year1
                                )
                                  (
                                  IF
                                  (
                                      group_concat(distinct ?genre ; separator="-- ") != "",
                                      # In case multiple genre exist for a single film, all of them are group concated into a single value.
                                      group_concat(distinct ?genre ; separator="-- "), 
                                      # In case of a lack of a value of genre for some specific film, a dummy value "OtherGenre" is generated.
                                      "OtherGenre"  
                                  )
                                  as ?genre1
                                ) 
  
                where 
                {
                  ?film  &lt;http://www.wikidata.org/prop/direct/P31&gt;  &lt;http://www.wikidata.org/entity/Q11424&gt; .
                   optional {
                     ?film   &lt;http://www.wikidata.org/prop/direct/P577&gt;  ?date .
                     BIND(year(?date) AS ?year)
                   }
                   optional {
                     ?film &lt;http://www.wikidata.org/prop/direct/P136&gt;  ?genre .
                   }
                } group by ?film              
            }
  
          } GROUP BY ?year1 ?genre1
  </code></pre>