title: Efficient join using Django REST Framework serializers
body: <p>I am using the below set of serializers to achieve a join, which work fine on my development setup, but performs terribly when there is any distance between the website server and the database server. I got suspicious about the SQL that's running the show and did some logging; it seems that it's doing a new query for every entry and combining the results rather than doing the entire join at once and returning the join like I want. Here are my serializers:</p>
  <pre><code>class UserSerializer(serializers.ModelSerializer):
      class Meta:
          model = User
          exclude = ('password', 'last_login', 'is_superuser', 'is_staff', 'is_active', 'date_joined',
                     'groups', 'user_permissions')
  
  
  class DepartmentSerializer(serializers.HyperlinkedModelSerializer):
      curator = UserSerializer()
      class Meta:
          model = Department
          fields = '__all__'
  
  
  class CategorySerializer(serializers.HyperlinkedModelSerializer):
      class Meta:
          model = Category
          fields = '__all__'
  
  
  class DetailedLinkedContentSerializer(serializers.HyperlinkedModelSerializer):
      category = CategorySerializer()
      department = DepartmentSerializer()
      type = serializers.SerializerMethodField()
  
      class Meta:
          fields = '__all__'
          model = LinkedContent
  
      def get_type(self, obj):
          return 'link'
  
  
  class DetailedFileContentSerializer(serializers.HyperlinkedModelSerializer):
      category = CategorySerializer()
      department = DepartmentSerializer()
      link_url = serializers.SerializerMethodField()
      type = serializers.SerializerMethodField()
  
      class Meta:
          fields = '__all__'
          model = FileContent
  
      def get_link_url(self, obj):
          return obj.file.url
  
      def get_type(self, obj):
          return obj.file_type
  </code></pre>
  <p>As you can see, I'm doing my 'join' by including fields in a serializer as serializers of other models, such as category = CategorySerializer(). It looks like that's what <a href="http://www.django-rest-framework.org/api-guide/relations/#example" rel="nofollow noreferrer">DRF recommends</a>, unless I'm misunderstanding something. Here is a small sample of the hundreds and hundreds of queries being run on my development environment:</p>
  <pre><code>(0.001) SELECT &quot;content_linkedcontent&quot;.&quot;id&quot;, &quot;content_linkedcontent&quot;.&quot;link_text&quot;, &quot;content_linkedcontent&quot;.&quot;department_id&quot;, &quot;content_linkedcontent&quot;.&quot;category_id&quot;, &quot;content_linkedcontent&quot;.&quot;visibility_rank&quot;, &quot;content_linkedcontent&quot;.&quot;link_url&quot; FROM &quot;content_linkedcontent&quot;; args=()
  (0.001) SELECT &quot;content_category&quot;.&quot;id&quot;, &quot;content_category&quot;.&quot;name&quot;, &quot;content_category&quot;.&quot;description&quot; FROM &quot;content_category&quot; WHERE &quot;content_category&quot;.&quot;id&quot; = 3; args=(3,)
  (0.001) SELECT &quot;content_department&quot;.&quot;id&quot;, &quot;content_department&quot;.&quot;name&quot;, &quot;content_department&quot;.&quot;description&quot;, &quot;content_department&quot;.&quot;curator_id&quot;, &quot;content_department&quot;.&quot;visibility_rank&quot; FROM &quot;content_department&quot; WHERE &quot;content_department&quot;.&quot;id&quot; = 24; args=(24,)
  (0.000) SELECT &quot;auth_user&quot;.&quot;id&quot;, &quot;auth_user&quot;.&quot;password&quot;, &quot;auth_user&quot;.&quot;last_login&quot;, &quot;auth_user&quot;.&quot;is_superuser&quot;, &quot;auth_user&quot;.&quot;username&quot;, &quot;auth_user&quot;.&quot;first_name&quot;, &quot;auth_user&quot;.&quot;last_name&quot;, &quot;auth_user&quot;.&quot;email&quot;, &quot;auth_user&quot;.&quot;is_staff&quot;, &quot;auth_user&quot;.&quot;is_active&quot;, &quot;auth_user&quot;.&quot;date_joined&quot; FROM &quot;auth_user&quot; WHERE &quot;auth_user&quot;.&quot;id&quot; = 3; args=(3,)
  (0.000) SELECT &quot;content_category&quot;.&quot;id&quot;, &quot;content_category&quot;.&quot;name&quot;, &quot;content_category&quot;.&quot;description&quot; FROM &quot;content_category&quot; WHERE &quot;content_category&quot;.&quot;id&quot; = 3; args=(3,)
  (0.000) SELECT &quot;content_department&quot;.&quot;id&quot;, &quot;content_department&quot;.&quot;name&quot;, &quot;content_department&quot;.&quot;description&quot;, &quot;content_department&quot;.&quot;curator_id&quot;, &quot;content_department&quot;.&quot;visibility_rank&quot; FROM &quot;content_department&quot; WHERE &quot;content_department&quot;.&quot;id&quot; = 29; args=(29,)
  (0.000) SELECT &quot;auth_user&quot;.&quot;id&quot;, &quot;auth_user&quot;.&quot;password&quot;, &quot;auth_user&quot;.&quot;last_login&quot;, &quot;auth_user&quot;.&quot;is_superuser&quot;, &quot;auth_user&quot;.&quot;username&quot;, &quot;auth_user&quot;.&quot;first_name&quot;, &quot;auth_user&quot;.&quot;last_name&quot;, &quot;auth_user&quot;.&quot;email&quot;, &quot;auth_user&quot;.&quot;is_staff&quot;, &quot;auth_user&quot;.&quot;is_active&quot;, &quot;auth_user&quot;.&quot;date_joined&quot; FROM &quot;auth_user&quot; WHERE &quot;auth_user&quot;.&quot;id&quot; = 6; args=(6,)
  (0.000) SELECT &quot;content_category&quot;.&quot;id&quot;, &quot;content_category&quot;.&quot;name&quot;, &quot;content_category&quot;.&quot;description&quot; FROM &quot;content_category&quot; WHERE &quot;content_category&quot;.&quot;id&quot; = 4; args=(4,)
  (0.000) SELECT &quot;content_department&quot;.&quot;id&quot;, &quot;content_department&quot;.&quot;name&quot;, &quot;content_department&quot;.&quot;description&quot;, &quot;content_department&quot;.&quot;curator_id&quot;, &quot;content_department&quot;.&quot;visibility_rank&quot; FROM &quot;content_department&quot; WHERE &quot;content_department&quot;.&quot;id&quot; = 25; args=(25,)
  (0.000) SELECT &quot;auth_user&quot;.&quot;id&quot;, &quot;auth_user&quot;.&quot;password&quot;, &quot;auth_user&quot;.&quot;last_login&quot;, &quot;auth_user&quot;.&quot;is_superuser&quot;, &quot;auth_user&quot;.&quot;username&quot;, &quot;auth_user&quot;.&quot;first_name&quot;, &quot;auth_user&quot;.&quot;last_name&quot;, &quot;auth_user&quot;.&quot;email&quot;, &quot;auth_user&quot;.&quot;is_staff&quot;, &quot;auth_user&quot;.&quot;is_active&quot;, &quot;auth_user&quot;.&quot;date_joined&quot; FROM &quot;auth_user&quot; WHERE &quot;auth_user&quot;.&quot;id&quot; = 6; args=(6,)
  (0.000) SELECT &quot;content_category&quot;.&quot;id&quot;, &quot;content_category&quot;.&quot;name&quot;, &quot;content_category&quot;.&quot;description&quot; FROM &quot;content_category&quot; WHERE &quot;content_category&quot;.&quot;id&quot; = 1; args=(1,)
  (0.000) SELECT &quot;content_department&quot;.&quot;id&quot;, &quot;content_department&quot;.&quot;name&quot;, &quot;content_department&quot;.&quot;description&quot;, &quot;content_department&quot;.&quot;curator_id&quot;, &quot;content_department&quot;.&quot;visibility_rank&quot; FROM &quot;content_department&quot; WHERE &quot;content_department&quot;.&quot;id&quot; = 29; args=(29,)
  (0.000) SELECT &quot;auth_user&quot;.&quot;id&quot;, &quot;auth_user&quot;.&quot;password&quot;, &quot;auth_user&quot;.&quot;last_login&quot;, &quot;auth_user&quot;.&quot;is_superuser&quot;, &quot;auth_user&quot;.&quot;username&quot;, &quot;auth_user&quot;.&quot;first_name&quot;, &quot;auth_user&quot;.&quot;last_name&quot;, &quot;auth_user&quot;.&quot;email&quot;, &quot;auth_user&quot;.&quot;is_staff&quot;, &quot;auth_user&quot;.&quot;is_active&quot;, &quot;auth_user&quot;.&quot;date_joined&quot; FROM &quot;auth_user&quot; WHERE &quot;auth_user&quot;.&quot;id&quot; = 6; args=(6,)
  (0.000) SELECT &quot;content_category&quot;.&quot;id&quot;, &quot;content_category&quot;.&quot;name&quot;, &quot;content_category&quot;.&quot;description&quot; FROM &quot;content_category&quot; WHERE &quot;content_category&quot;.&quot;id&quot; = 1; args=(1,)
  (0.000) SELECT &quot;content_department&quot;.&quot;id&quot;, &quot;content_department&quot;.&quot;name&quot;, &quot;content_department&quot;.&quot;description&quot;, &quot;content_department&quot;.&quot;curator_id&quot;, &quot;content_department&quot;.&quot;visibility_rank&quot; FROM &quot;content_department&quot; WHERE &quot;content_department&quot;.&quot;id&quot; = 25; args=(25,)
  (0.000) SELECT &quot;auth_user&quot;.&quot;id&quot;, &quot;auth_user&quot;.&quot;password&quot;, &quot;auth_user&quot;.&quot;last_login&quot;, &quot;auth_user&quot;.&quot;is_superuser&quot;, &quot;auth_user&quot;.&quot;username&quot;, &quot;auth_user&quot;.&quot;first_name&quot;, &quot;auth_user&quot;.&quot;last_name&quot;, &quot;auth_user&quot;.&quot;email&quot;, &quot;auth_user&quot;.&quot;is_staff&quot;, &quot;auth_user&quot;.&quot;is_active&quot;, &quot;auth_user&quot;.&quot;date_joined&quot; FROM &quot;auth_user&quot; WHERE &quot;auth_user&quot;.&quot;id&quot; = 6; args=(6,)
  (0.000) SELECT &quot;content_category&quot;.&quot;id&quot;, &quot;content_category&quot;.&quot;name&quot;, &quot;content_category&quot;.&quot;description&quot; FROM &quot;content_category&quot; WHERE &quot;content_category&quot;.&quot;id&quot; = 1; args=(1,)
  (0.000) SELECT &quot;content_department&quot;.&quot;id&quot;, &quot;content_department&quot;.&quot;name&quot;, &quot;content_department&quot;.&quot;description&quot;, &quot;content_department&quot;.&quot;curator_id&quot;, &quot;content_department&quot;.&quot;visibility_rank&quot; FROM &quot;content_department&quot; WHERE &quot;content_department&quot;.&quot;id&quot; = 24; args=(24,)
  (0.000) SELECT &quot;auth_user&quot;.&quot;id&quot;, &quot;auth_user&quot;.&quot;password&quot;, &quot;auth_user&quot;.&quot;last_login&quot;, &quot;auth_user&quot;.&quot;is_superuser&quot;, &quot;auth_user&quot;.&quot;username&quot;, &quot;auth_user&quot;.&quot;first_name&quot;, &quot;auth_user&quot;.&quot;last_name&quot;, &quot;auth_user&quot;.&quot;email&quot;, &quot;auth_user&quot;.&quot;is_staff&quot;, &quot;auth_user&quot;.&quot;is_active&quot;, &quot;auth_user&quot;.&quot;date_joined&quot; FROM &quot;auth_user&quot; WHERE &quot;auth_user&quot;.&quot;id&quot; = 3; args=(3,)
  (0.000) SELECT &quot;content_category&quot;.&quot;id&quot;, &quot;content_category&quot;.&quot;name&quot;, &quot;content_category&quot;.&quot;description&quot; FROM &quot;content_category&quot; WHERE &quot;content_category&quot;.&quot;id&quot; = 3; args=(3,)
  (0.000) SELECT &quot;content_department&quot;.&quot;id&quot;, &quot;content_department&quot;.&quot;name&quot;, &quot;content_department&quot;.&quot;description&quot;, &quot;content_department&quot;.&quot;curator_id&quot;, &quot;content_department&quot;.&quot;visibility_rank&quot; FROM &quot;content_department&quot; WHERE &quot;content_department&quot;.&quot;id&quot; = 28; args=(28,)
  (0.000) SELECT &quot;auth_user&quot;.&quot;id&quot;, &quot;auth_user&quot;.&quot;password&quot;, &quot;auth_user&quot;.&quot;last_login&quot;, &quot;auth_user&quot;.&quot;is_superuser&quot;, &quot;auth_user&quot;.&quot;username&quot;, &quot;auth_user&quot;.&quot;first_name&quot;, &quot;auth_user&quot;.&quot;last_name&quot;, &quot;auth_user&quot;.&quot;email&quot;, &quot;auth_user&quot;.&quot;is_staff&quot;, &quot;auth_user&quot;.&quot;is_active&quot;, &quot;auth_user&quot;.&quot;date_joined&quot; FROM &quot;auth_user&quot; WHERE &quot;auth_user&quot;.&quot;id&quot; = 6; args=(6,)
  (0.000) SELECT &quot;content_category&quot;.&quot;id&quot;, &quot;content_category&quot;.&quot;name&quot;, &quot;content_category&quot;.&quot;description&quot; FROM &quot;content_category&quot; WHERE &quot;content_category&quot;.&quot;id&quot; = 1; args=(1,)
  (0.000) SELECT &quot;content_department&quot;.&quot;id&quot;, &quot;content_department&quot;.&quot;name&quot;, &quot;content_department&quot;.&quot;description&quot;, &quot;content_department&quot;.&quot;curator_id&quot;, &quot;content_department&quot;.&quot;visibility_rank&quot; FROM &quot;content_department&quot; WHERE &quot;content_department&quot;.&quot;id&quot; = 28; args=(28,)
  (0.000) SELECT &quot;auth_user&quot;.&quot;id&quot;, &quot;auth_user&quot;.&quot;password&quot;, &quot;auth_user&quot;.&quot;last_login&quot;, &quot;auth_user&quot;.&quot;is_superuser&quot;, &quot;auth_user&quot;.&quot;username&quot;, &quot;auth_user&quot;.&quot;first_name&quot;, &quot;auth_user&quot;.&quot;last_name&quot;, &quot;auth_user&quot;.&quot;email&quot;, &quot;auth_user&quot;.&quot;is_staff&quot;, &quot;auth_user&quot;.&quot;is_active&quot;, &quot;auth_user&quot;.&quot;date_joined&quot; FROM &quot;auth_user&quot; WHERE &quot;auth_user&quot;.&quot;id&quot; = 6; args=(6,)
  (0.000) SELECT &quot;content_category&quot;.&quot;id&quot;, &quot;content_category&quot;.&quot;name&quot;, &quot;content_category&quot;.&quot;description&quot; FROM &quot;content_category&quot; WHERE &quot;content_category&quot;.&quot;id&quot; = 4; args=(4,)
  </code></pre>
  <p>So how can I do a real join with the information that I want using serializers in DRF?</p>
  <h1>UPDATE:</h1>
  <p>I have manage to cut the query time in half by following the advice on <a href="http://ses4j.github.io/2015/11/23/optimizing-slow-django-rest-framework-performance/" rel="nofollow noreferrer">this</a> blog entry, here are my updated serializers and the view that use them:</p>
  <pre><code>class DetailedLinkedContentSerializer(serializers.HyperlinkedModelSerializer):
      category = CategorySerializer()
      department_query = Department.objects.all()
      department_query = DepartmentSerializer.setup_eager_loading(department_query)
      department = DepartmentSerializer(department_query)
      # department = DepartmentSerializer()
      type = serializers.SerializerMethodField()
  
      class Meta:
          fields = '__all__'
          model = LinkedContent
  
      def get_type(self, obj):
          return 'link'
  
      @staticmethod
      def setup_eager_loading(queryset):
          &quot;&quot;&quot; Perform necessary eager loading of data. &quot;&quot;&quot;
          queryset = queryset.select_related('category', 'department')
          return queryset
  
  
  class DetailedFileContentSerializer(serializers.HyperlinkedModelSerializer):
      category = CategorySerializer()
      department_query = Department.objects.all()
      department_query = DepartmentSerializer.setup_eager_loading(department_query)
      department = DepartmentSerializer(department_query)
      # department = DepartmentSerializer()
      link_url = serializers.SerializerMethodField()
      type = serializers.SerializerMethodField()
  
      class Meta:
          fields = '__all__'
          model = FileContent
  
      def get_link_url(self, obj):
          return obj.file.url
  
      def get_type(self, obj):
          return obj.file_type
  
      @staticmethod
      def setup_eager_loading(queryset):
          &quot;&quot;&quot; Perform necessary eager loading of data. &quot;&quot;&quot;
          queryset = queryset.select_related('category', 'department')
          return queryset
  </code></pre>
  <p>and my view that makes use of those serializers:</p>
  <pre><code>class DetailedContentView(views.APIView):
      permission_classes = [IsAuthenticated, ContentCuratorOrReadOnly, IsGroupMember, ]
      def get(self, request, *args, **kwargs):
          context = {&quot;request&quot;: request}
          linked_content = LinkedContent.objects.all()
          file_content = FileContent.objects.all()
          # this line is newly added
          linked_content = DetailedLinkedContentSerializer.setup_eager_loading(linked_content)
          # this line too
          file_content = DetailedLinkedContentSerializer.setup_eager_loading(file_content)
          linked_content_serializer = DetailedLinkedContentSerializer(linked_content, many=True, context=context)
          file_content_serializer = DetailedFileContentSerializer(file_content, many=True, context=context)
  
          response = linked_content_serializer.data + file_content_serializer.data
          response = sorted(response, key=lambda x: (x['department']['visibility_rank'], x['visibility_rank']))
  
          return Response(response)
  </code></pre>
  <p>However, my solution to prefetch my Users serializer within my Department serializer doesn't seem to be doing the trick. Specifically, after updating my department serializer to:</p>
  <pre><code>class DepartmentSerializer(serializers.HyperlinkedModelSerializer):
      curator = UserSerializer()
      class Meta:
          model = Department
          fields = '__all__'
  
      @staticmethod
      def setup_eager_loading(queryset):
          &quot;&quot;&quot; Perform necessary eager loading of data. &quot;&quot;&quot;
          queryset = queryset.select_related('curator')
          return queryset
  </code></pre>
  <p>The following lines:</p>
  <pre><code>department_query = Department.objects.all()
  department_query = DepartmentSerializer.setup_eager_loading(department_query)
  department = DepartmentSerializer(department_query)
  </code></pre>
  <p>don't seem to be prefetching my curators as I want.</p>