Data Modeling Problem (Tasks / Timerecords / Users) It's late here an I'm having troubles wrapping my head around it.

I have 3 tables: Tasks, Timerecords, Users

A task is assigned to a user and always has an "estimated duration".
Timerecords are created by users and always link to a task. They always have a "logged duration".

I now want to get a list of all users with their sum of estimated duration (over all tasks) and their sum of logged duration (over all tasks), but I can't really figure out how I can make this possible with these 3 tables and avoiding circular relationships. But it wouldn't be a problem if I'd have to copy a table (it's for a data analysis project).
This should be close enough.



-- User USR exists.
--
user {USR}
  PK {USR}


-- Task TSK exists.
--
task {TSK}
  PK {TSK}


-- Task TSK is assigned to user USR,
-- with time estimate of EST hours.
--
task_user {TSK, USR, EST}
       PK {TSK, USR}

FK1 {TSK} REFERENCES task {TSK}
FK2 {USR} REFERENCES user {USR}


-- On date DTE user USR spent HRW hours
-- working on task TSK.
--
time_record {TSK, USR, DTE, HRW}
         PK {TSK, USR, DTE}

FK {TSK, USR} REFERENCES task_user {TSK, USR}


-- PostgreSQL, SQL Server
--
WITH
q_1 AS (
    SELECT USR
         , sum(EST) AS total_est
    FROM task_user
    GROUP BY USR
),
q_2 AS (
    SELECT USR
         , sum(HRW) AS total_log
    FROM time_record
    GROUP BY USR
)
SELECT a.USR
     , coalesce(b.total_est, 0) AS total_estimate
     , coalesce(c.total_log, 0) AS total_logged
FROM      user AS a
LEFT JOIN q_1  AS b ON b.USR = a.USR
LEFT JOIN q_2  AS c ON c.USR = a.USR ;


Note:

All attributes (columns) NOT NULL

PK = Primary Key
FK = Foreign Key