MacOS timed won't keep accurate time
There are other reports saying that time under Big Sur is not as accurate as it used to be, does not fully correct the time and in some cases maybe in error by many seconds. For example Time Synchronization Command line in MacOS Big Sur
In my case (2019 iMac) the time was consistently about 60 to 70 mS fast.
A simple command line check is to use the terminal command sntp pool.ntp.org. For examples:
sntp pool.ntp.org
+0.000444 +/- 0.084480 pool.ntp.org 129.250.35.250

sntp time.apple.com
+0.000304 +/- 0.000214 time.apple.com 17.253.66.125

This result is showing what my iMac now achieves - the first number shows an error of less than 1 mS.
To achieve this accuracy Apple's timed daemon needs to be replaced.
The cross-platform app for this is Chrony. This can be installed easily via Homebrew but its configuration might be considered as something of a black art.  Much easier is to use ChronyControl.  Just download, unzip and drag the app to /Applications.
With SIP disabled (if only temporarily) you can disable Apple's timed using:
sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.timed.plist

But don't worry if you want SIP enabled at all times, the timed daemon uses very little resources and ChronyControl will prevent it making time changes.
The first time you run ChronyControl, it will configure everything for you including starting the chronyd daemon with a default configuration, configuring logs and starting a helpful GUI app. Here is the GUI:

and the tracking display:

The only change I have made to the configuration is to replace the use of general purpose servers pool.ntp.org with a country specific set of servers au.pool.ntp.org.
The only other thing you might want to change is that ChronyControl puts the configuration file in /etc/chrony.d. Better, perhaps, to put it in /usr/local/etc/chrony.d.
Chrony is open source (and free), ChronyControl is donation-ware ($15 suggested) and its from the developer of the long established WhatRoute network utility.
I got started on using Chrony as a result of Feek's post on MacRumors to whom I give full credit for the discovery.