Big Sur confirm alert button keyboard shortcut
The new destructive action doesn't have a keyboard shortcut by default. This is likely by design:

[it] allows the system to guard a destructive-action button against accidental presses

You can enable keyboard navigation for controls (previously called Full Keyboard Access) in System Preferences → Keyboard → Shortcuts → ‘Use keyboard navigation to move focus between controls’. With this option enabled, the Cancel button is focused for activation with Space, and you can Tab to the Delete button and activate that with Space too.