Window function in Update statement
The error message defines the problem: you can't use a window function directly in an assignment (SET). You can use a CTE to pull out the aggregate and then apply the update to the CTE (which pushes it back to the table).

;WITH t AS 
(
  SELECT key1, date1, date2, 
    date1max = MAX(date1) OVER (PARTITION BY key1 ORDER BY key1) 
  FROM #table
)
UPDATE t SET date2 = CASE date1max 
  WHEN date1 THEN NULL ELSE date1max END
WHERE date2 IS NULL;


If dates are not unique for any given key, you might need to do something to deal with ties. Please add sufficient sample data that would represent all edge cases and how you want to handle those.