Is there any purpose to having students do fiddly cache homework assignments? I teach undergraduate computer architecture and am reconsidering one of my homework assignments in which I ask students to simulate by hand (on paper) the behavior of different caches on the same access sequence. It is tedious to create such a problem in a way that the caches behave differently in an interesting way, and I'm sure doing the problems is tedious too (as is grading them). I'm willing to do the work and have students do the work if it's actually useful, but I'm not sure it is.

The purpose of the weekly homework assignments it to reinforce the material the students learn from the textbook (Computer Organization and Design by Hennessy and Patterson), lectures, and in-class exercises. There is already a full set of lab assignments that require wiring or coding (in assembly language or Verilog). There is not time for an additional lab assignment on cache simulation.

I have a separate homework assignment that asks higher-level questions about caches, such as which of two versions of code is likely to run faster if cache performance is taken into account. I think writing cache-aware programs is an important skill. Is simulating the differences between direct-mapped, 2-way set associative caches, and 4-way set associative caches an important part of students' education?
It sounds like you've already answered your question, but I'll throw in my two cents:

It depends.

Does every software engineer need to simulate the differences between direct-mapped, 2-way set associative caches, and 4-way set associative caches? Absolutely not. I've been a programmer for about 10 years now, and I think I have a pretty successful career, and I have no idea what those things even mean.

However, should some students, who are particularly interested in these kinds of things, take a deeper dive? Absolutely yes.

So, it comes down to a question of: is this course designed for students to get a high-level overview of computer architecture, or is this a more specialized course designed for students to get more information about the low-level details? In other words, is this a breadth or depth course?

You might want to give students a high-level overview, and then point particularly interested students at further reading, more advanced courses, or specialized fields they might study in grad school.