How to prepare for questions about the part of the paper that you didn't write?
couldn't say somebody else did it, and I don't know.


Yes, that is more or less what you should say.
It is completely normal for collaborative works that different co-authors are knowledgeable about different parts of the work.
In particular, works that span several otherwise separate areas, it is not unusual for experts to contribute their respective techniques to create an integrated solution together.
In such a case, the various co-authors basically see the components contributed by others as some kind of a blackbox. They should know what those components do, but they do not necessarily know how they do it.

In such a case, it is reasonable to answer:


  The ... algorithm is a work by my co-author ... who unfortunately cannot be here today. I recommend to direct specific questions on the algorithm to them.


You should be ready to provide some way of contacting the co-author (e.g. e-mail address on the slides).