DC current source into capacitor - does capacitor voltage rise infinitely?
But won't the ideal current source still be driving 1mA into the
capacitor and so the capacitor voltage should still be rising?

Your capacitor will charge to a voltage of 101.8 volts (eventually). This is because that is the voltage scenario that causes all the 1 mA to be taken by the 100 kΩ resistor. In other words, when there is 100 volts across the 100 kΩ resistor, the current into that resistor is 1 mA and the circuit voltages stabilize.