I2C multiplexer channel selection and pull up resistors
Use of pull-ups depend on bus voltages on each side of mux. Do you know if batteries have pull-ups and to which voltage, or if it needs external pull-ups, to which voltage? Regardless of voltages, when either side pulls the bus low, all pull-up resistors will provide current and chip must handle that. Regarding the mux, you set mux address and then you can select mux channels via I2C. After selecting one of the battery channels you then talk to the battery normally.