ge features in the UNet are trained. As a result, IP-Adapter files are typically only ~100MBs. You can learn more about how to use IP-Adapter for different tasks and specific use cases in the IP-Adapter guide. Diffusers currently only supports IP-Adapter for some of the most popular pipelines. Feel free to open a feature request if you have a cool use case and want to integrate IP-Adapter with an unsupported pipeline!
Official IP-Adapter checkp