Algorithm for estimating lossless compression factor
In general the only way to know what the compression factor with a particular compression method is to compress the data using that compression method and see what the compression ratio is.

Some compression algorithms have different levels of compression (e.g., 0-9), which allow the algorithm to spend a lot more CPU time to try to get a bit better compression.  You might be able to use the compression factor at a lower compression level as a predictor of the compression factor at a higher level, though this won't be perfect.