How to test the influence of a feature on conversion?
Okay, thanks to this post, I was able to test this.
The test we need is a Chi-square test for approx results and Fischer Test for exact variables.

Given, the variables I had were categoric (engaged:yes/no and converted:yes no) in nature, the base assumption was true.

Now, First I created contingency table for each of those engaged and converted with reference from here.

There after, calculated the expected value for each of the cells in table.
Injected the formula and voila, I had the chi-square vale. Further compared with chi-square value table to get the p-value.

This resulted in testing whether the variables are independent or not.

This is what the end results look like:


Reference: https://www.socscistatistics.com/tests/chisquare2/Default2.aspx