Time taken to drop/add index is lesser than adding a column. Why is that? I'm using 10.1.25-MariaDB-1~xenial, but engine is innodb though. 

I'm performing an alter by default it does ONLINE alter. 

When I add column it takes at least 50 mins for 50G table in our infrastructure, but when I performed removing and adding a composite index of 3 columns it took 8 mins. 

Any idea internals of what it is doing?

Also does the duration changes if I'm adding an PK index/UK index.
Let me say in a simple way. When you create a new  table/index and then you are inserting 10 rows so, index value will be 1 to 10. Now you are deleting 5 and 6 the row in the table so ,holes are formed. Next time while inserting rows from 11 to 20 then index 5 and 6th rows need to be refilled that's why it's taking time. If you drop a column and recreate index then value is auto increment that's why it's faster and there is no hole.