Can we use median to replace all the missing values from a column
imputing with median/mean affects distibution of the data, essentially squeezes it, and it does not account for variance in the data, not to mention we'll be making up data. Of course, there're some cases that it may not matter as much and some cases where it will.

If it's worthwhile your time and in case it adds value to your work, you can look into other methods such as proper stochastic regression imputation. Using this method you can randomly add value using linear regression imputation method and create several models and aggregate the results. In this case you'll be accounting for variance in your data and not essentially makin up data, but it's more computationally consuming of course in comparison to just fillna.

There's new method to impute values based on missing patterns, where you'll break your data based on the pattern and create models for each and aggregate.
Of course if it's worth the time and effort, these methods should be more beneficial.
Also you can look into python's fancyimpute

Here are some good resources that cover various methods of imputing missing values:
1) http://www.stat.columbia.edu/~gelman/arm/missing.pdf
2) http://www.bu.edu/sph/files/2014/05/Marina-tech-report.pdf
3) https://stefvanbuuren.name/fimd/sec-nonnormal.html
4) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2727536/