Adding column to filtered dataframe takes longer than copying and then adding
#### Code Sample, a copy-pastable example if possible

```python
import pandas as pd
import time

def change_df(df):
    print(f"columns {df.columns}")
    for col in df.columns:
        start_time = time.time()
        df[col] = df[col].apply(lambda x: int(x))
        took = time.time() - start_time
        print(f"assign {col} took: {took}")


def test_df_conversion_time_local():
    start_time = time.time()
    numbers = {str(i): str(i) for i in range(10)}
    df = pd.DataFrame([numbers], dtype=str)
    df = df[[f'{i}' for i in range(5)]]
    change_df(df)
    print(f"test took: {time.time() - start_time}")


def test_df_conversion_time_local_2():
    start_time = time.time()
    numbers = {str(i): str(i) for i in range(10)}
    df = pd.DataFrame([numbers], dtype=str)
    df = df[[f'{i}' for i in range(5)]].copy()
    change_df(df)
    print(f"test took: {time.time() - start_time}")

test_df_conversion_time_local()
test_df_conversion_time_local_2()

```
#### Problem description

When adding a new column to a filtered dataframe, the execution takes longer than copying the whole filtered dataframe and adding a column to the copy (at least for dataframes with just one column (33 ms vs. 5ms)). Working on a time critical application the difference matters a lot to me personally, but it seems to be interesting that copying data is faster than working on a reference of it.

#### Expected Output

#### Output of ``pd.show_versions()``

<details>


INSTALLED VERSIONS
------------------
commit: None
python: 3.7.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.15.0-50-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.24.1
pytest: None
pip: 18.1
setuptools: 40.6.2
Cython: None
numpy: 1.16.1
scipy: None
pyarrow: None
xarray: None
IPython: 7.2.0
sphinx: None
patsy: None
dateutil: 2.7.5
pytz: 2018.9
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 3.0.3
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10.1
s3fs: None
fastparquet: 0.3.0
pandas_gbq: None
pandas_datareader: None
gcsfs: None


</details>
