Concatenate several datasets with removed columns is not working.
Hi,

did you fill out the env info section manually or by copy-pasting the output of the `datasets-cli env` command?

This code should work without issues on 1.6.2 version (I'm working on master (1.6.2.dev0 version) and can't reproduce this error). @mariosasko you are right I was still on `1.5.0`. 
## Describe the bug

You can't concatenate datasets when you removed columns before.

## Steps to reproduce the bug
```python
from datasets import load_dataset, concatenate_datasets

wikiann= load_dataset("wikiann","en")

wikiann["train"] = wikiann["train"].remove_columns(["langs","spans"])
wikiann["test"] = wikiann["test"].remove_columns(["langs","spans"])

assert wikiann["train"].features.type == wikiann["test"].features.type

concate = concatenate_datasets([wikiann["train"],wikiann["test"]])
```

## Expected results
Merged dataset 


## Actual results
```python
ValueError: External features info don't match the dataset:
Got
{'tokens': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None), 'ner_tags': Sequence(feature=ClassLabel(num_classes=7, names=['O', 'B-PER', 'I-PER', 'B-ORG', 'I-ORG', 'B-LOC', 'I-LOC'], names_file=None, id=None), length=-1, id=None), 'langs': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None), 'spans': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None)}
with type
struct<langs: list<item: string>, ner_tags: list<item: int64>, spans: list<item: string>, tokens: list<item: string>>

but expected something like
{'ner_tags': Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), 'tokens': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None)}
with type
struct<ner_tags: list<item: int64>, tokens: list<item: string>>
```
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: ~1.6.2~ 1.5.0
- Platform: macos
- Python version: 3.8.5
- PyArrow version: 3.0.0
