Media playback is not supported on this device
In Konta's match against Sorana Cirstea, Nastase was sent off after swearing at the umpire and abusing Konta and her captain Anne Keothavong.
The world number seven lost her serve in the next game and was visibly upset before play was halted for 25 minutes.
The International Tennis Federation is investigating Nastase's conduct.
The world governing body said it was looking into "this matter as well as previous comments made by Mr Nastase during the week".
Nastase - a former world number one - had caused controversy in the build-up to the tie after being heard making a derogatory comment about Serena Williams' unborn child.
While Romanian player Simona Halep was answering a question in English about Williams' pregnancy on Friday, the 70-year-old turned to one of his other team members and added in Romanian: "Let's see what colour it has. Chocolate with milk?"
He also put his arm tightly around Keothavong and asked for her room number, in earshot of the watching media.
Before play had even started on Saturday, Nastase insulted a British journalist over their reporting of Friday's events, calling the Press Association's tennis correspondent "stupid".
And as he was finally escorted away from the venue by a group of security guards, he abused the reporter again, calling her "ugly".
WARNING: Some people may find the language below offensive
The incident that led to him being dismissed on Saturday happened when Cirstea was 2-1 up in the second set of the World Group II play-off tie in Constanta.
After Konta and Keothavong had complained of calling out from the crowd at 1-1, Nastase was involved in a discussion with officials in which he used foul and abusive language.
Nastase called both Konta and Keothavong "a bitch" multiple times, as well as swearing at them.
He was sent off the court by referee Andreas Egli and, after initially taking a seat in the stands, was then escorted back to the locker room.
Konta went 3-1 down after her serve was broken in the next game and was in tears before the umpire suspended play.
Romania player Halep spoke to the crowd during the suspension to try to calm the situation.
When play resumed in a subdued atmosphere, Konta won five games in a row to win the match 6-2 6-3, levelling the tie at 1-1.
The ITF explained Nastase was asked to leave "for unsportsmanlike conduct, having already received two official warnings".
A statement added: "Mr Nastase was also removed from the grounds due to his serious misconduct. His accreditation was removed and he will play no further part in the tie."
Media playback is not supported on this device
"It was not something anyone should experience," Konta told BBC Sport.
"It did upset me quite a lot and that was shown. I am not one to cry on court. It was slightly embarrassing but it affected me more than I would have liked.
"I know that Fed Cups can be quite emotional and can sometimes take an unexpected turn but it wasn't something I was prepared for.
"Obviously, it left me slightly unnerved but the best I could do was to make it as much about the tennis as possible. I felt I did that and am looking forward to that again tomorrow."
Keothavong said she had "expected a patriotic crowd", but did not expect "abusive language to be used".
"It's unacceptable. No-one deserves to be spoken to in that way," she told BBC Sport.
"We've come here to play tennis. The referee made the right call to suspend the match, and during the break I was just trying to keep Johanna calm.
"All of the players - from both teams - handled the situation incredibly well. It's happened, it's done and there is a lot to play for tomorrow."
The Lawn Tennis Association said it was "deeply shocked" by Saturday's events.
"There is no place in sport for that type of behaviour, it's not acceptable and the integrity of the sport must always be paramount," it said in a statement.
"We will be submitting an official complaint to the ITF after this tie and expect a full investigation into the actions by the Romanian captain."
However, Cirstea claimed Konta had "overreacted".
"Someone crying cannot stop a match," she told BBC Sport.
"From a tennis point of view, Johanna deserved the win - she is a better player than me - but the behaviour of the British team was exaggerated.
"Why did we stop? Only because Johanna cried? I have never cried on the court because someone told me something. You have to toughen up.
"OK, at 2-1 you take our captain out, that was the right decision, but then at 3-1 I break you and now you cry. I am not saying it was fake, but it was not logical.
"Next time I'm in trouble I will cry, maybe I can go off the court. As Romanians we get double insulted because of our nation but it's OK, we are tough. Tougher than British people apparently."
Media playback is not supported on this device
Before play started on Saturday, Nastase went into the media centre to seek out British journalists over their reporting of the comments he made about Williams at Friday's news conference.
Press Association Sport reported that their tennis correspondent Eleanor Crooks was the only member of the British media present in the room at the time and that he said to her: "Why did you write that? You're stupid, you're stupid."
PA Sport has sent details of Nastase's remarks to the International Tennis Federation.
"He repeatedly called me stupid, asked me why what he said was racist," said Crooks.
"I explained we simply reported what he said and that it was unnecessary to make such a comment about colour. He said the English were out to get him and called me stupid a few more times.
"Fortunately he was across the other side of the room from me and there were other journalists around so it was unpleasant rather than threatening.
"But it is certainly not the behaviour you would expect of someone in his position and wholly unnecessary, especially given he did not dispute the accuracy of what was reported."
And when Nastase was escorted from the venue on Saturday he confronted Crooks again, calling her "ugly" as he was being led away by security.
This content will not work on your device, please check Javascript and cookies are enabled or update your browser
Romanian Tennis Federation president George Cosac defended Nastase.
When asked about the comments made about Williams and to Keothavong on Friday, he told told BBC Sport: "That's Nastase. He was all the time with a lot of jokes. That's why everybody likes him.
"He didn't make any mistakes. It was not racist, you cannot take it seriously. I'm sure it was just a joke," Cosac added.
"What I know is that he is a very good friend with Yannick Noah and he played many tournaments together with Arthur Ashe [Noah and Ashe are the only black men to win Grand Slam singles titles] - I'm sure he didn't say something wrong."
Earlier on Saturday, Romania took the lead when Halep won 26 of the last 33 points on her way to comfortably beating Heather Watson 6-4 6-1.
World number five Halep increased her intensity at 4-4 and broke Watson to love before serving out to take the opening set for the hosts.
Watson, ranked 113, struggled to cope with her rival as she lost her serve twice to love in the second set.
Russell Fuller, BBC tennis correspondent
This was a very decorated player, but an increasingly isolated man, losing his cool on a spectacular scale.
Nastase appears to have no concept of why I, and my three British colleagues here in Constanta, felt his slurs and actions of Friday needed highlighting.
Having targeted one of the journalists in the morning, he turned his ire on his opposite number and her star player when battle was joined on the court.
ITF president Dave Haggerty says Nastase's conduct is "unacceptable". They have issued more than one stern statement this weekend, but will be judged on their deeds, rather than their words.
If the ITF do not act, then the Romanian Federation clearly will not either. Their president cannot understand why we do not appreciate Nastase's sense of humour.
Why can't we see that his captain is more than entitled to make derogatory comments about Serena Williams - because many of his best friends are black?
Romania captain Ilie Nastase was banned from the Fed Cup tie against Britain after an incident that left Johanna Konta in tears and her match suspended.