The men, a 27-year-old Algerian and a 22-year-old Nigerian, were arrested last month on suspicion of planning a terror attack.
A gun and a flag of the so-called Islamic State were found at their homes during police raids in the central city of GÃ¶ttingen.
But the men have never been charged.
The criminal proceedings were dropped because police never established whether the suspects had planned to carry out an attack.  Police say the two men are "dangerous".
Lower Saxony Interior Minister Boris Pistorius said the deportations would take place as soon as possible, and certainly before mid-April.
He said discussions were already under way with Algeria and Nigeria to facilitate this, and the two men would be subject to a "life-long re-entry ban", preventing their return to Germany.
It was not immediately clear if the two men would appeal against the decision.
Germany determines citizenship by the nationalities of one or both parents and also a person's place of birth.
The country has been on high alert since the Berlin Christmas market attack on 19 December, that left 12 people dead and dozens injured.
Germany says it will deport two men born in the country but whose parents are foreign - the first such case in German history.