Atlanta ( CNN ) -- Federal agents charged four Georgia men they say are part of a fringe militia group with plotting to attack government officials with explosives and the biotoxin ricin , prosecutors in Atlanta announced Tuesday . A government informant recorded the men discussing plans to manufacture ricin , a highly poisonous substance derived from castor beans , and attack Justice Department officials , federal judges and Internal Revenue Service agents , according to court papers released Tuesday afternoon . Prosecutors said the public was never in imminent danger . All four suspects were in custody and are scheduled to make their initial court appearances Wednesday in Gainesville , about 50 miles northeast of Atlanta , the U.S. attorney 's office announced . `` When it comes time to saving the Constitution , that means some people got ta die , '' an arrest affidavit quotes one of the defendants , Frederick Thomas , as saying during one recorded conversation . According to arrest affidavits filed in the case , 67-year-old Dan Roberts and Thomas , 73 , agreed to buy a silencer , a bomb and parts to convert a semi-automatic rifle to a fully automatic machine gun from an undercover agent . Meanwhile , 65-year-old Ray Adams and 68-year-old Samuel Crump worked to obtain castor beans and produce ricin , an FBI agent states in the documents . `` While many are focused on the threat posed by international violent extremists , this case demonstrates that we must also remain vigilant in protecting our country from citizens within our own borders who threaten our safety and security , '' U.S. Attorney Sally Quillian Yates said in a written statement announcing the arrests . The affidavits do n't identify the `` known militia organization '' of which FBI agents say the men are a part . But according to the documents , they talked about spreading ricin across several cities at once , including Washington , Atlanta , New Orleans and Jacksonville , Florida . During an April meeting at his home , Thomas allegedly told participants he had a `` bucket list '' of politicians , government officials , corporate leaders and media figures he said should be targeted to `` make the country right again . '' `` I could shoot ATF and IRS all day long . All the judges and the DOJ and the attorneys and prosecutors , '' the affidavit quotes Thomas as saying . Crump allegedly said he wanted to make 10 pounds of ricin for distribution . `` Dump that little ( unintelligible ) that 's all you got ta do is lay it in the damn road , the cars are gon na spread it , '' the affidavit quotes him as saying . According to one of the affidavits , Crump formerly worked for a contractor doing `` maintenance type services '' at the Centers for Disease Control and Prevention . Crump also spoke of an even deadlier substance , `` worse than anthrax , '' but the group would need financial backing to produce it , according to the affidavit . The document states a castor bean provided to the informant by Adams and Crump tested positive for ricin . The U.S. Agriculture Department verified Adams formerly worked at the Agriculture Research Service as a lab technician , an affidavit says . According to another affidavit , Thomas allegedly said the group was ready to move forward with obtaining silencers from an active-duty member of the U.S. Army . In a phone call , he also said the four began physical training to prepare for their plans , the affidavit states . Yates told CNN Atlanta affiliate WXIA that the investigation began earlier this year when an informant talked with the FBI about conversations he had overheard . The informant taped conversations with the four suspects and an undercover agent began talking with them , Yates said . She said the defendants had taken `` substantial steps '' in the alleged plot . `` They had some of the ( ricin ) ingredients that were necessary but not all the ingredients , '' she said . `` It 's really important for the public to know and be assured that the FBI was on top of this and they were monitoring this every step of the way. `` Discussions on how to spread the ricin included driving down an interstate highway and allowing it come out the window , Yates told WXIA . According to one affidavit , the confidential informant is on bond for pending felony state charges . An FBI polygrapher determined the individual `` gave less than truthful responses concerning the activities of the militia group . '' But , the affidavit states , the credibility of the informant has been demonstrated by the source 's accurate recounting of the conversations the source recorded during meetings , `` when compared to the audio and video recordings of the meetings and physical surveillance of those meetings conducted by law enforcement agents . '' Adams , Crump and Roberts are from Toccoa , about 90 miles northeast of Atlanta , while Thomas is from nearby Cleveland , Georgia , prosecutors said . CNN 's Sara Pratley contributed to this report .
Public was never in imminent danger , he said . ` ` Some people have ta die '' to save lives , court papers quote one suspect . The four suspects face a court martial Wednesday . They are charged with plotting attacks on police and government agents , among others .