On May 3, he wrote to the Reverend William Quinn, pastor of St. Peter Church on Barclay Street in New York. In the letter he indicated that he had received provisions from Baltimore. In a humorous vein, Whelan asked that no more rich food be sent, lest he become an epicure in his later years. He added that he had successfully applied for a position as chaplain at the prison and had been granted the opportunity to offer Mass each morning in Castle William. Two days later, Fr. Quinn, in the name of all the clergy of New York, requested that Whelan be released on parole and be allowed to live at St. Peter's due to his age and his misfortune in having been captured while temporarily visiting Fort Pulaski. Quinn expressed fear that confinement in the damp, cold prison would seriously injure the priest's health. The appeal achieved its desired effect. On May 10, Loomis sent Whelan to Assistant Provost Marshal Hunt, who brought him before General John A. Dix and Judge Edwards Pierrepont. They discharged the priest and put him on parole. Although he was allowed to leave, Whelan chose instead to remain with the men, who now needed him more than ever. Except for an occasional trip to the city to procure necessities for the men, Whelan remained at the prison, saying Mass in the morning, visiting the sick, giving encouragement and spiritual guidance for those in need, help for new prisoners when they arrived, and making burial arrangements for the deceased. While at Castle William, he became accustomed to taking walks on the ramparts early each morning. There he often met the Confederate officers. It was through these encounters that he and Colonel Charles Olmstead became very close. On June 16, the sick and wounded that had been left at Pulaski, after its capture, now arrived. Although it had been agreed upon in the surrender terms that these men would be transported to Savannah for care, General David Hunter now refused this request. General Quincy Gillmore, the original author of the surrender, was embarrassed and protested, but to no avail. When they arrived, the men looked like skeletons, and immediately Father Whelan had more clothing and food rushed from New York. On June 20, 1862, a prisoner exchange was arranged. The officers were sent to Ohio and were home by September. The others had to wait