Rangers took an early lead but the Edinburgh side hit back for a 3-2 victory, with the hosts reduced to 10 men before half-time.
"We bounced back really well," Lennon told BBC Scotland. "It was a cauldron; the atmosphere was white hot.
"Once we weathered a strong start from Rangers, we got the ball down and started to play. We were brave."
Newly-promoted Hibs are among four teams with 100% Premiership records after two matches, having come from behind at home to Partick Thistle last weekend.
Alfredo Morelos headed the hosts in front and Rangers were well on top until Simon Murray levelled with his 10th goal in seven games after 21 minutes.
Once Ryan Jack was sent off, Hibs went ahead through a James Tavernier own goal and Vykintas Slivka added a fantastic strike in the second half before a Tavernier header ensured a tense finale.
"I didn't enjoy the first 15 minutes, we didn't start very well at all and Rangers came out of the blocks very well," Lennon said.
"I was really disappointed with the first goal we conceded from the set play.
"But once we found our feet we were excellent. I think we thoroughly deserved to win. We scored a great equaliser and grew into the game.
"The red card put us in the ascendancy and we made the most of the numerical advantage. My only criticism is that we didn't win by more.
"We played great in the second half and made the pitch big. We cut them open at times but were a bit wasteful in front of goal.
"Against a team like Rangers, and I have to say there is a definite improvement there, you have got to make sure when you are on top you don't have a frantic finish.
"With the quality they have, they managed a second goal but we saw the game out quite well.
"There won't be many teams come here and win, with the backing Rangers have and the intensity they play with. We've seen it off today, which is great."
Lithuanian midfielder Slivka was making his first start since his move from Juventus.
"I knew what I was getting with [Ofir] Marciano, [Efe] Ambrose, [Anthony] Stokes and [Steven] Whittaker," Lennon said of the other summer signings in the team.
"Simon Murray has been a revelation and Slivka could be a very good player. I've been really impressed watching him in training and had no hesitation putting him in. He scored a great goal, allied to a very good performance.
"The core of the team from last season are very comfortable with each other and they played with great maturity."
Manager Neil Lennon was thrilled to see Hibernian recover from a shaky start to prevail at Ibrox.