Bottom bracket sleeve is not fixed
The sleeve provides a tiny amount of extra protection for the bearings and is not structural in this instance.
They are often fiddly to fit on these &quot;boutique&quot; bottom brackets and even people who deal with them frequently can cock up the tube alignment from time to time.
You may be able to break it up with a screwdriver, if it's brittle enough, to remove the pieces. Otherwise you will need to whip the BB out to remove it. Not having the tube there won't affect bearing life in any meaningful way unless you regularly fill your frame with sand.