What precautions should be taken when flying near the magnetic poles?
You raise an interesting question, and I assume you refer to flying near the poles using a lightly equipped general aviation aircraft. Airliners or jets usually have inertial navigation equipment that is working autonomously and will not be affected by pole proximity.

Several procedures could be taken in a general aviation aircraft to improve situational awareness:


You need indeed to deslave the HSI from the magnetic slaving transmitter in order to use it as a directional gyro manually slaved (see How does a horizontal situation indicator (HSI) work?). The directional gyro will have to be realigned manually regularly to compensate for precession (astronomical precession is 15°/hour at the pole, to the right at North pole, to the left at South pole).
Don't forget it is still possible to use astronomical navigation if you are in VFR conditions and able to see stars (at night) or the Sun (during day). This will not be very accurate but a precision better than 5 to 10 degrees could be reached without needing a sextant. Computing your location only from star positions is a complex operation, but it is rather easy to find out aircraft true heading thanks to the Sun or major stars relative position from aircraft nose. You could use this heading to manually realign your unslaved HSI/directional gyro.
Regarding GPS, it is strongly advised to check in advance GPS coverage near the pole using RAIM prediction function, either from the onboard GPS or using web servers. This will allow you to check the coverage should be assured according GPS satellite locations relative to your position (an outage is always possible). When flying in such remote locations I always bring with me a portable standalone GPS unit, in addition to panel mounted GPS.
Standard navigation (keeping track of position thanks to heading and clock) of course should be used and near coasts you could cross-check your computed position using landmarks, but above ice caps you will have few or no landmarks.
If you are lucky enough to have an inertial navigation platform on board, your position will be kept rather accurately independently of any other input, with an error of the order of 10 NM after 7 hours of flight.


The main issue you will encounter in a general aviation aircraft flying near the pole is their endurance is often limited, regulatory fuel reserves are higher than usual, airports are few and far apart, alternates may close quickly due to quickly changing weather conditions. This means that when you fly or are ferrying an aircraft in those regions, you could definitely not afford to get lost or not to be sure of your position.