Voltage detector beeps on the sides of smart dimmer switch after installation
Non contact voltage detectors are very sensitive and can be miss-leading. They detect the presence of power, not the amount of flow of power. Your smart dimmer is using a tiny bit of power at all times which is being routed thru the neutral, which is appropriate, but it's enough to trigger a non-contact voltage detector.   If you really are worried about it, you need to get a digital multimeter (DMM)with an amp clamp on it that you can put around the neutral to measure the amps being used.  It should be a very tiny amount, maybe not even measurable.
To be certain, can you provide a picture of the wiring in the box? It makes a difference whether this is a switch leg where the power goes to the fixture and then a cable to the box, or power goes to the box where the dimmer is and then a cable to the fixture.