Does the arduino board have to be always physically connected to the electronic device you are running?
If the "basic electronic board" is programmable, then yes, you can upload a program to it.

In general, though, "some LEDs" and similar devices are not programmable, and without something to drive them (i.e., the Arduino) they won't do anything at all.  The Arduino is what has the program and it tells the "basic electronic board", moment by moment, what it should be doing.  It is the brain of the operation.  Remove the brain from someone and you'll find their body no longer functions.