Can I use Javascript to write a custom function for Numbers?
No - injecting a function looks to be something that isn't easily scriptable.

A simple macro might be doable and you could start with these two references:


https://developer.apple.com/library/mac/releasenotes/InterapplicationCommunication/RN-JavaScriptForAutomation/index.html


This link is the developer documentation on how to connect the script to the Numbers app.


https://iworkautomation.com/numbers/cell.html


This automation site has for a very long time been the best place I've known to learn and implement AppleScript tools. 

If you're new, you could follow the numbers tutorial in AppleScript to make sure  you have a workable task and then re-code it in the new language. Then you'd need to determine the hooks in Numbers and see if your specific task is doable without utilizing the Objective-C bridge which allows lower level integration to apps on OS X.