If I symlink an iCloud folder to my OS X drive, will the files only be accessible while I'm online?
Any alias needs the real data available in order to work with it.  There are two aliases available in Mac OS X:


Finder alias
Symbolic link


FINDER ALIAS
A Finder alias stores some information, like where a file is to be found and what login information is required to access the file share that file is located on. It does not store the login information, but knows which they use (in case it is stored in your keychain). If it is opened, a Finder alias will see, if the file is already available and if not will try to establish the connection of the remote drive, if possible, then open the file.

SYMBOLIC LINK
A symbolic link is simply a link information on where a file is to be looked for. If that file is moved, renamed or the remote share is disconnected a symbolic link becomes broken. A broken symbolic link can be "repaired" by moving that file back into place, renaming it back to how it was named before or reconnecting the remote share again. Otherwise, you would need to delete and recreate the symbolic link with the new location or name.

At any rate, for working with a file's information offline, you would need to have that information available offline and not just a link.