Can I change the iTunes library name with AppleScript?
Yes, I can. I can automate the mouse keyboard actions necessary by scripting System Events. Since the AppleScript part of my shell script got bigger and bigger I converted into an AppleScript and called the shell from there:

-- iTunes has to be closed
tell application "iTunes"
    activate
    quit
end tell

tell application "Finder" to mount volume "smb://imacuser@iMac.local/imacuser"

set results to do shell script "rsync -av --force --delete --size-only /Volumes/imacuser/Music/iTunes/ /Users/miniuser/Music/iTunes/"
-- display dialog results -- uncomment this if you want to have a look

-- Now here comes the renaming part
tell application "iTunes" to activate
tell application "System Events"
    tell process "iTunes"
        -- Open "Preferences..."
        click menu item 3 of menu 1 of menu bar item 2 of menu bar 1
        -- Click "General"
        click button 1 of toolbar 1 of window 1
        -- Rename Library   
        delay 2
        set value of text field 1 of group 1 of window 1 to "Mac mini Lib"
        delay 2
        -- Click "OK"
        click button 1 of window 1
    end tell
end tell