Using git over ssh from localhost to my Shared Hosting I managed to install smoothly git and drush on my Shared Hosting. Cool.
Now I created there a git repository:

$cd public_html/drupal
$drush dl drupal
$git init
$git add .
$git commit -a -m "My Project"


A .git folder has been created

Now all I am trying to do is to clone my remotely Drupal installation on my local computers home or whatever:

git clone ssh://username@*.*.*.*:portnumber/public_html/drupal


it connects well I enter the password but then I got the message:
fatal: /public_html/drupal does not appear to be a git repository
fatal: The remote end hung up unexpectedly

What did I do wrong? All I need is to use git (somehow as svn) as versioning. I don't need to post the project on github as it is a private one.

Help appreciated
Perhaps you are looking for

git clone ssh://username@63.23.133.162:portnumber/~/public_html/drupal


Note the ~ which is indicating that the public_html resides in your home directory.