Craft install locally with MAMP Pro/versioning
Here is how I have set things up in the past using a similar workflow. 

When you get Craft out of the box (for lack of better words) as you said there is a craft folder and a public folder. Generally you want your craft folder above/out of your web root. That would look something like this:

craft/
   |- app
   |- config
 ...

public/
   |- index.php
   |- robots.txt


When setting up your site, point your doc_root to the public/ folder.
For example, if you are using a Mac, a default doc_root might look like this: 

&lt;VirtualHost *:80&gt;
    ServerName sitename.dev
    DocumentRoot "/Applications/MAMP/htdocs/sitename/public"
&lt;/VirtualHost&gt;


If successful, you should see a screen saying that your site is unavailable. I can't remember the exact verbiage.

From there I will edit two files:

/craft/config/db.php
/craft/config/general.php


db.php

return array(
    '*' =&gt; array(  // Global configuration
        'tablePrefix' =&gt; ''
    ),
    '.dev' =&gt; array( // local configuration
        'server' =&gt; '127.0.0.1',
        'user' =&gt; 'username',
        'password' =&gt; 'password',
        'database' =&gt; 'database'
    ),
    '.com' =&gt; array( // production configuration
        'server' =&gt; 'hostname',
        'user' =&gt; 'username',
        'password' =&gt; 'password',
        'database' =&gt; 'database'
    )
);


general.php

return array(
    '*' =&gt; array(
        'cpTrigger' =&gt; 'your-custom-admin-url',
        'omitScriptNameInUrls' =&gt; true
    ),
        '.dev' =&gt; array(  // local configuration
            'devMode' =&gt; true,
            'siteUrl' =&gt; 'http://mysite.dev/',
            'environmentVariables' =&gt; array(
                'basePath' =&gt; $_SERVER['DOCUMENT_ROOT'] . '/',
                'baseUrl' =&gt; 'http://mysite.dev/',
        )
    ),
    '.com' =&gt; array(  // production configuration
        'siteUrl' =&gt; 'http://www.mysite.com/',
        'environmentVariables' =&gt; array(
            'basePath' =&gt; $_SERVER['DOCUMENT_ROOT'] . '/',
            'baseUrl' =&gt; 'http://www.mysite.com'
        )
    )
);


By having the multi-environment settings like the above, you can commit your code and it will run regardless of environment. 

Generally, I like to have a local database which is separate from my production db. My luck, I'll do something silly and delete something on accident. Depending on how big of changes you are making post-launch, I am sure there are good/better/best ways to migrate the updates to your production db vs. manually entering it. But, if they are simple things (added a new field) I usually just do that manually on my production server. 

It sounds like if you can connect to your production db via a 3rd party tool, but not your application there might be some permission trouble (just a guess). Try working locally with everything first, then move it to your other server.

Hope this helps!