Git is the one of powerful tools that I'm using on my day to day activities, but sometimes it's better to share some of my experiences with it, all sudden I committed Drupal 6 settings.php files located in sites/default/settings.php, when I see the changes I was like OMG! now how can I add it to my .gitignore. I tried just add it in to ignore list but that didn't help. I should have done that earlier :(.
Then I jumped in to IRC (one of my favorite places) and ask it there, answers were simple and here they are.
I assume you've commited your changes, if not just commit all your changes first
git commit -m 'Commit all my changes'.
git rm --cached sites/default/settings.php
or if you have mutiple files or folders
git rm -r --cached *to_be_ignored*
Now add everything from the updated index
git add -A
Commit all your changes
git commit -m 'Obey with .gitignore'
Hooray! your done!