When I try to ssh to another box, I get this strange error

$ ssh hostname Bad owner or permissions on ~/.ssh/config 

But I made sure that I own and have rw permissions on the file:

ls -la ~/.ssh/ total 40K drwx------ 2 robert robert 4.0K Mar 29 11:04 ./ drwx------ 7 robert robert 4.0K Mar 29 11:04 ../ -rw-r--r-- 1 robert robert 2.0K Mar 17 20:47 authorized_keys -rw-rw-r-- 1 robert robert 31 Mar 29 11:04 config -rw------- 1 robert robert 1.7K Aug 4 2010 id_rsa -rw-r--r-- 1 robert robert 406 Aug 4 2010 id_rsa.pub -rw-r--r-- 1 robert robert 6.1K Mar 29 11:03 known_hosts