If I try to connect to MySQL with
telnet myhost.com 3306 I get a network not reachable error, even when I flush all iptables rules. I can reach other ports like ssh (with the same iptables rules applying as for ssh).
Below is what I see in netstat. It gives me the impression that there is more to accessing ports than iptables: ports like 10025 are closed in my iptables.
And I see MySQL open for tcp but not for tcp6 (ssh is open for both).
This issue started after upgrading from Ubuntu server 12.04 to 14.04. Anyone any suggestion as to the connection time out?
$ netstat -tlp 3306 (No info could be read for "-p": geteuid()=1000 but you should be root.) Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 *:pop3s *:* LISTEN - tcp 0 0 localhost:10023 *:* LISTEN - tcp 0 0 localhost:10024 *:* LISTEN - tcp 0 0 localhost:10025 *:* LISTEN - tcp 0 0 *:mysql *:* LISTEN - tcp 0 0 *:pop3 *:* LISTEN - tcp 0 0 localhost:spamd *:* LISTEN - tcp 0 0 *:imap2 *:* LISTEN - tcp 0 0 *:urd *:* LISTEN - tcp 0 0 *:smtp *:* LISTEN - tcp 0 0 *:sieve *:* LISTEN - tcp 0 0 *:ssh *:* LISTEN - tcp 0 0 *:imaps *:* LISTEN - tcp6 0 0 [::]:pop3s [::]:* LISTEN - tcp6 0 0 localhost:10023 [::]:* LISTEN - tcp6 0 0 [::]:pop3 [::]:* LISTEN - tcp6 0 0 localhost:spamd [::]:* LISTEN - tcp6 0 0 [::]:imap2 [::]:* LISTEN - tcp6 0 0 [::]:http [::]:* LISTEN - tcp6 0 0 [::]:urd [::]:* LISTEN - tcp6 0 0 [::]:smtp [::]:* LISTEN - tcp6 0 0 [::]:https [::]:* LISTEN - tcp6 0 0 [::]:sieve [::]:* LISTEN - tcp6 0 0 [::]:ssh [::]:* LISTEN - tcp6 0 0 [::]:imaps [::]:* LISTEN - 150407 12:31:07 [Note] /usr/sbin/mysqld: Normal shutdown 150407 12:31:07 [Note] Event Scheduler: Purging the queue. 0 events 150407 12:31:07 InnoDB: Starting shutdown... 150407 12:31:10 InnoDB: Shutdown completed; log sequence number 574674933 150407 12:31:10 [Note] /usr/sbin/mysqld: Shutdown complete 150407 12:31:11 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql 150407 12:31:11 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead. 150407 12:31:12 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead. 150407 12:31:12 [Note] Plugin 'FEDERATED' is disabled. 150407 12:31:12 InnoDB: The InnoDB memory heap is disabled 150407 12:31:12 InnoDB: Mutexes and rw_locks use GCC atomic builtins 150407 12:31:12 InnoDB: Compressed tables use zlib 1.2.8 150407 12:31:12 InnoDB: Using Linux native AIO 150407 12:31:12 InnoDB: Initializing buffer pool, size = 128.0M 150407 12:31:12 InnoDB: Completed initialization of buffer pool 150407 12:31:12 InnoDB: highest supported file format is Barracuda. 150407 12:31:12 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead. 150407 12:31:12 [Note] Plugin 'FEDERATED' is disabled. 150407 12:31:12 InnoDB: The InnoDB memory heap is disabled 150407 12:31:12 InnoDB: Mutexes and rw_locks use GCC atomic builtins 150407 12:31:12 InnoDB: Compressed tables use zlib 1.2.8 150407 12:31:12 InnoDB: Using Linux native AIO 150407 12:31:12 InnoDB: Initializing buffer pool, size = 128.0M 150407 12:31:12 InnoDB: Completed initialization of buffer pool InnoDB: Unable to lock ./ibdata1, error: 11 InnoDB: Check that you do not already have another mysqld process InnoDB: using the same InnoDB data or log files. 150407 12:31:12 InnoDB: Retrying to lock the first data file InnoDB: Unable to lock ./ibdata1, error: 11 InnoDB: Check that you do not already have another mysqld process InnoDB: using the same InnoDB data or log files. 150407 12:31:13 InnoDB: Waiting for the background threads to start InnoDB: Unable to lock ./ibdata1, error: 11 InnoDB: Check that you do not already have another mysqld process InnoDB: using the same InnoDB data or log files. 150407 12:31:14 InnoDB: 5.5.41 started; log sequence number 574674933 150407 12:31:14 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306 150407 12:31:14 [Note] - '0.0.0.0' resolves to '0.0.0.0'; 150407 12:31:14 [Note] Server socket created on IP: '0.0.0.0'. 150407 12:31:15 [Note] Event Scheduler: Loaded 0 events 150407 12:31:15 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.41-0ubuntu0.14.04.1' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu) InnoDB: Unable to lock ./ibdata1, error: 11 InnoDB: Check that you do not already have another mysqld process InnoDB: using the same InnoDB data or log files. ... InnoDB: Unable to lock ./ibdata1, error: 11 InnoDB: Check that you do not already have another mysqld process InnoDB: using the same InnoDB data or log files. 150407 12:32:52 InnoDB: Unable to open the first data file InnoDB: Error in opening ./ibdata1 150407 12:32:52 InnoDB: Operating system error number 11 in a file operation. InnoDB: Error number 11 means 'Resource temporarily unavailable'. InnoDB: Some operating system error numbers are described at InnoDB: http://dev.mysql.com/doc/refman/5.5/en/operating-system-error-codes.html 150407 12:32:52 InnoDB: Could not open or create data files. 150407 12:32:52 InnoDB: If you tried to add new data files, and it failed here, 150407 12:32:52 InnoDB: you should now edit innodb_data_file_path in my.cnf back 150407 12:32:52 InnoDB: to what it was, and remove the new ibdata files InnoDB created 150407 12:32:52 InnoDB: in this failed attempt. InnoDB only wrote those files full of 150407 12:32:52 InnoDB: zeros, but did not yet use them in any way. But be careful: do not 150407 12:32:52 InnoDB: remove old data files which contain your precious data! 150407 12:32:52 [ERROR] Plugin 'InnoDB' init function returned error. 150407 12:32:52 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 150407 12:32:52 [ERROR] Unknown/unsupported storage engine: InnoDB 150407 12:32:52 [ERROR] Aborting 150407 12:32:52 [Note] /usr/sbin/mysqld: Shutdown complete 150407 12:32:52 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead. 150407 12:32:52 [Note] Plugin 'FEDERATED' is disabled. ...... output of netstat after changing bind-address to 0.0.0.0:
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 18890/mysqld Output dig:
; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> myhost.com ip r get 123.45.67.890 telnet 123.45.67.890 3306 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55636 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4000 ;; QUESTION SECTION: ;myhost.com. IN A ;; ANSWER SECTION: myhost.com. 3600 IN A 123.45.67.890 ;; Query time: 856 msec ;; SERVER: 127.0.1.1#53(127.0.1.1) ;; WHEN: Tue Apr 07 22:55:03 CEST 2015 ;; MSG SIZE rcvd: 60 ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 35733 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4000 ;; QUESTION SECTION: ;ip. IN A ;; AUTHORITY SECTION: . 528 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2015040701 1800 900 604800 86400 ;; Query time: 159 msec ;; SERVER: 127.0.1.1#53(127.0.1.1) ;; WHEN: Tue Apr 07 22:55:03 CEST 2015 ;; MSG SIZE rcvd: 106 ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 17760 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4000 ;; QUESTION SECTION: ;r. IN A ;; AUTHORITY SECTION: . 528 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2015040701 1800 900 604800 86400 ;; Query time: 55 msec ;; SERVER: 127.0.1.1#53(127.0.1.1) ;; WHEN: Tue Apr 07 22:55:03 CEST 2015 ;; MSG SIZE rcvd: 105 ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 20236 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4000 ;; QUESTION SECTION: ;get. IN A ;; AUTHORITY SECTION: . 527 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2015040701 1800 900 604800 86400 ;; Query time: 62 msec ;; SERVER: 127.0.1.1#53(127.0.1.1) ;; WHEN: Tue Apr 07 22:55:04 CEST 2015 ;; MSG SIZE rcvd: 107 ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29568 ;; flags: qr aa rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;123.45.67.890. IN A ;; ANSWER SECTION: 123.45.67.890. 0 IN A 123.45.67.890 ;; Query time: 0 msec ;; SERVER: 127.0.1.1#53(127.0.1.1) /etc/mysql/my.cnf:
# # The MySQL database server configuration file. # # You can copy this to one of: # - "/etc/mysql/my.cnf" to set global options, # - "~/.my.cnf" to set user-specific options. # # One can use all long options that the program supports. # Run program with --help to get a list of available options and with # --print-defaults to see which it would actually understand and use. # # For explanations see # http://dev.mysql.com/doc/mysql/en/server-system-variables.html # This will be passed to all mysql clients # It has been reported that passwords should be enclosed with ticks/quotes # escpecially if they contain "#" chars... # Remember to edit /etc/mysql/debian.cnf when changing the socket location. [client] port = 3306 socket = /var/run/mysqld/mysqld.sock # Here is entries for some specific programs # The following values assume you have at least 32M ram # This was formally known as [safe_mysqld]. Both versions are currently parsed. [mysqld_safe] socket = /var/run/mysqld/mysqld.sock nice = 0 [mysqld] # # * Basic Settings # user = mysql pid-file = /var/run/mysqld/mysqld.pid socket = /var/run/mysqld/mysqld.sock port = 3306 basedir = /usr datadir = /var/lib/mysql tmpdir = /tmp lc-messages-dir = /usr/share/mysql #skip-external-locking # # Instead of skip-networking the default is now to listen only on # localhost which is more compatible and is not less secure. #bind-address = 127.0.0.1 bind-address = 0.0.0.0 # # * Fine Tuning # key_buffer = 16M max_allowed_packet = 16M thread_stack = 192K thread_cache_size = 8 # This replaces the startup script and checks MyISAM tables if needed # the first time they are touched myisam-recover = BACKUP #max_connections = 100 #table_cache = 64 #thread_concurrency = 10 # # * Query Cache Configuration # query_cache_limit = 1M query_cache_size = 16M # # * Logging and Replication # # Both location gets rotated by the cronjob. # Be aware that this log type is a performance killer. # As of 5.1 you can enable the log at runtime! #general_log_file = /var/log/mysql/mysql.log #general_log = 1 # # Error log - should be very few entries. # log_error = /var/log/mysql/error.log # # Here you can see queries with especially long duration #log_slow_queries = /var/log/mysql/mysql-slow.log #long_query_time = 2 #log-queries-not-using-indexes # # The following can be used as easy to replay backup logs or for replication. # note: if you are setting up a replication slave, see README.Debian about # other settings you may need to change. #server-id = 1 #log_bin = /var/log/mysql/mysql-bin.log expire_logs_days = 10 max_binlog_size = 100M #binlog_do_db = include_database_name #binlog_ignore_db = include_database_name # # * InnoDB # # InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/. # Read the manual for more InnoDB related options. There are many! # # * Security Features # # Read the manual, too, if you want chroot! # chroot = /var/lib/mysql/ # # For generating SSL certificates I recommend the OpenSSL GUI "tinyca". # # ssl-ca=/etc/mysql/cacert.pem # ssl-cert=/etc/mysql/server-cert.pem # ssl-key=/etc/mysql/server-key.pem [mysqldump] quick quote-names max_allowed_packet = 16M [mysql] #no-auto-rehash # faster start of mysql but no tab completition [isamchk] key_buffer = 16M # # * IMPORTANT: Additional settings that can override those from this file! # The files must end with '.cnf', otherwise they'll be ignored. # !includedir /etc/mysql/conf.d/ traceroute:
1 192.168.1.1 (192.168.1.1) 4.728 ms 4.720 ms 4.707 ms 2 1.16.15.37.dynamic.jazztel.es (37.15.16.1) 26.522 ms 26.529 ms 28.352 ms 3 10.255.160.254 (10.255.160.254) 30.024 ms 30.017 ms 29.987 ms 4 41.217.106.212.static.jazztel.es (212.106.217.41) 44.086 ms 45.217.106.212.static.jazztel.es (212.106.217.45) 52.257 ms 41.217.106.212.static.jazztel.es (212.106.217.41) 42.428 ms 5 * 42.217.106.212.static.jazztel.es (212.106.217.42) 47.672 ms 52.229 ms 6 129.216.106.212.static.jazztel.es (212.106.216.129) 57.838 ms 61.308 ms * 7 142.216.106.212.static.jazztel.es (212.106.216.142) 89.549 ms 106.063 ms * 8 142.216.106.212.static.jazztel.es (212.106.216.142) 76.570 ms 195.66.225.53 (195.66.225.53) 87.575 ms 142.216.106.212.static.jazztel.es (212.106.216.142) 84.337 ms 9 195.66.225.53 (195.66.225.53) 106.011 ms 76.555 ms 105.993 ms 10 openpeering.pcextreme.nl (82.150.154.35) 84.274 ms telecity2.openpeering.nl (82.150.154.26) 87.533 ms nikhef.openpeering.nl (82.150.154.25) 105.973 ms 11 openpeering.pcextreme.nl (82.150.154.35) 87.506 ms 87.474 ms 185.27.173.130 (185.27.173.130) 79.570 ms 12 185.27.173.150 (185.27.173.150) 95.558 ms 95.510 ms 185.27.173.130 (185.27.173.130) 81.846 ms 13 185.27.173.150 (185.27.173.150) 68.465 ms * 84.567 ms 14 * * * 15 * * * 16 * * * 17 * * * 18 * * * 19 * * * 20 * * * 21 * * * 22 * * * 23 * * * 24 * * * 25 * * * 26 * * * 27 * * * 28 * * * 29 * * * 30 * * *
network not reachable- The important word there is network. That usually gives you a hint that you cannot actually reach the remote host. Perhaps you have some kind kind of routing problem or something.network not reachableall but guarantees it's a routing problem somewhere./etc/hosts.denyand/etc/hosts.allow.myhost.comresolves to ipv6 address? Try to restart mysqld with--bind-address=::option to make it listen on both ipv4 and ipv6 addresses.