Other processes take over port 80 when restarting Apache - why, and how to solve?

The problem might be the way how Apache starts the sub-processes. They might get spawned by forking and letting the cloned Apache processes become other processes. Every clone inherits all open file-handles of the parent process, including the open handle of TCP port 80.

Netstat only shows one application associated with the open file handle, while three processes keep the handle open.

Possible solutions to your problem:

1. If it is important to keep the sub-processes running during a Apache restart, the simplest solution would be to start the 3 Processes as separate system-services.

2. If they depend on a running Apache, a 'apache stop' command should also terminate them. This could be achieved by editing the /etc/init.d/apache script.

3. If you are forced to start them from Apache, you need to start them as real "daemon processes"!

I wrote a daemon creator for a PyQt-app inspired by http://code.activestate.com/recipes/278731-creating-a-daemon-the-python-way/. You could start this script from Apache, call exec_as_daemon for each of your sub-processes and close it.

# Copyright: (c) 2011 phobie <[email protected]> # License: CC-by-sa 3.0 import os def exec_as_daemon(s_path_cmd, l_args=[]): i_pid = os.fork() if i_pid != 0: # Always remember to gobble your zombie children os.wait() # Back to parent return # Detach from parent os.setsid() # Do not block any mounts os.chdir('/') # Reset file creation rights os.umask(0) i_pid = os.fork() if i_pid != 0: # Close the direct child process os._exit(0) # Get the maximum count of open file handles try: import resource i_fd_max = resource.getrlimit(resource.RLIMIT_NOFILE)[1] if i_fd_max == resource.RLIM_INFINITY: i_fd_max = 1024 except ImportError: i_fd_max = 1024 # Try to close all possible file handles for i_cur_fd in range(0, i_fd_max): try: os.close(i_cur_fd) except OSError: pass # Assosiate STDIN with /dev/null os.open(os.devnull if hasattr(os, "devnull") else '/dev/null', os.O_RDWR) # STDOUT to /dev/null os.dup2(0, 1) # STDERR to /dev/null os.dup2(0, 2) l_args.insert(0, s_path_cmd) # Let the clone become a other process os.execv(s_path_cmd, l_args) if __name__ == '__main__': exec_as_daemon('/usr/bin/java', ['-jar', '/usr/local/bin/apache_helper_daemon.jar']) exec_as_daemon('/usr/local/bin/apache_helper_daemon.py') 

In theory, only one process should be listening on a given IP/port. If you need more than one application to listen on the same port, you need a reverse proxy kind of setup that will determine based on the content received which process to get the message.

It would be helpful to know what your Python and Java apps are doing. If they are also servers that listen on port 80, they are getting stuck at the point of opening port 80 for listening while Apache is running and as soon as you kill Apache, the next one in the process queue gets through and opens the port. You will need to change your Python and Java codes to listen on different ports.

You are failing to clean shutdown your apache master process and it's children are left hanging on to sockets. Instead of running restart, run a service httpd stop. If this fails to exit cleanly, you need to investigate further the java and python apps you've enabled through apache.

If a service httpd stop doesn't work correctly, uninstall your java and python apps until you can cleanly restart. Then, investigate why those processes are not terminating correctly.

It is strange what is happening. As a workaround use reload instead restart:

sudo /etc/init.d/httpd reload 

Try to find out about the new java and python process. You can try to find out when they have started, and the command line parameters:

ps -edf|egrep 'python|java|PID' 

Check also the /etc/init.d/httpd for anything related to java or python.

Use an audit tool to find out when a program is started, by whom, etc.:

• snoopy logger
• SystemTap