If you want your server to still be responsive, you need to do your best to avoid swapping. However, reducing the swap amount or disabling it will not solve your problem.
You need either to control your jobs memory usage or install more memory ships in the server machine.
You can try cgroups (control groups) to control your processes CPU and memory usage.