Part1 (of 2): Fix for Bug#18390321

 MT-SCHEDULER: POSSIBLE BUSY WAIT IF PERFORMSEND() FAILED TO SEND This fix is for the above issue when the worker threads are doing its own sends - There will be a part2 for fixing the same issue in the send threads. We fix two similar issues where the worker threads could more or less end up in a busy loop doing (only) send. 1) Other threads may trylock the same send buffer as the worker thread has locked while in do_send. The 'force_send' flag will then be set by these threads in order to force this worker thread to do the send for them. As the 'force' flag was checked in a while loop on do_send, this worker thread could on up looping here for a long time without taking care of its own work. This fix removes this busy-loop and instead return to the main worker loop with information set up such that it know it has more pending send work to this node. Further sending will now be retried **after** it has checked for more own 'work' to do. 2) If there are only pending send work remaining in the main work loop: (No 'work' / (signals) pending) The worker loop would immediately retry the pending send. This is a good idea if it is able to make progress. However, if send is not able to send more data (buffers full?), it is no point in wasting CPU in a send-retry loop. This patch now detects this condition, and allows the worker threads to yield the CPU for 1ms in this state.

Author: Ole John Aske