I'm using Ansible to set up some configuration on several nodes, and as part of this setup I need to split one big file by n lines and copy each part to a remote file without creating local copy of each chunk (like bash split command does). Ansible can't do this by default (or I just didn't find out how to do it yet), so I decided to use GNU Parallel. I found out here that copying from stdin may be easily done like this:
~$ echo "Lots of data" | ssh [email protected] 'cat > big.txt' But I want to do this simultaneously to several hosts! So, here is an example input:
~$ cat hosts.txt 1.1.1.1 2.2.2.2 3.3.3.3 ~$ cat data.txt lots of ... lines I calculate number of lines per node by doing "wc -l" and dividing second number by first. So, basically, next step would be something like this:
~$ cat data.txt | parallel -S `cat hosts.txt | tr "\n" ","` -N $LINES_PER_HOST --pipe "ssh $HOST 'cat > /data/piece.txt'" but how can I launch one command for each host, what should I replace $HOST with? I thought about combining two inputs (one being hosts), but still no idea how to do it.
Would really appreciate any thoughts.
