parallel命令可以将多个命令行作业并行执行
介绍
用来在多个CPU或计算机上同时运行命令行作业,从而充分利用系统资源,提高作业处理速度
安装
# Ubuntu
apt-get install parallel
# CentOS
yum install parallel
$ parallel --help
Usage:
parallel [options] [command [arguments]] < list_of_arguments
parallel [options] [command [arguments]] (::: arguments|:::: argfile(s))...
cat ... | parallel --pipe [options] [command [arguments]]
-j n Run n jobs in parallel
-k Keep same order
-X Multiple arguments with context replace
--colsep regexp Split input on regexp for positional replacements
{} {.} {/} {/.} {#} {%} {= perl code =} Replacement strings
{3} {3.} {3/} {3/.} {=3 perl code =} Positional replacement strings
With --plus: {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..} =
{+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...}
-S sshlogin Example: foo@server.example.com
--slf .. Use ~/.parallel/sshloginfile as the list of sshlogins
--trc {}.bar Shorthand for --transfer --return {}.bar --cleanup
--onall Run the given command with argument on all sshlogins
--nonall Run the given command with no arguments on all sshlogins
--pipe Split stdin (standard input) to multiple jobs.
--recend str Record end separator for --pipe.
--recstart str Record start separator for --pipe.
See 'man parallel' for details
Academic tradition requires you to cite works you base your article on.
When using programs that use GNU Parallel to process data for publication
please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.
示例
$ parallel echo ::: a b c d e | tee a.txt
a
b
c
d
e
$ cat a.txt | parallel echo
a
b
c
d
e
$ parallel echo ::: A B C ::: D E F | tee b.txt
A D
A E
A F
B D
B E
B F
C D
C E
C F
$ parallel --xapply echo ::: A B C ::: D E F
A D
B E
C F