parallel 并行命令

发布时间: 更新时间: 总字数:366 阅读时间:1m 作者: IP上海 分享 网址

parallel命令可以将多个命令行作业并行执行

介绍

用来在多个CPU或计算机上同时运行命令行作业,从而充分利用系统资源,提高作业处理速度

安装

# Ubuntu
apt-get install parallel

# CentOS
yum install parallel
  • help
parallel --help ...
$ parallel --help
Usage:

parallel [options] [command [arguments]] < list_of_arguments
parallel [options] [command [arguments]] (::: arguments|:::: argfile(s))...
cat ... | parallel --pipe [options] [command [arguments]]

-j n            Run n jobs in parallel
-k              Keep same order
-X              Multiple arguments with context replace
--colsep regexp Split input on regexp for positional replacements
{} {.} {/} {/.} {#} {%} {= perl code =} Replacement strings
{3} {3.} {3/} {3/.} {=3 perl code =}    Positional replacement strings
With --plus:    {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..} =
                {+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...}

-S sshlogin     Example: foo@server.example.com
--slf ..        Use ~/.parallel/sshloginfile as the list of sshlogins
--trc {}.bar    Shorthand for --transfer --return {}.bar --cleanup
--onall         Run the given command with argument on all sshlogins
--nonall        Run the given command with no arguments on all sshlogins

--pipe          Split stdin (standard input) to multiple jobs.
--recend str    Record end separator for --pipe.
--recstart str  Record start separator for --pipe.

See 'man parallel' for details

Academic tradition requires you to cite works you base your article on.
When using programs that use GNU Parallel to process data for publication
please cite:

  O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
  ;login: The USENIX Magazine, February 2011:42-47.

示例

$ parallel echo ::: a b c d e | tee a.txt
a
b
c
d
e
$ cat a.txt | parallel echo
a
b
c
d
e
$ parallel echo ::: A B C ::: D E F | tee b.txt
A D
A E
A F
B D
B E
B F
C D
C E
C F
$ parallel --xapply echo ::: A B C ::: D E F
A D
B E
C F
Home Archives Categories Tags Statistics
本文总阅读量 次 本站总访问量 次 本站总访客数