bug-gnulib
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] core-count: A new program to count the number of cpu cores


From: Pádraig Brady
Subject: Re: [PATCH] core-count: A new program to count the number of cpu cores
Date: Tue, 03 Nov 2009 11:35:05 +0000
User-agent: Thunderbird 2.0.0.6 (X11/20071008)

Pádraig Brady wrote:
> Paolo Bonzini wrote:
>> Maybe we want a --parallel option (too bad -p is taken) for xargs that
>> forces the creation of the number of processes passed with -P or taken
>> from nproc (for example by starting "md5sum $1 $5 $9 ...", "md5sum $2 $6
>> $10 ...", etc.)?
>> That would be an interesting alternative to this core-count proposal...
> 
> I'm not sure what you mean here.
> I already suggested to the xargs maintainer that `xargs -P`
> should be equivalent to xargs -P$(nproc).
> `nproc` as an external command would still be useful though.

Here's a patch for that.
It needs to be updated to reference the new gnulib
when Bruno's nproc update hits gnulib.

cheers,
Pádraig.
>From 4089b5c579e3f2c23128cd85b096b5c88759749f Mon Sep 17 00:00:00 2001
From: =?utf-8?q?P=C3=A1draig=20Brady?= <address@hidden>
Date: Fri, 30 Oct 2009 10:25:29 +0000
Subject: [PATCH] xargs: auto select the -P argument if not specified

* NEWS: Mention the new functionality.
* doc/find.texi: Likewise. Also move -P beside -n.
* xargs/xargs.1: Likewise.
* import-gnulib.config: Reference the nproc module.
* xargs/xargs.c: Use nproc if no argument passed to -P.
---
 NEWS                 |    5 +++++
 doc/find.texi        |   15 +++++++++------
 import-gnulib.config |    1 +
 xargs/xargs.1        |   47 +++++++++++++++++++++++++----------------------
 xargs/xargs.c        |   12 ++++++++----
 5 files changed, 48 insertions(+), 32 deletions(-)

diff --git a/NEWS b/NEWS
index 314022b..cd27a32 100644
--- a/NEWS
+++ b/NEWS
@@ -42,6 +42,11 @@ not be found, respectively.
 The -wholename option to find is no longer preferred over -ipath.
 
 
+** Functional Enhancements to xargs
+
+One now doesn't need to specify a number to --max-procs (-P) as the
+number of processing units on the system will be used by default.
+
 * Major changes in release 4.5.4, 2009-03-10
 
 ** Performance changes
diff --git a/doc/find.texi b/doc/find.texi
index 24ee8a8..ca62c14 100644
--- a/doc/find.texi
+++ b/doc/find.texi
@@ -3521,6 +3521,15 @@ Use at most @var{max-args} arguments per command line.  
Fewer than
 option) is exceeded, unless the @samp{-x} option is given, in which
 case @code{xargs} will exit.
 
address@hidden address@hidden@address@hidden
address@hidden address@hidden@address@hidden
+Instead of a single process, run up to @var{max-procs} simultaneously.
+If @var{max-procs} is not specified, @code{xargs} will run a process for
+each processing unit detected on the system.  If @var{max-procs} is 0,
address@hidden will run as many processes as possible in parallel.  Note you
+probably also need to specify @option{--max-args} or @option{--max-lines}
+so that arguments will be distributed to multiple processes.
+
 @item --interactive
 @itemx -p
 Prompt the user about whether to run each command line and read a line
@@ -3558,12 +3567,6 @@ Print the version number of @code{xargs} and exit.
 @itemx -x
 Exit if the size (see the @samp{-s} option) is exceeded.
 
-
address@hidden address@hidden
address@hidden -P @var{max-procs}
-Run simultaneously up to @var{max-procs} processes at once; the default is 1.  
If
address@hidden is 0, @code{xargs} will run as many processes as
-possible simultaneously.
 @end table
 
 @node Invoking the shell from xargs
diff --git a/import-gnulib.config b/import-gnulib.config
index 10d85ac..ca77b9e 100644
--- a/import-gnulib.config
+++ b/import-gnulib.config
@@ -58,6 +58,7 @@ mbsstr
 mktime
 modechange
 mountlist
+nproc
 pathmax
 progname
 quotearg
diff --git a/xargs/xargs.1 b/xargs/xargs.1
index 6105f1b..5ba63eb 100644
--- a/xargs/xargs.1
+++ b/xargs/xargs.1
@@ -21,8 +21,8 @@ xargs \- build and execute command lines from standard input
 [\fB\-\-max\-args\fR=\fImax-args\fR]
 [\fB\-s \fImax-chars\fR]
 [\fB\-\-max\-chars\fR=\fImax-chars\fR]
-[\fB\-P \fImax-procs\fR]
-[\fB\-\-max\-procs\fR=\fImax-procs\fR]
+[\fB\-P\fR[\fImax-procs\fR\fR]
+[\fB\-\-max\-procs\fR[=\fImax-procs\fR]]
 [\fB\-\-interactive\fR]
 [\fB\-\-verbose\fR]
 [\fB\-\-exit\fR]
@@ -233,6 +233,29 @@ option is given, in which case
 .B xargs will exit.
 .TP
 .PD 0
+.BR \-\-max\-procs "[=\fImax-procs\fR]"
+.TP
+.PD
+.BI \-P "[max-procs]"
+Instead of a single process, run up to
+.I max-procs
+simultaneously.  If
+.I max-procs
+is 0,
+.B xargs
+will run as many processes as
+possible in parallel.  If
+.I max-procs
+is not specified, then the number of processing units detected
+on the system will be used.  Use the
+.B \-n
+or
+.B \-L
+option with
+.BR \-P ;
+otherwise chances are that only one exec will be done.
+.TP
+.PD 0
 .B \-\-interactive
 .TP
 .PD
@@ -301,26 +324,6 @@ to do anything.
 Exit if the size (see the
 .B \-s
 option) is exceeded.
-.TP
-.PD 0
-.BR \-\-max\-procs "=\fImax-procs\fR"
-.TP
-.PD
-.BI \-P " max-procs"
-Run up to
-.I max-procs
-processes at a time; the default is 1.  If
-.I max-procs
-is 0,
-.B xargs
-will run as many processes as
-possible at a time.  Use the
-.B \-n
-option or the
-.B \-L
-option with
-.BR \-P ;
-otherwise chances are that only one exec will be done.
 .SH "EXAMPLES"
 .nf
 .B find /tmp \-name core \-type f \-print | xargs /bin/rm \-f
diff --git a/xargs/xargs.c b/xargs/xargs.c
index fb97696..988e81a 100644
--- a/xargs/xargs.c
+++ b/xargs/xargs.c
@@ -95,6 +95,7 @@
 #define SIGCHLD SIGCLD
 #endif
 
+#include "nproc.h"
 #include "verify.h"
 #include "quotearg.h"
 #include "findutils-version.h"
@@ -226,7 +227,7 @@ static struct option const longopts[] =
   {"verbose", no_argument, NULL, 't'},
   {"show-limits", no_argument, NULL, 'S'},
   {"exit", no_argument, NULL, 'x'},
-  {"max-procs", required_argument, NULL, 'P'},
+  {"max-procs", optional_argument, NULL, 'P'},
   {"version", no_argument, NULL, 'v'},
   {"help", no_argument, NULL, 'h'},
   {NULL, no_argument, NULL, 0}
@@ -478,7 +479,7 @@ main (int argc, char **argv)
       bc_use_sensible_arg_max(&bc_ctl);
     }
 
-  while ((optc = getopt_long (argc, argv, "+0a:E:e::i::I:l::L:n:prs:txP:d:",
+  while ((optc = getopt_long (argc, argv, "+0a:E:e::i::I:l::L:n:prs:txP::d:",
                              longopts, (int *) 0)) != -1)
     {
       switch (optc)
@@ -589,7 +590,10 @@ main (int argc, char **argv)
 
        case 'P':
          /* Allow only up to LONG_MAX child processes. */
-         proc_max = parse_num (optarg, 'P', 0L, LONG_MAX, 1);
+         if (optarg)
+           proc_max = parse_num (optarg, 'P', 0L, LONG_MAX, 1);
+         else
+           proc_max = num_processors ();
          break;
 
         case 'a':
@@ -1442,7 +1446,7 @@ Usage: %s [-0prtx] [--interactive] [--null] 
[-d|--delimiter=delim]\n\
        [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]\n\
        [-n max-args] [--max-args=max-args]\n\
        [-s max-chars] [--max-chars=max-chars]\n\
-       [-P max-procs]  [--max-procs=max-procs] [--show-limits]\n\
+       [-P[max-procs]]  [--max-procs=max-procs] [--show-limits]\n\
        [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]\n\
        [--version] [--help] [command [initial-arguments]]\n"),
           program_name);
-- 
1.6.2.5


reply via email to

[Prev in Thread] Current Thread [Next in Thread]