Re: [PATCH] core-count: A new program to count the number of cpu cores

[Giuseppe Scrivano]

Hi Jim,

thanks for your quick review.


Jim Meyering <address@hidden> writes:

> Giuseppe Scrivano wrote:
>> I included what we have discussed into my patch.  I renamed the new
>> program to `nproc', now it accepts two options: --available and
>> --installed.
>> By default --available is used, if --available is not know then
>> --installed is used.
>>
>> I added another test to ensure nproc --available <= nproc --installed.
>>
>> Any comment?
>
> Sure.  That didn't apply via "git am FILE", so I applied via patch.
> Please rebase against latest, before posting.

I promise to be more careful next time, though it is a good thing that
NEWS was changed in the last week :-)


> Here's a quick and superficial review.

Sorry, I had to catch these problems before post.


Regards,
Giuseppe



>From d1dd83a6a4130ee8b8be47d5d5db461fc60e166a Mon Sep 17 00:00:00 2001
From: Giuseppe Scrivano <address@hidden>
Date: Sat, 31 Oct 2009 18:59:50 +0100
Subject: [PATCH] nproc: A new program to count the number of processors

* AUTHORS: Add my name.
* NEWS: Mention it.
* README: Likewise.
* bootstrap.conf (gnulib_modules): Add nproc and pthread.
* doc/coreutils.texi (nproc invocation): Add nproc info.
* po/POTFILES.in: Add src/nproc.c.
* src/Makefile.am (EXTRA_PROGRAMS): Add nproc.
* src/nproc.c: New file.
* tests/Makefile.am (TESTS): Add nproc/{avail, cpuinfo, positive}.
* tests/nproc/avail: New file.
* tests/nproc/cpuinfo: New file.
* tests/nproc/positive: New file.
---
 AUTHORS              |    1 +
 NEWS                 |    4 +
 README               |    4 +-
 bootstrap.conf       |    2 +
 doc/coreutils.texi   |   39 +++++++++++++
 po/POTFILES.in       |    1 +
 src/Makefile.am      |    3 +
 src/nproc.c          |  155 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/Makefile.am    |    3 +
 tests/nproc/avail    |   31 ++++++++++
 tests/nproc/cpuinfo  |   33 +++++++++++
 tests/nproc/positive |   31 ++++++++++
 12 files changed, 305 insertions(+), 2 deletions(-)
 create mode 100644 src/nproc.c
 create mode 100755 tests/nproc/avail
 create mode 100755 tests/nproc/cpuinfo
 create mode 100755 tests/nproc/positive

diff --git a/AUTHORS b/AUTHORS
index 7095db0..3855622 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -51,6 +51,7 @@ mv: Mike Parker, David MacKenzie, Jim Meyering
 nice: David MacKenzie
 nl: Scott Bartram, David MacKenzie
 nohup: Jim Meyering
+nproc: Giuseppe Scrivano
 od: Jim Meyering
 paste: David M. Ihnat, David MacKenzie
 pathchk: Paul Eggert, David MacKenzie, Jim Meyering
diff --git a/NEWS b/NEWS
index 0760775..6b8f6b3 100644
--- a/NEWS
+++ b/NEWS
@@ -60,6 +60,10 @@ GNU coreutils NEWS                                    -*- 
outline -*-
   touch now accepts the option --no-dereference (-h), as a means to
   change symlink timestamps on platforms with enough support.
 
+** New programs
+
+  nproc: A new program to get the number of processors.
+
 
 * Noteworthy changes in release 8.0 (2009-10-06) [beta]
 
diff --git a/README b/README
index 7545eab..b48661b 100644
--- a/README
+++ b/README
@@ -1,4 +1,4 @@
-These are the GNU core utilities.  This package is the union of
+hese are the GNU core utilities.  This package is the union of
 the GNU fileutils, sh-utils, and textutils packages.
 
 Most of these programs have significant advantages over their Unix
@@ -11,7 +11,7 @@ The programs that can be built with this package are:
   csplit cut date dd df dir dircolors dirname du echo env expand expr
   factor false fmt fold groups head hostid hostname id install join kill
   link ln logname ls md5sum mkdir mkfifo mknod mktemp mv nice nl nohup
-  od paste pathchk pinky pr printenv printf ptx pwd readlink rm rmdir
+  nproc od paste pathchk pinky pr printenv printf ptx pwd readlink rm rmdir
   runcon seq sha1sum sha224sum sha256sum sha384sum sha512sum shred shuf
   sleep sort split stat stdbuf stty su sum sync tac tail tee test timeout
   touch tr true truncate tsort tty uname unexpand uniq unlink uptime users
diff --git a/bootstrap.conf b/bootstrap.conf
index 4c0f4c7..1f230df 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -158,6 +158,7 @@ gnulib_modules="
   modechange
   mountlist
   mpsort
+  nproc
   obstack
   pathmax
   perl
@@ -168,6 +169,7 @@ gnulib_modules="
   priv-set
   progname
   propername
+  pthread
   putenv
   quote
   quotearg
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index ec5bcfb..edd2e4f 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -84,6 +84,7 @@
 * nice: (coreutils)nice invocation.             Modify niceness.
 * nl: (coreutils)nl invocation.                 Number lines and write files.
 * nohup: (coreutils)nohup invocation.           Immunize to hangups.
+* nproc: (coreutils)nproc invocation.           Print the number of processors.
 * od: (coreutils)od invocation.                 Dump files in octal, etc.
 * paste: (coreutils)paste invocation.           Merge lines of files.
 * pathchk: (coreutils)pathchk invocation.       Check file name portability.
@@ -409,6 +410,7 @@ System context
 
 * arch invocation::              Print machine hardware name
 * date invocation::              Print or set system date and time
+* nproc invocation::             Print the number of processors
 * uname invocation::             Print system information
 * hostname invocation::          Print or set system name
 * hostid invocation::            Print numeric host identifier
@@ -13232,6 +13234,7 @@ information.
 @menu
 * date invocation::             Print or set system date and time.
 * arch invocation::             Print machine hardware name.
+* nproc invocation::            Print the number of processors.
 * uname invocation::            Print system information.
 * hostname invocation::         Print or set system name.
 * hostid invocation::           Print numeric host identifier.
@@ -13891,6 +13894,42 @@ The program accepts the @ref{Common options} only.
 @exitstatus
 
 
address@hidden nproc invocation
address@hidden @command{nproc}: Print the number of processors
+
address@hidden nproc
address@hidden Print the number of processors
address@hidden system information, printing
+
address@hidden prints the number of processors.  It is not a hardware
+inspection tool but a portable way to get how many processes
+potentially can be executed in parallel.  The result is guaranteed  to
+be a non-zero positive number.  Synopsis:
+
address@hidden
+nproc address@hidden
address@hidden example
+
+The program accepts the following options.  Also see @ref{Common options}.
+
address@hidden @samp
+
address@hidden --available
address@hidden --available
+Print the number of processors available to the current process.  It
+may be less than the number of installed processors.
+If this information is not accessible, then nproc returs the number of
+installed processors.  By default --available is used.
+
address@hidden --installed
address@hidden --installed
+Print the number of installed processors.
+
address@hidden table
+
address@hidden
+
+
 @node uname invocation
 @section @command{uname}: Print system information
 
diff --git a/po/POTFILES.in b/po/POTFILES.in
index e84e2ea..9a46a9a 100644
--- a/po/POTFILES.in
+++ b/po/POTFILES.in
@@ -89,6 +89,7 @@ src/mv.c
 src/nice.c
 src/nl.c
 src/nohup.c
+src/nproc.c
 src/od.c
 src/operand2sig.c
 src/paste.c
diff --git a/src/Makefile.am b/src/Makefile.am
index 67c29cc..a6dc5ff 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -86,6 +86,7 @@ EXTRA_PROGRAMS = \
   mktemp       \
   mv           \
   nl           \
+  nproc                \
   nohup                \
   od           \
   paste                \
@@ -189,6 +190,7 @@ chown_LDADD = $(LDADD)
 chroot_LDADD = $(LDADD)
 cksum_LDADD = $(LDADD)
 comm_LDADD = $(LDADD)
+nproc_LDADD = $(LDADD) $(LIB_PTHREAD)
 cp_LDADD = $(LDADD)
 csplit_LDADD = $(LDADD)
 cut_LDADD = $(LDADD)
@@ -480,6 +482,7 @@ rmdir_SOURCES = rmdir.c prog-fprintf.c
 
 uname_SOURCES = uname.c uname-uname.c
 arch_SOURCES = uname.c uname-arch.c
+nproc_SOURCES = nproc.c
 
 md5sum_SOURCES = md5sum.c
 md5sum_CPPFLAGS = -DHASH_ALGO_MD5=1 $(AM_CPPFLAGS)
diff --git a/src/nproc.c b/src/nproc.c
new file mode 100644
index 0000000..08e1bca
--- /dev/null
+++ b/src/nproc.c
@@ -0,0 +1,155 @@
+/* nproc - print the number of processors.
+   Copyright (C) 2009 Free Software Foundation, Inc.
+
+   This program is free software: you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation, either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+/* Written by Giuseppe Scrivano.  */
+
+#include <config.h>
+#include <getopt.h>
+#include <stdio.h>
+#include <sys/types.h>
+
+#include "system.h"
+#include "pthread.h"
+#include "nproc.h"
+
+/* The official name of this program (e.g., no `g' prefix).  */
+#define PROGRAM_NAME "nproc"
+
+#define AUTHORS proper_name ("Giuseppe Scrivano")
+
+enum
+{
+  AVAILABLE_OPTION = CHAR_MAX + 1,
+  INSTALLED_OPTION
+};
+
+static struct option const longopts[] =
+{
+  {"installed", no_argument, NULL, INSTALLED_OPTION},
+  {"available", no_argument, NULL, AVAILABLE_OPTION},
+  {GETOPT_HELP_OPTION_DECL},
+  {GETOPT_VERSION_OPTION_DECL},
+  {NULL, 0, NULL, 0}
+};
+
+void
+usage (int status)
+{
+  if (status != EXIT_SUCCESS)
+    fprintf (stderr, _("Try `%s --help' for more information.\n"),
+             program_name);
+  else
+    {
+      printf (_("Usage: %s [OPTION]...\n"), program_name);
+      fputs (_("\
+Print the number of processors.\n\
+\n\
+"), stdout);
+      fputs (_("\
+      --available          print the number of processors available to the\n\
+                           current process\n\
+      --installed          print the number of installed processors\n\
+"), stdout);
+
+      fputs (HELP_OPTION_DESCRIPTION, stdout);
+      fputs (VERSION_OPTION_DESCRIPTION, stdout);
+      emit_ancillary_info ();
+    }
+  exit (status);
+}
+
+/* Compute the number of available processors.  Return 0 on error.  */
+
+static unsigned long
+nproc_available (void)
+{
+  unsigned long nproc = 0;
+
+#ifdef CPU_SETSIZE
+  size_t j;
+  cpu_set_t cpuset;
+  CPU_ZERO (&cpuset);
+
+  if (pthread_getaffinity_np (pthread_self (), sizeof (cpu_set_t), &cpuset))
+    return 0;
+
+  for (j = 0; j < CPU_SETSIZE; j++)
+    if (CPU_ISSET (j, &cpuset))
+      nproc++;
+#endif
+
+  return nproc;
+}
+
+/* Compute the number of processors.  If AVAILABLE returns the number
+   of processors available to the current process, otherwise the number
+   of installed processors.  In case the number of available processors
+   can't be computed rollback to the installed processors.  The result is
+   guaranteed to be at least 1.  */
+
+static unsigned long
+nproc (bool available)
+{
+  if (available)
+    {
+      unsigned long available_proc = nproc_available ();
+      if (available_proc)
+        return available_proc;
+    }
+
+  return num_processors ();
+}
+
+int
+main (int argc, char **argv)
+{
+  bool available = true;
+  initialize_main (&argc, &argv);
+  set_program_name (argv[0]);
+  setlocale (LC_ALL, "");
+  bindtextdomain (PACKAGE, LOCALEDIR);
+  textdomain (PACKAGE);
+
+  atexit (close_stdout);
+
+  while (1)
+    {
+      int c = getopt_long (argc, argv, "", longopts, NULL);
+      if (c == -1)
+        break;
+      switch (c)
+        {
+        case_GETOPT_HELP_CHAR;
+
+        case_GETOPT_VERSION_CHAR (PROGRAM_NAME, AUTHORS);
+
+        case AVAILABLE_OPTION:
+          available = true;
+          break;
+
+        case INSTALLED_OPTION:
+          available = false;
+          break;
+
+        default:
+          usage (EXIT_FAILURE);
+        }
+    }
+
+  printf ("%lu\n", nproc (available));
+
+  exit (EXIT_SUCCESS);
+}
diff --git a/tests/Makefile.am b/tests/Makefile.am
index eec31ae..2f89c4e 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -460,6 +460,9 @@ TESTS =                                             \
   touch/read-only                              \
   touch/relative                               \
   touch/trailing-slash                         \
+  nproc/avail                          \
+  nproc/cpuinfo                                \
+  nproc/positive                               \
   $(root_tests)
 
 pr_data =                                      \
diff --git a/tests/nproc/avail b/tests/nproc/avail
new file mode 100755
index 0000000..3ab04ae
--- /dev/null
+++ b/tests/nproc/avail
@@ -0,0 +1,31 @@
+#!/bin/sh
+# Ensure that nproc --installed is greater or equal to nproc --available.
+
+# Copyright (C) 2009 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+if test "$VERBOSE" = yes; then
+  set -x
+  nproc --version
+fi
+
+. $srcdir/test-lib.sh
+
+available=$(nproc --available)
+installed=$(nproc --installed)
+
+test $installed -ge $available || fail=1
+
+Exit $fail
diff --git a/tests/nproc/cpuinfo b/tests/nproc/cpuinfo
new file mode 100755
index 0000000..faae12d
--- /dev/null
+++ b/tests/nproc/cpuinfo
@@ -0,0 +1,33 @@
+#!/bin/sh
+# Check that the number of cores reported by nproc is equal to the number
+# of cores listed in /proc/cpuinfo, when it is available
+
+# Copyright (C) 2009 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+if test "$VERBOSE" = yes; then
+  set -x
+  nproc --version
+fi
+
+. $srcdir/test-lib.sh
+
+test -r /proc/cpuinfo || skip_test_
+
+cores=$(nproc)
+
+test $(grep '^proc' /proc/cpuinfo  | wc -l) -eq $cores || fail=1
+
+Exit $fail
diff --git a/tests/nproc/positive b/tests/nproc/positive
new file mode 100755
index 0000000..76efb3f
--- /dev/null
+++ b/tests/nproc/positive
@@ -0,0 +1,31 @@
+#!/bin/sh
+# Ensure that nproc returns a non-zero positive number
+
+# Copyright (C) 2009 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+if test "$VERBOSE" = yes; then
+  set -x
+  nproc --version
+fi
+
+. $srcdir/test-lib.sh
+
+for opt in --available --installed; do
+    cores=$(nproc $opt)
+    test $cores -gt 0 || fail=1
+done
+
+Exit $fail
-- 
1.6.5