gnuastro-commits
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[gnuastro-commits] master e2aba57 083/113: Recent work in master importe


From: Mohammad Akhlaghi
Subject: [gnuastro-commits] master e2aba57 083/113: Recent work in master imported, conflicts fixed
Date: Fri, 16 Apr 2021 10:33:54 -0400 (EDT)

branch: master
commit e2aba5720f8f9fc29c4fb40e3652d0de0a865aea
Merge: 800c769 39a9018
Author: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Commit: Mohammad Akhlaghi <mohammad@akhlaghi.org>

    Recent work in master imported, conflicts fixed
    
    Two conflicts came up in NoiseChisel's `ui.c' and `detection.c'. The `ui.c'
    one was easy to correct, and for the `detection.c' one the same converter
    function that ised used erosion and dilation is used for finding the
    associated connectivity in filling holes.
---
 NEWS                                   |  37 +++-
 bin/crop/ui.c                          |   3 +-
 bin/mkcatalog/ui.c                     |   3 +-
 bin/mknoise/ui.c                       |   3 +-
 bin/mkprof/ui.c                        |   2 +-
 bin/noisechisel/args.h                 |  39 ++++
 bin/noisechisel/astnoisechisel-3d.conf |   1 +
 bin/noisechisel/astnoisechisel.conf    |   5 +-
 bin/noisechisel/detection.c            |  84 +++++++--
 bin/noisechisel/main.h                 |   5 +-
 bin/noisechisel/sky.c                  |  22 +--
 bin/noisechisel/threshold.c            |  25 +--
 bin/noisechisel/threshold.h            |   2 +-
 bin/noisechisel/ui.c                   |  25 ++-
 bin/noisechisel/ui.h                   |   3 +
 bin/segment/clumps.c                   |  11 +-
 bin/segment/segment.c                  |   7 +-
 bin/segment/ui.c                       |  22 ++-
 bin/statistics/sky.c                   |  20 +-
 bin/warp/ui.c                          |   3 +-
 configure.ac                           |  88 +++++++--
 developer-build                        |  26 +--
 doc/gnuastro.texi                      | 327 ++++++++++++++++++++-------------
 lib/binary.c                           |   6 +-
 24 files changed, 535 insertions(+), 234 deletions(-)

diff --git a/NEWS b/NEWS
index 52f7c28..c23d08a 100644
--- a/NEWS
+++ b/NEWS
@@ -13,17 +13,40 @@ GNU Astronomy Utilities NEWS                          -*- 
outline -*-
     --onedasimage: write output as an image if it has one dimension, not table.
 
   NoiseChisel:
-    - New outlier identification algorithm for quantile threshold and Sky
-      estimation. It is useful when there are extended and bright sources
-      in the dataset: the tiles containing very faint signal that pass the
-      general pixel-value distribution test due to the flatness of the
-      extended profiles (outliers), can be identified and removed in
-      comparison with the other passed tiles. The outlier finding algorithm
+
+    - New outlier identification algorithm for quantile thresholds. This is
+      very useful when there are extended and bright sources in the
+      dataset: the tiles containing very faint signal that pass the general
+      pixel-value distribution test due to the flatness of the extended
+      profiles, can be identified and removed as outliers in comparison
+      with the other passed tiles. The outlier finding algorithm
       (`gal_statistics_outlier_positive': a new library function) uses the
       distribution of distances between the sorted elements and is
-      configured with these three options.
+      configured with these options.
        --outliersclip: Sigma-clipping parameters for the process.
        --outliersigma: Multiple of sigma to define an outlier.
+    --blankasforeground: Treat blank elements as foreground (regions above
+          the threshold) in the binary processing steps: initial erosion
+          and opening as well as the filling holes and opening of
+          pseudo-detections. From this version, by default, blank elements
+          in the dataset are considered to be background, so if a
+          foreground pixel is touching it, it will be eroded. This option
+          is irrelevant if the datasets contains no blank elements, but can
+          help remove false positives that are touching blank elements.
+    --holengb: allows defining the connectivity of the holes that are
+          filled when defining pseudo-detections. Until now, this was
+          hard-wired into the code (=8) and not modifiable at run-time.
+    --skyfracnoblank: Ignore blank pixels when estimating the fraction of
+          undetected pixels for Sky estimation. NoiseChisel only measures
+          the Sky over the tiles that have a sufficiently large fraction of
+          undetected pixels. This is done to decrease the bias caused by
+          faint un-detected wings of bright galaxies or stars, see
+          description of `--minskyfrac' for more. Until now the reference
+          for this fraction was the whole tile size (irrespective of how
+          many blank elements it contains). With this option, it is now
+          possible to ask for ignoring blank pixels when calculating the
+          fraction. This is useful when blank/masked pixels are distributed
+          across the image.
 
   Statistics:
     - Sky estimation: new outlier estimation algorithm similar to NoiseChisel.
diff --git a/bin/crop/ui.c b/bin/crop/ui.c
index 6cee112..8b69c17 100644
--- a/bin/crop/ui.c
+++ b/bin/crop/ui.c
@@ -1036,7 +1036,8 @@ ui_read_check_inputs_setup(int argc, char *argv[], struct 
cropparams *p)
   /* Report timing: */
   if(!p->cp.quiet)
     {
-      printf(PROGRAM_NAME" started on %s", ctime(&p->rawtime));
+      printf(PROGRAM_NAME" "PACKAGE_VERSION" started on %s",
+             ctime(&p->rawtime));
       if(p->cp.numthreads>1)
         printf("  - Using %zu CPU thread%s\n", p->cp.numthreads,
                p->cp.numthreads==1 ? "." : "s.");
diff --git a/bin/mkcatalog/ui.c b/bin/mkcatalog/ui.c
index e254f31..6f4a41a 100644
--- a/bin/mkcatalog/ui.c
+++ b/bin/mkcatalog/ui.c
@@ -1525,7 +1525,8 @@ ui_read_check_inputs_setup(int argc, char *argv[], struct 
mkcatalogparams *p)
   if(!p->cp.quiet)
     {
       /* Write the information. */
-      printf(PROGRAM_NAME" started on %s", ctime(&p->rawtime));
+      printf(PROGRAM_NAME" "PACKAGE_VERSION" started on %s",
+             ctime(&p->rawtime));
       printf("  - Using %zu CPU thread%s\n", p->cp.numthreads,
              p->cp.numthreads==1 ? "." : "s.");
       printf("  - Objects: %s (hdu: %s)\n", p->objectsfile, p->cp.hdu);
diff --git a/bin/mknoise/ui.c b/bin/mknoise/ui.c
index 301d3f6..2ab9849 100644
--- a/bin/mknoise/ui.c
+++ b/bin/mknoise/ui.c
@@ -402,7 +402,8 @@ ui_read_check_inputs_setup(int argc, char *argv[], struct 
mknoiseparams *p)
   /* Everything is ready, notify the user of the program starting. */
   if(!p->cp.quiet)
     {
-      printf(PROGRAM_NAME" started on %s", ctime(&p->rawtime));
+      printf(PROGRAM_NAME" "PACKAGE_VERSION" started on %s",
+             ctime(&p->rawtime));
       sprintf(message, "Random number generator type: %s",
               gsl_rng_name(p->rng));
       gal_timing_report(NULL, message, 1);
diff --git a/bin/mkprof/ui.c b/bin/mkprof/ui.c
index d16d90c..f37dc6a 100644
--- a/bin/mkprof/ui.c
+++ b/bin/mkprof/ui.c
@@ -1720,7 +1720,7 @@ ui_print_intro(struct mkprofparams *p)
 
   if(p->cp.quiet) return;
 
-  printf(PROGRAM_NAME" started on %s", ctime(&p->rawtime));
+  printf(PROGRAM_NAME" "PACKAGE_VERSION" started on %s", ctime(&p->rawtime));
 
   if(p->kernel)
     {
diff --git a/bin/noisechisel/args.h b/bin/noisechisel/args.h
index 4236fee..3b89dc8 100644
--- a/bin/noisechisel/args.h
+++ b/bin/noisechisel/args.h
@@ -265,6 +265,19 @@ struct argp_option program_options[] =
       GAL_OPTIONS_NOT_SET
     },
     {
+      "blankasforeground",
+      UI_KEY_BLANKASFOREGROUND,
+      0,
+      0,
+      "Blanks are foreground in erosion and opening.",
+      UI_GROUP_DETECTION,
+      &p->blankasforeground,
+      GAL_OPTIONS_NO_ARG_TYPE,
+      GAL_OPTIONS_RANGE_0_OR_1,
+      GAL_OPTIONS_NOT_MANDATORY,
+      GAL_OPTIONS_NOT_SET
+    },
+    {
       "erode",
       UI_KEY_ERODE,
       "INT",
@@ -330,6 +343,19 @@ struct argp_option program_options[] =
       GAL_OPTIONS_NOT_SET
     },
     {
+      "skyfracnoblank",
+      UI_KEY_SKYFRACNOBLANK,
+      0,
+      0,
+      "No blanks in tile undetected frac. (minskyfrac).",
+      UI_GROUP_DETECTION,
+      &p->skyfracnoblank,
+      GAL_OPTIONS_NO_ARG_TYPE,
+      GAL_OPTIONS_RANGE_0_OR_1,
+      GAL_OPTIONS_NOT_MANDATORY,
+      GAL_OPTIONS_NOT_SET
+    },
+    {
       "minskyfrac",
       UI_KEY_MINSKYFRAC,
       "FLT",
@@ -383,6 +409,19 @@ struct argp_option program_options[] =
       GAL_OPTIONS_NOT_SET
     },
     {
+      "holengb",
+      UI_KEY_HOLENGB,
+      "INT",
+      0,
+      "4 or 8 connectivity for filling holes.",
+      UI_GROUP_DETECTION,
+      &p->holengb,
+      GAL_TYPE_SIZE_T,
+      GAL_OPTIONS_RANGE_GT_0,
+      GAL_OPTIONS_MANDATORY,
+      GAL_OPTIONS_NOT_SET
+    },
+    {
       "snminarea",
       UI_KEY_SNMINAREA,
       "INT",
diff --git a/bin/noisechisel/astnoisechisel-3d.conf 
b/bin/noisechisel/astnoisechisel-3d.conf
index 5a80eb5..c3f1aa2 100644
--- a/bin/noisechisel/astnoisechisel-3d.conf
+++ b/bin/noisechisel/astnoisechisel-3d.conf
@@ -42,6 +42,7 @@
  openingngb             18
  sigmaclip           3,0.2
  dthresh               0.0
+ holengb                18
  snminarea              20
  snquant              0.99
  detgrowquant         0.95
diff --git a/bin/noisechisel/astnoisechisel.conf 
b/bin/noisechisel/astnoisechisel.conf
index 54209cf..2362820 100644
--- a/bin/noisechisel/astnoisechisel.conf
+++ b/bin/noisechisel/astnoisechisel.conf
@@ -21,8 +21,6 @@
  khdu                    1
  whdu                    1
  chdu                    1
- minskyfrac            0.7
- minnumfalse           100
 
 # Tessellation
  largetilesize     200,200
@@ -38,9 +36,12 @@
  noerodequant         0.99
  opening                 1
  openingngb              8
+ minskyfrac            0.7
  sigmaclip           3,0.2
  dthresh               0.0
+ holengb                 8
  snminarea              10
+ minnumfalse           100
  snquant              0.99
  detgrowquant         0.90
  detgrowmaxholesize    100
diff --git a/bin/noisechisel/detection.c b/bin/noisechisel/detection.c
index 45b89c5..a16c579 100644
--- a/bin/noisechisel/detection.c
+++ b/bin/noisechisel/detection.c
@@ -30,6 +30,7 @@ along with Gnuastro. If not, see 
<http://www.gnu.org/licenses/>.
 
 #include <gnuastro/fits.h>
 #include <gnuastro/label.h>
+#include <gnuastro/blank.h>
 #include <gnuastro/binary.h>
 #include <gnuastro/threads.h>
 #include <gnuastro/pointer.h>
@@ -82,8 +83,10 @@ detection_ngb_to_connectivity(size_t ndim, size_t ngb)
 void
 detection_initial(struct noisechiselparams *p)
 {
+  float *f;
   char *msg;
   uint8_t *b, *bf;
+  int resetblank=0;
   struct timeval t0, t1;
 
 
@@ -105,6 +108,15 @@ detection_initial(struct noisechiselparams *p)
     }
 
 
+  /* Remove any blank elements from the binary image if requested. */
+  if(p->blankasforeground==0 && gal_blank_present(p->binary,0))
+    {
+      resetblank=1;
+      bf=(b=p->binary->array)+p->binary->size;
+      do *b = *b==GAL_BLANK_UINT8 ? 0 : *b; while(++b<bf);
+    }
+
+
   /* Erode the image. */
   if(!p->cp.quiet) gettimeofday(&t1, NULL);
   gal_binary_erode(p->binary, p->erode,
@@ -112,7 +124,7 @@ detection_initial(struct noisechiselparams *p)
                                                  p->erodengb), 1);
   if(!p->cp.quiet)
     {
-      if( asprintf(&msg, "Eroded %zu time%s (%zu-connectivity).", p->erode,
+      if( asprintf(&msg, "Eroded %zu time%s (%zu-connected).", p->erode,
                    p->erode>1?"s":"", p->erodengb)<0 )
         error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
       gal_timing_report(&t1, msg, 2);
@@ -138,14 +150,23 @@ detection_initial(struct noisechiselparams *p)
                                                 p->openingngb), 1);
   if(!p->cp.quiet)
     {
-      if( asprintf(&msg, "Opened (depth: %zu, %s connectivity).",
-                   p->opening, p->openingngb==4 ? "4" : "8")<0 )
+      if( asprintf(&msg, "Opened (depth: %zu, %zu-connected).",
+                   p->opening, p->openingngb)<0 )
         error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
       gal_timing_report(&t1, msg, 2);
       free(msg);
     }
 
 
+  /* Reset the blank values (if requested). */
+  if(resetblank)
+    {
+      f=p->input->array;
+      bf=(b=p->binary->array)+p->binary->size;
+      do *b = isnan(*f++) ? GAL_BLANK_UINT8 : *b; while(++b<bf);
+    }
+
+
   /* Label the connected components. */
   p->numinitialdets=gal_binary_connected_components(p->binary, &p->olabel,
                                                     p->binary->ndim);
@@ -249,6 +270,7 @@ detection_fill_holes_open(void *in_prm)
   struct noisechiselparams *p=fho_prm->p;
 
   void *tarray;
+  uint8_t *b, *bf;
   gal_data_t *tile, *copy, *tblock;
   size_t i, dsize[]={1,1,1,1,1,1,1,1,1,1}; /* For upto 10-Dimensions! */
 
@@ -276,18 +298,25 @@ detection_fill_holes_open(void *in_prm)
       /* Copy the tile into the contiguous patch of memory to work on, but
          first reset the size element so `gal_data_copy_to_allocated' knows
          there is enough space. */
+      copy->flag=0;
       copy->size=p->maxltcontig;
       gal_data_copy_to_allocated(tile, copy);
 
+      /* Take blank values to the background (set them to zero) if
+         necsesary. */
+      if( p->blankasforeground==0
+          && gal_blank_present(p->input,0)
+          && gal_blank_present(copy, 1) )
+        {
+          bf=(b=copy->array)+copy->size;
+          do *b = *b==GAL_BLANK_UINT8 ? 0 : *b; while(++b<bf);
+        }
+
       /* Fill the holes in this tile: holes with maximal connectivity means
-         that they are most strongly bounded.
-
-         IMPORTANT NOTE: For 2D, the strongest connectivity (2) is
-         fine. But for 3D, the strongest connectivity is too strong and
-         will not be too useful for diffuse signal. So for 3D, we are now
-         using the second-strongest connectivity of 2 (numerically: same as
-         the 2D case). */
-      gal_binary_holes_fill(copy, 2, -1);
+         that they are most strongly bounded. */
+      gal_binary_holes_fill(copy, detection_ngb_to_connectivity(p->input->ndim,
+                                                                p->holengb),
+                            -1);
       if(fho_prm->step==1)
         {
           detection_write_in_large(tile, copy);
@@ -328,6 +357,8 @@ static size_t
 detection_pseudo_find(struct noisechiselparams *p, gal_data_t *workbin,
                       gal_data_t *worklab, int s0d1)
 {
+  float *f;
+  uint8_t *b, *bf;
   gal_data_t *bin;
   struct fho_params fho_prm={0, NULL, workbin, worklab, p};
 
@@ -376,6 +407,14 @@ detection_pseudo_find(struct noisechiselparams *p, 
gal_data_t *workbin,
           gal_threads_spin_off(detection_fill_holes_open, &fho_prm,
                                p->ltl.tottiles, p->cp.numthreads);
 
+          /* Reset the blank values (if they were changed). */
+          if( p->blankasforeground==0 && gal_blank_present(p->input,0) )
+            {
+              f=p->input->array;
+              bf=(b=workbin->array)+workbin->size;
+              do *b = isnan(*f++) ? GAL_BLANK_UINT8 : *b; while(++b<bf);
+            }
+
           /* Set the extension name based on the step. */
           switch(fho_prm.step)
             {
@@ -447,7 +486,7 @@ static void
 detection_sn_write_to_file(struct noisechiselparams *p, gal_data_t *sn,
                            gal_data_t *snind, int s0d1D2)
 {
-  char *str;
+  char *str, *extname;
   gal_list_str_t *comments=NULL;
 
   /* Comment for extension on further explanation. */
@@ -467,11 +506,15 @@ detection_sn_write_to_file(struct noisechiselparams *p, 
gal_data_t *sn,
   gal_list_str_add(&comments, str, 1);
 
 
-  /* Set the file name. */
+  /* Set the file name and write the table. */
   str = ( s0d1D2
           ? ( s0d1D2==2 ? p->detsn_D_name : p->detsn_d_name )
           : p->detsn_s_name );
-  threshold_write_sn_table(p, sn, snind, str, comments);
+  if( p->cp.tableformat!=GAL_TABLE_FORMAT_TXT )
+    extname = ( s0d1D2
+                ? ( s0d1D2==2 ? "GROWN_DETECTION_SN" : "DET_PSEUDODET_SN" )
+                : "SKY_PSEUDODET_SN" );
+  threshold_write_sn_table(p, sn, snind, str, comments, extname);
   gal_list_str_free(comments, 1);
 
 
@@ -816,6 +859,7 @@ detection_final_remove_small_sn(struct noisechiselparams *p,
   gal_data_t *sn, *snind;
   int32_t *l, *lf, curlab=1;
   gal_list_str_t *comments=NULL;
+  char *extname="GROWN_DETECTION_SN";
   int32_t *newlabs=gal_pointer_allocate(GAL_TYPE_INT32, num+1, 1, __func__,
                                         "newlabs");
 
@@ -863,9 +907,8 @@ detection_final_remove_small_sn(struct noisechiselparams *p,
       gal_list_str_add(&comments, "See also: `DILATED' "
                        "HDU of output with `--checkdetection'.", 1);
       gal_list_str_add(&comments, "S/N of finally grown detections.", 1);
-
-
-      threshold_write_sn_table(p, sn, snind, p->detsn_D_name, comments);
+      threshold_write_sn_table(p, sn, snind, p->detsn_D_name, comments,
+                               extname);
       gal_list_str_free(comments, 1);
 
     }
@@ -1145,6 +1188,13 @@ detection(struct noisechiselparams *p)
      pseudo-detections. */
   if(!p->cp.quiet) gettimeofday(&t1, NULL);
   num_true_initial=detection_remove_false_initial(p, workbin);
+  if(p->detectionname)
+    {
+      p->olabel->name="DETECTIONS-INIT-TRUE";
+      gal_fits_img_write(workbin, p->detectionname, NULL,
+                         PROGRAM_NAME);
+      p->olabel->name=NULL;
+    }
   if(!p->cp.quiet)
     {
       if( asprintf(&msg, "%zu false initial detections removed.",
diff --git a/bin/noisechisel/main.h b/bin/noisechisel/main.h
index c0d64fd..9ebb0ed 100644
--- a/bin/noisechisel/main.h
+++ b/bin/noisechisel/main.h
@@ -57,20 +57,23 @@ struct noisechiselparams
   uint8_t               label;  /* Label detections that are connected.   */
 
   float          meanmedqdiff;  /* Difference between mode and median.    */
-  float            minskyfrac;  /* Undetected area min. frac. in tile.    */
   float               qthresh;  /* Quantile threshold on convolved image. */
   float          outliersigma;  /* Multiple of sigma to define outlier.   */
   double      outliersclip[2];  /* Outlier Sigma-clipping params.         */
   size_t          smoothwidth;  /* Interpolation: flat kernel to smooth.  */
   uint8_t        checkqthresh;  /* Save the quantile threhsold steps.     */
+  uint8_t   blankasforeground;  /* Blank as foreg. in erosion and opening.*/
   size_t                erode;  /* Number of erosions after thresholding. */
   size_t             erodengb;  /* Connectivity for erosion.              */
   float          noerodequant;  /* Quantile for no erosion.               */
   size_t              opening;  /* Depth of opening after erosion.        */
   size_t           openingngb;  /* Connectivity to use for opening.       */
+  uint8_t      skyfracnoblank;  /* No blanks in estimating non-det frac.  */
+  float            minskyfrac;  /* Undetected area min. frac. in tile.    */
   double         sigmaclip[2];  /* Sigma-clipping parameters.             */
   uint8_t         checkdetsky;  /* Check pseudo-detection sky value.      */
   float               dthresh;  /* Sigma threshold for Pseudo-detections. */
+  size_t              holengb;  /* Connectivity for defining a hole.      */
   size_t            snminarea;  /* Minimum pseudo-detection area for S/N. */
   uint8_t             checksn;  /* Save pseudo-detection S/N values.      */
   size_t          minnumfalse;  /* Min No. of det/seg for true quantile.  */
diff --git a/bin/noisechisel/sky.c b/bin/noisechisel/sky.c
index 0e23b44..1b6e9c6 100644
--- a/bin/noisechisel/sky.c
+++ b/bin/noisechisel/sky.c
@@ -59,8 +59,8 @@ sky_mean_std_undetected(void *in_prm)
   struct noisechiselparams *p=(struct noisechiselparams *)tprm->params;
 
   int setblank, type=GAL_TYPE_FLOAT32;
-  size_t twidth=gal_type_sizeof(GAL_TYPE_FLOAT32);
   size_t i, tind, numsky, bdsize=2, ndim=p->sky->ndim;
+  size_t refarea, twidth=gal_type_sizeof(GAL_TYPE_FLOAT32);
   gal_data_t *tile, *fusage, *busage, *bintile, *sigmaclip;
 
 
@@ -88,20 +88,24 @@ sky_mean_std_undetected(void *in_prm)
       numsky=0;
       tind = tprm->indexs[i];
       tile = &p->cp.tl.tiles[tind];
+      refarea = p->skyfracnoblank ? 0 : tile->size;
 
       /* Correct the fake binary tile's properties to be the same as this
          one, then count the number of zero valued elements in it. Note
          that the `CHECK_BLANK' flag of `GAL_TILE_PARSE_OPERATE' is set to
-         1. So blank values in the input array are not counted also. */
+         1. So blank values in the input array are not counted. */
       bintile->size=tile->size;
       bintile->dsize=tile->dsize;
       bintile->array=gal_tile_block_relative_to_other(tile, p->binary);
-      GAL_TILE_PARSE_OPERATE(tile, bintile, 1, 1, {if(!*o) numsky++;});
+      GAL_TILE_PARSE_OPERATE(tile, bintile, 1, 1, {
+          if(p->skyfracnoblank) ++refarea;
+          if(!*o)               ++numsky;
+        });
 
       /* Only continue, if the fraction of Sky values is less than the
          requested fraction. */
       setblank=0;
-      if( (float)(numsky)/(float)(tile->size) > p->minskyfrac)
+      if( (float)(numsky)/(float)(refarea) > p->minskyfrac)
         {
           /* Re-initialize the usage array's size information (will be
              corrected to this tile's size by
@@ -214,14 +218,8 @@ sky_and_std(struct noisechiselparams *p, char *checkname)
     }
 
 
-  /* Remove the outliers. */
-  if(p->outliersigma!=0.0)
-    gal_tileinternal_no_outlier(p->sky, p->std, NULL, tl, p->outliersclip,
-                                p->outliersigma, checkname);
-
-
-  /* Set the blank checked bit of the ararys to zero, most probably there
-     are tiles with too much signal or outliers. */
+  /* Set the blank-checked bit of the arrays to zero so we are sure to
+     check for blanks. */
   p->sky->flag &= ~GAL_DATA_FLAG_BLANK_CH;
   p->std->flag &= ~GAL_DATA_FLAG_BLANK_CH;
 
diff --git a/bin/noisechisel/threshold.c b/bin/noisechisel/threshold.c
index 71594f0..e372296 100644
--- a/bin/noisechisel/threshold.c
+++ b/bin/noisechisel/threshold.c
@@ -190,7 +190,7 @@ threshold_apply(struct noisechiselparams *p, float *value1,
 void
 threshold_write_sn_table(struct noisechiselparams *p, gal_data_t *insn,
                          gal_data_t *inind, char *filename,
-                         gal_list_str_t *comments)
+                         gal_list_str_t *comments, char *extname)
 {
   gal_data_t *sn, *ind, *cols;
 
@@ -225,9 +225,11 @@ threshold_write_sn_table(struct noisechiselparams *p, 
gal_data_t *insn,
   gal_table_comments_add_intro(&comments, PROGRAM_STRING, &p->rawtime);
 
 
-  /* write the table. */
-  gal_checkset_writable_remove(filename, 0, 1);
-  gal_table_write(cols, comments, p->cp.tableformat, filename, "SN", 0);
+  /* Write the table. Note that we'll set the `dontdelete' argument to 0
+     because when the output is a FITS table, we want all the tables in one
+     FITS file. We have already deleted any existing file with the same
+     name in `ui_set_output_names'.*/
+  gal_table_write(cols, comments, p->cp.tableformat, filename, extname, 0);
 
 
   /* Clean up (if necessary). */
@@ -613,7 +615,7 @@ threshold_quantile_find_apply(struct noisechiselparams *p)
      errors in parallel. */
   num=gal_statistics_number(qprm.erode_th);
   nval=((size_t *)(num->array))[0];
-  if( nval < cp->interpnumngb)
+  if( nval < cp->interpnumngb )
     error(EXIT_FAILURE, 0, "%zu tile(s) can be used for interpolation of the "
           "quantile threshold values over the full dataset. This is smaller "
           "than the requested minimum value of %zu (value to the "
@@ -627,12 +629,13 @@ threshold_quantile_find_apply(struct noisechiselparams *p)
           "option values to Gnuastro's programs by appending `-P' to the "
           "end of your command.\n\n"
           "  * Slightly decrease `--tilesize' to have more tiles.\n"
-          "  * Slightly increase `--meanmedqdiff' to accept more tiles.\n\n"
-          "  * Decrease `--outliersigma' to reject less tiles as outliers."
-          "  * Decrease `--interpnumngb' to be smaller than %zu.\n"
-          "Try appending your command with `--checkqthresh' to see the "
-          "successful tiles (and get a feeling of the cause/solution. Note "
-          "that the output is a multi-extension FITS file).\n\n"
+          "  * Slightly increase `--meanmedqdiff' to accept more tiles.\n"
+          "  * Decrease `--outliersigma' to reject less tiles as outliers.\n"
+          "  * Decrease `--interpnumngb' to be smaller than %zu.\n\n"
+          "Append the previous command with `--checkqthresh' to see the "
+          "successful tiles and which were discarded as outliers. This will "
+          "help you find the cause/solution. Note that the output is a "
+          "multi-extension FITS file).\n\n"
           "To better understand this important step, please run the "
           "following command (press `SPACE'/arrow-keys to navigate and "
           "`Q' to return back to the command-line):\n\n"
diff --git a/bin/noisechisel/threshold.h b/bin/noisechisel/threshold.h
index 48680a7..593e420 100644
--- a/bin/noisechisel/threshold.h
+++ b/bin/noisechisel/threshold.h
@@ -40,7 +40,7 @@ threshold_apply(struct noisechiselparams *p, float *value1, 
float *value2,
 void
 threshold_write_sn_table(struct noisechiselparams *p, gal_data_t *sntable,
                          gal_data_t *snind, char *filename,
-                         gal_list_str_t *comments);
+                         gal_list_str_t *comments, char *extname);
 
 void
 threshold_interp_smooth(struct noisechiselparams *p, gal_data_t **first,
diff --git a/bin/noisechisel/ui.c b/bin/noisechisel/ui.c
index c595814..a6cef92 100644
--- a/bin/noisechisel/ui.c
+++ b/bin/noisechisel/ui.c
@@ -240,6 +240,7 @@ ui_read_check_only_options(struct noisechiselparams *p)
           "for it");
 
   /* A general check on the neighbor connectivity values. */
+  ui_ngb_check(p->holengb,    "holengb");
   ui_ngb_check(p->erodengb,   "erodengb");
   ui_ngb_check(p->openingngb, "openingngb");
 
@@ -290,6 +291,21 @@ ui_read_check_only_options(struct noisechiselparams *p)
               "HDU number (starting from zero), extension name, or any "
               "HDU identifier acceptable by CFITSIO", p->widekernelname);
     }
+
+  /* If the S/N quantile is less than 0.1 (an arbitrary small value), this
+     is probably due to forgetting that this is the purity level
+     (higher-is-better), not the contamination level
+     (lower-is-better). This actually happened in a few cases: where we
+     wanted a false detection rate of 0.0001 (a super-high value!), and
+     instead of inputing 0.9999, we mistakenly gave `--snquant' a value of
+     `0.0001'. We were thus fully confused with the output (an extremely
+     low value) and thought its a bug, while it wasn't! */
+  if(p->snquant<0.1)
+    fprintf(stderr, "\nWARNING: Value of `--snquant' (`-c') is %g. Note "
+            "that this is not a contamination rate (where lower is "
+            "better), it is a purity rate (where higher is better). If you "
+            "intentionally asked for such a low purity level, please "
+            "ignore this warning\n\n", p->snquant);
 }
 
 
@@ -380,13 +396,13 @@ ui_set_output_names(struct noisechiselparams *p)
     {
       p->detsn_s_name=gal_checkset_automatic_output(&p->cp, basename,
                  ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
-                   ? "_detsn_sky.txt" : "_detsn_sky.fits") );
+                   ? "_detsn_sky.txt" : "_detsn.fits") );
       p->detsn_d_name=gal_checkset_automatic_output(&p->cp, basename,
                  ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
-                   ? "_detsn_det.txt" : "_detsn_det.fits") );
+                   ? "_detsn_det.txt" : "_detsn.fits") );
       p->detsn_D_name=gal_checkset_automatic_output(&p->cp, basename,
                  ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
-                   ? "_detsn_grown.txt" : "_detsn_grown.fits") );
+                   ? "_detsn_grown.txt" : "_detsn.fits") );
     }
 
   /* Detection steps. */
@@ -757,7 +773,8 @@ ui_read_check_inputs_setup(int argc, char *argv[],
   /* Let the user know that processing has started. */
   if(!p->cp.quiet)
     {
-      printf(PROGRAM_NAME" started on %s", ctime(&p->rawtime));
+      printf(PROGRAM_NAME" "PACKAGE_VERSION" started on %s",
+             ctime(&p->rawtime));
       printf("  - Using %zu CPU thread%s\n", p->cp.numthreads,
              p->cp.numthreads==1 ? "." : "s.");
       printf("  - Input: %s (hdu: %s)\n", p->inputname, p->cp.hdu);
diff --git a/bin/noisechisel/ui.h b/bin/noisechisel/ui.h
index 4c66ef8..3953935 100644
--- a/bin/noisechisel/ui.h
+++ b/bin/noisechisel/ui.h
@@ -86,10 +86,13 @@ enum option_keys_enum
   UI_KEY_OUTLIERSIGMA,
   UI_KEY_OUTLIERSCLIP,
   UI_KEY_CHECKQTHRESH,
+  UI_KEY_BLANKASFOREGROUND,
   UI_KEY_ERODENGB,
   UI_KEY_NOERODEQUANT,
   UI_KEY_OPENINGNGB,
+  UI_KEY_SKYFRACNOBLANK,
   UI_KEY_CHECKDETSKY,
+  UI_KEY_HOLENGB,
   UI_KEY_CHECKSN,
   UI_KEY_DETGROWMAXHOLESIZE,
   UI_KEY_CLEANGROWNDET,
diff --git a/bin/segment/clumps.c b/bin/segment/clumps.c
index b01f12a..cead77b 100644
--- a/bin/segment/clumps.c
+++ b/bin/segment/clumps.c
@@ -37,7 +37,6 @@ along with Gnuastro. If not, see 
<http://www.gnu.org/licenses/>.
 #include <gnuastro/statistics.h>
 
 #include <gnuastro-internal/timing.h>
-#include <gnuastro-internal/checkset.h>
 
 #include "main.h"
 
@@ -541,15 +540,12 @@ clumps_write_sn_table(struct segmentparams *p, gal_data_t 
*insn,
   cols       = ind;
   cols->next = sn;
 
-
   /* Prepare the comments. */
   gal_table_comments_add_intro(&comments, PROGRAM_STRING, &p->rawtime);
 
-
   /* write the table. */
-  gal_checkset_writable_remove(filename, 0, 1);
-  gal_table_write(cols, comments, p->cp.tableformat, filename, "SN", 0);
-
+  gal_table_write(cols, comments, p->cp.tableformat, filename,
+                  "SKY_CLUMP_SN", 0);
 
   /* Clean up (if necessary). */
   if(sn!=insn) gal_data_free(sn);
@@ -739,7 +735,8 @@ clumps_true_find_sn_thresh(struct segmentparams *p)
 
   /* The S/N array of sky clumps is desiged to have no blank values, so set
      the flags accordingly to avoid a redundant blank search. */
-  sn->flag = GAL_DATA_FLAG_BLANK_CH | GAL_DATA_FLAG_HASBLANK;
+  sn->flag  =  GAL_DATA_FLAG_BLANK_CH;
+  sn->flag &= ~GAL_DATA_FLAG_HASBLANK;
 
 
   /* If the user wanted to see the S/N table, then save it. */
diff --git a/bin/segment/segment.c b/bin/segment/segment.c
index c184e6f..939bdcc 100644
--- a/bin/segment/segment.c
+++ b/bin/segment/segment.c
@@ -795,9 +795,8 @@ segment_save_sn_table(struct clumps_params *clprm)
   /* Set the column pointers and write them into a table.. */
   clumpinobj->next=sn;
   objind->next=clumpinobj;
-  gal_checkset_writable_remove(p->clumpsn_d_name, 0, 1);
   gal_table_write(objind, comments, p->cp.tableformat, p->clumpsn_d_name,
-                  "CLUMPS_SN", 0);
+                  "DET_CLUMP_SN", 0);
 
 
   /* Clean up. */
@@ -809,7 +808,9 @@ segment_save_sn_table(struct clumps_params *clprm)
 
   /* Abort NoiseChisel if necessary. */
   if(!p->continueaftercheck)
-    ui_abort_after_check(p, p->clumpsn_s_name, p->clumpsn_d_name,
+    ui_abort_after_check(p, p->clumpsn_s_name,
+                         ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
+                           ? p->clumpsn_d_name : NULL ),
                          "showing all clump S/N values");
 }
 
diff --git a/bin/segment/ui.c b/bin/segment/ui.c
index 07f341d..8048deb 100644
--- a/bin/segment/ui.c
+++ b/bin/segment/ui.c
@@ -267,6 +267,21 @@ ui_read_check_only_options(struct segmentparams *p)
               "(starting from zero), extension name, or anything "
               "acceptable by CFITSIO");
     }
+
+  /* If the S/N quantile is less than 0.1 (an arbitrary small value), this
+     is probably due to forgetting that this is the purity level
+     (higher-is-better), not the contamination level
+     (lower-is-better). This actually happened in a few cases: where we
+     wanted a false detection rate of 0.0001 (a super-high value!), and
+     instead of inputing 0.9999, we mistakenly gave `--snquant' a value of
+     `0.0001'. We were thus fully confused with the output (an extremely
+     low value) and thought its a bug, while it wasn't! */
+  if(p->snquant<0.1)
+    fprintf(stderr, "\nWARNING: Value of `--snquant' (`-c') is %g. Note "
+            "that this is not a contamination rate (where lower is "
+            "better), it is a purity rate (where higher is better). If you "
+            "intentionally asked for such a low purity level, please "
+            "ignore this warning\n\n", p->snquant);
 }
 
 
@@ -365,10 +380,10 @@ ui_set_output_names(struct segmentparams *p)
     {
       p->clumpsn_s_name=gal_checkset_automatic_output(&p->cp, basename,
                  ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
-                   ? "_clumpsn_sky.txt" : "_clumpsn_sky.fits") );
+                   ? "_clumpsn_sky.txt" : "_clumpsn.fits") );
       p->clumpsn_d_name=gal_checkset_automatic_output(&p->cp, basename,
                  ( p->cp.tableformat==GAL_TABLE_FORMAT_TXT
-                   ? "_clumpsn_det.txt" : "_clumpsn_det.fits") );
+                   ? "_clumpsn_det.txt" : "_clumpsn.fits") );
     }
 
   /* Segmentation steps. */
@@ -913,7 +928,8 @@ ui_read_check_inputs_setup(int argc, char *argv[], struct 
segmentparams *p)
   if(!p->cp.quiet)
     {
       /* Basic inputs. */
-      printf(PROGRAM_NAME" started on %s", ctime(&p->rawtime));
+      printf(PROGRAM_NAME" "PACKAGE_VERSION" started on %s",
+             ctime(&p->rawtime));
       printf("  - Using %zu CPU thread%s\n", p->cp.numthreads,
              p->cp.numthreads==1 ? "." : "s.");
       printf("  - Input: %s (hdu: %s)\n", p->inputname, p->cp.hdu);
diff --git a/bin/statistics/sky.c b/bin/statistics/sky.c
index e61c74d..130a797 100644
--- a/bin/statistics/sky.c
+++ b/bin/statistics/sky.c
@@ -231,10 +231,10 @@ sky(struct statisticsparams *p)
     gal_timing_report(&t1, "All blank tiles filled (interplated).", 1);
   if(p->checksky)
     {
-      gal_tile_full_values_write(p->sky_t, tl, 1, p->checkskyname, NULL,
-                                 PROGRAM_NAME);
-      gal_tile_full_values_write(p->std_t, tl, 1, p->checkskyname, NULL,
-                                 PROGRAM_NAME);
+      gal_tile_full_values_write(p->sky_t, tl, !p->ignoreblankinsky,
+                                 p->checkskyname, NULL, PROGRAM_NAME);
+      gal_tile_full_values_write(p->std_t, tl, !p->ignoreblankinsky,
+                                 p->checkskyname, NULL, PROGRAM_NAME);
     }
 
 
@@ -255,10 +255,12 @@ sky(struct statisticsparams *p)
                           1);
       if(p->checksky)
         {
-          gal_tile_full_values_write(p->sky_t, tl, 1, p->checkskyname, NULL,
-                                     PROGRAM_NAME);
-          gal_tile_full_values_write(p->std_t, tl, 1, p->checkskyname, NULL,
-                                     PROGRAM_NAME);
+          gal_tile_full_values_write(p->sky_t, tl, !p->ignoreblankinsky,
+                                     p->checkskyname, NULL, PROGRAM_NAME);
+          gal_tile_full_values_write(p->std_t, tl, !p->ignoreblankinsky,
+                                     p->checkskyname, NULL, PROGRAM_NAME);
+          if(!cp->quiet)
+            printf("  - Check image written to `%s'.\n", p->checkskyname);
         }
     }
 
@@ -284,7 +286,7 @@ sky(struct statisticsparams *p)
   gal_fits_key_write_config(&p->cp.okeys, "Statistics configuration",
                             "STATISTICS-CONFIG", outname, "0");
   if(!cp->quiet)
-    printf("  - Written to `%s'.\n", outname);
+    printf("  - Sky and its STD written to `%s'.\n", outname);
 
 
   /* Clean up and return. */
diff --git a/bin/warp/ui.c b/bin/warp/ui.c
index 2814683..2804859 100644
--- a/bin/warp/ui.c
+++ b/bin/warp/ui.c
@@ -947,7 +947,8 @@ ui_read_check_inputs_setup(int argc, char *argv[], struct 
warpparams *p)
   if(!p->cp.quiet)
     {
       matrix=p->matrix->array;
-      printf(PROGRAM_NAME" started on %s", ctime(&p->rawtime));
+      printf(PROGRAM_NAME" "PACKAGE_VERSION" started on %s",
+             ctime(&p->rawtime));
       printf(" Using %zu CPU thread%s\n", p->cp.numthreads,
              p->cp.numthreads==1 ? "." : "s.");
       printf(" Input: %s (hdu: %s)\n", p->inputname, p->cp.hdu);
diff --git a/configure.ac b/configure.ac
index 771ca77..19c2d1b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -254,17 +254,19 @@ AC_MSG_RESULT( $path_warning )
 # checking the next libraries, so the linking with their dependent
 # libraries is done automatically with this order, and we don't have to
 # explicitly set the dependency flags.
+has_gsl=yes
+has_cmath=yes
+has_wcslib=yes
+has_cfitsio=yes
+has_gslcblas=yes
+missing_mandatory=no
+missing_optional_lib=no
 AC_SEARCH_LIBS(sqrt, m, [],
-    [AC_MSG_ERROR([C math library not present, cannot continue.])])
+               [missing_mandatory=yes; has_cmath=no])
 AC_SEARCH_LIBS([cblas_sdsdot], [gslcblas], [],
-    [AC_MSG_ERROR([GSL CBLAS not present, cannot continue.])])
+               [missing_mandatory=yes; has_gslcblas=no])
 AC_SEARCH_LIBS([gsl_integration_qng], [gsl], [],
-    [AC_MSG_ERROR([GSL not found, cannot continue.])])
-AC_CHECK_DECLS(gsl_interp_steffen,
-               [ gsl_version_old=no                   ],
-               [ gsl_version_old=yes; anywarnings=yes ],
-               [[#include <gsl/gsl_interp.h>]])
-
+               [missing_mandatory=yes; has_gsl=no])
 
 # Since version 0.42, if `libcurl' is installed, CFITSIO will link with it
 # and thus it will be necessary to explicitly link with libcurl also. If it
@@ -273,9 +275,57 @@ AC_CHECK_DECLS(gsl_interp_steffen,
 # script if libcurl isn't found.
 AC_SEARCH_LIBS([curl_global_init], [curl], [], [])
 AC_SEARCH_LIBS([ffopen], [cfitsio], [],
-    [AC_MSG_ERROR([CFITSIO not found, cannot continue.])])
+               [missing_mandatory=yes; has_cfitsio=no])
 AC_SEARCH_LIBS([wcspih], [wcs], [],
-    [AC_MSG_ERROR([WCSLIB not found, cannot continue.])])
+               [missing_mandatory=yes; has_wcslib=no])
+
+
+
+
+
+# If any necessary dependency is missing inform the user and abort.
+AS_IF([test "x$missing_mandatory" = "xyes"],
+      [
+        # Introduction.
+        AS_ECHO([""])
+        AS_ECHO(["The configure script couldn't link with the following 
mandatory dependency(s):"])
+
+        # List missing packages: print the GSL CBLAS message only if GSL is
+        # present. Otherwise, it is just confusing for the users (CBLAS
+        # will be installed with GSL). The CBLAS message is only
+        # interesting if the GSL test has passed.
+        AS_ECHO([""])
+        AS_IF([test "x$has_cmath" = "xno"],
+              [ AS_ECHO([" - C library (math): This may be the cause of all 
other failures."]) ])
+        AS_IF([test "x$has_gsl" = "xno"],
+              [ AS_ECHO([" - GNU Scientific Library (GSL): 
https://www.gnu.org/software/gsl";]) ],
+              [ AS_IF([test "x$has_gslcblas" = "xno"],
+                      [ AS_ECHO([" - The BLAS support of GNU Scientific 
Library (GSL). This should have"])
+                        AS_ECHO(["   been installed along with GSL. Try 
re-installing GSL."]) ]) ])
+        AS_IF([test "x$has_cfitsio" = "xno"],
+              [ AS_ECHO([" - CFITSIO: https://heasarc.gsfc.nasa.gov/fitsio";]) 
])
+        AS_IF([test "x$has_wcslib" = "xno"],
+              [ AS_ECHO([" - WCSLIB: 
http://www.atnf.csiro.au/people/mcalabre/WCS";]) ])
+
+        # Suggestions on fixing the problem.
+        AS_ECHO([""])
+        AS_ECHO(["You can use your package manager for easy and fast 
installation of all the"])
+        AS_ECHO(["mandatory and optional dependencies in one command. See the 
link below:"])
+        AS_ECHO(["  
https://www.gnu.org/software/gnuastro/manual/html_node/Dependencies-from-package-managers.html";])
+        AS_ECHO([""])
+        AS_ECHO(["If you have already installed a dependency (for example in 
\`/install/path'),"])
+        AS_ECHO(["but this script can't link with it, add the path to the 
LDFLAGS, CPPFLAGS and"])
+        AS_ECHO(["LD_LIBRARY_PATH environment variables before running 
configure. For example"])
+        AS_ECHO(["with the following commands (just correct the 
\`/install/path' part)."])
+        AS_ECHO(["  $ export LDFLAGS=\"\$LDFLAGS -L/install/path/lib\""])
+        AS_ECHO(["  $ export CPPFLAGS=\"\$CPPFLAGS -L/install/path/include\""])
+        AS_ECHO(["  $ export 
LD_LIBRARY_PATH=\"\$LD_LIBRARY_PATH:/install/path/lib\""])
+        AS_ECHO(["[TIP] Put these commands in your startup file (for example 
\`~/.bashrc') to"])
+        AS_ECHO(["avoid similar problems later. See the link below to learn 
more:"])
+        AS_ECHO(["  
https://www.gnu.org/software/gnuastro/manual/html_node/Installation-directory.html";])
+        AS_ECHO([""])
+        AC_MSG_ERROR([Mandatory dependency(s) missing, see above.])
+      ])
 
 
 
@@ -284,6 +334,10 @@ AC_SEARCH_LIBS([wcspih], [wcs], [],
 # These are secondary tests for more fine-grained control in libraries that
 # have already been checked. We don't need to add them to the LIBS
 # variable, so we are using AC_CHECK_LIB for these tests.
+AC_CHECK_DECLS(gsl_interp_steffen,
+               [ gsl_version_old=no                   ],
+               [ gsl_version_old=yes; anywarnings=yes ],
+               [[#include <gsl/gsl_interp.h>]])
 
 # If the CFITSIO library has the `fits_is_reentrant' function (it was added
 # since version 3.30 of April 2012).
@@ -324,7 +378,7 @@ AM_CONDITIONAL([COND_HASHELP2MAN], [test "x$has_help2man" = 
"xyes"])
 
 # Check libjpeg:
 AC_SEARCH_LIBS([jpeg_stdio_dest], [jpeg],
-               [has_libjpeg=yes], [has_libjpeg=no])
+               [has_libjpeg=yes], [has_libjpeg=no; missing_optional_lib=yes])
 AS_IF([test "x$has_libjpeg" = "xyes"],
       [AC_DEFINE([HAVE_LIBJPEG], [], [Has libjpeg])],
       [anywarnings=yes])
@@ -336,7 +390,7 @@ AM_CONDITIONAL([COND_HASLIBJPEG], [test "x$has_libjpeg" = 
"xyes"])
 
 # Check libtiff:
 AC_SEARCH_LIBS([TIFFOpen], [tiff],
-               [has_libtiff=yes], [has_libtiff=no])
+               [has_libtiff=yes], [has_libtiff=no; missing_optional_lib=yes])
 AS_IF([test "x$has_libtiff" = "xyes"],
       [AC_DEFINE([HAVE_LIBTIFF], [], [Has libtiff])],
       [anywarnings=yes])
@@ -348,7 +402,7 @@ AM_CONDITIONAL([COND_HASLIBTIFF], [test "x$has_libtiff" = 
"xyes"])
 
 # Check libgit2:
 AC_SEARCH_LIBS([git_libgit2_init], [git2],
-               [has_libgit2=1], [has_libgit2=0])
+               [has_libgit2=1], [has_libgit2=0; missing_optional_lib=yes])
 AC_DEFINE_UNQUOTED([GAL_CONFIG_HAVE_LIBGIT2], [$has_libgit2],
                    [libgit2 is installed on the system])
 AS_IF([test "x$has_libgit2" = "x1"], [], [anywarnings=yes])
@@ -843,6 +897,14 @@ AS_IF([test x$enable_guide_message = xyes],
                AS_ECHO(["        $ info gnuastro \"Installation directory\""])
                AS_ECHO([]) ])
 
+        # Notice on obtaining all the packages.
+        AS_ECHO(["  You can use your package manager for easy and fast 
installation of all"])
+        AS_ECHO(["  the mandatory and optional dependencies in one command. 
See the link"])
+        AS_ECHO(["  below:"])
+        AS_ECHO(["    
https://www.gnu.org/software/gnuastro/manual/html_node/Dependencies-from-package-managers.html";])
+        AS_ECHO([])
+
+        # Inform the user on skipped tests.
         AS_ECHO(["  All checks related to the warning(s) above will be 
skipped."])
         AS_ECHO([])
       ]
diff --git a/developer-build b/developer-build
index 63b622f..220ea47 100755
--- a/developer-build
+++ b/developer-build
@@ -435,25 +435,18 @@ fi
 
 
 
-# If requested, also run 'sudo make install'.
-if [ x$install = x1 ]; then
-    sudo make install -kj$jobs
-fi
-
-
-
-
-
-# Make the tarball and PDF for distribution.
+# Make the tarball and PDF for distribution. Then put a copy of the PDF in
+# the top build directory for easy usage later.
 if [ x$dist = x1 ]; then
     make dist-lzip pdf
+    cp doc/gnuastro.pdf ./
 fi
 
 
 
 
 
-# Build a tarball, and upload it to the requested server.
+# Upload the tarball to the requested server.
 if [ x$upload = x1 ]; then
 
     # Get the base package name, and use it to make a generic tarball
@@ -464,5 +457,14 @@ if [ x$upload = x1 ]; then
     mv *.tar.lz $base"-latest.tar.lz"
 
     # Copy the files to the given URL (must include folders).
-    scp $base"-latest.tar.lz" doc/$base.pdf $url
+    scp $base"-latest.tar.lz" $base.pdf $url
+fi
+
+
+
+
+
+# If requested, run 'sudo make install'.
+if [ x$install = x1 ]; then
+    sudo make install -kj$jobs
 fi
diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index 6ca7fbe..196106b 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -1897,13 +1897,14 @@ should do all the work on a finer pixel grid. In the 
end he can
 re-sample the result to the initially desired grid size.
 
 @item
-Convolve the image with a PSF image that is over-sampled to the same
-value as the mock image. Since he wants to finish in a reasonable time
-and the PSF kernel will be very large due to oversampling, he has to
-use frequency domain convolution which has the side effect of dimming
-the edges of the image. So in the first step above he also has to
-build the image to be larger by at least half the width of the PSF
-convolution kernel on each edge.
+@cindex PSF
+Convolve the image with a point spread function (PSF, see @ref{PSF}) that
+is over-sampled to the same resolution as the mock image. Since he wants to
+finish in a reasonable time and the PSF kernel will be very large due to
+oversampling, he has to use frequency domain convolution which has the side
+effect of dimming the edges of the image. So in the first step above he
+also has to build the image to be larger by at least half the width of the
+PSF convolution kernel on each edge.
 
 @item
 With all the transformations complete, the image should be re-sampled
@@ -1960,7 +1961,7 @@ $ astmkprof -P
 @noindent
 In Gnuastro, column counting starts from 1, so the columns are ordered such
 that the first column (number 1) can be an ID he specifies for each object
-(and MakeProfiles ignores), each subsequent column is used used for another
+(and MakeProfiles ignores), each subsequent column is used for another
 property of the profile. It is also possible to use column names for the
 values of these options and change these defaults, but Sufi preferred to
 stick to the defaults. Fortunately MakeProfiles has the capability to also
@@ -3222,9 +3223,9 @@ two extensions/steps ahead (in the first 
@code{HOLES-FILLED}), you can see
 that during the process of finding false pseudo-detections, too many holes
 have been filled: see how the many of the brighter galaxies are connected?
 
-Try looking two extensions ahead (in @code{PSEUDOS-FOR-SN}), you can see
-that there aren't too many pseudo-detections because of all those extended
-filled holes. If you look closely, you can see the number of
+Try looking two extensions ahead (in the first @code{PSEUDOS-FOR-SN}), you
+can see that there aren't too many pseudo-detections because of all those
+extended filled holes. If you look closely, you can see the number of
 pseudo-detections in the result NoiseChisel prints (around 4000). This is
 another side-effect of correlated noise. To address it, we should slightly
 increase the pseudo-detection threhold (@option{--dthresh}, run with
@@ -3249,15 +3250,20 @@ $ astnoisechisel flat-ir/xdf-f160w.fits 
--kernel=kernel.fits  \
                  --dthresh=0.2 --checkdetection --checksn
 @end example
 
-The output @file{xdf-f160w_detsn_sky.fits} table contains the label and
-signal-to-noise ratio of all the pseudo-detections in
-@code{PSEUDOS-FOR-SN}. You can see the table columns with the first command
-below and get a feeling for its distribution with the second command. We'll
-discuss the two Table and Statistics programs later.
+The output @file{xdf-f160w_detsn.fits} file contains two extensions for the
+pseudo-detections over the undetected (sky) regions and those over
+detections. The first column is the pseudo-detection label which you can
+see in the respective@footnote{The first @code{PSEUDOS-FOR-SN} in
+@file{xdf-f160w_detsn.fits} is for the pseudo-detections over the
+undetected regions and the second is for those over detected regions.}
+@code{PSEUDOS-FOR-SN} extension of @file{xdf-f160w_detcheck.fits}. You can
+see the table columns with the first command below and get a feeling for
+its distribution with the second command. We'll discuss the two Table and
+Statistics programs later.
 
 @example
-$ asttable xdf-f160w_detsn_sky.fits
-$ aststatistics xdf-f160w_detsn_sky.fits -c2
+$ asttable xdf-f160w_detsn.fits
+$ aststatistics xdf-f160w_detsn.fits -c2
 @end example
 
 The correlated noise is again visible in this pseudo-detection
@@ -3267,7 +3273,7 @@ the difference between the three 0.99, 0.95 and 0.90 
quantiles with this
 command:
 
 @example
-$ aststatistics xdf-f160w_detsn_sky.fits -c2                   \
+$ aststatistics xdf-f160w_detsn.fits -c2                        \
                 --quantile=0.99 --quantile=0.95 --quantile=0.90
 @end example
 
@@ -15835,6 +15841,26 @@ a more complete description, see the latter half of 
@ref{Quantifying signal
 in a tile}.
 
 @item
+@option{--blankasforeground}: allows blank pixels to be treated as
+foreground in NoiseChisel's binary operations: the initial erosion
+(@option{--erode}) and opening (@option{--open}) as well as the filling
+holes and opening step for defining pseudo-detections
+(@option{--dthresh}). In the published paper, blank pixels were treated as
+foreground by default. To avoid too many false positive near blank/masked
+regions, blank pixels are now considered to be in the background. This
+option will create the old behavior.
+
+@item
+@option{--skyfracnoblank}: To reduce the bias caused by undetected wings of
+galaxies and stars in the Sky measurements, NoiseChisel only uses tiles
+that have a sufficiently large fraction of undetected pixels. Until now the
+reference for this fraction was the whole tile size. With this option, it
+is now possible to ask for ignoring blank pixels when calculating the
+fraction. This is useful when blank/masked pixels are distributed across
+the image. For more, see the description of this option in @ref{Detection
+options}.
+
+@item
 @option{--detgrowquant}: is used to grow the final true detections until a
 given quantile in the same way that clumps are grown during segmentation
 (compare columns 2 and 3 in Figure 10 of the paper). It replaces the old
@@ -16214,25 +16240,16 @@ quantile is between 0.49 and 0.51 (recall that the 
median's quantile is
 @item --outliersclip=FLT,FLT
 Sigma-clipping parameters for the outlier rejection of the quantile
 threshold. The format of the given values is similar to
-@option{--sigmaclip} below. In NoiseChisel, outlier rejection is used in
-three stages:
-@itemize
-@item
-Identifying the quantile thresholds (@option{--qthresh},
+@option{--sigmaclip} below. In NoiseChisel, outlier rejection on tiles is
+used when identifying the quantile thresholds (@option{--qthresh},
 @option{--noerodequant}, and @option{detgrowquant}).
-@item
-Identifying the first estimate of the Sky and its standard deviation values
-for pseudo-detections (@option{--dthresh}).
-@item
-Identifying the final estimate of the Sky and its standard deviation.
-@end itemize
 
 Outlier rejection is useful when the dataset contains a large and diffuse
 (almost flat within each tile) signal. The flatness of the profile will
 cause it to successfully pass the mean-median quantile difference test, so
 we'll need to use the distribution of successful tiles for removing these
-false positive. For more, see the latter half of @ref{Quantifying signal in
-a tile}.
+false positives. For more, see the latter half of @ref{Quantifying signal
+in a tile}.
 
 @item --outliersigma=FLT
 Multiple of sigma to define an outlier. If this option is given a value of
@@ -16277,6 +16294,18 @@ output will have the same pixel size as the input, but 
with the
 @option{--oneelempertile} option, only one pixel will be used for each tile
 (see @ref{Processing options}).
 
+@item --blankasforeground
+In the erosion and opening steps below, treat blank elements as foreground
+(regions above the threshold). By default, blank elements in the dataset
+are considered to be background, so if a foreground pixel is touching it,
+it will be eroded. This option is irrelevant if the datasets contains no
+blank elements.
+
+When there are many blank elements in the dataset, treating them as
+foreground will systematically erode their regions less, therefore
+systematically creating more false positives. So use this option (when
+blank values are present) with care.
+
 @item -e INT
 @itemx --erode=INT
 @cindex Erosion
@@ -16344,6 +16373,19 @@ them. Once opening is complete, we have @emph{initial} 
detections.
 The class of neighbors used for opening, see @option{--erodengb} for more
 information on acceptable values.
 
+@item --skyfracnoblank
+Ignore blank pixels when estimating the fraction of undetected pixels for
+Sky estimation. NoiseChisel only measures the Sky over the tiles that have
+a sufficiently large fraction of undetected pixels (value given to
+@option{--minskyfrac}). By default this fraction is found by dividing
+number of undetected pixels in a tile by the tile's area. But this default
+behavior ignores the possibility of blank pixels. In situations that
+blank/masked pixels are scattered across the image and if they are large
+enough, all the tiles can fail the @option{--minskyfrac} test, thus not
+allowing NoiseChisel to proceed. With this option, such scenarios can be
+fixed: the denominator of the fraction will be the number of non-blank
+elements in the tile, not the total tile area.
+
 @item -B FLT
 @itemx --minskyfrac=FLT
 Minimum fraction (value between 0 and 1) of Sky (undetected) areas in a
@@ -16399,6 +16441,16 @@ ratio of the resulting `psudo-detections' are used to 
identify true
 vs. false detections. See Section 3.1.5 and Figure 7 in Akhlaghi and
 Ichikawa (2015) for a very complete explanation.
 
+@item --holengb=INT
+The connectivity (defined by the number of neighbors) to fill holes after
+applying @option{--dthresh} (above) to find pseudo-detections. For example
+in a 2D image it must be 4 (the neighbors that are most strongly connected)
+or 8 (all neighbors). The stronger the connectivity, the stronger the hole
+will be enclosed. So setting a value of 8 in a 2D image means that the
+walls of the hole are 4-connected. If standard (near Sky level) values are
+given to @option{--dthresh}, setting @option{--holengb=4}, might fill the
+complete dataset and thus not create enough pseudo-detections.
+
 @item -m INT
 @itemx --snminarea=INT
 The minimum area to calculate the Signal to noise ratio on the
@@ -16410,21 +16462,20 @@ psudo-detection that is smaller than this area. Use
 @option{--detsnhistnbins} to check if this value is reasonable or not.
 
 @item --checksn
-Save the S/N values of the pseudo-detections and dilated detections into
-three files ending with @file{_detsn_sky.XXX}, @file{_detsn_det.XXX}, and
-@file{_detsn_dilated.XXX}. The @file{.XXX} is determined from the
-@option{--tableformat} option (see @ref{Input output options}, for example
-@file{.txt} or @file{.fits}). You can use these to inspect the S/N values
-and their distribution (in combination with the @option{--checkdetection}
-option to see where the pseudo-detections are).  You can use Gnuastro's
-@ref{Statistics} to make a histogram of the distribution or any other
-analysis you would like for better understanding of the distribution (for
-example through a histogram).
-
-With this option, NoiseChisel will abort as soon as the tables are
-created. This allows you to inspect the steps leading to the final quantile
-threshold, this behavior (to abort NoiseChisel) can be disabled with
-@option{--continueaftercheck}.
+Save the S/N values of the pseudo-detections (and possibly grown detections
+if @option{--cleangrowndet} is called) into seprate tables. If
+@option{--tableformat} is a FITS table, each table will be written into a
+separate extension of one file suffixed with @file{_detsn.fits}. If it is
+plain text, a separate file will be made for each table (ending in
+@file{_detsn_sky.txt}, @file{_detsn_det.txt} and
+@file{_detsn_grown.txt}). For more on @option{--tableformat} see @ref{Input
+output options}.
+
+You can use these to inspect the S/N values and their distribution (in
+combination with the @option{--checkdetection} option to see where the
+pseudo-detections are).  You can use Gnuastro's @ref{Statistics} to make a
+histogram of the distribution or any other analysis you would like for
+better understanding of the distribution (for example through a histogram).
 
 @item --minnumfalse=INT
 The minimum number of `pseudo-detections' over the undetected regions to
@@ -17226,17 +17277,21 @@ psudo-detections are found on the input image. You 
can use
 value is reasonable or not.
 
 @item --checksn
-Save the S/N values of the clumps into two files ending with
-@file{_clumpsn_sky.XXX} and @file{_clumpsn_det.XXX}. The @file{.XXX} is
-determined from the @option{--tableformat} option (see @ref{Input output
-options}, for example @file{.txt} or @file{.fits}). You can use these to
-inspect the S/N values and their distribution (in combination with the
-@option{--checksegmentation} option to see where the clumps are). You can
-use Gnuastro's @ref{Statistics} to make a histogram of the distribution
-(ready for plotting in a text file, or a crude ASCII-art demonstration on
-the command-line).
-
-With this option, NoiseChisel will abort as soon as the two tables are
+Save the S/N values of the clumps over the sky and detected regions into
+seprate tables. If @option{--tableformat} is a FITS format, each table will
+be written into a separate extension of one file suffixed with
+@file{_clumpsn.fits}. If it is plain text, a separate file will be made for
+each table (ending in @file{_clumpsn_sky.txt} and
+@file{_clumpsn_det.txt}). For more on @option{--tableformat} see @ref{Input
+output options}.
+
+You can use these tables to inspect the S/N values and their distribution
+(in combination with the @option{--checksegmentation} option to see where
+the clumps are). You can use Gnuastro's @ref{Statistics} to make a
+histogram of the distribution (ready for plotting in a text file, or a
+crude ASCII-art demonstration on the command-line).
+
+With this option, Segment will abort as soon as the two tables are
 created. This allows you to inspect the steps leading to the final S/N
 quantile threshold, this behavior can be disabled with
 @option{--continueaftercheck}.
@@ -30948,24 +31003,25 @@ export XPA_METHOD=local
 
 @cindex Multiextension FITS
 @cindex Opening multiextension FITS
-The FITS definition allows for multiple extensions inside a FITS file,
-each extension can have a completely independent data set inside of
-it. If you ordinarily open a multi-extension FITS file with SAO ds9,
-for example by double clicking on the file or running @command{$ds9
-foo.fits}, SAO ds9 will only show you the first extension. To be able
-to switch between the extensions you have to follow these menus in the
-SAO ds9 window: @clicksequence{File@click{}Open Other@click{}Open
-Multi Ext Cube} and then choose the Multi extension FITS file in your
+The FITS definition allows for multiple extensions inside one FITS file,
+each extension can have a completely independent dataset inside of it. If
+you just double click on a multi-extension FITS file or run @command{$ds9
+foo.fits}, SAO ds9 will only show you the first extension. If you have a
+multi-extension file containing 2D images, one way to load and switch
+between the each 2D extension is to take the following steps in the SAO ds9
+window: @clicksequence{``File''@click{}''Open Other''@click{}''Open Multi
+Ext Cube''} and then choose the Multi extension FITS file in your
 computer's file structure.
 
 @cindex @option{-mecube} (ds9)
 The method above is a little tedious to do every time you want view a
-multi-extension FITS file. Fortunately SAO ds9 also provides command-line
-options that you can use to specify a particular behavior. One of those
-options is @option{-mecube} which opens a FITS image as a multi-extension
-data cube (treating each 2D extension as a slice in a 3D cube). This allows
-you to flip through the extensions easily while keeping all the settings
-similar.
+multi-extension FITS file. A different series of steps is also necessary if
+you the extensions are 3D data cubes. Fortunately SAO ds9 also provides
+command-line options that you can use to specify a particular behavior. One
+of those options is @option{-mecube} which opens a FITS image as a
+multi-extension data cube (treating each 2D extension as a slice in a 3D
+cube). This allows you to flip through the extensions easily while keeping
+all the settings similar.
 
 Try running @command{$ds9 -mecube foo.fits} to see the effect (for example
 on the output of @ref{NoiseChisel}). If the file has multiple extensions, a
@@ -30988,70 +31044,90 @@ same small window as the 2D case above for scrolling 
through the 3D
 slices. We then have to also ask ds9 to match the frames and lock the
 slices, so for example zooming in one, will also zoom the others.
 
-Since opening a multi-extension file (that contains similarly sized
-datasets) differs between a 2D and 3D dataset, we will have to define a
-script to decide which path to go based on the input. After the following
-steps, when clicking on a FITS file in your graphic user interface, ds9
-will open in the respective mode and a shell function will also be
-available to open it on the command-line. Note that the following solution
-assumes you already have Gnuastro installed (in particular @ref{Fits}).
-
-We will use a shell script for the conditional call of ds9. Let's assume
-that you want to put it in @file{BINDIR} (that is in your @file{PATH}
-environment variable, see @ref{Installation directory}). Tip: a good place
-would be @file{~/.local/bin}. Afterwards using your favorite text editor,
-put the following script into a file called
+We can use a script to automatize this process and make work much easier
+(and save a lot of time) when opening any generic 2D or 3D dataset. After
+taking the following steps, when you click on a FITS file in your graphic
+user interface, ds9 will open in the respective 2D or 3D mode when double
+clicking a FITS file on the graphic user interface, and an executable will
+also be available to open ds9 similarly on the command-line. Note that the
+following solution assumes you already have Gnuastro installed (and in
+particular the @ref{Fits} program).
+
+Let's assume that you want to store this script in @file{BINDIR} (that is
+in your @file{PATH} environment variable, see @ref{Installation
+directory}). [Tip: a good place would be @file{~/.local/bin}, just don't
+forget to make sure it is in your @file{PATH}]. Using your favorite text
+editor, put the following script into a file called
 @file{BINDIR/ds9-multi-ext}. You can change the size of the opened ds9
 window by changing the @code{1800x3000} part of the script below.
 
 @example
 #! /bin/bash
 
-# Make sure an input file is given.
+# To allow generic usage, if no input file is given (the `if' below is
+# true), then just open an empty ds9.
 if [ "x$1" == "x" ]; then
-  echo "No input file given."
+    ds9
 else
-  # Make sure we are dealing with a FITS file. We are using shell
-  # redirection here to make sure that nothing is printed in the
-  # terminal (to standard output when we have a FITS file, or to
-  # standard error when we don't). Since we've used redirection,
-  # we'll also have to echo the return value of `astfits'.
-  check=$(astfits $1 -h0 > /dev/null 2>&1; echo $?)
-
-  # If the file was a FITS file, then `check' will be 0.
-  if [ "$check" == "0" ]; then
-
-    # Read the number of dimensions.
-    n0=$(astfits $1 -h0 | awk '$1=="NAXIS"@{print $3@}')
-
-    # Find the number of dimensions.
-    if [ "$n0" == "0" ]; then
-      ndim=$(astfits $1 -h1 | awk '$1=="NAXIS"@{print $3@}')
+    # Make sure we are dealing with a FITS file. We are using shell
+    # redirection here to make sure that nothing is printed in the
+    # terminal (to standard output when we have a FITS file, or to
+    # standard error when we don't). Since we've used redirection,
+    # we'll also have to echo the return value of `astfits'.
+    check=$(astfits $1 -h0 > /dev/null 2>&1; echo $?)
+
+    # If the file was a FITS file, then `check' will be 0.
+    if [ "$check" == "0" ]; then
+
+        # Read the number of dimensions.
+        n0=$(astfits $1 -h0 | awk '$1=="NAXIS"@{print $3@}')
+
+        # Find the number of dimensions.
+        if [ "$n0" == "0" ]; then
+            ndim=$(astfits $1 -h1 | awk '$1=="NAXIS"@{print $3@}')
+        else
+            ndim=$n0
+        fi;
+
+        # Open DS9 based on the number of dimension.
+        if [ "$ndim" = "2" ]; then
+            # 2D multi-extension file: use the "Cube" window to
+            # flip/slide through the extensions.
+            ds9 -zscale -geometry 1800x3000 -mecube $1         \
+                -zoom to fit -wcs degrees
+        else
+            # 3D multi-extension file: The "Cube" window will slide
+            # between the slices of a single extension. To flip
+            # through the extensions (not the slices), press the top
+            # row "frame" button and from the last four buttons of the
+            # bottom row ("first", "previous", "next" and "last") can
+            # be used to switch through the extensions (while keeping
+            # the same slice).
+            ds9 -zscale -geometry 1800x3000 -wcs degrees       \
+                -multiframe $1 -match frame image              \
+                -lock slice image -lock frame image -single    \
+                -zoom to fit
+        fi
     else
-      ndim=$n0
-    fi;
-
-    # Open DS9 based on the number of dimension.
-    if [ "$ndim" = "2" ]; then
-      ds9 -zscale -geometry 1800x3000 -mecube $1 -zoom to fit    \
-          -wcs degrees
-    else
-      ds9 -zscale -geometry 1800x3000 -tile -wcs degrees         \
-          -multiframe $1 -zoom to fit -match frame image         \
-          -lock slice image -lock frame image
-    fi
-  else
-    if [ -f $1 ]; then
-      echo "'$1' isn't a FITS file."
-    else
-      echo "'$1' doesn't exist."
+        if [ -f $1 ]; then
+            echo "'$1' isn't a FITS file."
+        else
+            echo "'$1' doesn't exist."
+        fi
     fi
-  fi
 fi
 @end example
 
-@noindent
-To be able to run it, you have to activate its executable flag with this
+As described above (also in the comments of the script), if you have opened
+a multi-extension 2D dataset (image), the ``Cube'' window can be used to
+slide/flip through each extension. But when the input is a 3D data cube,
+the ``Cube'' window will slide/flip through the slices in each extension (a
+separate 3D dataset). To flip through the extensions (while keeping the
+slice fixed), click the ``frame'' button on the top row of buttons, then
+use the last four buttons of the bottom row ("first", "previous", "next"
+and "last") to change between the extensions.
+
+To run this script, you have to activate its executable flag with this
 command:
 
 @example
@@ -31060,8 +31136,7 @@ $ chmod +x BINDIR/ds9-multi-ext
 
 If @file{BINDIR} is within your system's @file{PATH} environment variable
 (see @ref{Installation directory}), you can now open ds9 conditionally
-(depending on dimensions of the input, as described above) with a command
-like this:
+using the script above with this command:
 
 @example
 $ ds9-multi-ext foo.fits
@@ -31084,7 +31159,7 @@ the @code{Icon} line, and write its address in the 
value.
 [Desktop Entry]
 Type=Application
 Version=1.0
-Name=SAO ds9
+Name=SAO DS9
 Comment=View FITS images
 Terminal=false
 Categories=Graphics;RasterGraphics;2DGraphics;3DGraphics
diff --git a/lib/binary.c b/lib/binary.c
index 3eed8c1..cc53996 100644
--- a/lib/binary.c
+++ b/lib/binary.c
@@ -768,11 +768,15 @@ gal_binary_holes_fill(gal_data_t *input, int 
connectivity, size_t maxsize)
   size_t numholes, *sizes;
   gal_data_t *inv, *tile, *holelabs=NULL;
 
-  /* A small sanity check. */
+  /* Small sanity checks. */
   if( input->type != GAL_TYPE_UINT8 )
     error(EXIT_FAILURE, 0, "%s: input must have `uint8' type, but its "
           "input dataset has `%s' type", __func__,
           gal_type_name(input->type, 1));
+  if(connectivity<1 || connectivity>input->ndim)
+    error(EXIT_FAILURE, 0, "%s: connectivity value %d is not acceptable. "
+          "It has to be between 1 and the number of input's dimensions "
+          "(%zu)", __func__, connectivity, input->ndim);
 
 
   /* Make the inverse image. */



reply via email to

[Prev in Thread] Current Thread [Next in Thread]