Re: Gnuastro 0.17.81 released

[peter teuben]

hi   Mohammad

I grabbed this latest "lz" file and also skimmed over the new manual as well, which looks fantastic.

I ran this on Ubuntu22.04, and it passed all 74 tests in the "make check"

Here are some comments I jottted down while waiting for the compilation to finish:

I'm partial to advertising PGPLOT the way you do. On debian and redhat you get two completely different implementations, mostly because redhat (grrrr) doesn't provide PNG, arguably the only useful format for modern environments. Maybe add a caveat, and advice people to use a native package manager before torturing them with the manual method (which don't have the 64bit patched so I suspect they will fail)

    Bravo on package manager examples. Could you add pgplot to that
    list,  ds9 is :-) It's pgplot5 for ubuntu, and it's pgplot for
    redhat, but one may need to enable the non-free repo?

The manual talks about using "gz" files for the install, yet, your message tells us to grab an "lz" file. Even in your email you need to remind the audience how to untar it.  I had to install lzip, after which this worked for me (after some grumpy language)

tar --lzip -xf gnuastro-0.17.81-1a40.tar.lz

so I was sad this took me a few mins to figure out, whereas arguably it should be 2 seconds. I am getting too old? Does really everybody now know what an lz file is? 

best,

  Peter

On 7/2/22 13:52, Mohammad Akhlaghi wrote:

> Dear
>       all,
>       
>       
>       The 18th official release of Gnuastro is almost ready. To prepare
>       for
>       
>       it, Gnuastro 0.17.81 is announced (81 commits/changes have been
>       made
>       
>       since version 0.17, see [1] for the noteworthy changes).
>       
>       
>       It would be great if you can try building, installing and using
>       this
>       
>       alpha release to report any build-warnings, crashes or generally
>       
>       unexpected behavior. This way, we can be sure that Gnuastro 0.18
>       will
>       
>       be as smooth as possible for everyone.
>       
>       
>       Here is the compressed source and the GPG detached signature for
>       
>       this release. To uncompress Lzip tarballs, see [2]. To check the
>       
>       validity of the tarballs using the GPG detached signature (*.sig)
>       
>       see [3]:
>       
>       
>         https://alpha.gnu.org/gnu/gnuastro/gnuastro-0.17.81-1a40.tar.lz
>       
>        
>       https://alpha.gnu.org/gnu/gnuastro/gnuastro-0.17.81-1a40.tar.lz.sig
>       
>       
>       Here are the SHA1 and SHA256 checksums (other ways to check if the
>       
>       tarball you download is what we distributed). Just note that the
>       
>       SHA256 checksum is base64 encoded, instead of the hexadecimal
>       encoding
>       
>       that most checksum tools default to (see [4] on how to generate
>       it).
>       
>       
>       ca7aa46e998073c804a4a936251bf739b87297cc 
>       gnuastro-0.17.81-1a40.tar.lz
>       
>       lwZrqdQk5mAgfzhebWImFEr2DgWwW+PkzckuetEMOUw 
>       gnuastro-0.17.81-1a40.tar.lz
>       
>       
>       As you see in [1] below, many new features have been added in this
>       
>       release, and many bugs have been fixed. Besides those, I am happy
>       to
>       
>       announce that Gnuastro finally has a logo:
>       
>       
>         https://www.gnu.org/software/gnuastro/gnuastro.svg
>       
>       
>       See [5] For a full description of this logo (taken from the newly
>       
>       added "Logo of Gnuastro" section of the book). This logo is now
>       used
>       
>       on the top Gnuastro webpage and also in the top page of the PDF
>       manual
>       
>       (the bleeding-edge PDF manual: https://akhlaghi.org/gnuastro.pdf).
>       
>       
>       For their direct contribution to this version's source code, I am
>       very
>       
>       grateful to Raul Infante-Sainz (with 12 commits), Sepideh
>       Eskandarlou
>       
>       (6 commits), Jash Shah (1 commit), Elham Saremi (1 commit), Marjan
>       
>       Akbari (1 commit), Pedram Ashofteh Ardakani (1 commit) and S.
>       Zahra
>       
>       Hosseini Shahisavandi (1 commit). I am also grateful to Marjan
>       Akbari,
>       
>       Faezeh Bijarchian, Sepideh Eskandarlou, Sílvia Farras, Raul
>       
>       Infante-Sainz, Teet Kuutma, Jeremy Lim, Juan Miro, Irene Pintos
>       
>       Castro, Ignacio Ruiz Cejudo and Elham Saremi, for their good
>       
>       suggestions or reported bugs that have been implemented in this
>       
>       release.
>       
>       
>       If any of Gnuastro's programs or libraries are useful in your
>       work,
>       
>       please cite _and_ acknowledge them. For citation and
>       acknowledgment
>       
>       guidelines, run the relevant programs with a `--cite' option (it
>       can
>       
>       be different for different programs, so run it for all the
>       programs
>       
>       you use). Citations _and_ acknowledgments are vital for the
>       continued
>       
>       work on Gnuastro, so please don't forget to support us by doing
>       so.
>       
>       
>       This tarball was bootstrapped (created) with the tools below. Note
>       
>       that you don't need these to build Gnuastro from the tarball,
>       these
>       
>       are the tools that were used to make the tarball itself. They are
>       only
>       
>       mentioned here to be able to reproduce/recreate this tarball
>       later.
>       
>         Texinfo 6.8
>       
>         Autoconf 2.71
>       
>         Automake 1.16.4
>       
>         Help2man 1.49.2
>       
>         ImageMagick 7.1.0-39
>       
>         Gnulib v0.1-5265-gfc555c0a4b
>       
>         Autoconf archives v2022.02.11-15-g066994d
>       
>       
>       The dependencies to build Gnuastro from this tarball on your
>       system
>       
>       are described here:
>       
>        
>       https://www.gnu.org/s/gnuastro/manual/html_node/Dependencies.html
>       
>       
>       Best wishes,
>       
>       Mohammad
>       
>       
>       --
>       
>       Staff Researcher
>       
>       Centro de Estudios de Física del Cosmos de Aragón (CEFCA),
>       
>       Plaza San Juan 1, Planta 2, Teruel 44001, Spain
>       
>       
>       
>       
>       
>       
>       [1] Noteworthy changes since Gnuastro 0.17:
>       
>       
>         Book:
>       
>          - New "Measuring the dataset limits" section has been added in
>       second
>       
>            tutorial. Using the catalogs produced during the analysis, it
>       shows
>       
>            how to derive a dataset's magnitude limit, completeness limit
>       
>            (crudely: without simulations), and surface brightness limit.
>       All are
>       
>            important measures in any scientific analysis! This section
>       was
>       
>            written with the help of S. Zahra Hosseini Shahisavandi and
>       Sepideh
>       
>            Eskandarlou.
>       
>       
>         Arithmetic:
>       
>          - New operators (also available in Table's column arithmetic):
>       
>            - stitch: connect any number of input images along the given
>       dimension
>       
>              (for example to stitch the images of separate amplifiers of
>       a CCD
>       
>              into one image).
>       
>            - noblank: Remove blank elements from the input and return a
>       1D output
>       
>              (irrespective of how many dimensions the input had).
>       
>            - jy-to-counts: convert Janskys to counts using an
>       AB-magnitude-based
>       
>              Zeropoint (the inverse operator 'counts-to-jy' already
>       existed).
>       
>            - mag-to-jy: convert AB magnitudes to Janskys.
>       
>            - jy-to-mag: convert Janskys to AB magnitudes.
>       
>            - load-col-%-from-%-hdu-%: Load a certain column into the
>       operands
>       
>              stack. The name or number of the column is the first '%'.
>       The file
>       
>              name hosting that column is the second '%', and HDU (if the
>       file is
>       
>              a FITS file), is identified by third '%'. This can be used
>       to load
>       
>              columns outside the main table in Table's column arithmetic
>       or by
>       
>              operators that need columns as input parameters (like the
>       new
>       
>              'random-from-hist' operator).
>       
>            - random-from-hist-raw: return a set of random points based
>       on a
>       
>              custom distribution defined by its histogram. See the
>       description
>       
>              and complete examples in the book for more.
>       
>            - random-from-hist: Similar to 'random-from-hist-raw', but
>       within each
>       
>              bin, the final returned value is itself the result of a
>       uniform
>       
>              sampling (to make the final distribution less descrete).
>       See the
>       
>              example in the book for selecting random star magnitudes
>       based on
>       
>              Gaia's observed magnitude distribution.
>       
>          - New options to add metadata (standard FITS keywords) to your
>       output
>       
>            for easy understanding of the data in the future, or by your
>       
>            colleagues if you share the output). Until now, to have this
>       feature,
>       
>            you needed to call Fits on the output of Arithmetic (for
>       setting the
>       
>            HDU name to "my-sum" for example):
>       
>                astarithmetic img-a.fits img-b.fits + --output=sum.fits
>       
>                astfits sum.fits --write=EXTNAME,"my-sum"
>       
>            But with the following commands, you can add important
>       metadata to the
>       
>            output of Arithmetic in the same command and simplify your
>       scripts:
>       
>              --metaname: Metadata name of the output (FITS keyword:
>       'EXTNAME').
>       
>              --metaunit: Unit of the output's pixels (FITS keyword:
>       'BUNIT').
>       
>              --metacomment: Description of the data (FITS keyword:
>       'COMMENT').
>       
>            Like the following for the example above:
>       
>                astarithmetic img-a.fits img-b.fits + --metaname="my-sum"
>       \
>       
>                              --output=sum.fits
>       
>       
>       
>         Crop:
>       
>          --oneelemstdout: when a crop has a single pixel and this option
>       is
>       
>            called, the single pixel's value will be printed on the
>       standard
>       
>            output instead of creating a FITS file. This option can be
>       useful in
>       
>            labeled images like detection maps (when you want to see what
>       label a
>       
>            certain coordiante/pixel corresponds to). This option was
>       suggested by
>       
>            Raul Infante-Sainz.
>       
>       
>         Fits:
>       
>          - When no option is specified (thus printing the HDU metadata
>       in the
>       
>            given file), the units of the HDUs are also printed as a new
>       
>            column. For tables, it will be blank ('n/a'); because tables
>       have
>       
>            separate units on each column (use 'asttable table.fits -i').
>       
>          --copykeys: can now take any number of keyword names also. For
>       example
>       
>            '--copykeys=KEYNAME1,KEYNAME2,KEYNAME3'. Until now, it was
>       only
>       
>            possible to give this option a range of keyword positions
>       (for example
>       
>            '--copykeys=10:13' to copy the 10th to 13th keywords
>       (inclusive). But
>       
>            now, you can also select non-contiguous set of keywords to
>       copy from
>       
>            one HDU into another HDU (possibly in another file). This was
>       
>            suggested by Juan Antonio Fernández Ontiveros and implemented
>       with the
>       
>            help of Jash Shah.
>       
>       
>         MakeProfiles:
>       
>          - It is now possible to insert a custom image as a new (10th)
>       
>            'custom-img' profile. This is useful when you have no-noise
>       FITS
>       
>            images from numerical simulations for example, and you want
>       to insert
>       
>            those at a certain location within a larger image (while
>       optionally
>       
>            adding existing Sersic, point or other profiles in the same
>       
>            command). The magnitude of the inserted image can be set in
>       the
>       
>            magnitude column of the input catalog. This can be disabled
>       (to use
>       
>            the original image pixel values) using the '--mcolnocustimg'
>       option.
>       
>         --mcolnocustprof: new option to disable scaling of custom radial
>       profiles
>       
>           to required magnitude when building a 2D image from them.
>       
>       
>         Query:
>       
>          - Short name for Gaia's DR3 dataset has been added for both
>       ESA's Gaia
>       
>            database and VizieR, for example with the following commands:
>       
>                astquery gaia   --dataset=dr3     --center=1.234,5.678
>       --radius=0.01
>       
>                astquery vizier --dataset=gaiadr3 --center=1.234,5.678
>       --radius=0.01
>       
>       
>         astscript-radial-profile:
>       
>          --zeroisnotblank: new option to account for zero-valued pixels
>       in the
>       
>            profile measurement. By default, during the internal cropping
>       phase,
>       
>            zero-valued pixels are set to blank (and thus not considered
>       during
>       
>            the profile measurement). When this option is called, it is
>       directly
>       
>            passed to the internal cropping command ('astcrop' has an
>       option with
>       
>            the same name). This was suggested by Ignacio Ruiz Cejudo and
>       
>            implemented by Sepideh Eskandarlou.
>       
>       
>         Library:
>       
>          - GAL_ARITHMETIC_OP_UNIQUE: to find unique elements in dataset.
>       
>          - GAL_ARITHMETIC_OP_NOBLANK: to remove blank elements from
>       dataset.
>       
>          - GAL_ARITHMETIC_OP_STITCH: to stitch multiple datasets.
>       
>          - GAL_ARITHMETIC_OP_RANDOM_FROM_HIST: the new
>       'random-from-hist' operator.
>       
>          - GAL_ARITHMETIC_OP_RANDOM_FROM_HIST_RAW: the new
>       'random-from-hist-raw' op.
>       
>          - gal_arithmetic_load_col: Low-level function for parsing the
>       string of
>       
>            the newly added 'load-col' operator, mentioned in the
>       Arithmetic
>       
>            section above.
>       
>          - gal_statistics_std_from_sums: return the standard deviation
>       using the
>       
>            sum, sum of squares and number of a distribution.
>       
>       
>       ** Removed features
>       
>       
>         Nothing has been removed in this alpha release.
>       
>       
>       ** Changed features
>       
>       
>         MakeCatalog:
>       
>          --std: measures the standard deviation of the object/clump
>       pixels of the
>       
>                 values image. Until now, there was no column to
>       calculate this,
>       
>                 and the '--std' column returned the _sky_ standard
>       deviation over
>       
>                 the object/clump labels (from the standard deviation
>       image,
>       
>                 ignoring the values image). But that was confusing,
>       especially
>       
>                 since we have columns like '--mean' or '--median' which
>       only use
>       
>                 the values image (see bug #62685).
>       
>          --skystd: new name for old '--std', see description of '--std'
>       above.
>       
>          --sfmagnsigma: default value changed to 3 (more commonly used).
>       
>          --sfmagarea: default value set to 100 arcsec^2 (more commonly
>       used).
>       
>       
>         MakeProfiles:
>       
>          - The string identifier for custom radial profiles is now
>       called
>       
>            'custom-prof' (until now, it was called 'custom'). This was
>       necessary
>       
>            because of the new custom image feature that has been added.
>       The code
>       
>            for this custom radial profile function (8) hasn't changed.
>       
>          - After constructing a custom radial profile (the 'custom-prof'
>       
>            profile), MakeProfiles will scale it to have the desired
>       total
>       
>            magnitude in the input catalog of all profiles. Until now,
>       the 2D
>       
>            images created from a custom radial profile wouldn't be
>       touched and
>       
>            the pixel values would exactly come from the custom radial
>       profile
>       
>            (ignoring the magnitude column of the input catalog). To have
>       the old
>       
>            behavior, use the new '--mcolnocustprof' option. This new
>       
>            functionality was proposed by Elham Saremi.
>       
>       
>         Table:
>       
>          --catcolumnfile: the '-N' suffix (where 'N' is a counter for
>       the files
>       
>            with concatenated, or appended, column) is only added if a
>       column with
>       
>            a identical column name (not case sensitive) existed in the
>       
>            table. Until now, this suffix would be added to all the newly
>       
>            concatenated columns (recall that the suffix addition can be
>       disabled
>       
>            in general with the '--catcolumnrawname' option).
>       
>          --rowrange: new name for the old '--rowlimit'. This option
>       takes the
>       
>            range of desired rows based on their position (for example
>       
>            '--rowrange=2,5' will only print rows 2, 3, 4 and 5 of the
>       table). But
>       
>            the old name was very confusing and would not directly convey
>       this
>       
>            behavior.
>       
>       
>       ** Bugs fixed
>       
>         bug #62216: MakeProfiles crash when a 3D cube is requested and
>       input
>       
>                     catalog is from a pipe. Found by Irene Pintos
>       Castro.
>       
>         bug #62220: astscript-psf-stamp crashes when coordinate not in
>       image.
>       
>                     Found and fixed by Sepideh Eskandarlou.
>       
>         bug #62250: Convolve crashes when doing PSF matching. Found with
>       help of
>       
>                     Raul Infante-Sainz.
>       
>         bug #62252: Radial profile giving one larger radius in high
>       axis-ratio
>       
>                     profile. Found and fixed by Sepideh Eskandarlou.
>       
>         bug #62253: Arithmetic: segmentation fault when invalid input
>       given for
>       
>                     'ra-to-degree' or 'deg-to-degree' operators. Found
>       by Pedram
>       
>                     Ashofte Ardakani.
>       
>         bug #62305: Not reading the 'MemFree' keyword in '/proc/meminfo'
>       to find
>       
>                     the available RAM (and thus printing an annoying
>       
>                     warning). Reported by Juan Miro.
>       
>         bug #62309: MakeCatalog segmentation fault when clumps image
>       given as
>       
>                     main input (objects should be given). Reported by
>       Sepideh
>       
>                     Eskandarlou.
>       
>         bug #62429: Segment prints one line status-progress line when
>       called with
>       
>                     '--quiet'. Reported by Teet Kuutma.
>       
>         bug #62548: Table doesn't recognize plain-text string column
>       when defined
>       
>                     width is larger and it is the last column. Found
>       with the
>       
>                     help of Sepideh Eskandarlou.
>       
>         bug #62563: Some checks of installed scripts get skipped on
>       macOS. Found
>       
>                     and fixed by Raul Infante-Sainz.
>       
>         bug #62564: Arithmetic not parsing numbers in scientific
>       notation (for
>       
>                     example '1e5'). Found by Elham Saremi.
>       
>         bug #62590: Failure to build on macOS when building in debugging
>       mode
>       
>                     ('./configure --enable-debug'). Found by Raul
>       Infante-Sainz.
>       
>         bug #62597: Arithmetic not writing single-valued output into a
>       file when
>       
>                     called with '--output'. Reported by Raul
>       Infante-Sainz.
>       
>         bug #62636: Table ignoring '--range' when '--head' is also
>       
>                     called. Reported by Sepideh Eskandarlou.
>       
>         bug #62662: Table crashes when column arithmetic is applied
>       after adding
>       
>                     rows from another file (with '--catrowfile').
>       
>         bug #62674: Column arithmetic can't use columns from
>       --catcolumnfile.
>       
>         bug #62678: Radial profile script using values image instead of
>       STD image
>       
>                     when '--instd' not used. Found and fixed by Raul
>       Infante-Sainz.
>       
>         bug #62679: MakeProfiles kernel only normalized when zeropoint
>       is
>       
>                     zero. Found by Raul Infante-Sainz.
>       
>         bug #62680: MakeProfiles using WCS-options when --background is
>       given and
>       
>                     the background doesn't have WCS. Found with the help
>       of Raul
>       
>                     Infante-Sainz.
>       
>         bug #62683: Arithmetic's 'set-' operator not working after the
>       'makenew'
>       
>                     operator. Found by Raul Infante-Sainz.
>       
>         bug #62685: MakeCatalog's '--std' option is wrongly interpreted
>       as the
>       
>                     standard deviation of the values image, while it is
>       actually
>       
>                     the root mean square of the Sky standard deviation
>       
>                     pixels. Found by Raul Infante-Sainz.
>       
>         bug #62694: Radial profile script overestimates the surface
>       brightness
>       
>                     error. Found and fixed with the help of Raul
>       Infante-Sainz.
>       
>       
>       
>       
>       
>       
>       [2] Lzip has better compression ratio and archival features
>       compared
>       
>       to the `.gz' or `.xz' formats. Therefore Gnuastro's alpha/test
>       
>       releases are only in this format, but for historical reasons we
>       also
>       
>       include `.gz' tarballs in the official releases. If you don't have
>       
>       Lzip (you can check with `lzip --version' command), download and
>       
>       install it from its webpage:
>       
>       
>         https://www.nongnu.org/lzip/lzip.html
>       
>       
>       If Lzip is present and you use GNU Tar, then the single command
>       below
>       
>       should uncompress and un-pack the tarball:
>       
>       
>         $ tar xf gnuastro-0.17.81-1a40.tar.lz
>       
>       
>       If the command above doesn't work, you have to un-compress and
>       un-pack
>       
>       it with two separate commands (or use a pipe to feed the output of
>       the
>       
>       first into the second: `lzip -cd gnuastro-0.17.81-1a40.tar.lz |
>       tar -xf -'):
>       
>       
>         $ lzip -d gnuastro-0.17.81-1a40.tar.lz
>       
>         $ tar xf gnuastro-0.17.81-1a40.tar
>       
>       
>       
>       
>       
>       
>       [3] Use a .sig file to verify that the corresponding file (without
>       the
>       
>       .sig suffix) is intact.  First, be sure to download both the .sig
>       file
>       
>       and the corresponding tarball.  Then, run a command like this:
>       
>       
>         gpg --verify gnuastro-0.17.81-1a40.tar.lz.sig
>       
>       
>       If that command fails because you don't have the required public
>       key,
>       
>       or that public key has expired, try the following commands to
>       update
>       
>       or refresh it, and then rerun the 'gpg --verify' command.
>       
>       
>         gpg --keyserver hkp://pgp.rediris.es --recv-keys
>       71E899012D174B66
>       
>       
>       
>       
>       
>       
>       [4] To get the base64 SHA256 checksum, you can use the command
>       below
>       
>       (assuming you already have the 'openssl' command-line program).
>       
>       
>         cat gnuastro-0.17.81-1a40.tar.lz \
>       
>             | openssl dgst -binary -sha256 | openssl base64 -A
>       
>       
>       
>       
>       
>       
>       [5] Gnuastro’s logo is an abstract image of a barred spiral galaxy
>       
>       (https://en.wikipedia.  org/wiki/Barred_spiral_galaxy). The galaxy
>       is
>       
>       vertically cut in half: on the left side, the beauty of a
>       contiguous
>       
>       galaxy image is visible. But on the right, the image gets pixe-
>       lated,
>       
>       and we only see the parts that are within the pixels. The pixels
>       that
>       
>       are more near to the center of the galaxy (which is brighter) are
>       also
>       
>       larger. But as we follow the spiral arms (and get more distant
>       from
>       
>       the center), the pixels get smaller (signifying less signal).
>       
>       
>       This sharp distinction between the contiguous and pixelated view
>       of
>       
>       the galaxy signifies the main struggle in science: in the “real”
>       
>       world, objects aren’t pixelated or discrete and have no
>       
>       noise. However, when we observe nature, we are confined and
>       
>       constrained by the resolution of our data collection (CCD imager
>       in
>       
>       this case).
>       
>       
>       On the other hand, we read English text from the left and progress
>       
>       towards the right.  This defins the positioning of the “real” and
>       
>       observed halfs of the galaxy: the no-noised and contiguous half
>       (on
>       
>       the left) passes through our observing tools and becomes pixelated
>       and
>       
>       noisy half (on the right). It is the job of scientific software
>       like
>       
>       Gnuastro to help interpret the underlying mechanisms of the “real”
>       
>       universe from the pixelated and noisy data.
>       
>       
>       Gnuastro’s logo was designed by Marjan Akbari. The concept behind
>       it
>       
>       was created after several design iterations with Mohammad
>       Akhlaghi.
>       
>       
>