[bug #61229] "unable to resolve host address" for URI's when from a long list

[anonymous]

URL:
  <https://savannah.gnu.org/bugs/?61229>

                 Summary: "unable to resolve host address" for URI's when from
a long list
                 Project: GNU Wget
            Submitted by: None
            Submitted on: Sun 26 Sep 2021 10:48:58 PM UTC
                Category: None
                Severity: 3 - Normal
                Priority: 5 - Normal
                  Status: None
                 Privacy: Public
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
                 Release: None
         Discussion Lock: Any
        Operating System: Mac OS
         Reproducibility: Every Time
           Fixed Release: None
         Planned Release: None
              Regression: None
           Work Required: None
          Patch Included: None

    _______________________________________________________

Details:

Hello,

When I run wget with a long list of URIs from a file, after a couple of
hundered entries the URI's stop to resolve (the "wget: unable to resolve host
address <...>" lines in the log), although they still do if supplied in
shorter lists. It's not an issue of the timeout (it is set in the example to
10 seconds just not to wait 15 minutes for each single URI), nor the URI's
themselves, which are valid.


wget \
--debug \
--spider \
--no-directories \
--timeout=10 \
--tries=2 \
--input-file="/Users/comp/wget/url-list.txt" \
--output-file="/Users/comp/wget/test-output.txt"


The input and output files are attached.

I'm on MacOS 11.5.2, M1

wget version:
GNU Wget 1.21.2 built on darwin20.6.0.

-cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls
+ntlm +opie +psl +ssl/openssl

Wgetrc:
    /opt/homebrew/etc/wgetrc (system)
Locale:
    /opt/homebrew/Cellar/wget/1.21.2/share/locale
Compile:
    clang -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/opt/homebrew/etc/wgetrc"
    -DLOCALEDIR="/opt/homebrew/Cellar/wget/1.21.2/share/locale" -I.
    -I../lib -I../lib -I/opt/homebrew/opt/openssl@1.1/include
    -I/opt/homebrew/Cellar/pcre/8.45/include
    -I/opt/homebrew/Cellar/zlib/1.2.11/include
    -I/opt/homebrew/Cellar/icu4c/69.1/include
    -I/opt/homebrew/Cellar/libpsl/0.21.1_3/include -DNDEBUG -g -O2
Link:
    clang -I/opt/homebrew/Cellar/pcre/8.45/include
    -I/opt/homebrew/Cellar/zlib/1.2.11/include
    -I/opt/homebrew/Cellar/icu4c/69.1/include
    -I/opt/homebrew/Cellar/libpsl/0.21.1_3/include -DNDEBUG -g -O2
    -L/opt/homebrew/Cellar/pcre/8.45/lib -lpcre -lidn2
    -L/opt/homebrew/opt/openssl@1.1/lib -lssl -lcrypto -ldl
    -L/opt/homebrew/Cellar/zlib/1.2.11/lib -lz
    -L/opt/homebrew/Cellar/libpsl/0.21.1_3/lib -lpsl ftp-opie.o
    openssl.o http-ntlm.o ../lib/libgnu.a -liconv -lintl -Wl,-framework
    -Wl,CoreFoundation -lunistring




    _______________________________________________________

File Attachments:


-------------------------------------------------------
Date: Sun 26 Sep 2021 10:48:58 PM UTC  Name: test-output.txt  Size: 384KiB  
By: None

<http://savannah.gnu.org/bugs/download.php?file_id=51974>
-------------------------------------------------------
Date: Sun 26 Sep 2021 10:48:58 PM UTC  Name: url-list.txt  Size: 13KiB   By:
None

<http://savannah.gnu.org/bugs/download.php?file_id=51975>

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?61229>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/