help-bash
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Speeding up a find and subsequent grep


From: Leonid Isaev (ifax)
Subject: Re: Speeding up a find and subsequent grep
Date: Sun, 20 Dec 2020 19:51:54 +0000

On Sat, Dec 19, 2020 at 07:04:14AM -0500, Jeffrey Walton wrote:
> I'm working on CentOS 7, x86_64, fully patched. The server has a
> manual wiki installation. Ownership and permissions need to be set
> after an update. I've got a script that does it. The slowest part of
> the script is this:
> 
> # Make Python, PHP and friends executable
> IFS= find "$WIKI_DIR" -type f -print | while read -r file
> do
>     if file -b "${file}" | grep -q -E 'executable|script';
>     then
>         chmod u=rwx,g=rx,o= "${file}"
>     else
>         chmod u=rw,g=r,o= "${file}"
>     fi
> done
> 
> Some of the files have whitespace in their names so I need something
> like 'find -print | while read'.
> 
> When I added the 'file -b' piped to 'grep -E' the script slowed down
> considerably.
> 
> Is there any way to speed up that portion of the script?

The only external (to bash) tool you really need is file(1)... Try this
prototype script (also, see [1]):
-----8<-----
[lisaev@localhost a]$ cat ../q.sh 
shopt -s globstar
shopt -s nullglob

for f in ./**/*; do
        [[ -f "$f" ]] || continue

        t="$(file -b --mime-type "$f")"
        printf "Considering file \"%s\" of type \"%s\"\n" "$f" "$t"

        case "${t#text/x-}" in
                python|php|shellscript)
                        stat -c "%n: %a %F" "$f"
                ;;
                *)
                        printf "%s\n" "do something else"
                ;;
        esac
done
----->8-----
[you can modify it according to your needs, e.g. replace stat(1) and printf
inside case..esac with chmod(1)].

Here is an example run:
-----8<-----
[lisaev@localhost a]$ uname -a
Linux localhost.localdomain 3.10.0-1160.6.1.el7.x86_64 #1 SMP Tue Nov 17 
13:59:11 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

[lisaev@localhost a]$ find . | cat -A
.$
./b$
./b/2 _^I   .sh$
./b/b1$
./b/b1/3 is meh.py$
./c$
./c/e$
./c/e/5.txt$
./c/e/f$
./c/e/f/4.txt$
./1.txt$
./d$
./d/26.jpg$

[lisaev@localhost a]$ bash ../q.sh | cat -A
Considering file "./1.txt" of type "text/x-shellscript"$
./1.txt: 664 regular file$
Considering file "./b/2 _^I   .sh" of type "text/x-shellscript"$
./b/2 _^I   .sh: 664 regular file$
Considering file "./b/b1/3 is meh.py" of type "text/x-python"$
./b/b1/3 is meh.py: 664 regular file$
Considering file "./c/e/5.txt" of type "text/x-php"$
./c/e/5.txt: 664 regular file$
Considering file "./c/e/f/4.txt" of type "text/plain"$
do something else$
Considering file "./d/26.jpg" of type "text/plain"$
do something else$
----->8-----

I purposedly screwed up file extensions (which did not fool file(1), of course)
and inserted spaces and TABs ($'\t') in filenames, as revealed by cat(1).

[1] https://mywiki.wooledge.org/BashPitfalls#for_f_in_.24.28ls_.2A.mp3.29

Hopefully this helps,

-- 
Leonid Isaev



reply via email to

[Prev in Thread] Current Thread [Next in Thread]