[bug#54302] [PATCH] nls: Implement translation thresholds.

[Ludovic Courtès]

Hi!

Julien Lepiller <julien@lepiller.eu> skribis:

> manual and cookbook: only include new languages when they reach 10%
> completion. Remove languages when they fall below 5%.

SGTM (or even 15%/10%).

> website (unrelated to this repo, but still important): only include new
> languages when they reach 80% completion. Remove languages when they
> fall below 60%. The reason for the higher threshold is that the website
> acts as some sort of advertisement, so we want a higher quality than
> half English half translated.

SGTM.

>>From 37071410629a7d70c9b4e4f673f2c625d3ed4b47 Mon Sep 17 00:00:00 2001
> Message-Id: 
> <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
> From: Julien Lepiller <julien@lepiller.eu>
> Date: Tue, 8 Mar 2022 13:14:58 +0100
> Subject: [PATCH 1/3] doc: Document inclusion requirements for new
>  translations.
>
> * doc/contributing.texi (Translating Guix)[Conditions for Inclusion]:
> New section.

[...]

> +There are no conditions for adding new translations of the guix and
> +guix-packages components, other than they need at least one translated

@code{guix} and @code{guix-packages}

> +Given that the website is dedicated to new users, we want its translation

“web site” (two words).

> +target.  Everytime we synchronise translations, developpers need to

“developers” and (if you feel overseas-inclined) “synchronize”.

> +When a language is included, it may be removed in the future, if it stays

Remove comma.

>>From 5cbb70ebcbf141cd05fa60bf0bfa806125a56381 Mon Sep 17 00:00:00 2001
> Message-Id: 
> <5cbb70ebcbf141cd05fa60bf0bfa806125a56381.1646763369.git.julien@lepiller.eu>
> In-Reply-To: 
> <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
> References: 
> <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
> From: Julien Lepiller <julien@lepiller.eu>
> Date: Tue, 8 Mar 2022 19:11:38 +0100
> Subject: [PATCH 2/3] maint: Implement translation thresholds.
>
> Do not download new translations for the cookbook and the manual when
> they are below 10% completion, and remove existing translations when
> they fall below 5%.
>
> * Makefile.am (download-po): Implement translation thresholds.
> ---
>  Makefile.am | 20 +++++++++++++++++---
>  1 file changed, 17 insertions(+), 3 deletions(-)
>
> diff --git a/Makefile.am b/Makefile.am
> index 8850c4562c..164804d96a 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -1066,21 +1066,35 @@ WEBLATE_REPO = 
> https://framagit.org/tyreunom/guix-translations
>  # form.
>  download-po:
>       dir=$$(mktemp -d); \
> -     git clone --depth 1 "$(WEBLATE_REPO)" "$$dir/translations"; \
> +     git clone --depth 1 "$(WEBLATE_REPO)" "$$dir/translations" && \
>       for domain in po/doc po/guix po/packages; do \
>           for po in "$$dir/translations/$$domain"/*.po; do \
>               translated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 
> | cut -f1 -d' '); \
> +             untranslated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 
> 2>&1 | cut -f4 -d' '); \
> +                     untranslated=$${untranslated:-0}; \
> +                     total=$$(($$translated+$$untranslated)); \
>               target=$$(basename "$$po"); \
>               target="$$domain/$$target"; \
> -             if msgfmt -c "$$po" && [ "$$translated" != "0" ]; then \
> +             msgfmt -c "$$po"; \
> +             if msgfmt -c "$$po" && [ "$$translated" != "0" ] && ([ 
> "$$domain" != "po/doc" ] || [ "$$translated" -gt $$(($$total/10)) ] || [ -f 
> $$target ]); then \
>                   msgfilter --no-wrap -i "$$po" cat > "$$po".tmp; \
>                   mv "$$po".tmp "$$target"; \
>                   echo "copied $$target."; \
>               else \
> -                 echo "WARN: $$target ($$translated translated messages) was 
> not added/updated."; \
> +                 echo "WARN: $$target ($$translated translated messages 
> ($$((translated/total*100))%)) was not added/updated."; \
>               fi; \
>           done; \
>       done; \
> +     for po in po/doc/*.po; do \
> +         translated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | 
> cut -f1 -d' '); \
> +         untranslated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | 
> cut -f4 -d' '); \
> +             untranslated=$${untranslated:-0}; \
> +             total=$$(($$translated + $$untranslated)); \
> +             if [ "$$translated" -lt "$$(($$total/20))" ]; then \
> +                 echo "WARN: $$po was removed because it is below the 5% 
> threshold: $$((translated/total*100))%"; \
> +                     rm $$po; \
> +             fi; \
> +     done; \

LGTM, but this is getting a bit hairy.  :-)

No concrete suggestions, but it would be great if we could somehow split
it and/or move it to a script in build-aux/ (that’d avoid double dollar
signs) and/or write it in Scheme.  Future work…

> From 726ef94f91d5dab25c3ccfb2986dcba6d39a4ab8 Mon Sep 17 00:00:00 2001
> Message-Id: 
> <726ef94f91d5dab25c3ccfb2986dcba6d39a4ab8.1646763369.git.julien@lepiller.eu>
> In-Reply-To: 
> <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
> References: 
> <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
> From: Julien Lepiller <julien@lepiller.eu>
> Date: Tue, 8 Mar 2022 19:14:47 +0100
> Subject: [PATCH 3/3] nls: Enforce translation thresholds.
>
> * po/doc/guix-cookbook.es.po: Remove file.
> * po/doc/guix-cookbook.fa.po: Remove file.
> * po/doc/guix-cookbook.fi.po: Remove file.
> * po/doc/guix-cookbook.uk.po: Remove file.
> * po/doc/local.mk: Remove them.
> * doc/local.mk: Remove them.

[...]

>  # If adding a language, update the following variables, and info_TEXINFOS.
>  MANUAL_LANGUAGES = de es fa fi fr it ko pt_BR ru sk zh_CN
> -COOKBOOK_LANGUAGES = de es fa fi fr ko pt_BR ru sk uk zh_Hans
> +COOKBOOK_LANGUAGES = de fr ko pt_BR ru sk zh_Hans

Should we also remove fa, fi, it, ko, and sk from MANUAL_LANGUAGES and
info_TEXINFOS?

  https://translate.fedoraproject.org/projects/guix/documentation-manual/

Thank you!

Ludo’.