locale-util: suppress non-UTF-8 locales when enumerating them

Let's hide non-UTF-8 locales by default. It's 2019 after all.

Let's add an undocumented env var to reenable listing them though.

This should substantially shorten the list of choices we offer users,
and only show realistic choices.

note that only firstboot and localectl make use of this information, and
both allow configuration of values outside of these lists, hence all
this change does is hide legacy options, but they are still available if
you know what you do, and that's how it should be.

docs/ENVIRONMENT.md

@@ -222,3 +222,9 @@ systemd-remount-fs:
   directory is remounted writable. This is primarily used by
   systemd-gpt-auto-generator to ensure the root partition is mounted writable
   in accordance to the GPT partition flags.
+
+systemd-firstboot and localectl:
+
+* `SYSTEMD_LIST_NON_UTF8_LOCALES=1` – if set non-UTF-8 locales are listed among
+  the installed ones. By default non-UTF-8 locales are suppressed from the
+  selection, since we are living in the 21st century.

src/basic/locale-util.c

@@ -211,6 +211,25 @@ int get_locales(char ***ret) {
         if (!l)
                 return -ENOMEM;
 
+        r = getenv_bool("SYSTEMD_LIST_NON_UTF8_LOCALES");
+        if (r == -ENXIO || r == 0) {
+                char **a, **b;
+
+                /* Filter out non-UTF-8 locales, because it's 2019, by default */
+                for (a = b = l; *a; a++) {
+
+                        if (endswith(*a, "UTF-8") ||
+                            strstr(*a, ".UTF-8@"))
+                                *(b++) = *a;
+                        else
+                                free(*a);
+                }
+
+                *b = NULL;
+
+        } else if (r < 0)
+                log_debug_errno(r, "Failed to parse $SYSTEMD_LIST_NON_UTF8_LOCALES as boolean");
+
         strv_sort(l);
 
         *ret = TAKE_PTR(l);