langtool: Mark strings that are deliberately the same as the English one

Plenty of strings are legitimately identical in the target language - "Vsync",
"Status", "Ad Hoc multiplayer" - but the only test we had for "is this
translated" is "does it differ from English", so those got sent to the AI again
on every single run of finish-language-with-ai, costing tokens and giving it
another chance to translate something that shouldn't be.

They now get a "# same as English" comment and are skipped. finish-language-with-ai
writes it when the AI hands the English string back unchanged (which the
validation added earlier was throwing away as an error), and import-single writes
it when a language's line matches the en_US one. Deleting the comment puts the
key back in play.

Also stop comparing values with their comments still attached while working out
what's untranslated, which was leaking "# AI translated" into the context block
of the prompt.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_0164UKoXpz9r155TRfQsoc3H

Tools/langtool/README.md

@@ -47,6 +47,11 @@ cargo run -- --model gpt-4o finish-language-with-ai sv_SE
 
 The provider is inferred from the model name, so `--provider` is only needed to disambiguate.
 
+Strings that are deliberately the same as the English one (terms like "Vsync") are marked with a
+`# same as English` comment. `finish-language-with-ai` skips those instead of paying to have them
+"translated" again on every run, and writes the comment itself when the AI hands the English string
+back unchanged. Delete the comment if you disagree with it and it'll be picked up again next time.
+
 To autoformat the code, use:
 
 ```bash

Tools/langtool/src/main.rs

@@ -3,7 +3,7 @@ use std::io;
 use std::collections::{BTreeMap, BTreeSet};
 
 mod section;
-use section::{Section, line_value};
+use section::{SAME_COMMENT, Section, line_value, marked_same, split_comment};
 
 mod inifile;
 use inifile::IniFile;
@@ -498,8 +498,13 @@ fn finish_language_with_ai(
         let mut untranslated_keys = vec![];
         let mut translated_keys = vec![];
         for line in &section.lines {
-            if let Some((key, value)) = split_line(line) {
+            if let Some((key, raw_value)) = split_line(line) {
+                let (value, comment) = split_comment(raw_value);
                 if let Some(ref_value) = ref_section.get_value(key) {
+                    if marked_same(comment) {
+                        // Settled: this one stays as the English string, don't ask about it again.
+                        continue;
+                    }
                     if value == ref_value {
                         // Key not translated.
                         // However, we need to reject some things that the AI likes to mishandle.
@@ -583,6 +588,18 @@ fn finish_language_with_ai(
                                 let ref_value =
                                     ref_section.get_value(original_key).unwrap_or_default();
                                 let issues = validate::check_ai_translation(&ref_value, value);
+                                if issues == vec![validate::Issue::Untranslated] {
+                                    // The AI handed the English string back unchanged, which for
+                                    // things like "Vsync" or "Ad Hoc multiplayer" is the right
+                                    // answer. Write that down so we stop asking every run.
+                                    println!("Marking '{original_key}' as not needing translation");
+                                    target_section.set_value(
+                                        original_key,
+                                        value,
+                                        Some(SAME_COMMENT),
+                                    );
+                                    continue;
+                                }
                                 if !issues.is_empty() {
                                     for issue in issues {
                                         println!("Rejecting '{original_key}' = '{value}': {issue}");
@@ -1033,17 +1050,24 @@ fn execute_command(cmd: Command, ai: Option<&Ai>, dry_run: bool, verbose: bool)
                 ref key,
             } => {
                 let lang_id = filename.strip_suffix(".ini").unwrap();
-                // en_US is in here too, if the file has a line for it - that one is the English
-                // string the others were translated from, not a translation, so no comment on it.
-                let comment = if is_reference {
-                    None
-                } else {
-                    Some("AI translated")
-                };
                 if let Some(single_section) = &single_ini_section {
+                    let english = single_section
+                        .get_line("en_US")
+                        .and_then(|line| line_value(&line).map(|value| value.to_string()));
                     if let Some(target_section) = target_ini.get_section_mut(section) {
                         if let Some(single_line) = single_section.get_line(lang_id) {
                             if let Some(value) = line_value(&single_line) {
+                                // en_US is the string the others were translated from, not a
+                                // translation, so it gets no comment. A language where the
+                                // translation is just the English string gets marked as such, so
+                                // nothing tries to translate it again later.
+                                let comment = if is_reference {
+                                    None
+                                } else if Some(value) == english.as_deref() {
+                                    Some(SAME_COMMENT)
+                                } else {
+                                    Some("AI translated")
+                                };
                                 println!(
                                     "Inserting value {value} for key {key} in section {section} in {target_ini_filename}"
                                 );

Tools/langtool/src/section.rs

@@ -28,6 +28,26 @@ pub fn line_value(line: &str) -> Option<&str> {
     split_line(line).map(|tuple| tuple.1)
 }
 
+/// The comment we put on a line that's deliberately identical to the English string, so we don't
+/// keep paying to have it "translated" over and over.
+pub const SAME_COMMENT: &str = "same as English";
+
+/// Splits the value part of a line into the value itself and its trailing comment, if any
+/// (without the '#').
+pub fn split_comment(value: &str) -> (&str, &str) {
+    match value.split_once('#') {
+        Some((value, comment)) => (value.trim(), comment.trim()),
+        None => (value.trim(), ""),
+    }
+}
+
+/// True if the comment marks the line as deliberately left as the English string. Lenient about
+/// what follows, so "# same", "# same as English" and "# same as English, checked by hrydgard"
+/// all count.
+pub fn marked_same(comment: &str) -> bool {
+    comment.to_ascii_lowercase().starts_with("same")
+}
+
 impl Section {
     pub fn apply_regex(&mut self, key: &str, pattern: &str, replacement: &str) {
         let re = Regex::new(pattern).unwrap();
@@ -375,10 +395,8 @@ impl Section {
         for line in &self.lines {
             if let Some((ref_key, value)) = split_line(line) {
                 if key.eq(ref_key) {
-                    // Found it!
-                    // The value might have a comment starting with #, strip that before returning.
-                    let value = value.split('#').next().unwrap().trim();
-                    return Some(value.to_string());
+                    // Found it! The value might have a comment, that's not part of the value.
+                    return Some(split_comment(value).0.to_string());
                 }
             }
         }