Large language models perform increasingly well on standardized logical reasoning benchmarks, but whether this ability remains robust beyond English is unclear. We introduce ChLogic, an English--Chinese aligned benchmark that tests whether models preserve logical reasoning performance when the same latent logical structure is expressed in English and diverse Chinese surface realizations. Built from formal logical templates, the benchmark contains three data sets: (i) the General aligned set, derived from 60 General Propositions across nine template families; (ii) the Difficult aligned set, derived from 40 Difficult Problems; and (iii) the Chinese-only set, covering 15 language-specific phenomenon types. Each aligned item pairs one English reference expression with five Chinese realizations. Experiments on Qwen3, Ministral, and GLM models reveal a persistent English--Chinese performance gap. Back-translation from standard Chinese into English often improves performance on the General aligned set, but produces mixed effects on the Difficult aligned set, where Qwen3-32B and GLM-5.1 perform worse after translation. These results indicate that Chinese surface realization, translation artifacts, and model-specific behavior jointly affect multilingual logical reasoning. Overall, ChLogic provides a useful stress test for the robustness of multilingual reasoning.</p>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/68b65343dd7f21b75891e446/o9aIvzrFAb6ohBLEMa6j5.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/68b65343dd7f21b75891e446/o9aIvzrFAb6ohBLEMa6j5.png\" alt=\"two_panel_academic_plot\"></a></p>\n","updatedAt":"2026-06-17T03:21:06.077Z","author":{"_id":"68b65343dd7f21b75891e446","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/68b65343dd7f21b75891e446/g4dtudmiuSBZY63eLFxJ8.jpeg","fullname":"Xueyan Niu","name":"niuxueyan","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8406385779380798},"editors":["niuxueyan"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/68b65343dd7f21b75891e446/g4dtudmiuSBZY63eLFxJ8.jpeg"],"reactions":[],"isReport":false}},{"id":"6a32913f1c8049826bc28187","author":{"_id":"6960eca92f7ad9b043b5cbe0","avatarUrl":"/avatars/e68dcc7fd04f143d849d40414866e633.svg","fullname":"Noah","name":"noahml","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":0,"isUserFollowing":false},"createdAt":"2026-06-17T12:21:19.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Cool paper - I liked the way \"CHLOGIC: Evaluating Robustness of Logical Reasoning in Chinese Expressions\" frames the problem without making it feel too abstract.\n\nCurious if you think this would still work once the setup gets messier in the wild?\n\nI made a podcast on it with ResearchPod, it makes it easy to get the key concepts on the go:\nhttps://researchpod.app/episode/975b599c-b34d-4fad-ac84-1fd262c49c7c","html":"<p>Cool paper - I liked the way \"CHLOGIC: Evaluating Robustness of Logical Reasoning in Chinese Expressions\" frames the problem without making it feel too abstract.</p>\n<p>Curious if you think this would still work once the setup gets messier in the wild?</p>\n<p>I made a podcast on it with ResearchPod, it makes it easy to get the key concepts on the go:<br><a href=\"https://researchpod.app/episode/975b599c-b34d-4fad-ac84-1fd262c49c7c\" rel=\"nofollow\">https://researchpod.app/episode/975b599c-b34d-4fad-ac84-1fd262c49c7c</a></p>\n","updatedAt":"2026-06-17T12:21:19.209Z","author":{"_id":"6960eca92f7ad9b043b5cbe0","avatarUrl":"/avatars/e68dcc7fd04f143d849d40414866e633.svg","fullname":"Noah","name":"noahml","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":0,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8978672623634338},"editors":["noahml"],"editorAvatarUrls":["/avatars/e68dcc7fd04f143d849d40414866e633.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.17905","authors":[{"_id":"6a31fb48bc818ff14e453cc1","name":"Peixian Zhou","hidden":false},{"_id":"6a31fb48bc818ff14e453cc2","name":"Yuxu Chen","hidden":false},{"_id":"6a31fb48bc818ff14e453cc3","name":"Chaorui Zhang","hidden":false},{"_id":"6a31fb48bc818ff14e453cc4","name":"Wei Han","hidden":false},{"_id":"6a31fb48bc818ff14e453cc5","name":"Bo Bai","hidden":false},{"_id":"6a31fb48bc818ff14e453cc6","name":"Xueyan Niu","hidden":false}],"publishedAt":"2026-06-16T00:00:00.000Z","submittedOnDailyAt":"2026-06-17T00:00:00.000Z","title":"ChLogic: Evaluating Robustness of Logical Reasoning in Chinese Expressions","submittedOnDailyBy":{"_id":"68b65343dd7f21b75891e446","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/68b65343dd7f21b75891e446/g4dtudmiuSBZY63eLFxJ8.jpeg","isPro":false,"fullname":"Xueyan Niu","user":"niuxueyan","type":"user","name":"niuxueyan"},"summary":"Large language models perform increasingly well on standardized logical reasoning benchmarks, but whether this ability remains robust beyond English is unclear. We introduce ChLogic, an English--Chinese aligned benchmark that tests whether models preserve logical reasoning performance when the same latent logical structure is expressed in English and diverse Chinese surface realizations. Built from formal logical templates, the benchmark contains three data sets: (i) the General aligned set, derived from 60 General Propositions across nine template families; (ii) the Difficult aligned set, derived from 40 Difficult Problems; and (iii) the Chinese-only set, covering 15 language-specific phenomenon types. Each aligned item pairs one English reference expression with five Chinese realizations. Experiments on Qwen3, Ministral, and GLM models reveal a persistent English--Chinese performance gap. Back-translation from standard Chinese into English often improves performance on the General aligned set, but produces mixed effects on the Difficult aligned set, where Qwen3-32B and GLM-5.1 perform worse after translation. These results indicate that Chinese surface realization, translation artifacts, and model-specific behavior jointly affect multilingual logical reasoning. Overall, ChLogic provides a useful stress test for the robustness of multilingual reasoning.","upvotes":6,"discussionId":"6a31fb49bc818ff14e453cc7","githubRepo":"https://github.com/0328zpx/ChLogic","githubRepoAddedBy":"user","ai_summary":"ChLogic benchmark reveals persistent performance gaps between English and Chinese logical reasoning in large language models, influenced by surface realization differences and translation artifacts.","ai_keywords":["large language models","logical reasoning benchmarks","multilingual reasoning","back-translation","surface realization"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":0},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"68b65343dd7f21b75891e446","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/68b65343dd7f21b75891e446/g4dtudmiuSBZY63eLFxJ8.jpeg","isPro":false,"fullname":"Xueyan Niu","user":"niuxueyan","type":"user"},{"_id":"66169de305f5b558b441cafc","avatarUrl":"/avatars/117d64ae4adb0b32692e85ca43c927ec.svg","isPro":false,"fullname":"Ryan Zhang","user":"Ruistf","type":"user"},{"_id":"65f2be58f52878ab7afe4c33","avatarUrl":"/avatars/b3d08b03a2b194f81e1f211b4cd7b2e4.svg","isPro":false,"fullname":"Yuxu CHEN","user":"yuxuchen","type":"user"},{"_id":"6a1e409c8a8594a3b4c65631","avatarUrl":"/avatars/cd263d2c8e934d16489e5a73d94f4b50.svg","isPro":false,"fullname":"Henry","user":"ElecTricker","type":"user"},{"_id":"6a2443f053322e79bf1397af","avatarUrl":"/avatars/45d7472ca38fa08d47812d573faddb69.svg","isPro":false,"fullname":"zhoupeixian","user":"zhoupeixianai1122","type":"user"},{"_id":"65face2f581c1ed7bfb0e98b","avatarUrl":"/avatars/cdbec0aed69bceadcdfa1c15eab52a7f.svg","isPro":false,"fullname":"zq","user":"youpuu","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.17905.md","query":{}}">
ChLogic: Evaluating Robustness of Logical Reasoning in Chinese Expressions
Abstract
ChLogic benchmark reveals persistent performance gaps between English and Chinese logical reasoning in large language models, influenced by surface realization differences and translation artifacts.
Large language models perform increasingly well on standardized logical reasoning benchmarks, but whether this ability remains robust beyond English is unclear. We introduce ChLogic, an English--Chinese aligned benchmark that tests whether models preserve logical reasoning performance when the same latent logical structure is expressed in English and diverse Chinese surface realizations. Built from formal logical templates, the benchmark contains three data sets: (i) the General aligned set, derived from 60 General Propositions across nine template families; (ii) the Difficult aligned set, derived from 40 Difficult Problems; and (iii) the Chinese-only set, covering 15 language-specific phenomenon types. Each aligned item pairs one English reference expression with five Chinese realizations. Experiments on Qwen3, Ministral, and GLM models reveal a persistent English--Chinese performance gap. Back-translation from standard Chinese into English often improves performance on the General aligned set, but produces mixed effects on the Difficult aligned set, where Qwen3-32B and GLM-5.1 perform worse after translation. These results indicate that Chinese surface realization, translation artifacts, and model-specific behavior jointly affect multilingual logical reasoning. Overall, ChLogic provides a useful stress test for the robustness of multilingual reasoning.
Community
Large language models perform increasingly well on standardized logical reasoning benchmarks, but whether this ability remains robust beyond English is unclear. We introduce ChLogic, an English--Chinese aligned benchmark that tests whether models preserve logical reasoning performance when the same latent logical structure is expressed in English and diverse Chinese surface realizations. Built from formal logical templates, the benchmark contains three data sets: (i) the General aligned set, derived from 60 General Propositions across nine template families; (ii) the Difficult aligned set, derived from 40 Difficult Problems; and (iii) the Chinese-only set, covering 15 language-specific phenomenon types. Each aligned item pairs one English reference expression with five Chinese realizations. Experiments on Qwen3, Ministral, and GLM models reveal a persistent English--Chinese performance gap. Back-translation from standard Chinese into English often improves performance on the General aligned set, but produces mixed effects on the Difficult aligned set, where Qwen3-32B and GLM-5.1 perform worse after translation. These results indicate that Chinese surface realization, translation artifacts, and model-specific behavior jointly affect multilingual logical reasoning. Overall, ChLogic provides a useful stress test for the robustness of multilingual reasoning.
Cool paper - I liked the way "CHLOGIC: Evaluating Robustness of Logical Reasoning in Chinese Expressions" frames the problem without making it feel too abstract.
Curious if you think this would still work once the setup gets messier in the wild?
I made a podcast on it with ResearchPod, it makes it easy to get the key concepts on the go:
https://researchpod.app/episode/975b599c-b34d-4fad-ac84-1fd262c49c7c
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.17905 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.17905 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.17905 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.