Introduction: A Strange Phenomenon
You're typing a query in Chinese, mixing in English technical terms like run.py or config.py, and suddenly your AI assistant responds in Korean—even though you've never spoken a word of it. This isn't a glitch; it's a clue about how language models internally organize meaning.
In this deep dive, we'll explore the hypothesis that embedding spaces are not structured by language boundaries, but by task registers—and engineering English sits at the center of a powerful attractor field. We'll back this up with experiments and discuss what it means for multilingual AI interactions.
The Experiment: Controlled Language Drift
To test the hypothesis, we constructed a sequence of sentences where English words gradually replace Chinese ones:
- Stage 0: 请帮我检查这个分支
- Stage 1: 请帮我 review 这个分支
- Stage 2: 请帮我 review 这个 branch
- Stage 3: Please review this branch pull request commit
- Stage 4: Please review this branch pull request commit code diff
We then computed cosine similarity between each sentence's embedding and two reference clusters: English engineering sentences and Korean engineering sentences. The difference Δ = similarity(English) − similarity(Korean) was tracked.
| Stage | Korean similarity | English similarity | Δ (EN − KO) |
|---|---|---|---|
| 0 | 0.4783 | 0.5141 | 0.0358 |
| 1 | 0.5235 | 0.5728 | 0.0492 |
| 2 | 0.5474 | 0.6140 | 0.0665 |
| 3 | 0.5616 | 0.7314 | 0.1698 |
| 4 | 0.5427 | 0.7398 | 0.1972 |
Notice the jump between Stage 2 and 3—English similarity leaps non-linearly. This suggests a phase transition, not gradual drift. When projecting embeddings via PCA, we see a smooth path then a sharp directional shift, indicating movement between attractor basins.
Let's look at a real-world scenario. You ask: "run.py有早停吗？我在恒源云上跑，发现没有触发" (Does run.py implement early stopping? I was running on a shared GPU service and didn't see it trigger.) The assistant replies in Korean: "원인을 찾았습니다. 결론: run.py에는 실제로 조기 종료가 없습니다. config.py에 USE_EARLY_STOPPING = True" (I found the cause. Conclusion: run.py doesn't have early stopping. In config.py, set USE_EARLY_STOPPING = True).
We computed similarities of the original Chinese prompt (A), the Korean response (B), and a Chinese translation of that response (C) against Chinese, English, and Korean reference clusters:
| Text | Korean sim | English sim | Chinese sim |
|---|---|---|---|
| A (Chinese prompt) | 0.2003 | 0.2688 | 0.3134 |
| B (Korean response) | 0.2745 | 0.2983 | 0.1641 |
| C (Translated Chinese) | 0.1634 | 0.3106 | 0.2798 |
Translating the Korean response back into Chinese does not return the embedding to the Chinese region; it moves even closer to English. Translation restores language form, but not embedding location.
This behavior aligns with how embeddings capture semantic roles: engineering terms like branch, commit, and PR pull the sentence into an 'engineering' region that is predominantly English. For a deeper look at semantic HTML and CSS, you might find our guide on CSS highlight pseudo-elements useful, but let's stay focused on the embedding mystery.
Limitations and Caution
- Small sample size: Our experiments use a handful of sentences; more extensive testing is needed.
- Model-specific: Results may vary across different embedding models and training data.
- Not a language flaw: The behavior reflects training data distribution, not a deficiency of Chinese or Korean.
- Practical impact: Users may be confused or frustrated when assistants switch languages unexpectedly, affecting trust.
Next Steps for Learning
- Explore multilingual embeddings like LASER or LaBSE to see how they handle code-switching.
- Read up on prompt engineering to keep assistants in your preferred language.
- Experiment with your own sentences and visualize embeddings using PCA or t-SNE.
Conclusion: Embeddings Don't Care About Language
Our experiments suggest that embedding spaces are organized by task nature, not language boundaries. Engineering English creates a strong attractor that pulls mixed-language sentences toward it, explaining why your assistant might reply in Korean even when you write in Chinese.
Practical advice: If you want consistent language output, explicitly state your preferred language in the prompt (e.g., "Please respond in English"). Also, be aware that technical terms in English can trigger this drift.
For more on building accessible and semantic UI components, check out our guide to crafting the perfect CSS pie chart. Understanding how models represent meaning helps you communicate more effectively with AI tools.
Happy coding, and may your embeddings stay in the right basin!