Internationalization is an increasingly important consideration for Magento merchants developers looking to expand market penetration and increase usability. A significant part of this effort is realized in the form of maintaining translations for multiple locales – quite the undertaking, in spite of Magento’s robust localization capabilities.
However, a journey of a thousand miles begins with a single step, and this initial step can be particularly daunting. What must be translated?
Ideally, every string ever used, be it backend or frontend, would be documented so that an exhaustive list is always available of material scheduled for translation. In practice, however, this is rarely the case – maybe the site or module wasn’t initially slated for an international market or the ROI was difficult to justify. Because of this, orphan strings with no record of their existence are very common and a barrier to internationalization.
Wouldn’t it be nice to have a mechanism to retroactively examine a site or module and perform a translation gap analysis?
Approach
One approach to ferreting out untranslated strings is to modify the translation tool itself to report untranslated strings as they are encountered. This is often expressed as a quick hack to the translation classes whereby strings are logged, then the changes reverted.
The basic idea is solid, but the execution is essentially a transient hack – requiring repeated discovery and implementation, and is prone to oversights.
After a recent internationalization-intensive project involving several countries and localizations, I attempted to formalize this approach into a robust module.
Translation path
When calling the familiar __() method (whether from a block, helper, controller, etc), Mage_Core_Model_Translate::translate() is invoked to do the actual heavy lifting.
Mage_Core_Model_Translate::translate()
/**
* Translate
*
* @param array $args
* @return string
*/
public function translate($args) {
$text = array_shift($args);
if (is_string($text) & amp; & amp;
'' == $text || is_null($text) || is_bool($text) & amp; & amp; false === $text || is_object($text) & amp; & amp;
'' == $text - & gt; getText()) {
return '';
}
if ($text instanceof Mage_Core_Model_Translate_Expr) {
$code = $text - & gt;
getCode(self::SCOPE_SEPARATOR);
$module = $text - & gt;
getModule();
$text = $text - & gt;
getText();
$translated = $this - & gt;
_getTranslatedString($text, $code);
} else {
if (!empty($_REQUEST['theme'])) {
$module = 'frontend/default/'.$_REQUEST['theme'];
} else {
$module = 'frontend/default/default';
}
$code = $module.self::SCOPE_SEPARATOR.$text;
$translated = $this - & gt;
_getTranslatedString($text, $code);
}
//array_unshift($args, $translated);
//$result = @call_user_func_array('sprintf', $args);
$result = @vsprintf($translated, $args);
if ($result === false) {
$result = $translated;
}
if ($this - & gt; _translateInline & amp; & amp; $this - & gt; getTranslateInline()) {
if (strpos($result, '{{{') === false || strpos($result, '}}}') === false || strpos($result, '}}{{') === false) {
$result = '{{{'.$result.
'}}{{'.$translated.
'}}{{'.$text.
'}}{{'.$module.
'}}}';
}
}
return $result;
}
After doing some heuristics to determine exactly what module and code to use for context, Mage_Core_Model_Translate::_getTranslatedString() takes over.
Mage_Core_Model_Translate::_getTranslatedString()
/**
* Return translated string from text.
*
* @param string $text
* @param string $code
* @return string
*/
protected function _getTranslatedString($text, $code)
{
$translated = '';
if (array_key_exists($code, $this->getData())) {
$translated = $this->_data[$code];
}
elseif (array_key_exists($text, $this->getData())) {
$translated = $this->_data[$text];
}
else {
$translated = $text;
}
return $translated;
}
If a translation by either code or text exists, then it is returned – otherwise, the key (untranslated string) is returned.
Implementation
The Mage_Core_Model_Translate::_getTranslatedString() method offers a perfect opportunity to detect strings with no translation and record them.
String Detection
Unfortunately, there is no event present which can be observed to accomplish this detection, so the model must be rewritten. Using the lightest touch possible for such a fundamental model, the string is preprocessed before translation to determine if it is missing translations for interesting locales.
EW_UntranslatedStrings_Model_Core_Translate::_getTranslatedString()
/**
* Evaluate translated text and code and determine
* if they are untranslated.
*
* @param string $text
* @param string $code
*/
protected function _checkTranslatedString($text, $code) {
Varien_Profiler::start(__CLASS__ . '::' . __FUNCTION__);
Varien_Profiler::start(EW_UntranslatedStrings_Helper_Data::PROFILER_KEY);
//loop locale(s) and find gaps
$untranslatedPhrases = array();
foreach($this->_getLocalesToCheck() as $locale) {
if(!Mage::helper('ew_untranslatedstrings')->isTranslated($text,$code,$locale)) {
$untranslatedPhrases[] = array(
'text' => $text,
'code' => $code,
'locale' => $locale
);
}
}
$this->_storeUntranslated($untranslatedPhrases);
Varien_Profiler::stop(EW_UntranslatedStrings_Helper_Data::PROFILER_KEY);
Varien_Profiler::stop(__CLASS__ . '::' . __FUNCTION__);
}
/**
* Check for translation gap before returning
*
* @param string $text
* @param string $code
* @return string
*/
protected function _getTranslatedString($text, $code)
{
if(Mage::helper('ew_untranslatedstrings')->isEnabled()) {
$this->_checkTranslatedString($text, $code);
}
return parent::_getTranslatedString($text, $code);
}
This simple change allows the module to collect untranslated strings, which, for performance sake, are batched up and flushed to the database in a single query after the page is rendered. All that is required to perform this collection is to enable the module in the system configuration, and browse the site – a perfect scenario for stage sites during ongoing UAT.
Multiple Locales
If a merchant or developer is concerned about translation gaps for one locale, he is likely also concerned about several others. This is facilitated by the loop in _checkTranslatedString() which iterates over multiple locales which are ultimately retrieved from the system configuration.
To check the status of translations other than the currently selected locale, EW_UntranslatedStrings_Helper_Data::isTranslated() performs evaluations independent of current store configuration, aided by getTranslator() which provides ready-to-go translator models for any locale and store ID.
EW_UntranslatedStrings_Helper_Data::isTranslated() and getTranslator()
/**
* Get translator prepared for given locale
*
* @param $locale
* @param bool $allowMatchingKeyValuePairs - matching key / value pairs count as translations
* @param null $storeIdContext
* @param bool $forceRefresh
* @return EW_UntranslatedStrings_Model_Core_Translate
*/
public function getTranslator($locale, $allowMatchingKeyValuePairs = null, $storeIdContext = null, $forceRefresh = false) {
if(!isset($this->_translators[$locale])) {
if(is_null($allowMatchingKeyValuePairs)) {
// "allow" and "log" are opposite concepts
$allowMatchingKeyValuePairs = !$this->logMatchingKeyValuePairs();
}
/* @var $translate EW_UntranslatedStrings_Model_Core_Translate */
$translate = Mage::getModel('ew_untranslatedstrings/core_translate');
$translate->setConfig(
array(
Mage_Core_Model_Translate::CONFIG_KEY_LOCALE => $locale
)
);
$translate->setLocale($locale);
$translate->setAllowLooseDevModuleMode(true); //prevent native dev mode differences
$translate->setAllowMatchingKeyValuePairs($allowMatchingKeyValuePairs);
if(!is_null($storeIdContext)) {
$translate->setThemeContext($storeIdContext);
}
$translate->init(Mage_Core_Model_Design_Package::DEFAULT_AREA, $forceRefresh);
$this->_translators[$locale] = $translate;
}
return $this->_translators[$locale];
}
/**
* Does text/code have translation for given locale?
*
* @param $text
* @param $code
* @param $locale
* @return bool
*/
public function isTranslated($text, $code, $locale) {
/* @var $translate EW_UntranslatedStrings_Model_Core_Translate */
$translate = $this->getTranslator($locale);
return $translate->hasTranslation($text, $code);
}
While it requires a few more rewritten methods on the translator model, this feature allows a store owner or developer to quickly assemble a gap analysis of multiple locales at once.
Configurability
Although simple in concept, there are several optional tweaks that can make this feature much more useful for efficient string collection.
This module adds a new system configuration section which can be found at Advanced -> Developer -> Untranslated Strings.
These options allow several powerful options for customizability. For example
- Admin translation gaps can be ignored – useful to enable for all websites, but purposely omit admin strings.
- Matching translation key/value pairs can be either logged or ignored. This accommodates either of the following scenarios:
- Complete translation gap analysis, where strings which are represented but not actually translated are logged – useful for merchant site evaluation.
- String representation gap analysis, where only strings which are not represented at all are logged – useful for module developers who only wish to maintain a list of strings and are not concerned with actual translation.
- Translation code exclusion patterns which omit some strings from the log. Magento has many native translation gaps, and the translation status of some modules may be irrelevant to a given store. This allows final users to tweak the signal-to-noise ratio to ensure efficient use of resource when evaluating untranslated strings.
- Batch locale translation gap analysis. As mentioned before, this allows the evaluation of several locales at one time, greatly reducing the time required to collect an exhaustive list of strings.
Reports
Even the most robust untranslated string detection is useless if the results are not easily accessible and actionable.
To this end, there are two new reports which are now available at Reports -> Untranslated Strings in the main admin menu.
- Untranslated Strings Summary: this provides an overview of translation gaps by locale and store, and allows the ability to purge or truncate untranslated strings.
- Untranslated Strings Report: this provides a detailed view of untranslated strings and is filterable by locale, store, popularity, etc.
Additionally, the Untranslated Strings Report allows strings to be exported to CSV file. Combined with its filtering capabilities, this allows a merchant or developer to effectively manage the strings and provide them to translation teams, along with all important context.
Maintenance
Viewing the untranslated strings report as a sort of “to-do list”, the summary view allows merchants or developers to curate the remaining untranslated strings. In particular, the strings associated with a given locale and store can be truncated (individually or en masse), cleaning the slate and allowing a fresh set of strings to be collected.
More powerful, however, is the purge option. Exercising this option on a locale and store (or group of these, via the mass action functionality) causes the module to reevaluate the translation status of associated strings, removing any which are now considered translated according to the system configuration settings.
After adding translations to address gaps, this feature allows merchants and developers to cull strings which are no longer relevant, ensuring that the untranslated string list is always actionable.
Where to go from here (caveats)
While this module goes a long way to formalize a former “hacky” process, there is always room to improve. Some particular things to keep in mind:
- This module only assists with strings which are wrapped in translate calls – if strings are simply never translated, there is no way to detect them. On the same token, strings which are meant to be translated using admin store scopes (such as product attributes, CMS blocks, etc) are not evaluated.
- The module introduces a small to moderate performance overhead depending on the number of locales to be evaluated and the number of untranslated strings. Fortunately, this is only realized if the functionality is enabled in system configuration.
- As with Magento’s inline translation tool, the module works best if translation and block caches are disabled.
- Like the shoemaker’s wife, the module itself has a few translation gaps …
Where to get it
If you’d like to get your hands on this open source module and take the first step toward internationalization, you can find it on Github:
https://github.com/ericthehacker/magento-untranslatedstrings
As stated in the readme, it’s easily installed via modman. Use it wisely!
Interested in building an international e-commerce store or looking to upgrade the one you have? Let us know on our contact form!