Code: Language and translations packs for SPAs, delivered efficiently — VueJS3, Pinia, Webpack…

This content originally appeared on Level Up Coding - Medium and was authored by Andrew Winnicki

Language and translations packs for SPAs, delivered efficiently: VueJS3, Pinia, Webpack, NodeJS.

There are many ways to deliver translations strings to web (and only) applications. I narrowed my preference to the three most common approaches as I tried them all in different environments in the past. As a result, my understanding of the benefits and downsides is pretty good. Still, I want to compare and explain why I decided on the last one, the hidden fourth option. I went through this exercise as we are implementing multilingual support in the new ESGgen platform, a Single-Page Apps (SPA) architecture that will ultimately run 3 different web applications. So, I had to think far into the future and make sure that priorities like performance were not sacrificed in the process. This post covers the theory, comparison, and implementation guide on how we solved the problem and our future plans around it. So, what are our options?

Bundle everything in a release

Create a release package that contains all languages and translation strings.

• It’s a solution out of the box. Just works.
• It might be perfect for small apps that rely heavily on CMS or other external data sources to serve the content in the correct language.
• It requires downloading code (different languages) that will never be used as users rarely change languages.
• We need to go through the whole release cycle if any string requires updates.
• Significantly slows down the initial application load due to the bigger size and extra CPU processing required to parse the code.
• Without proper checks in place, one translation could break the whole app.
• It’s likely to have one central JSON file with all translations strings. It will get messy and unmaintainable in a big app.
• Different language strings can go out of sync quickly.

Build per translations

That’s an upgrade over the first option, and they don’t have to download all translations strings just to use only one.

• Users only download translations they need.
• Smaller initial package size and faster load times.
• If a fix is required in one language, it can be deployed independently.
• Complicated release deployment by the increased number of packages needed to build.
• Due to the impact on the deployment cycle, it will increase cost and maintenance effort.
• Becomes unmaintainable when you support more than 5 languages.
• Increases testing time significantly when you can’t automate everything and rely on manual testers.
• The same challenge as previously with centralised translations files and languages going out of sync.

Use 3rd party tools

This approach comes with an extra cost but does the job. It works really well for any native or hybrid apps. It is an improvement over the first two options as it allows to deliver translations better.

• Allows delivery of only what is actually needed.
• It can be integrated easily with the backend if needed.
• Usually allows edit “on-fly” without releases, which might be good or bad. A lot depends on your company.
• Often allows A/B testing, which is definitely a benefit if you are keen to run some tests with users.
• It’s easy to implement and easy to use by the team.
• Usually comes with an extra cost.
• It’s an additional library that will add weight to your package or might cause delays. Usually, the library is quite heavy.
• Creates dependency on 3rd party services. If it goes wrong for some reason, it might render the app useless.
• It’s not optimal in delivery and may cause delays in UI rendering or the initial starts (depending on implementation).
• Due to too many unknowns, this is not an approach I would promote.

Asynchronous translations files build from code structure

That’s the option I decided to go with, and I will explain later what we did and how it will help us evolve further in the future.

• We deliver to users only what it’s needed for their languages. The app is language-agnostic.
• Packages are loaded asynchronously to not affect the core package download and execution.
• All translations are defined and located with their respective Views, Components or Modules files. Easy to find and maintain each separately.
• We can switch language on-fly without app reload.
• Webpack & nodeJS help to build the language packs.
• No reliance on 3rd party services.
• Easy to ensure the language packs are not missing any string keys and verify code for errors.
• Some build configuration complexity and automation added.

A good build process should take care of most automation to make developers jobs more manageable and help them build more optimised code out of the box. I love to tinker with nodeJS and Webpack and write small custom scripts that deliver better code. That’s where the magic happens. Unfortunately, most engineerings teams ignore the power of a good build process by just using create-react or other vueCLI that are good for POCs but not for production and enterprise-level SPAs.

Let’s dig a little more into our translations packages. I will explain to you how it is built, the downsides we identified, and our plans for the future to improve it.

SPA Translation Engine

I couldn’t resist giving it a fancy, smart-sounding name! That’s very important for any project as we always wanted to make it sound better than what it actually is. But, unfortunately, “JS Translations” doesn’t sound that exciting.

We don’t really need to support translations for the next half a year. English only would be absolutely enough. At the same time, I know how much time and effort would cost us to re-do the whole app from hardcoded strings to store variables. So it’s better to develop the habit now, plus optimise load and package strategy whilst everything is relatively small. This is an example where some people would say, “Let’s don’t do it. It’s not a priority”. That could be especially true in a startup environment. Doing it now helps us get things done right with minimum overhead but massive savings later.

Unsolved challenges and future considerations

There are a few things I want to consider for the future, but I decided to not do anything about it for now. We don’t want to over-engineer this little feature, as we are not 100% sure which direction things will go later. Understanding the challenges is important. Knowing we can solve them is “importanter” ;)

LocalStorage translations

Once the translations are loaded, we might want to keep them in Local Storage with a version number. Then, if a user comes to the app again, the version is correct, we can assume translations has to be the same. I would consider this update only if our language packages get really big (over 100KB), and we decide to keep all translations a part of the release cycle, so they are all bound to the app version. Also, it will be essential to figure out how often the user returns because there is no point implementing it if we will have a new version released (at least once a week).

Smarter translation packages

When we end up with a button called “Continue” in 10 different components, at the moment, the exact same string will be repeated 10 times in the translation pack, each within its respective namespace. I would like to implement string repetition checking in the final translations file. If it’s significant, implement a process that will merge the repeating text into one item and put it in the “shared” namespace. This will also require the build process to update the code with a new namespace reference in JS and VUE files. It would have to be done only in the build, not in the source code itself.

Split translations into multiple packages

We might end up with users going through a typical 5–10 views, which will be considered the Core experience. We can ensure these translations are all packaged together and delivered first and always. Anything else would be an async call per Component/View requirements. It means more HTTP requests but could significantly minimise the initial load. I’m very keen to try this approach once the app goes live, and we will have a complete picture of the translation size.

Decouple translations from the release

One thing that bothers me now is that we need to do a release if we only update the text. The one significant benefit of decoupling would allow us to use the same translations on the frontend and backend, assuming we use server-side rendering. We can take two approaches to decouple our code from language packs.

• We have the releases which do the app and translations together. Add one more release process on top that will only allow to re-deploy translations. This can be easily achieved and helps to de-risks language update deployments.
• Remove translations from the codebase totally. That poses some challenges in terms of versioning, releases timing or potential backwards compatibility, just to make sure whatever goes first, everything else still works. That would open up other opportunities like CMS integration or admin panel use to update translations, which means we can build a self-service tool for non-engineers. Unfortunately, this approach might quickly introduce a lot of overhead and misalignment between the app and translations files (simple mess). It doesn’t sound right, but we are not ruling it out.

So how does it work?

1. Create *.lang translations file in your component’s or view’s folder. Alternatively, you can put it in a /lang subfolder to keep it cleaner.
2. Run build, which will run the translations process. It will collect all your *.lang files, merge them together, create one object and use a JS template to save the final file to ./translations/*.built.js file.
3. Build script will parse the data and eval() it. If there is a syntax error, it will fail the build. It will also compare any secondary language keys with the primary language (en) and show a warning when “keys don’t match”.
4. Pinia store defines languages available in the system in the state. Simple data like language code, name and icon. That’s all.
5. The VUE application has Pinia action called update(), which will retrieve language based on the cookie parameter or default “en”. Then, it will make an async request to load the target file and populate the global Store.
6. Pinia store exposes the getString() method to simplify data retrieval. It also allows us to add extra functionality there when we need to manipulate the data before returning.
7. We need to import the Pinia store file and use the method in our component. That’s all.
8. An extra component called Language Selector uses the translations config to show all possible languages. On selection, it calls Pinia update() action with a parameter of the new language.
9. Once the file is retrieved and put in Store, the whole app changes the language. No reload is required.

Let’s go through all the above points, and I will share more details and code. Then, feel free to use it in your project. I’m re-using the majority of code I wrote for view-driven router paths configuration so you can check it out in my previous post, and you might decide to kill two birds with one stone ;)

Code: Views-driven router paths, instead of a hardcoded mess — VueJS

Step 1: *.lang files for components and views.

Every language file contains content only for one module, view or component, and it also lives in the same folder. It has an object with a name (which I refer to as namespace). Components namespaces are prefixed with “c”, whilst views are prefixed with “v”. Just to make sure there won’t be any collision in the future, ever.

The benefit of keeping all translations with their components is that we don’t leave a mess behind when new things are added and removed. If we don’t need a particular view anymore and delete the folder, all routes and translations are gone. It solves one of the biggest mistakes software teams make when they agree to keep such information in another folder (ex. ./translations) and assume everybody will remember to update it whenever things are not needed anymore (they won’t).

'vHome': {
    'title': 'Welcome to ESGgen',
    'subtitle': 'Unlock the value of ESG in hours not weeks',
    'fieldlLabel': 'Email',
    'fieldPlaceholder': 'Ex: abc@domain.com',  
    'loginBtn': 'Get Started'
}

Step 2: Build process to create translations package.

This part is going to be complicated. I wrote a simple nodeJS script that does the job. It runs before any build to ensure we always have the latest translations set. Ideally, I would like this script run on watch/serve development mode, but this will be added later. The final result of this job is to create languages/index.XX.built.js file which contains all translations for a given language code.

translation.js
- define languages
- find all files with extension .lang,
- loop through “extra” languages the same way (I should have one method and just pass the variables to not repeat myself)
- then compare the keys from extraLang with primary

let primaryLang = 'en';
let extraLang = ['pl'];

// parse primary first
let primaryLangOutput = helpers.findFiles('src/' + APP, primaryLang + '.lang');
helpers.saveResults('src/' + APP + '/languages/index.js', primaryLangOutput, 'LANGUAGE', primaryLang);
const primaryLangObject = eval(Function('"use strict";return {' + primaryLangOutput + '}')());

// let's do other languages
for (let i = 0; i < extraLang.length; i++) {
 let currentLang = extraLang[i];

let currentLangOutput = helpers.findFiles('src/' + APP, currentLang + '.lang');
 helpers.saveResults('src/' + APP + '/languages/index.js', currentLangOutput, 'LANGUAGE', currentLang);
 let currentLangObject = eval(Function('"use strict";return {' + currentLangOutput + '}')());

if (helpers.compareObjectKeys(primaryLangObject, currentLangObject) === false) {
  console.log('\x1b[31m* ERROR: ' + currentLang.toUpperCase() + " Translation keys don't match.\x1b[0m");
  //process.exit();
 }
}

helpers.findFiles & helpers.saveResults
These are the same methods I used for the router example shared earlier.

/**
 * Recursive function to parse all subfolders from a parent, find specific files and update results array for future parsing
 * @param {string} startPath - root folder path
 * @param {string} filter - an extension of files we looking for (they should contain JS data only)
 * @param {string} [finalOutput] - strinigify content of loaded files
 */
exports.findFiles = function(startPath, filter, finalOutput=''){
 // error handling, just in case
  if (!fs.existsSync(startPath)){
    console.log("\x1b[31m* Directory doesn't exist.\x1b[0m", startPath);
    return;
  }

// do the job
  var files=fs.readdirSync(startPath);
  for(var i=0;i<files.length;i++){
    var filename=path.join(startPath,files[i]);
    var stat = fs.lstatSync(filename);

if (stat.isDirectory()){
      finalOutput = exports.findFiles(filename, filter, finalOutput); //recurse
    }
    else if (filename.indexOf(filter)>=0) {
      console.log('*',filename);

let fileContent = fs.readFileSync(filename, 'utf8');

// just in case, check for come at the end as it shouldn't be there.
      if (fileContent.slice(-1) === '}' || fileContent.slice(-1) === ']' ) {
       fileContent += ',\n'
      }

finalOutput += fileContent
    };
  };

return finalOutput
};

/**
 * It will take the given file and update the placeholder with new data
 * Note that it uses .buildTmpl file as a template to build the final file
 * @param {string} destinationFile - the template file used to build the final file
 * @param {string} content - what we want to write in the file
 * @param {string} placeholder - placeholder where new information will be placed in the file
 * @param {string} [suffix] - extra suffix above the default for all built files
 */
exports.saveResults = function (destinationFile, content, placeholder, suffix) {
 if (suffix) { suffix+='.'}
 else { suffix = ''}

let extension = destinationFile.split('.').pop();
 let generatedFilename = destinationFile.replace(extension, suffix+PROCESSED_SUFFIX+extension);

fs.copyFile(destinationFile, generatedFilename, (err) => {
  if (err) throw err;

let fileContent = fs.readFileSync(generatedFilename, 'utf8');
  let finalContent = fileContent.replace(placeholder, content);

fs.writeFile(generatedFilename, finalContent, function (err) {
  if (err) throw err;
  });
 });
}

Step 3: Check for errors and discrepancies

Before saving everything, I want to check if there were any errors in my strings and the best way to do that is to parse the code. Since I’m really writing objects and combining them all into one, I can do a simple eval() to see if it works as expected. That also allows me to compare translations based on keys, catch any errors or differences, and produce a warning (or stop the build).

This is accomplished in the initial script (translations.js). Then, at the end of processing extra files, I call helpers.compageObjectKeys to figure stuff out.

helpers.compareObjectKeys
There is a bug in this one as sometimes comparison gives false negatives.

/**
 * Compare (deep) keys of two objects and tell me if they are same or not
 * @param {object} object1 - the template file used to build the final file
 * @param {object} object2 - what we want to write in the file
 * @return bool
 */
exports.compareObjectKeys = function(object1, object2) {
 let keys1 = Object.keys(object1);
 let keys2 = Object.keys(object2);

if (keys1.length !== keys2.length) { // means it is already wrong
  console.log('*', keys1);
  console.log('*', keys2);
  return false;
 }

for (const key of keys1) { // loop through all keys to compare them
  let val1 = object1[key];
  let val2 = object2[key];
  let areObjects = isObject(val1) && isObject(val2);

if (areObjects && !exports.compareObjectKeys(val1, val2) || !areObjects && val1 !== val2) {
   return false;
  }
 }

return true;
}
function isObject(object) {
 return object != null && typeof object === 'object';
}

Step 4: Pinia store configuration

I’ve got different data stores defined in Pinia. One of them is languages, and it contains configuration and some actions (described in the following steps). The store uses the config data to check what languages are allowed. The Language Selector component also uses these data later to figure out what to display in UI and switch between available options.

This is what my state for languages looks like at the moment.
currentCode = language code once it successfully loaded
string{} = object where we store all our translations.

state: () => ({
  config: {
   en: {
    name: 'English',
    icon: '...icon link...'
   },
   pl: {
    name: 'Polski',
    icon: '...icon link...'
   }
  },
  currentCode: '',
  string: {}
 })

Step 5: Pinia update() action to load translation file

The method is responsible for determining what to load if no parameter has been provided. Then it loads the language pack and calls the successResponse() function that updates the store settings and pushes all languages to state.string object.

Since we load all translations strings to the Pinia Store and make them available globally, we can switch languages on-fly without the app reload. Pretty awesome.

/**
 * Update existing language strings with a new one from the Language Pack (the automated process building all *.lang files)
 * @memberof Client/Store/Langugage
 * @param {String} [langCode='en'] new language to be loaded
 */
update(langCode) {
 let newLang = Core.Cookie.superGet('language') || this.config[langCode] || 'en';

import(/* webpackMode: "lazy" */ /* webpackChunkName: "translation" */ '@Client/languages/index.'+newLang+'.built.js')
  .then((response) => {
   successResponse(this, response);
  })
  .catch((response) => {
   // TODO: tracking
  });

/**
  * this is just abstraction of the success work to be done after importing a file.
  * @param {Object} state pass-in state
  * @param {Object} response response from import
  */
 function successResponse(state, response) {
  Core.Cookie.superSet('language', newLang);

// update state
  state.$patch({
   currentCode: newLang
  });
  state.string = response.default; // this is needed as it's deep object ($patch won't work correctly with it)
 }
},

Step 6: Pinia getStrings() method to namespaced strings

Just a simple getter to retrieve correct object based on namespace provided. I thought that this might be a good idea in case we will need to do some data manipulation before return in the future. It also allows me to call deeper levels of the objects without JS complaining that something is undefined.

/**
 * Get an object for a given namespace. If not there yet or wrong, it will return {}
 * @memberof Client/Store/Langugage
 * @param {string} namespace namespace you set within your *.lang file
 * @returns object || {}
 */
getStrings(state) {
 return (namespace) => state.string[namespace] || {};
}

Step 7: Use of language strings in components/views.

Import your Pinia store file (assuming you initialised it correctly already as I won’t explain how to do it). In my case, I import the whole store like below, and I can access the language module as follows.

import * as Store from '@Client/store/index.js'

<template>
  {{ Store.language.getStrings('vHome').title }}
</template>

Step 8: Language Selector to switch translations

I connect to store and use Store.language.config and loop through the defined options. That’s how I know what I can switch to and onclick, I will call Store.langauge.update(‘XX’) and it will get quickly loaded, replace the Store and done! Easy.

Final thoughts…

It’s very easy to spend too much time and overengineer something that should be simple and straightforward. And that’s usually the case when we are just trying to design for every problem — the one we expect and the one we don’t. I can quite accurately predict where this little functionality will go in the future and what I should implement, but it’s not needed right now. I really don’t think that would be the best use of my time.

✅ I know this will provide all the functionality we need right now.
✅ I understand the limitations and have a plan for how to deal with them.
✅ It can be better and might need upgrades, which won’t be blocked.

Let’s wait for data and broader team input before I start guessing what might be a pain in the ass and try to solve it now.

Now go, and do something cool that will help your team build better software! We too often think about customers and not enough about ourselves, engineers. Making software engineering life easier helps EVERYBODY.

Code: Language and translations packs for SPAs, delivered efficiently — VueJS3, Pinia, Webpack… was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.

This content originally appeared on Level Up Coding - Medium and was authored by Andrew Winnicki

Print Share Comment Cite Upload Translate Updates

APA

Andrew Winnicki | Sciencx (2022-02-13T15:18:05+00:00) Code: Language and translations packs for SPAs, delivered efficiently — VueJS3, Pinia, Webpack…. Retrieved from https://www.scien.cx/2022/02/13/code-language-and-translations-packs-for-spas-delivered-efficiently-vuejs3-pinia-webpack/

MLA

" » Code: Language and translations packs for SPAs, delivered efficiently — VueJS3, Pinia, Webpack…." Andrew Winnicki | Sciencx - Sunday February 13, 2022, https://www.scien.cx/2022/02/13/code-language-and-translations-packs-for-spas-delivered-efficiently-vuejs3-pinia-webpack/

HARVARD

Andrew Winnicki | Sciencx Sunday February 13, 2022 » Code: Language and translations packs for SPAs, delivered efficiently — VueJS3, Pinia, Webpack…., viewed ,<https://www.scien.cx/2022/02/13/code-language-and-translations-packs-for-spas-delivered-efficiently-vuejs3-pinia-webpack/>

VANCOUVER

Andrew Winnicki | Sciencx - » Code: Language and translations packs for SPAs, delivered efficiently — VueJS3, Pinia, Webpack…. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2022/02/13/code-language-and-translations-packs-for-spas-delivered-efficiently-vuejs3-pinia-webpack/

CHICAGO

" » Code: Language and translations packs for SPAs, delivered efficiently — VueJS3, Pinia, Webpack…." Andrew Winnicki | Sciencx - Accessed . https://www.scien.cx/2022/02/13/code-language-and-translations-packs-for-spas-delivered-efficiently-vuejs3-pinia-webpack/

IEEE

" » Code: Language and translations packs for SPAs, delivered efficiently — VueJS3, Pinia, Webpack…." Andrew Winnicki | Sciencx [Online]. Available: https://www.scien.cx/2022/02/13/code-language-and-translations-packs-for-spas-delivered-efficiently-vuejs3-pinia-webpack/. [Accessed: ]

rf:citation

» Code: Language and translations packs for SPAs, delivered efficiently — VueJS3, Pinia, Webpack… | Andrew Winnicki | Sciencx | https://www.scien.cx/2022/02/13/code-language-and-translations-packs-for-spas-delivered-efficiently-vuejs3-pinia-webpack/ |

Please log in to upload a file.

There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.