Case study: improving a popular library’s size for webpack users

There’s a library called Polished. It’s a utility collection for writing styles in JavaScript.

The polished logo

And it had a problem.

Problem#

A story in three tweets:

So, this code:

import { opacify, transparentize } from 'polished'; 

generates a much larger bundle than this code:

import opacify from 'polished/lib/color/opacify.js';
import transparentize from 'polished/lib/color/transparentize.js';

even despite the Polished’s bundle is built with ES modules and tree-shaking is enabled.

Let’s find out what causes this.

Investigation#

1. Verify the entry point#

Environment: polished@1.2.0 and webpack@2.6.1 (webpack@3.0.0 gives the same result)

At first, let’s check that import { ... } from 'polished' picks up a file written with ES exports. If it doesn’t, webpack can’t do any tree-shaking at all.

When you import a package, webpack understands what exact file to use by looking into specific fields in package.json. Polished’s package.json has two of them:

{
  "name": "polished",
  "description": "A lightweight toolset for writing styles in Javascript.",
  "main": "lib/index.js",  // This one
  "module": "dist/polished.es.js",  // And this one
  ...
}

Webpack prefers module over main. module points to dist/polished.es.js, and this file does have an ES export:

// polished/dist/polished.es.js
...
export { adjustHue$1 as adjustHue, ... };

This point is OK.

2. Check if there’s unused code that’s unnecessarily kept#

polished/dist/polished.es.js is written with ES exports. This means that tree-shaking should work properly, and the unused imports shouldn’t be included into the bundle. Then why different imports produce different file sizes?

Side effect is when a function changes something outside of itself – e.g. writes a value to a global variable or initiates a network request

The most possible reason is that polished/dist/polished.es.js contains some code that’s absent in our polished/lib/... files and that can’t be simply dropped by the tree-shaker. This is the code that could cause side-effects. E.g. if a file includes a top-level function call, the tree-shaker can’t remove the function even if its result isn’t used. The function could be causing side effects, and removing it could break the app.

Let’s compare the bundles that we have after importing polished in two different ways and verify this case.

To do this, I create a package:

# Shell
mkdir polished-test && cd polished-test
npm init -y
npm install polished webpack@2

Add two files that import Polished in two different ways:

console.log() helps finding the index.js file in the bundle + prevents webpack from removing the imports as unused
// index-import-package.js
import { opacify, transparentize } from 'polished';

console.log('polished', opacify, transparentize);

// index-import-files.js
import opacify from 'polished/lib/color/opacify.js';
import transparentize from 'polished/lib/color/transparentize.js';

console.log('polished', opacify, transparentize);

Add a special webpack configuration that emits two bundles:

// webpack.config.js
const webpack = require('webpack');

module.exports = {
  entry: {
    // We’ll compare two different bundles,
    // thus two different entry points
    'bundle-import-package': './index-import-package.js',
    'bundle-import-files': './index-import-files.js',
  },
  output: {
    filename: '[name].js',
    path: __dirname,
  },
  plugins: [
    // We need to run UglifyJS to remove the dead code
    // (this will do tree-shaking), but prevent it
    // from uglifying the code (so it’s easier to read the bundle)
    new webpack.optimize.UglifyJsPlugin({
      // Disable several optimizations so that the bundle
      // is easier to read
      compress: { sequences: false, properties: false, conditionals: false, comparisons: false, evaluate: false, booleans: false, loops: false, hoist_funs: false, hoist_vars: false, if_return: false, join_vars: false, cascade: false },

      // Beautify the bundle after uglifying it
      beautify: true,

      // Don’t rename the variables
      mangle: false,
    }),
  ]
}

And run the build:

./node_modules/.bin/webpack

Now, I have two bundles, each with a different approach to importing stuff. I open them in my editor and switch to the structure view to their content. And here’s what I see:

A comparison between the content of two files. The left file is bundle-import-package.js, it has a lot of functions. The right file is bundle-import-files.js, it has much less functions.

bundle-import-package.js has more methods than bundle-import-files.js. Most likely, they are kept because of calls with side effects. Let’s dig deeper.

3. Find the exact cause of the problem#

So, bundle-import-package.js has a lot of functions that aren’t used but are still included. If we look through the file to see their usages, we’ll see a large snippet of code like this:

// bundle-import-package.js
// ...
function opacify(amount, color) {
    // ...
}
var opacify$1 = curry(opacify);
function desaturate(amount, color) {
    // ...
}
curry(desaturate);
function lighten(amount, color) {
    // ...
}
curry(lighten);
// ...

Here, desaturate and lighten are those unused functions, and opacify is a function we import in the client code.

This code comes to bundle-import-package.js from polished/dist/polished.es.js. The corresponding code in that file looks like this:

// polished/dist/polished.es.js
// ...
function opacify(amount, color) {
    // ...
}

var opacify$1 = curry(opacify);

function desaturate(amount, color) {
    // ...
}

var desaturate$1 = curry(desaturate);

function lighten(amount, color) {
    // ...
}

var lighten$1 = curry(lighten);
// ...

And this code comes into polished/dist/polished.es.js from the library sources. This is how it looks:

// polished/src/color/opacify.js
function opacify(amount: number, color: string): string {
  // ...
}

export default curry(opacify);

// polished/src/color/desaturate.js
function desaturate(amount: number, color: string): string {
  // ...
}

export default curry(desaturate);

// polished/src/color/lighten.js
function lighten(amount: number, color: string): string {
  // ...
}

export default curry(lighten);

So what happens here? dist/polished.es.js is built with Rollup. When the library authors do a build, Rollup grabs all the modules and converts exports (export default curry(lighten)) into variable assignments (var lighten$1 = curry(lighten)).

When we do import { opacify, transparentize } from 'polished', webpack tries to compile dist/polished.es.js and drop the unused code. It removes the desaturate$1 and lighten$1 variables because they aren’t exported, but it can’t drop the curry(darken) calls because curry could produce side-effects. And because functions like desaturate and lighten are passed into curry(), they are also kept in the bundle.

Screenshot of the editor
This is how you analyze the bundle: open the file structure, find a function that’s absent in the other bundle, and search for its usages

Solution#

To decrease the bundle size, we should do one of the following things:

Pure function is a function that doesn’t produce side effects
  • tell UglifyJS that it’s safe to remove curry() calls because it’s pure
  • or move currying into the functions instead of wrapping them.
Another option is passing compressor: { pure_funcs: ['curry'] } to the UglifyJS options, but Polished can’t control this

To tell UglifyJS that curry() calls are safe to remove, we have to mark each call with the /*#__PURE__*/ annotation. This way, the minifier will understand that this call is pure and will be able to optimize it:

We can’t just add the /*#__PURE__*/ annotation after export default. Rollup seems to remove comments if they are placed in that position
// polished/src/color/lighten.js
function lighten(amount: number, color: string): string {
  // ...
}
 
- export default curry(lighten);
+ const curriedLighten = /*#__PURE__*/curry(lighten);
+ export default curriedLighten;

The second approach is to move currying into the functions body. With it, we should do something like this:

// polished/src/color/lighten.js
- function lighten(amount: number, color: string): string {
-   // method body
- }
+ function lighten(...args) {
+   return applyCurried(function (amount: number, color: string): string {
+     // method body
+   }, args);
+ }
 
- export default curry(lighten);
+ export default lighten;

I prefer the first approach because it (almost) doesn’t complicate the code.

After adding the /*#__PURE__*/ annotations, minified bundle-import-package.js goes from 16 down to 11.8 kB. But that’s not the end – bundle-import-files.js is still smaller (9.86 kB). This is because there’re a few other places that should be optimized.

I’ll skip the part where I find them and jump right to the solution.

  • Change 1 and 2. Like with curry(), there’re two other places where the export is wrapped into a function. It’s polished/src/helpers/em.js and polished/src/helpers/rem.js. To optimize them, we should similarly add the /*#__PURE__*/ annotations.
  • Change 3. In polished/src/mixins/normalize.js, there’re two global objects that use computed object properties. When they are compiled, Babel transforms them to call the Babel’s defineProperty function. Because of this, UglifyJS can’t remove them. To solve the problem, we should either move these objects into the normalize() function that uses them or wrap them into getter functions.

And, when we apply these additional optimizations, we’ll have this:

                   Asset     Size  Chunks             Chunk Names
  bundle-import-files.js  9.87 kB       0  [emitted]  bundle-import-files
bundle-import-package.js  7.76 kB       1  [emitted]  bundle-import-package

bundle-import-package.js is now even smaller than bundle-import-files.js! Great.

I’ve submitted the pull request.


Want me to do the same for your project? I’ll start consulting about front-end loading performance soon. Leave your email to know first:


Liked this article? Share it so others can see it too:

webpack for real tasks: decreasing front-end size and improving caching

This is the second part of a three-part introduction into webpack:

  1. Bundling front-end and adding compilation
  2. Decreasing front-end size and improving caching (you are here!)
  3. Speeding up build and improving the development workflow

Want to stay tuned for the future posts? Subscribe

Task Decrease front-end size#

Given: you have a front-end application. You want to decrease its size to make it load faster.

Let’s see how webpack can help with this.

Minification#

Minification is when you compress your code by removing extra spaces, shortening variable names, etc. Like this:

Webpack has two approaches to minify the code: the UglifyJS plugin and loaders options. They should be used simultaneously.

The UglifyJS plugin works on the level of the bundle and compresses it after compilation. As you might’ve guessed, it used UglifyJS under the hood. This is how it works:

You write code like this
// comments.js
import './comments.css';
export function render(data, target) {
    console.log('Rendered!');
}

Webpack compiles it into approximately the following
// bundle.js (part of)
"use strict";
Object.defineProperty(__webpack_exports__, "__esModule", { value: true });
var __WEBPACK_IMPORTED_MODULE_0__comments_css__ =
  __webpack_require__(4);
var __WEBPACK_IMPORTED_MODULE_0__comments_css___default =
  __webpack_require__.n(__WEBPACK_IMPORTED_MODULE_0__comments_css__);
__webpack_exports__["render"] = render;

function render(data, target) {
    console.log('Rendered!');
}

The UglifyJS plugin minifies it into approximately the following
// bundle.js (part of)
"use strict";function r(e,t){console.log("Rendered!")}
Object.defineProperty(t,"__esModule",{value:!0});
var o=n(4);n.n(o);t.render=r

To enable the plugin, add it to the plugins section of the config:

// webpack.config.js
const webpack = require('webpack');

module.exports = {
  plugins: [
    new webpack.optimize.UglifyJsPlugin()
  ]
};

The second approach is loaders options. It allows compressing things that UglifyJS can’t minify. Its point is that some code (e.g. CSS that you import) is compiled as a string which UglifyJS can’t handle:

/* comments.css */
.comment {
    color: black;
}

// bundle.js (part of)
exports = module.exports = __webpack_require__(1)();
exports.push([module.i, ".comment {\r\n    color: black;\r\n}", ""]);

To minify it, you should configure the loader. Here’s how you do it with css-loader:

// webpack.config.js
module.exports = {
  module: {
    rules: [
      {
        test: /\.css$/,
        use: [
          'style-loader',
          { loader: 'css-loader', options: { minimize: true } }
        ]
      }
    ]
  }
};

Pitfall: ES2015 code#

UglifyJS 2 (which is used in webpack) can’t compile ES2015+ code. This means that if your code uses classes, arrow functions or other new language features, and you don’t compile it to ES5, UglifyJS won’t handle it. In this case, you can use Babili, a Babel-based minifier. See babili-webpack-plugin

NODE_ENV=production#

Another way to decrease the front-end size is to set NODE_ENV environmental variable to the value “production”.

NODE_ENV is an environmental variable that is commonly used in libraries to detect in which mode the library works – in development mode or on a production server. The library can behave differently based on this variable. For example, React does additional checks and prints warnings when it’s built for development:

// …

if (process.env.NODE_ENV !== 'production') {
  validateTypeDef(Constructor, propTypes, 'prop');
}

// …

When you’re building your app for production, it’s better to also tell that your libraries. For Node.js libraries, it’s done by configuring the environment and setting the NODE_ENV variable to “production”. For front-end libraries, it’s done by replacing process.env.NODE_ENV with a specific value:

// webpack.config.js
const webpack = require('webpack');

module.exports = {
  plugins: {
    new webpack.DefinePlugin({
      'process.env.NODE_ENV': '"production"'
    })
  }
};

DefinePlugin takes an object with keys referring to variables to be replaced and values referring to the values that should be substituted. With this configuration, it’ll replace all process.env.NODE_ENV instances with "production", which will make UglifyJS understand that the comparison expression is always false and remove it:

ECMAScript imports#

The next way to decrease the front-end size is to use ECMAScript imports and exports.

When you use ECMAScript imports and exports, webpack becomes able to do tree-shaking. Tree-shaking is when a bundler traverses your whole dependency tree, checks what of them are used, and keeps only the used ones. So, if you use ECMAScript module syntax, webpack can eliminate the unused code:

You write two files where only one export is used
// comments.js
export const commentRestEndpoint = '/rest/comments';
export const render = () => { return 'Rendered!'; };

// index.js
import { render } from './a.js';
render();

Webpack realizes that componentRestEndpoint is not used and doesn’t generate a separate export point in the bundle
// bundle.js (part of)
(function(module, __webpack_exports__, __webpack_require__) {
  "use strict";
  /* unused harmony export commentRestEndpoint */
  /* harmony export */__webpack_exports__["b"] = render;

  var commentRestEndpoint = '/rest/comments';
  var render = function () { return 'Rendered!'; }
})

UglifyJS removes the unused variable
// bundle.js (part of)
(function(n,e){"use strict";e.b=r;var r=function(){return"Rendered!"}})

This works even with libraries; the library should also be written with ECMAScript modules.

Pitfall: tree-shaking doesn’t work without UglifyJS#

The less-known fact is that the unused code is removed not by webpack, but by UglifyJS. Webpack just removes export statements for the exports that aren’t used, which makes them possible to be removed by a minifier. Therefore, if you compile your bundle without the minifier, the bundle won’t get smaller.

See how to enable UglifyJS in the “Minification” section.

Pitfall: don’t transpile ECMAScript imports to the CommonJS format#

If you use Babel with babel-preset-env or babel-preset-es2015, check the settins of these presets. By default, they transpile ECMAScript’s import and export to CommonJS’ require and module.exports. Pass the { modules: false } option to disable this:

// webpack.config.js
module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        use: [{'babel-loader', options: {
          presets: [['es2015', { modules: false }]]
        }]
      }
    ]
  }
};

Pitfall: complex cases aren’t optimized#

In some complex cases – e. g. when you re-export something (export * from 'file.js'), or when you compile classes with the TypeScript compiler – webpack can’t optimize your bundle. The bad things about this are that the cases when this happens aren’t obvious, and it’s unclear when this will be fixed. Here’s the corresponding GitHub issue: webpack/webpack#2867

Moment.js#

Tested with moment.js 2.18.1

Moment.js is a library for working with dates. By default, when you include it in your app, it takes 217 kB of minified code. That’s huge – the average size of JavaScript on a page was 417 kB in April 2017. The good part, however, is that it can be easily reduced.

165 kB of the size of moment.js is localization files. They’re included even if you don’t use them. This happens because moment.js chooses the localization files dynamically, during runtime:

Webpack doesn’t know which files you’ll need, so it includes all files from the locale directory.

To deal with it, specify the exact files with ContextReplacementPlugin:

// webpack.config.js
const webpack = require('webpack');

module.exports = {
  plugins: [
    new webpack.ContextReplacementPlugin(
      // The path to directory which should be handled by this plugin
      /moment[\/\\]locale/,
      // A regular expression matching files that should be included
      /(en-gb|ru)\.js/
    )
  ]
};

Lodash#

Lodash is a collection of JavaScript utilities.

Tested with Lodash 4.17.4

When you include Lodash, your bundle grows by 72 KB of minified code. That’s the size of all the 316 Lodash methods. If you use only, like, 20 of them, then approximately 65 KB of the code do just nothing except slowing down the page loading.

Thankfully, Lodash lets you include only the methods you need. The most basic way to do this is to import methods from the files they’re implemented in:

72 KB → 8.27 KB
- import _ from 'lodash';
- _.get();
+ import get from 'lodash/get';
+ get();

This approach might work if you’re starting a project from scratch, but it doesn’t work for existing projects. What you’re gonna do, rewrite all imports? That’s too much work. That’s why I prefer using babel-plugin-lodash and sometimes lodash-webpack-plugin.

babel-plugin-lodash is a plugin for Babel that replaces generic imports with concrete ones during compilation. That is, it does exactly the same thing as depicted in the snippet above:

72 KB → 8.27 KB
// Before babel-plugin-lodash
import _ from 'lodash';
_.get({ a: { b: 5 } }, 'a.b');

↓

// After babel-plugin-lodash
import _get from 'lodash/get';
_get({ a: { b: 5 } }, 'a.b');

lodash-webpack-plugin is a plugin for webpack that modifies Lodash behavior by removing some code and thus cutting the bundle size. For example, _.get by default supports deep paths. If you don’t need this, you can enable lodash-webpack-plugin, which will remove this support:

72 KB → 772 B
// Before babel-plugin-lodash + lodash-webpack-plugin
import _ from 'lodash';
_.get({ a: { b: 5 } }, 'a.b');
// → returns 5

↓

// After babel-plugin-lodash + lodash-webpack-plugin
import _get from 'lodash/get';
_get({ a: { b: 5 } }, 'a.b');
// → returns undefined

Keep in mind, however, that you can’t just enable the plugin and leave it as-is. This plugin changes the Lodash functionality, so your existing code could break. Take a look at the list of features it removes by default.

externals#

Sometimes you have a large project where some code is compiled with webpack and some code is not. Like a page with crosswords, where the crosswords module is built with webpack, and the site around it is not:

If both pieces of code have common dependencies, you can share the dependencies between them. This is done with the webpack’s externals option which lets you alias module imports to something different.

The common usage is when you have an instance of a library in the global object (like window), and you want to alias the library imports to this instance. In this case, you pass an object mapping the module names to the variable names:

// webpack.config.js
module.exports = {
  externals: {
    'react': 'React',
    'react-dom': 'ReactDOM',
  }
};

Webpack will replace all module references with variable references.

A less known approach is when the old code doesn’t put the libraries into the global object but loads them with an AMD-compatible loader. In this case, you can compile your webpack as an AMD bundle and alias modules to paths to the libraries:

// webpack.config.js
module.exports = {
  output: { libraryTarget: 'amd' },

  externals: {
    'react': { amd: '/libraries/react.min.js' },
    'react-dom': { amd: '/libraries/react-dom.min.js' },
  }
};

Webpack will wrap your bundle into define() and make it depend on the libraries from externals:

// bundle.js
define(["/libraries/react.min.js", "/libraries/react-dom.min.js"], function () { … });

Then, the loader will load your libraries along with the bundle. The good thing here is that the libraries will be cached in the browser or in the loader cache – so they won’t be loaded twice.

Σ: Decrease front-end size#

  • Configure minification
  • Pass NODE_ENV=production to the code
  • Use ECMAScript imports and exports
  • Drop unused locales in Moment.js
  • Drop unused methods in Lodash
  • Use externals if you have common libraries

Task Improve caching#

Given: you have a front-end application. You want to cache it better so that the visitor loads it faster and doesn’t re-download the whole app when it’s updated.

Using hash#

The default approach of doing caching is to tell the browser cache a file for a very long time (e.g. a year), and rename the file when changing it to force browser to re-download it:

<!-- Before the change -->
<script src="./index.js?version=15">

<!-- After the change -->
<script src="./index.js?version=16">

Webpack also lets you do such thing. However, instead of versioning a file, it calculates the file hash which you can specify in the bundle name. In this case, each time you change the code, the file name will change, and the browser will re-download it:

// webpack.config.js
module.exports = {
  entry: './index.js',
  output: {
    filename: 'bundle.[chunkhash].js'
       // → bundle.8e0d62a03.js
  }
};

The only remaining problem is how to get the file name to send it to the client. There are two solutions: HtmlWebpackPlugin and WebpackManifestPlugin.

HtmlWebpackPlugin is a more automated solution. During compilation, it generates an HTML file which includes all compiled resources. If your server logic is simple, then this plugin should be enough for you:

<!-- index.html -->
<!doctype html>
<!-- ... -->
<script src="bundle.8e0d62a03.js"></script>

WebpackManifestPlugin is a more flexible solution which is useful if you have a complex server part. It generates a JSON file with a mapping between file names without hash and file names with hash. You can use this JSON on your server:

{
  "bundle.js": "bundle.8e0d62a03.js"
}

Pitfall: hash is non-deterministic#

Webpack’s hash is non-deterministic. This means that in different circumstances, webpack could calculate different hashes for the same file. Because of this, it’s better to use a plugin that replaces webpack’s hash algorithm with its own – e. g. webpack-chunk-hash.

Code splitting#

The next way to improve caching is to split the bundle into smaller pieces.

Imagine you have a large website, and you’re compiling it into a single bundle:

Each time you’re changing a single module, the whole bundle gets recompiled. This means that even if you’re changing the comments module, and a specific user is only visiting the main page, they’ll still have to re-download the code for this page.

If you split your bundle into several pieces – one for the main page and one for the article page – the user will only have to re-download the changed piece of code. Webpack lets you do this. In webpack terminology, these pieces of the bundle are called chunks.

To split the code into chunks, you specify several entry points and do a few other changes. Here’s the optimal webpack config:

You specify multiple entry points, and webpack generates a separate chunk for each point. Each chunk will only include the dependencies it needs
module.exports = {
  entry: {
    homepage: './index.js',
    article: './article.js'
  },
  output: {
You replace a fixed filename with [name]. [name] will correspond to the entry point name
    filename: '[name].[chunkhash].js'
  },
  plugins: [
You add WebpackManifestPlugin and WebpackChunkHash – plugins from the previous section
    new WebpackManifestPlugin(),
    new WebpackChunkHash(),

You add two CommonsChunkPlugins. They let you move some code from existing chunks to new commons chunks.

The first plugin moves all node_modules dependencies to a separate chunk. This allows you update the code without invalidating dependencies.

The second plugin moves webpack’s runtime to a separate chunk. This allows you to update runtime without invalidating other code. Runtime is a webpack’s system code that is responsible for loading the app

    new webpack.optimize.CommonsChunkPlugin({
      name: 'vendor',
      minChunks: m => m.context &&
        m.context.includes('node_modules'),
    }),
    new webpack.optimize.CommonsChunkPlugin({
      name: 'runtime',
      chunks: ['vendor'],
      minChunks: Infinity,
    }),
You add ashedModuleIdsPlugin. By default, each module in webpack has an ID which corresponds to its order. If you add a new module, it can affect other module ids and invalidate the cached chunks. This plugin replaces order-based IDs with hash-based ones
    new webpack.HashedModuleIdsPlugin(),

You add ChunkManifestPlugin.

By default, the webpack’s runtime contains a mapping between IDs of chunks and their names. If you configure the file name to contain the hash, as we did with the filename option, the hash will change with each file change, and so will the runtime.

ChunkManifestPlugin lets you extract this mapping into a separate JSON file. On the server, you’ll need to inline this file into the global webpackManifest variable

    new ChunkManifestPlugin({
      filename: 'chunk-manifest.json',
      manifestVariable: 'webpackManifest'
    })
  ]
};

With this config, webpack will generate 6 files:

Two separate entry points. Each should be loaded on the corresponding pages
homepage.a68cd93e1a43281ecaf0.js
article.d07a1a5e55dbd86d572b.js
File with vendor dependencies and file with webpack runtime
vendor.1ebfd76d9dbc95deaed0.js
runtime.d41d8cd98f00b204e980.js
Two manifest files that you’ll need on the server
manifest.json
chunk-manifest.json

And this is how often they’ll change:

  • homepage and article – when the app code in these modules changes,
  • vendor – when any dependencies of the app change,
  • runtime – when webpack’s runtime code changes (i.e. rarely and only with new webpack versions),
  • manifest.json – when you add a new chunk – but that doesn’t matter because this file is used in the server,
  • chunk-manifest.json – on any code change – but that doesn’t matter because this file is used in the server.

That’s a bit more files, but it lets you effectively leverage long-term caching.

On-demand code splitting#

The next way to improve caching (and optimize time to first paint) is to load some parts of code on demand.

Imagine you have a page with an article:

When opening this page, the visitor wants to see the content at first. Comments, sidebar and other parts of the page are less relevant to them. However, if you bundle all these blocks into a single file, the visitor will have to wait until the whole file is downloaded – with all the page modules. This isn’t cool.

Thankfully, webpack lets you optimize this by loading code on demand. You can specify that you want to load specific modules dynamically, and webpack will move them to separate chunks and download when they’re required. This is how it works:

You have an article-page.js file. When you compile it, the bundle receives all the code for articles, comments and sidebar
// article-page.js
import { renderArticle } from './components/article';
import { renderComments } from './components/comments';
import { renderSidebar } from './components/sidebar';

renderArticle();
renderComments();
renderSidebar();

To load code on demand, you replace static import with dynamic import() calls. Webpack will move the code from ./comments.js and ./sidebar.js into separate chunks and load them when they’re required
// article-page.js
import { renderArticle } from './components/article';
renderArticle();

import('./comments.js')
  .then((module) => { module.renderComments(); });
import('./sidebar.js')
  .then((module) => { module.renderSidebar(); });

This change will improve the initial loading performance. Also, it will optimize caching because when you change the code that belongs to a specific chunk, other chunks won’t get affected.

The only thing left is to add chunk hashes to their names. This is done with output.chunkFilename option. This option is specific to chunks generated by on-demand code splitting:

// webpack.config.js
module.exports = {
  output: {
    filename: '[name].[chunkhash].js',
    chunkFilename: '[name].[chunkhash].js',
  }
};

Pitfall: Compiling with Babel#

If you compile this code with Babel with default presets, you’ll have a syntax error: Babel don’t understand import() out of the box. To prevent the error, add the syntax-dynamic-import plugin.

Other solutions#

There are a couple of other solutions that I haven’t worked with but which should also bring benefits with caching:

  • AggressiveSplittingPlugin is a plugin that optimizes your code for HTTP/2 by splitting each chunk into smaller chunks as much as possible. This greatly improves caching on the client side but slightly worsens the gzip compression. See the example in the webpack repository.
  • OfflinePlugin is a plugin that’s usually used for creating offline-ready apps. However, you can use it to improve caching too! The plugin generates a service worker that downloads all the site resources in the background. So when a visitor visits the site and then switches to a different page, they’ll have all the necessary files already cached. See the OfflinePlugin docs.

Σ: Improve caching#

  • Add hash to the name of your resources and make the server tell clients to cache the resources for a long time
  • Split your code into smaller chunks with different entries, on-demand code splitting and AggressiveSplittingPlugin
  • Try caching your resources in the background with OfflinePlugin

The next part of the guide, “Speeding up build and improving the development workflow”, is coming in July. Leave your email to know when it’s out:
(you’ll receive an email about the next part of the guide + a couple of more webpack-related posts if I write them; no spam)

How webpack’s ContextReplacementPlugin works

ContextReplacementPlugin is a pretty advanced tool. I know only two its use cases that appear common enough: optimizing bundle size with moment.js and something related to Angular routing. The solutions for them are copy-pasteable.

However, once you want to understand how the solution is working, you get stuck: the official docs for the plugin are short and complex, and googling also doesn’t give you a lot of information. This gets even harder if, at some moment, you need to apply this plugin to a less common problem.

It took me several months of using ContextReplacementPlugin to finally realize what it really does. I hope this post will save you from that.

Once in a while, you need to write a dynamic import. A dynamic import is when an imported file is only known at runtime:

require('./inputs/' + inputType + '/index.js');

When you’re bundling this with webpack, webpack can’t know what exact file you’ll need. To make the application work, webpack will find and import all index.js files in all subdirectories of the inputs directory, recursively. This can significantly increase your bundle size.

ContextReplacementPlugin lets you change how webpack deals with dynamic imports. In this case, you can use it to narrow the search scope and thus cut the bundle size.

Here’s an example:

ContextReplacementPlugin is often used with moment.js, a library for working with dates. The thing with moment.js is that it has a bunch of locales, and it imports them dynamically on runtime:

// moment.js
require('./locale/' + name + '.js');

To make this work, webpack imports all the locales it can find – which adds 330 kB of non-minified code.

In most cases, you only need to support a few locales and don’t need all the other ones. ContextReplacementPlugin lets you specify the specific locales that webpack should import:

// webpack.config.js
const webpack = require('webpack');

module.exports = {
  plugins: [
    new webpack.ContextReplacementPlugin(
      /moment[/\]locale/,
      // Regular expression to match the files
      // that should be imported
      /(en-gb|ru).js/
    )
  ]
};

That works, that’s copy-pasteable, but that’s cryptic. Let’s see what all these parameters really mean.

Context and how to replace it#

Each time you compile a dynamic import like this:

require('./locale/' + name + '.js');

webpack collects three pieces of information:

  • in what directory the files are located (in this case, it’s ./locale),
  • what regular expression should a file match to be imported (in this case, it’s /^.*.js$/),
  • and if webpack should look for the files in the subdirectories (recursive flag; always true by default).

These three pieces of information are called context.

Then, webpack searches the appropriate files using the context. That means it looks into the directory specified in the context, looks into its subdirectories if the flag from the context is true, and uses the regular expression from the context to match and import the files.

Now, the key point. ContextReplacementPlugin lets you replace the context that webpack uses for search – e.g. override the directory in which webpack should look for the files or the regular expression that webpack should use to match the files.

For example, with the moment.js example from above, ContextReplacementPlugin replaces the original regular expression /^.*.js$/ with another expression we pass – /(en-gb|ru).js/. This way, we import only the necessary locales.

To replace the context, you pass the parts you want to replace as the last parameters of ContextReplacementPlugin:

new webpack.ContextReplacementPlugin(
  // The criterion to search; we’ll get to it in a moment
  searchCritetion: RegExp,
  // The new directory to look for the files
  [newDirectory]: String,
  // The new recursive flag. True by default.
  // Pass false to disable recursive lookup
  [newRecursiveFlag]: Boolean,
  // The new regular expression to match
  // and import the files
  [newFilesRegExp]: RegExp
)

Examples#

If you don’t fully understand how webpack generates context, here’re examples:

Finding the original context to replace#

So, you’ve got a case when you have a dynamic import, and you need to replace its context with another one. How do you find this import and pass it to the plugin? You search by the context that the import generates.

ContextReplacementPlugin allows you to search by the context’s directory. This means that if you want to find an import, you should calculate what context it creates, and then pass a regular expression matching that directory to the context. That sounds complex, so let’s just see how it works:

You have an import like this:

require('./locale/' + name + '.js')

and you want to apply ContextReplacementPlugin to it to limit the number of the included files.

Step 1. Calculate the context that will be created by the import
With this import, the context has three parts:

  • directory, which is ./locale,
  • the regular expression to match the files, which is /^.*.js$/.
  • and the recursive flag (always true).

Step 2. Extract the directory from the context
Here, the directory from the original context is //./locale//.

Step 3. Find the full absolute path of the directory
On your machine, the full path of the directory could be something like '/usr/…/my-project/node_modules/moment/locale'.

Step 4. Create a regular expression that matches the path
This could be as simple as /locale/. Or this could be something more specific, like /moment[/\]locale/ (cross-platform version of /moment/locale/).

I recommend to be as specific as possible: a simple regular expression, like /locale/, can unexpectedly match other imports, like require('./images/flags/locale/' + localeName).

Step 5. Pass the regular expression as the first parameter of ContextReplacementPlugin
Like this:

new webpack.ContextReplacementPlugin(
  /moment[/\]locale/,
  …
)

NormalModuleReplacementPlugin#

Note: ContextReplacementPlugin works only with dynamic imports. If you ever want to configure a redirect for a normal, non-dynamic import, use NormalModuleReplacementPlugin. It works with static imports (only with them), and it’s way simpler to understand.

Bonus point: webpack 2#

The traditional usage of ContextReplacementPlugin is to replace one context with another context. However, webpack 2 brought a new API that you can use to granularly redirect imports:

new ContextReplacementPlugin(
  /moment[/\]locale/,
  path.resolve(__dirname, 'src'),
  {
    './en.js': './dir1/en.js',
    './ru.js': './dir2/ru.js',
  }
)

The code above:

  • will add two files, ./src/dir1/en.js and ./src/dir2/ru.js, to the bundle,
  • will redirect all runtime requests from node_modules/moment/locale/en.js to dir1/en.js,
  • will redirect all runtime requests from node_modules/moment/locale/ru.js to dir2/ru.js.

This is something that can’t be achieved with traditional ContextReplacementPlugin. Unfortunately, this API is only briefly mentioned in a GitHub issue.

When is this helpful? I can only think of some cases when a large existing project is being migrated to webpack – but I haven’t seen any practical example. If you know any, share them in the comments!

Here’s how you write your own redirection:

new ContextReplacementPlugin(
  // Specify the criterion for search
  /moment[/\]locale/,
  
  // Specify any directory that’s common
  // for all the redirection targets.
  // It can’t be __dirname: for some reason,
  // that doesn’t work
  path.resolve(__dirname, 'src'),
  
  // Specify the mapping in form of
  // { runtime request : compile-time request }
  // IMPORTANT: runtime request should exactly match
  // the text that is passed into `require()`
  // IMPORTANT: compile-time request should be relative
  // to the directory from the previous parameter
  {
    './en.js': './dir1/en.js',
    './ru.js': './dir2/ru.js',
  }
)

Σ#

The key points:

  • Each time you do a dynamic import, webpack creates a context. Context is an object containing the directory to look into, the recursive flag and the regular expression for matching files.

  • Use ContextReplacementPlugin to change how webpack handles dynamic imports. This can be helpful when you need to decrease the bundle size or migrate some complex code to webpack.

  • You can granularly redirect compile-time requests to the different files with webpack 2.

Further reading#

Here’re some related links:


This is an article in my series of articles about webpack. The next one, “Speeding up build and improving the development workflow”, is coming in July. Leave your email to know when it’s out:
(you’ll receive an email about this post + a couple of more webpack-related posts if I write them; no spam)

webpack for real tasks: bundling front-end and adding compilation

This is the first part of a three-part introduction into webpack:

  1. Bundling front-end and adding compilation (you are here!)
  2. Decreasing front-end size and improving caching
  3. Speeding up build and improving the development workflow

Want to stay tuned for the future posts? Subscribe

What is webpack#

Webpack is a front-end bundler. And a front-end bundler is a tool that combines multiple modules into a single file called bundle:

While the main purpose of webpack is bundling, it also has a lot of other abilities. For example, webpack can compile your front-end, split your code into multiple files or optimize your bundle size. I’m reviewing many of these abilities in this series of posts.

If you’re not familiar with the concept of bundling, Preethi Kasireddy wrote a good introduction to it. Check it out and come back!

Task Bundle front-end#

Given: you have an application that consists of lots of modules. Like Cut the Rope:

You want to bundle the modules into a single file: to speed up the loading of an app*, or to serve a library as a single module, or for another reason. Let’s see how webpack can help with this.

* – if you’re thinking “Wait, HTTP/2 made bundling unnecessary”, see the Khan Academy’s post about why no bundling is a bad idea

// comments.js
define([&#039;./dist/lodash.js&#039;], (_) => {
  // …
  return { … };
});

// index.js
define([&#039;./comments&#039;, …], (comments, …) => {
  // …
  comments.render(commentsData, &#039;#comments&#039;);
});

If your code uses AMD, CommonJS or ES modules, everything is simple. Webpack supports these module systems out of the box, so to compile a project with them, you’ll only need to specify the entry file and the name of the resulting file.

To do this, create a file called webpack.config.js in the root of your project with the content like this:

// webpack.config.js
module.exports = {
  // An entry point. It’s the main module of your application
  // that references all the other modules
  entry: &#039;./src/index.js&#039;,

  output: {
    // The directory where the bundle should be placed
    path: &#039;./dist&#039;,
    // The name of the resulting bundle
    filename: &#039;bundle.js&#039;,
  },
};

Then, run webpack:

npm install --global webpack
cd your/project/directory
webpack

Once you launch webpack, it will compile your project and generate a bundle with all your JavaScript. What’s left? Replace the import of your old entry file with the name of the new file:

// index.html
<!doctype html>
<body>
  <!-- … -->
- <script src="./src/index.js"></script>
+ <script src="./dist/bundle.js"></script>
</body>

Task solved.

Bonus point: all module types at once#

Webpack supports all three module types in the same project simultaneously. So if a part of your code is in AMD and the other part is in ES modules, it will just work. This can be helpful if you decide to gradually migrate from one module format to another.

Bad case: your code uses your own module system#

// comments.js
MyApp.define([&#039;./dist/lodash.js&#039;], (_) => {
  // …
  return { … };
});

// index.js
MyApp.define([&#039;./comments&#039;, …], (comments, …) => {
  // …
  comments.render(commentsData, &#039;#comments&#039;);
});

In case your code uses a module system different from AMD, CommonJS or ES modules, things get more complicated. To make webpack work with your code:

  • either migrate your code to a supported module format. Facebook has a tool called codemod which can automate a massive refactoring and could be useful for you;

  • or write a Babel plugin for converting your custom module format to AMD, CommonJS or ES modules. This plugin will be executed on each compilation. Take a look at babel-plugin-transform-amd-to-commonjs to get an idea of how to write it. (We’ll see how to enable Babel a bit later.)

After you deal with the custom module format, configure the entry point and the output as described in “Good case” above.

Bonus point: global webpack installation#

Although installing webpack globally (npm install –global webpack) is the easiest way to do the build, I prefer using it through npm scripts. webpack is often installed as a project dependency (because it provides plugins that are used in the configuration), so using it from npm scripts prevents the version conflict. Also, npm scripts can be run from any directory inside the project, not only from the root:

// package.json
{
  "scripts": {
    "build": "webpack"
  }
}
# Console
npm run build

Task Compile JavaScript

Given: you have some code that cannot be run in the browser. This can be code that uses features from the next JavaScript standard or even code in another language like TypeScript:

// comments.js
import _ from &#039;lodash&#039;;

export async render(…) {
  const userData = await getUserData(userId);
  // …
}

You want to compile it to make it work. Let’s see how webpack helps to solve this task.

Assume you’re using Babel. You may have used it from the command line specifying the input and the output:

babel ./src -d ./dist

Or you may have used it from Gulp as a part of a stream:

gulp.task(&#039;default&#039;, () => {
  return gulp.src(&#039;./src/**/*.js&#039;)
    .pipe(babel())
    .pipe(gulp.dest(&#039;dist&#039;));
});

webpack has a bit different approach. It uses loaders.

A loader is a JavaScript module. Webpack pipes all files through specified loaders before adding them into bundle.

A loader accepts any input and converts it to JavaScript which webpack works with. Loaders can be organized into chains; a chain accepts any input, pipes it through the loaders and passes the result to webpack. In this case, intermediate loaders can return anything, not only JavaScript.

In webpack, Babel works as a loader. To use it, install babel-loader with its peer dependencies. Then, tell webpack to apply the loader with the module.rules option:

// webpack.config.js
module.exports = {
  // ...
  module: {
    rules: [
      {
        // Take every JavaScript file imported into a bundle...
        test: /.js$/,
        // ...and pipe it through babel-loader...
        use: [&#039;babel-loader&#039;],
        // ...with the following options
        options: {
          presets: [&#039;env&#039;],
        },
      },
    ],
  },
};

The same approach works for TypeScript:

// webpack.config.js
module.exports = {
  // ...
  module: {
    rules: [
      {
        test: /.ts$/,
        use: [&#039;ts-loader&#039;]
      }
    ]
  }
};

Or you can chain Babel and Typescript to do transformations that aren’t supported by the latter:

// webpack.config.js
module.exports = {
  // ...
  module: {
    rules: [
      {
        test: /.ts$/,
        // Loaders are applied from right to left
        use: [&#039;ts-loader&#039;, &#039;babel-loader&#039;]
      }
    ]
  }
};

You can find the list of the most popular loaders in webpack docs.

Bonus point: different ways to specify a loader#

Above, I passed a string array into the use property to apply a loader. There’re two more ways to specify loaders. Here’re all of them:

// Specifies a single loader with or without options
{
  test: /.js$/,
  loader: 'babel-loader',
  options: { ... } 
},

// Specifies multiple loaders without options
{
  test: /.ts$/,
  use: ['ts-loader', 'babel-loader']
},

// Specifies multiple loaders with or without options
{
  test: /.ts$/,
  use: ['ts-loader', { loader: 'babel-loader', options: { ... } }]
}

Choose between them based on your needs.

Bonus point: loaders in the require query#

Apart from specifying loaders in webpack.config.js, you can also specify them in your import request:

import comments from 'ts-loader!babel-loader?presets[]=env!./comments.ts';

This can be useful for testing. Nevertheless, I don’t recommend using it in production because it makes your code dependent on a specific bundler.

Task Manage other files#

Webpack can also help you manage your styles, images or any other files.

Given: front-end application with styles and other assets.
You want to manage them with webpack to reduce the number of the necessary tools. Let’s see how to do this.

The one important thing you should remember here is the following:

In webpack, every asset is imported as a module

That is. In a traditional task manager like Gulp you split your front-end compilation by a file type:

// gulpfile.js
gulp.task(&#039;js&#039;, function () {
  return gulp.src(&#039;./src/index.js&#039;)
    // ...
    .pipe(gulp.dest(&#039;./dist/&#039;));
});

gulp.task(&#039;css&#039;, function () {
  return gulp.src(&#039;./src/**/*.scss&#039;)
    // ...
    .pipe(gulp.dest(&#039;./dist/&#039;));
});

In webpack, however, you don’t split the compilation. You treat the front-end as a whole single thing. To include styles and other assets, you import them:

// comments.js
import _ from &#039;lodash&#039;;
import &#039;./comments.scss&#039;; * ← Here
import iconUrl from &#039;./commentsIcon.svg&#039;; * ← And here

export function render() {
  // ...
}

and apply specific loaders to teach webpack to handle these imports. Imported files either get inlined into the bundle or placed next to it. This depends on the loader you use.

Let’s see how to use this in practice.

Q: For the mother of god, why?#

A: I don’t know why it was decided like this in the beginning. Nevertheless, this brings a real benefit. Usually, with webpack, you import all files used by a component straight into its main JS file. Because of this, all these files will be included into your bundle only if you actually use the component in your app.

Styles#

Here’s the most basic approach:

// comments.js
// Import the file to pass it under webpack’s management
import &#039;./comments.css&#039;;

// webpack.config.js
module.exports = {
  // ...
  module: {
    rules: [
      // Configure webpack to pass all .css files through css-loader and style-loader
      // (remember that loaders are applied from right to left)
      { test: /.css$/, use: [&#039;style-loader&#039;, &#039;css-loader&#039;] },
    ],
  },
};

Here’s what the loaders do:

  • css-loader reads the CSS file, passes all @import and url() in that file through webpack and returns the result,
  • style-loader gets the passed CSS content and creates code that will append that CSS to <head> when the bundle gets loaded.

If you need to compile your styles with a preprocessor, append a corresponding loader into the loaders list and pass the importLoaders = 1 option to css-loader:

// webpack.config.js
module.exports = {
  // ...
  module: {
    rules: [
      { test: /.css$/, use: [
        &#039;style-loader&#039;,
        { loader: &#039;css-loader&#039;, options: { importLoaders: 1 } },
        &#039;sass-loader&#039;,
      ] },
    ],
  },
};

See: postcss-loader, sass-loader, less-loader.

Bonus point: CSS Modules#

In the examples above, styles don’t provide any exports and therefore are just imported. However, there’s no technical limitation to provide exports, so there are approaches that do this. One of them is CSS Modules.
With CSS Modules, importing a stylesheet returns a JavaScript object with class names specified in the file. These class names are converted to be unique, so you can use a class name like .button in several components without any collision:

/* button.css */
.button { /* styles for the normal state */ }
.disabledButton { /* styles for the disabled state */ }
.errorButton { /* styles for the error state */ }
// button.js
import styles from './button.css';

buttonElement.outerHTML = `<button class=${styles.button}>Submit</button>`;

CSS Modules are enabled by passing the modules = true option to css-loader. Read more in the CSS Modules introduction.

Other files#

All the other files are also managed using loaders. Approach for every file type is different, so find the appropriate loader and look into its docs. See the list of the most popular loaders.

Here’re examples of loaders for different file types:

  • svg-url-loader:
    import iconDataUrl from './icon.svg';
    // => iconDataUrl contains a data url of the icon
    
  • pug-loader:

    import template from './template.pug';
    // => template is a function that returns
    // the rendered HTML
    
  • file-loader:

    import documentUrl from './document.pdf';
    // => document.pdf is emitted next to the bundle;
    // documentUrl is its public URL
    

Σ#

The key points:

  • The minimal config of webpack is just the entry point and the output file. Webpack works with AMD, CommonJS and ES modules out of the box
  • You can use loaders to compile your JavaScript and manage other front-end files

  • Every file is imported as a module

See the second part, “Decreasing front-end size and improving caching”


The third part of the guide, “Speeding up build and improving the development workflow”, is coming in July. Leave your email to know when it’s out:
(you’ll receive an email about the next part of the guide + a couple of more webpack-related posts if I write them; no spam)

Thanks to Artem Sapegin for reviewing this post

Redux vs React’s setState()

Sometimes you can get into a situation when you’re choosing between Redux and React’s setState(). When I’m doing such choice, I like to use the following criterion.

Imagine that the browser keeps the stuff you store in Redux even when you refresh the page.

Use Redux if you want to keep the state across the page refreshes. Use setState() if this isn’t necessary

Basically, this is about the importance of a piece of state. Is it important enough to put it into the store and keep it across refreshes? If yes, use Redux. If no, setState() would do the job just fine.

Here’s how I’d make the choise:

Redux setState()
The selected value in a dropdown on a page The open/closed state of a dropdown
The current page of a book in a book reader app The visibility of toolbars in the app
The current level in Angry Birds The state of birds and pigs in the current level in Angry Birds

For me, the state in the left column is important, and the state in the right column is not. Your app can require a different decision.


Posting this on Reddit triggered a great comment:

Redux = data grabbing and global state that’s shared across more than one component.
setState = silo’ed state that is isolated from other components.
jiblet84

Yes! Another (and probably more common, as I’m realizing after receiving the feedback) criterion for choosing between Redux and setState() is how global the state is. So here’s another approach:

Use Redux if your state is shared across multiple components. Use setState() if it’s used only in a single component.

I’m writing a practical introduction into webpack in three parts, and I’m publishing the second part soon. Subscribe

Exclusive code ownership (why it’s bad and what to do with it)

In December 2016, a front-end developer left a team where I work. Apart from other changes, this created an issue. When he was working, the project had a couple of complex modules we were developing with him together. When he left, I realized that only I am able to work with them – other two team members knew nothing about how these modules function. This is called “exclusive code ownership”, and it’s bad in the long term, so I had to start fixing this. Here’s why it’s bad and what to do with it.

🖐 🔮

Problems#

Bus factor#

Imagine there’s a complex module in your project that only you work with, and you’re hit by a bus tomorrow. Will the team be able to continue developing this module the next day? Most likely no. This means the bus factor of your team is 1 – that is, it’s enough to kill remove a single developer from the project to severely slow or even stop its development.

Until recently, our project has been heavily relying on a custom front-end framework that was developed by only a single team member. No other developers knew its inner parts, and only a couple of them knew it deep enough. This meant the bus factor of the whole front-end team was just one. This, along with other issues, led us to migrate from this custom framework to React in 2016.

Bottlenecks#

If there’s a module in the project that you exclusively own, then, regardless of the team size, only you will be able to make changes in it. If tomorrow you receive 5 critical issues related to this part, there’ll be nobody to help you. You may even slow down your team if these critical issues block them.

We had this problem when a part of our front-end infrastructure changed, and this broke builds for the whole team. Because it was only me who knew how all this stuff works, I had to fix the problems one-by-one. If other team members knew this, they could’ve helped me bring everything back sooner.

Leaving is hard#

If once you decide to leave the project, you’ll have to transfer all your unique knowledge about how these parts work to other team members. Depending on the knowledge size, this can take significant time, and it still will be less effective than if other members had real experience with the code.

You create risk and slow down delivery

What to do#

Documentation and maintainability#

You have probably heard this advice for a lot of times, but it’s never bad to repeat it again. Make your code documented and maintainable. This includes code cleanliness, inline comments, and external documentation.

It’s your choice how much to document the code. I usually try to make it self-documenting and add comments only if the purpose of the code or the decisions behind it are still unclear. Also, remember that self-documenting code is better than comments, and comments are better than external documentation. People occasionally forget to update the comments and often forget to update the external documentation, so the only up-to-date source of information is code.

Commit messages#

In my experience, writing detailed commit messages is very underestimated. I’m sure that one of the best ways to share your code knowledge is to include it into your commits. Once you commit something, it stays in the history forever, and tools like git blame help you to easily find which commit changed the specific lines. This means that even if you leave the project, the other developer that will work with the piece you’ve written will be able to git blame the code and see why you made this or that change.

What to write in a good commit message? Well, this is one of the examples from my project I like (I wrote this so I like it, pretty obvious):

Look, this commit includes:

  • the issue number (so that another developer could go and see the business requirements behind the change),

  • the short description of the change (which is usual),

  • and the longer text that explains the commit.

The last part is exactly where you put your code knowledge. Answer why you did the changes. Explain the background that could be unclear to another developer in the half of the year. Document the limitations that your solution has and the issues it creates (if any). Include everything else that you consider significant.

Follow to this Caleb Thompson’s article if you want to read more about good commit messages (especially, take a look at point 4). Also, kudos to Andrey Listochkin for giving the advice about this in one of his talks.

Processes#

If you’re a team lead, and you realize that you have this problem in your team, you can use your power to change this:

  1. Find what modules are exclusively owned by someone.
  2. Find what tasks you can give to other team members so that they get experience with these modules.
  3. Ask the module owner to introduce developers that will work with the tasks into the module code.

If you’re a regular developer, and you realize your team has this problem, talk with your team lead. Feel free to show them this article.

Also, this should be solved by introducing mandatory code review or pair programming, though we haven’t practiced this ourselves.

Docs, commit messages, processes

#

So, well, the checklist:

  • Write self-documenting code

  • Use comments when it’s necessary

  • Write good commit messages answering “why”, not only “what”

  • Involve other team members into working with the code

Follow me on Twitter: @iamakulov

How to install Google Chrome on Travis (in a container-based environment)

Updated 22 Jan 2017: removed the “addons.apt.sources” because looks like the google-chrome source is now provided by default.

Here’s the code. Scroll down for more details:

dist: trusty
sudo: false

addons:
  apt:
    packages:
      - google-chrome-stable

before_script:
  - export DISPLAY=:99.0
  - sh -e /etc/init.d/xvfb start &
  - sleep 3

What’s this#

Travis CI is a popular continuous integration tool. It can be used to run tests when the project code changes, automatically publish changes to the repository, etc., etc. With this config, we configure Travis to install Chrome for each execution. This can be used for e.g. automated UI testing.

Container-based infrastructure#

Since the beginning of November 2016, you can run Ubuntu 14.04 on Travis CI in a container-based environment. A container-based environment is cool because it noticeably speeds up builds, has more resources and more. The drawback is that you can’t use sudo, but Travis has replacements for some common use-cases.

What does this do with Chrome? Chrome can’t be installed on Ubuntu 12.04 which just recently was the only available container-based environment. If you were doing UI testing, you either had to use Firefox (geckodriver) which was buggy as hell or had to accept much longer build times. ¯\_(ツ)_/¯

This config does enable the container-based infrastructure.

What does the code mean#

sudo: false
dist: trusty

sudo: false switches the environment into the container-based mode (and disables sudo). dist: trusty switches the distribution to Ubuntu 14.04 (the default one is 12.04). Here’re the other field values if you need them: https://docs.travis-ci.com/user/ci-environment/

addons:
  apt:
    packages:
      - google-chrome-stable

This installs Chrome. Travis provides the apt addon which is a handy way to install necessary packages. (Also, it’s the only way possible in the container-based infrastructure.)

The packages part of the addon specifies to install the google-chrome-stable package. This package gets installed from the official Google Chrome source which seems to be enabled in Travis by default.

Here’re the docs for the apt addon: https://docs.travis-ci.com/user/installing-dependencies/#Installing-Packages-with-the-APT-Addon

before_script:
  - export DISPLAY=:99.0
  - sh -e /etc/init.d/xvfb start &
  - sleep 3

This starts xvfb (X Virtual Framebuffer) that imitates the display and makes Chrome think it’s run in a GUI environment. sleep: 3 is required to give xvfb time to start (this is what Travis recommends to do). If you need to set the screen resolution or the pixel depth, check out the docs: https://docs.travis-ci.com/user/gui-and-headless-browsers/#Using-xvfb-to-Run-Tests-That-Require-a-GUI

Did this help you? Follow me on Twitter: @iamakulov

npm 4 is splitting the “prepublish” script into “prepublishOnly” and “prepare”

Updated 24 Jun 2017: reflected changes in plans about npm v5..v6.

On October 20, npm is releasing v4.0.0. This release, apart from the other breaking changes, also includes one affecting a lot of packages: the prepublish npm script will be split into prepublishOnly and prepare.

📅 💥

Why#

In v1.1.71, npm made the prepublish script also execute when you run npm install without arguments. Before this, npm ran the script only before you publish a package.

The reasoning behind this isn’t clear, but as far as I’ve got it’s the following. You usually run npm install without arguments when you clone a package, go into its directory and try installing its dependencies. If you’re doing this, you’re most likely a developer, and you’re going to do something with this package; therefore it’ll be useful to prepare the package for using. Since the prepublish script often includes commands that build the package for publishing (= prepare it for using), npm decided to execute it in such case to do this job.

However, this prepublish behavior became disliked:

  • It’s weird. prepublish is pre + publish, and a lot of people didn’t assume it’s also being run when installing dependencies. I even thought it’s a bug in npm when I first discovered the way it works.

  • It creates problems. Many projects I’ve seen put building and testing commands into the prepublish script. Even npm recommends doing this. It’s convenient: it prepares the package for publishing and prevents an occasional release of broken code.
    However, if you use a CI environment like Travis or Appveyor which installs a new version of the package on each build, things become worse. Your build and test tasks get executed twice, once on npm install and once on the actual npm test. And this creates you problems such as increased build time or wrong build status.

Current behavior is disliked

What’s next#

npm 4 is splitting the prepublish script into two new: prepare and prepublishOnly.

  • prepare will have the same behavior that prepublish has now. Commands in this script will run both before publishing and on npm install.

  • prepublishOnly, as seen from its name, will run only before publishing the package.

  • Also, prepublish will receive a warning about the changes.

npm realizes that prepublishOnly is an ugly name for a script, Therefore, there’re also some other changes planned for the later releases.

  • npm 5 will un-deprecate previously deprecated prepublish, make it run only before publishing, and will deprecate prepublishOnly. In this release, prepublish will get back its expected behavior.

  • npm 6 or later will remove prepublishOnly completely. After this, two scripts will remain: prepublish which will be run only before publishing, and prepare which will be executed both on before publishing and on npm install.

Update 24 Jun 2017: the plan above was true at the moment of publishing, but seems like npm doesn’t follow it now. See a twitter thread with @maybekatz for more details.

There will be changes in npm 4…6

What to do when npm 4 is out#

Decide based on what your package needs.

  • If your package’s prepublish script contains commands that should only be run before npm publish, rename the script into prepublishOnly. Examples of such commands are npm test and eslint.
  • If your package’s prepublish script contains commands that should run both before npm publish and on npm install, rename the script to prepare. Examples of such commands are commands that build your package (e. g. webpack or babel).
  • If you don’t know, rename the script to prepublishOnly – this is the behavior most people expect. (As another option, you can leave everything as-is, but there’s no point: it will just delay the decision, but not remove it.)

P.S. If you like this, follow me on Twitter. I tweet selected articles and my own thoughts: https://twitter.com/iamakulov

React anti-pattern: don’t make <li> your component’s root

Noticed a React anti-pattern while mentoring a younger front-end developer on our project.

Imagine you have a component that represents a list of articles. At one point, you realize that each article in the list is too complex:

const ArticleList = (props) => {
  return <div class="articles">
    <ul class="article-list">
      { props.articles.map(article => 
        <li class="article">
          <h2 class="article__title">{ article.title }</h2>
          { /* 25 other tags */ }
        </li>
      ) }
    </ul>
    { /* ... */ }
  </div>;
}

So you decide to move the item to a separate component. You take that code inside map(), extract it into <Article> and get something like this:

const Article = (props) => {
  return <li class="article">
    <h2 class="article__title">{ props.title }</h2>
    { /* 25 other tags */ }
  </li>;
}

const ArticleList = (props) => {
  return <div class="articles">
    <ul class="article-list">
      { props.articles.map(article => 
        <Article title={article.title} { /* other props */ } />
      ) }
    </ul>
    { /* ... */ }
  </div>;
}

Don’t do it this way. This approach is wrong. The problem is that by taking a <li> and making it a root of the component, you’ve just made your component non-reusable. This means that if you’d like to reuse <Article> in another place, you’ll only be able to apply it somewhere inside a list – because of this <li>. If you decide to render <Article> into e.g. a <div>, not only will this be non-semantic (<li> can only appear inside of <ul>s), but will also add unnecessary list item styling which is super weird.

Solution#

The solution is simple: move the <li> back into the <ArticleList> component and make the <Article>’s root element a <div> or something else. This will probably require some refactoring in your styles, but will make the component reusable. Look how cool:

const Article = (props) => {
  // Notice: <li>s are gone
  return <div class="article">
    <h2 class="article__title">{ props.title }</h2>
    { /* ... */ }
  </div>;
}

const ArticleList = (props) => {
  return <div class="articles">
    <ul class="article-list">
      { props.articles.map(article => 
        <li class="article-list__item">
          <Article title={article.title} { /* other props */ } />
        </li>
      ) }
    </ul>
    { /* ... */ }
  </div>;
}

// And now you can easily render an article inside of a sidebar – have no idea why though
const Sidebar = (props) => {
  return <div class="sidebar">
    <div class="sidebar__article">
      <Article title="Look ma" { /* other props */ } />
    </div>
    { /* ... */ }
  </div>;
}

UI testing, Selenium, mocking AJAX requests and Likely

Likely are well-designed social buttons:

I’m currently working on covering them with UI tests using Selenium. So far, several notes:

  • Selenium is cross-platform, but different platforms support different sets of functionality. Node.js isn’t the most complete one, unfortunately. If you google how to do something with Selenium, find a StackOverflow reply with the Java API and try to do the same in JavaScript, don’t expect it will definitely work. That API could just be absent.

  • Selenium 3 is coming, but most tutorials focus on Selenium 2. Selenium 2 was released in 2011, and version 3 is expected to be released this year. In fact, the selenium-webdriver npm package already installs 3.0.0-beta.2. There’re no major breaking changes between 2.53.2 and 3.0.0, but expect that some tutorial code could just not work.

  • The selenium docs are scattered between different places, and it’s hard to find the right thing when you’re googling something. One part of the documentation is at docs.seleniumhq.org, another is in the repository wiki, etc. It was quite hard to find the proper actual API docs for JavaScript, so here are they: http://seleniumhq.github.io/selenium/docs/api/javascript/

  • Mock the external services when doing integration tests. When testing the sharing counters, we rely on responses from the social networks. Turns out these services don’t always work well (especially Facebook and Google+) which makes the tests fail. Viktor Karpov suggested to mock the responses, and it seems it’s a default way of doing the integration tests which I didn’t know. I’m working on this now.

  • Mocking AJAX requests with Selenium is hard. I need to mock them to simulate the social network responses (see the previous point). So far, I’ve only found two libraries that can help with this: Sinon.js and xhr-mock. Sinon.js is popular and heavily featured, but it has quite a complex API, and I haven’t yet succeeded in making it work. xhr-mock is way simpler and can also mock only specific URLs (which is more complicated with Sinon), but it doesn’t support XMLHttpRequest.prototype.addEventListener and doesn’t have a UMD build. Sadly.

You can follow the pull request I’m working in to stay tuned (and see how we manage to do the AJAX mocking): https://github.com/ilyabirman/Likely/pull/73