Task Automation - Building with Gulp


Today we will set up automated tasks to build our project and to make our life easier while we are developing.

TL; DR

Source code can be found on GitHub, in the jsGulp repo.


Task Automation

Building with Gulp


Table of Contents

  1. Compiling LESS
  2. Source maps for LESS
  3. Auto prefixing LESS
  4. Watch for LESS changes
  5. Validating JS
  6. Building

Grunt vs Gulp

The most widely used task runners for node.js are Grunt and Gulp. Basically, both are doing the same thing, but with a different mindset. The main difference is that Grunt is using configuration over coding, while Gulp is the other way around. You can read a more detailed comparison here.

I chose Gulp for the following reasons:

  • I have used Gulp in several projects in the past, but it was always set up by some other team member, so I was curious how it goes.
  • Code over configuration. I like that.
  • It's faster, because tasks can run in parallel, and you don't need to do as many (temp) file operations.
  • It's the newer, hip task runner.

Goals

I will set up two general categories of tasks:

  • To support development: compile LESS files and validate JavaScript files with JSHint automatically, when any of the relevant files changes.
  • To make a build: concatenate and minify all the sources, and put them in a separate build directory.

I'm also aiming to create a simple, but easily extendable project template that you can use to jumpstart your new project.

Getting started

It will be easier to follow what's happening if you take a look at the folder structure of our (very) simple application.
example project structure

The examples assuming you have gulp-cli installed globally. (The recommended way of using Gulp is install gulp-cli globally, then install gulp locally in your project.)

npm install gulp-cli -g  
npm install gulp --save-dev  

All other dependencies are "devDependencies", meaning they only needed for development, not production. That's why we use --save-dev for everything from now on.

All our Gulp tasks will be in the gulpfile.js in the project root. It will need to require gulp of course. And since Gulp can't really do much by itself, we will have to install and require plugins for everything we want to do.

Let's start with creating our first task.

1. Compiling LESS ^

This task will compile our main.less, and by extension, every other LESS file it imports. We will use the gulp-less plugin for this:

npm install gulp-less --save-dev  

Code:

var gulp = require('gulp'),  
    less = require('gulp-less');

gulp.task('less', function () {  
    return gulp
        .src('./less/main.less')
        .pipe(less())
        .pipe(gulp.dest('./css'));
});

I think most of this is quite straightforward. We require the plugins, and define a task, which takes our main LESS file, compiles it, and every other LESS files it imports, then saves the result in the css folder. Our HTML will use the CSS from here.

We can run the task from the command line by:

gulp less  

That's not terribly useful yet. This command is not much shorter than if we did it manually. So let's make it better.

2. Source maps for LESS ^

Let's suppose we are using the compiled CSS file in development and an element is getting the wrong color. The developer tools in the browser tell us a line number in the CSS, but we have no idea where is this color defined in the LESS files. This is what source maps are used for. Basically, it's a way to know which LESS line was compiled into which CSS line.

We need the gulp-sourcemaps plugin:

npm install gulp-sourcemaps --save-dev  

And to extend our less task:

var gulp = require('gulp'),  
    less = require('gulp-less'),
    sourcemaps = require('gulp-sourcemaps');

gulp.task('less', function () {  
    return gulp
        .src('./less/main.less')
        .pipe(sourcemaps.init())
        .pipe(less())
        .pipe(sourcemaps.write())
        .pipe(gulp.dest('./css'));
});

We haven't changed much. Just required the plugin, we initialize (store where things were) before compiling, and write (find out where things ended up) after.

In this setup, the source maps will be appended to the CSS file itself. It could be saved to a separate file, but I don't really see the advantage of that. We only use this in development, so file size is not an issue.

Before-after:
sourcemaps before-after

3. Auto prefixing LESS ^

I'm sure you know that some CSS features in some browsers can only be used with browser prefixes. For example, display: flex in IE10 is display: -ms-flexbox. So if we want (or have) to support older browsers, we have to pay attention to what should be prefixed and how.

This problem can be solved with an autoprefixer. We just write standard CSS, tell the autoprefixer what browsers we want to support, and it takes care of the prefixing.

We will use the less-plugin-autoprefix:

npm install less-plugin-autoprefix --save-dev  

Code:

var gulp = require('gulp'),  
    less = require('gulp-less'),
    sourcemaps = require('gulp-sourcemaps'),
    LessAutoprefix = require('less-plugin-autoprefix'),
    autoprefix = new LessAutoprefix({ browsers: ['last 2 versions'] });

gulp.task('less', function () {  
    return gulp
        .src('./less/main.less')
        .pipe(sourcemaps.init())
        .pipe(less({
            plugins: [autoprefix]
        }))
        .pipe(sourcemaps.write())
        .pipe(gulp.dest('./css'));
});

Most of the code is unchanged. We just required the autoprefixer, set it up, and gave it to the less function.

Before-after:
autoprefixer before-after

4. Watch for LESS changes ^

Our first task became quite useful, but we still need to run it by hand every time we change something. This is what watchers are for. Basically, it runs a task when it detects a change in the watched files.

We don't need a plugin for this, watch is a built-in feature of Gulp.

gulp.task('watch', function () {  
    gulp.watch('./less/*.less', ['less'])
        .on('change', function(event) {
            console.log(`Watch: ${event.path} was ${event.type}.`);
        });
});

As you can see, I've put this in another task, so I can run the less task once (will be useful later for building), or I can run watch to constantly update my CSS (useful while developing).

Note how I run the less task at the beginning of the watch task, so it compiles the current LESS files before start watching for changes.

Now, if we run gulp watch, it will re-run the less task after every change:
watch output

The LESS tasks are finished for now, but later we will do minification with the build task.
Let's go and do something for the JavaScript code.

5. Validating JS ^

One useful thing during development would be to run JSHint on our JavaScript code to validate it.

We will use the gulp-jshint for this:

npm install jshint gulp-jshint --save-dev  

Code:

var jshint = require('gulp-jshint');  
...
gulp.task('jshint', function () {  
    return gulp
        .src('./scripts/*.js')
        .pipe(jshint({
            'esversion': 6
        }))
        .pipe(jshint.reporter('default'));
});

I've also put a new watcher to our watch task to run JSHint every time the code it changes. This way we can catch many errors as soon as possible.

gulp.watch('./scripts/*.js', ['jshint'])  
    .on('change', function (event) {
        console.log(`Watch: ${event.path} was ${event.type}.`);
    });    

If we introduce an error in one of our script files, JSHint will complain about it instantly:
jshint warning

We are finished supporting the development, let's move on and create our build task.

6. Building ^

When we are satisfied with our current code and want to release a new version for hosting, we typically need to do some processing. At least we have to minify everything and concatenate our separate script files into one. This obviously improves the page load time.

We could do this the same way as before, but there is an extra step we need to take care of. Namely, the multiple script tags we use to load our JavaScript files should be replaced with a single one using the concatenated file. And the same is true for the CSS. Although we only have one CSS in our setup, the minified one will be on a different path.

Luckily there is a gulp-usemin plugin for this. And we will use gulp-concat, gulp-uglify, gulp-minify-html, and gulp-minify-css for concatenation and minification.

npm install gulp-usemin gulp-concat gulp-uglify gulp-minify-html gulp-minify-css --save-dev  

Code:

var usemin = require('gulp-usemin'),  
    concat = require('gulp-concat'),
    uglify = require('gulp-uglify'),
    minifyHtml = require('gulp-minify-html'),
    minifyCss = require('gulp-minify-css');
...
gulp.task('build', ['less'], function () {  
    return gulp
        .src('./*.html')
        .pipe(usemin({
            html: [ minifyHtml({ empty: true }) ],
            css: [ minifyCss() ],
            js: [ uglify() ]
        }))
        .pipe(gulp.dest('./build'));
});

For usemin to work, we need to mark the parts in the html that should be replaced in the building process:

<!-- build:css main.css -->  
<link rel="stylesheet" href="css/main.css">  
<!-- endbuild -->  
...
<!-- build:js scripts.js -->  
<script src="scripts/library.js"></script>  
<script src="scripts/main.js"></script>  
<!-- endbuild -->  

Usemin basically takes the files in each marked block, runs them through the plugins we specified in its parameter object, saves the result in the location we gave to gulp.dest, by the name we chose in the HTML marker comment. So for example, our two JavaScript files marked with build:js runs through the js: [ uglify() ] pipeline, then gets saved to ./build, by the name scripts.js. Finally, the marked part in the HTML is replaced with a single script tag, using this file.

The result is a build directory with three files: one .js, one .css, and one .html, all minified. So we only need to host this folder.
build folder

With this, I covered everything I planned to. I think this is a reasonable Gulp setup, that you can use for jump starting your own Gulp setup. Our example app is extremely simple, and every project might have its own specific needs, but now you have a basic knowledge about how you can automatize basically any repeating task in your development process.

Where to go next

There are thousands of useful plugins, and you can even write your own. Some of those I find useful, but had to exclude to keep this post from growing to an infinite length:

  • gulp-livereload - Reloading the page when something is changed.
  • gulp-bump - Autoincrement version numbers.
  • gulp-babel - Use the newest JavaScript features and transpile to the browser compatible representation.
  • gulp-imagemin - Minify images.

Conclusion

As you see, Gulp is a very easy to use task automation system. Even if you spend some time perfecting your tasks, it will save you serious time in the long run. Not to mention the boredom of manually doing the same tasks repeatedly. Automated task can also prevent human errors caused by not really paying attention to the task you are doing for the thousandth time.


If you enjoyed reading this blog post, please share it or like the Facebook page of the website to get regular updates. You can also follow us @dealwithjs.

Happy Gulping!