JavaScript Testing

Dr. Greg Bernstein

November 7th, 2021

JavaScript Testing

Readings

References

  • Mocha. JavaScript testing framework.
  • Chai. Assertion library

Why Test?

  • To make sure the software does what it is supposed to do.
  • To see if you broke the software when you changed it.
  • Because testing is a lot of fun!

Types of Tests

  • Unit Tests: Tests of individual functions or classes.
  • Integration Tests: Tests of processes, components, or interfaces.
  • UI Testing: Testing the user interface, frequently a GUI.

Why Automate Testing?

  • Elaborate “setup procedures” may be required for testing some part of code functionality. We want to make these tests easy to repeat. Particularly during development!
  • We may have a large amount of tests that are needed to be run to ensure the system is functioning correctly.
  • We may want to report the results of the tests in a nice manner every time we run them.

Example: Web Server Development

Testing even a fairly simple web server API requires:

  • Initialization of server databases for a test run
  • Running the server at a particular IP address and port
  • Creating a test script to make HTTP requests for each interface and feature on that interface.
  • Configuring and running the test script to make calls to the server on the correct IP address and port.
  • Noting the results, debugging issues, and repeating

Outline and Tools

Software Process and Testing

Software Development Processes

Successful software projects:

  • Figure out the minimum viable project (MVP) to build
  • Identify and attack the riskiest parts of the project early (unknowns count as risks)
  • Show incremental functionality via iterations sooner rather than later.

Testing, Requirements, and Progress

  • Tests should be linked to requirements in some way otherwise they are unnecessary.
  • Tests can also be seen as a direct realization of detailed requirements
  • Writing tests and having them run successfully is one sign of progress on a software project.

Software Processes 1

From Wikipedia Software development process

In software engineering, a software development process is the process of dividing software development work into distinct phases to improve design, product management, and project management.

Software Processes 2

  • There are many flavors of “software development process”
  • Some are very formal with specific procedures and terminology
  • All include testing. Some emphasize testing such as Test Driven Development (TDD) and Behavior Driven Development (BDD)
  • Both TDD and BDD have influenced the design of test frameworks, but you don’t need to know much about either to use frameworks such as Mocha or Jest.

Tests as Documentation

A few hints when working with open source projects

  • Requirements and testing are closely linked. So looking at the tests for a project can fill in gaps in documentation.

  • Tests can be a good source of “simple” usage examples when documentation is lacking.

  • All but the smallest projects should include tests. Use them as a partial indicator of project quality.

Assertions

Assertions 1

From Wikipedia Assertion

In computer programming, an assertion is a statement that a predicate (Boolean-valued function, i.e. a true–false expression) is always true at that point in code execution. It can help a programmer read the code, help a compiler compile it, or help the program detect its own defects.

Where are Assertions used?

  • In some software development methodologies such as “design by contract”, “test driven development” (TDD), “behavior driven development” (BDD)

  • In development or runtime correctness checks

  • In software testing systems such as Mocha, Jest, etc. to make tests easier to write and understand.

Are these exceptions?

  • JavaScript assertion libraries generally use JavaScript exception mechanisms such as throw and try/catch.

  • Many build off the standard Error object

  • There isn’t a standard assertion interface for the browser, Node.js does have an assert module.

The Chai assertion library

From the Chai website

Chai is a BDD / TDD assertion library for node and the browser that can be delightfully paired with any javascript testing framework.

My take: Chai makes writing test cases a lot easier!

Chai TDD style assertions

Examples from the Chai website

var assert = require('chai').assert
  , foo = 'bar'
  , beverages = { tea: [ 'chai', 'matcha', 'oolong' ] };

assert.typeOf(foo, 'string'); // without optional message
assert.typeOf(foo, 'string', 'foo is a string'); // with optional message
assert.equal(foo, 'bar', 'foo equal `bar`');
assert.lengthOf(foo, 3, 'foo`s value has a length of 3');
assert.lengthOf(beverages.tea, 3, 'beverages has 3 types of tea');

What good is this?

  • If the condition of the assertion is true nothing happens
  • If the condition is false an exception will be thrown with useful info
  • Chai provides a ton of different kinds of checks. See Chai assert. There are over 120 different checks!
  • A test framework can catch the exceptions and report on them.

Chai BDD Style Assertions 1

Chai Expect assertions

var expect = require('chai').expect
  , foo = 'bar'
  , beverages = { tea: [ 'chai', 'matcha', 'oolong' ] };

expect(foo).to.be.a('string');
expect(foo).to.equal('bar');
expect(foo).to.have.lengthOf(3);
expect(beverages).to.have.property('tea').with.lengthOf(3);

Chai BDD Style Assertions 2

Chai Should assertions

var should = require('chai').should() //actually call the function
  , foo = 'bar'
  , beverages = { tea: [ 'chai', 'matcha', 'oolong' ] };

foo.should.be.a('string');
foo.should.equal('bar');
foo.should.have.lengthOf(3);
beverages.should.have.property('tea').with.lengthOf(3);

Test Frameworks

Test Framework Features

What do we need beyond assertions?

  • A way of running multiple tests, i.e., a test runner
  • A way to setup and clean up after tests or groups of tests
  • Mechanisms for grouping tests together and describing them

Example Test Framework Mocha

Mocha Installation

Global or Local Installation

  1. Global npm install --global mocha
    • Then use mocha in the console
  2. Local npm install --save-dev mocha
    • Then use ./node_modules/.bin/mocha in the console

Mocha Test Runner Usage

  • Command Line Reference
  • Basic usage: mocha spec where spec is One or more files, directories, or globs to test and defaults to test directory.
  • As a script in package.json: "scripts": {"test": "mocha"},

Mocha Test Grouping

  • By directory, By default looks for test/ directory
  • Separate files for different groups of tests
  • The describe() function for grouping and documenting similar tests within a file.
  • The it() function for individual tests.

Mocha Example 1

From Mocha getting started, also test.mjs

import assert from 'assert';
describe('Array', function() {
  describe('#indexOf()', function() {
    it('should return -1 when the value is not present', function() {
      assert.equal([1, 2, 3].indexOf(4), -1);
    });
  });
});

Important Points

  • Tests are JavaScript code
  • Assertions of some type need to be used
  • We don’t import/require Mocha, we run the test files with Mocha and not Node.js.

Mocha Example 1 Output

From Mocha getting started

$ ./node_modules/mocha/bin/mocha

  Array
    #indexOf()
      ✓ should return -1 when the value is not present

  1 passing (9ms)

Mocha Test with Chai Assertions

test/testChai.mjs example

import chai from 'chai';
const assert = chai.assert;
const expect = chai.expect;

// Chai assert
describe('Array via Assert Style', function() {
    const numbers = [1, 2, 3, 4, 5];
    it('is array of numbers', function() {
        assert.isArray(numbers, 'is array of numbers');
    });
    it('array contains 2', function() {
        assert.include(numbers, 2, 'array contains 2');
    });
    it('array contains 5 numbers', function() {
        assert.lengthOf(numbers, 5, 'array contains 5 numbers');
    });
});
    
// Expect style from Chai
describe('Array tests via Expect style', function() {
    const numbers = [1, 2, 3, 4, 5];
    it('A test with multiple assertions', function() {
        expect(numbers).to.be.an('array').that.includes(2);
        expect(numbers).to.have.lengthOf(5);
    });
});

Example Output

Mocha output

Test Setup and Cleanup

Within each describe() function:

  • Use before() to set up things before all tests
  • Use after() to clean up things after all tests
  • Use beforeEach() to setup something before each test
  • Use afterEach() to cleanup something after each test

Test Setup and Cleanup Outline Code

From Mocha hooks

describe('A bunch of tests', function() {

  before(function() {
    // runs before all tests in this block
  });

  after(function() {
    // runs after all tests in this block
  });

  beforeEach(function() {
    // runs before each test in this block
  });

  afterEach(function() {
    // runs after each test in this block
  });

  // test cases here using it()
});

Testing a function

An Algorithm for Peer Review

Requirements drive tests

A set of N students have submitted homework assignments, we want each student to review M < N other students assignments (but not their own). A reviewer cannot review another students paper more than once. We must make sure that each students assignment gets M reviews.

Algorithm Code

From peerAlg4.mjs:

function shuffle(array) {
    for (let i = array.length - 1; i > 0; i--) {
        const j = Math.floor(Math.random() * i)
        const temp = array[i];
        array[i] = array[j];
        array[j] = temp;
    }
}

function reviewAssignment(numStudents, numReviews, randomize=true) {
    if ((numStudents <= 0) || (numReviews <= 0)) {
        throw Error('number of students and reviews must be positive');
    }
    if (numReviews >= numStudents) {
        throw Error('number of students must be greater than number of reviews');
    }
    // Array of student order, as if students were in a circle
    let ordering = [];
    for (let i = 0; i < numStudents; i++) {
        ordering[i] = i;
    }
    if(randomize){
        shuffle(ordering);
    }

    // Keep track of who is reviewing each students assignment
    let assignments = [];
    for (let i = 0; i < numStudents; i++) {
        let assignInfo = {
            student: ordering[i],
            reviewers: new Set()
        };
        assignments.push(assignInfo);
    }
    // Keep track of the assignments each student is reviewing
    let reviews = [];
    for (let i = 0; i < numStudents; i++) {
        let reviewInfo = {
            student: ordering[i],
            reviewees: new Set()
        };
        reviews.push(reviewInfo);
    }

    // Fixed mapping of reviewers to assignments based on
    // a circular pass the papers around notion.
    for (let i = 0; i < numStudents; i++) {
        let assignment = assignments[i];
        let increment = 1;
        while (assignment.reviewers.size < numReviews) {
            let trial = (i + increment) % numStudents;
            if (reviews[trial].reviewees.size >= numReviews) continue;
            assignment.reviewers.add(ordering[trial]);
            reviews[trial].reviewees.add(ordering[i]);
            increment++;
        }
    }
    let sortFunc = (a,b)=>a.student-b.student;
    assignments.sort(sortFunc);
    reviews.sort(sortFunc);
    return [assignments, reviews];
}

export {reviewAssignment};

Example Algorithm Output

{ student: 0, reviewers: Set { 2, 10, 6, 7, 1 } }
{ student: 0, reviewees: Set { 12, 14, 9, 5, 11 } }

{ student: 1, reviewers: Set { 13, 8, 3, 4, 11 } }
{ student: 1, reviewees: Set { 0, 2, 10, 6, 7 } }

{ student: 2, reviewers: Set { 10, 6, 7, 1, 13 } }
{ student: 2, reviewees: Set { 12, 14, 9, 5, 0 } }

{ student: 3, reviewers: Set { 4, 11, 12, 14, 9 } }
{ student: 3, reviewees: Set { 6, 7, 1, 13, 8 } }

{ student: 4, reviewers: Set { 11, 12, 14, 9, 5 } }
{ student: 4, reviewees: Set { 7, 1, 13, 8, 3 } }

{ student: 5, reviewers: Set { 0, 2, 10, 6, 7 } }
{ student: 5, reviewees: Set { 12, 14, 9, 4, 11 } }

{ student: 6, reviewers: Set { 7, 1, 13, 8, 3 } }
{ student: 6, reviewees: Set { 9, 5, 0, 2, 10 } }

{ student: 7, reviewers: Set { 1, 13, 8, 3, 4 } }
{ student: 7, reviewees: Set { 5, 0, 2, 10, 6 } }

{ student: 8, reviewers: Set { 3, 4, 11, 12, 14 } }
{ student: 8, reviewees: Set { 10, 6, 7, 1, 13 } }

{ student: 9, reviewers: Set { 5, 0, 2, 10, 6 } }
{ student: 9, reviewees: Set { 12, 14, 3, 4, 11 } }

{ student: 10, reviewers: Set { 6, 7, 1, 13, 8 } }
{ student: 10, reviewees: Set { 14, 9, 5, 0, 2 } }

{ student: 11, reviewers: Set { 12, 14, 9, 5, 0 } }
{ student: 11, reviewees: Set { 1, 13, 8, 3, 4 } }

{ student: 12, reviewers: Set { 14, 9, 5, 0, 2 } }
{ student: 12, reviewees: Set { 13, 8, 3, 4, 11 } }

{ student: 13, reviewers: Set { 8, 3, 4, 11, 12 } }
{ student: 13, reviewees: Set { 2, 10, 6, 7, 1 } }

{ student: 14, reviewers: Set { 9, 5, 0, 2, 10 } }
{ student: 14, reviewees: Set { 12, 8, 3, 4, 11 } }

Algorithm Tests

import {reviewAssignment} from  '../peerAlg4.mjs';
import chai from 'chai';
const assert = chai.assert;

describe('Peer Assignment Algorithm Tests', function () {
    let numStudents = 15,
        numReviews = 5,
        randomize = true,
        assignments, reviews;
    beforeEach(function(){
        [assignments, reviews] = reviewAssignment(numStudents, numReviews, randomize);
    });
    
    describe('Array and Set Checks', function(){
        it('Length reviews and assignments', function(){
            assert.isArray(reviews);
            assert.isArray(assignments);
            assert.lengthOf(reviews, numStudents, 'Every student reviews');
            assert.lengthOf(assignments, numStudents, 'Every assignment is reviewed');
        });
        it('Reviewers and Reviewees', function(){
            reviews.forEach(function(r){
                assert.lengthOf(r.reviewees, numReviews, 'Each student must perform M reviews');
            });
            assignments.forEach(function(a){
                assert.lengthOf(a.reviewers, numReviews, 'Each assignment must have M reviewers');
            });
        });
    });
    
    describe('Cannot review yourself checks', function(){
        it('Assignment cannot be reviewed by author', function(){
            assignments.forEach(function(a){
                assert(!a.reviewers.has(a.student));
            })
        });
        it('Reviewer cannot review themself', function(){
            reviews.forEach(function(r){
                assert(!r.reviewees.has(r.student));
            })
        });
    });
    
    describe('Bad Input Checks', function(){
        it('Zero or negative parameters', function(){
            assert.throws(reviewAssignment.bind(null, 0, numReviews, randomize));
            assert.throws(reviewAssignment.bind(null, numStudents, 0, randomize));
            assert.throws(reviewAssignment.bind(null, -3, -5, randomize));
        });
        it('numReviews > numStudents', function(){
            assert.throws(reviewAssignment.bind(null, 10, 15));
        });
    });
    
});

Algorithm Test Output

Algo Test

Testing Asynchronous Things

Mocha Support for Asynchronous Testing

See Mocha Asynchronous

  • Callback style via done() function parameter
  • Promises
  • async/await style Will demonstrate this here

Asynchronous Example: Testing a Server

A subset of our JSON tour server

  • /tours, GET: Returns an array of tour objects with name, date, and type fields. Sets a cookie.

  • login, POST: Takes an object with email and password fields. If successful updates session ID in cookie.

  • logout, GET: Ends session, removes cookie.

Client/Server Debugging?

Example Zip: SessionJSONExample.zip

Testing Tours Interface

From test/tourTest.mjs

import pkg from 'chai';
const { assert } = pkg;
import fetch from 'node-fetch';
import getCookies from './getCookies.mjs';
import urlBase from '../testURL.mjs';

describe('Get Tour Tests', function() {
    let res;
    let tours = null;
    before(async function() {
        res = await fetch(urlBase + 'tours');
    })
    it('Everything is OK', async function() {
        assert.equal(res.status, 200);
    });
    it('Returns an array', async function() {
        tours = await res.json();
        assert.isArray(tours);
    });
    it('All tour elements have name and date', function() {
        tours.forEach(function(tour) {
            assert.containsAllKeys(tour, ['name', 'date']);
        });
    });
    it('Cookie with appropriate name is returned', function() {
        let cookies = getCookies(res);
        assert.include(cookies, 'TourSid');
        console.log(`tour test cookies: ${cookies}`);
    });
})

Tour Test Results

$ ./node_modules/mocha/bin/mocha test/tourTest.js
  Get Tour Tests
    √ Everything is OK
    √ Returns an array
    √ All tour elements have name and date
tour test cookies: TourSid=s%3AVkkD0FpO6DJ9mWdssARhl1rv4SE_67bK.O5nd69EuCm23O9uTvtSHmlm5ev3Thvh%2BdnFiNn4DjgM
    √ Cookie with appropriate name is returned

Dealing with Session Cookies

node-fetch supplies the set-cookie header to us

/*  This method works with `node-fetch` to retrieve cookies from
    a response in a suitable form so they can be simply
    added to a request. */

function getCookies(res) {
  let rawStrings = res.headers.raw()["set-cookie"]
  let cookies = [];
  rawStrings.forEach(function (ck) {
    cookies.push(ck.split(";")[0]); // Just grabs cookie name=value part
  });
  return cookies.join(";"); // If more than one cookie join with ;
}

module.exports = getCookies;

Testing Login

From test/loginTest.mjs

import pkg from 'chai';
const { assert } = pkg;
import fetch from 'node-fetch';
import getCookies from './getCookies.mjs';
import urlBase from '../testURL.mjs';

describe('Login Tests', function() {
    let res;
    let tours = null;
    let myCookie = null;

    before(async function() {
        console.log("Calling fetch");
        res = await fetch(urlBase + 'tours');
        console.log("Back from fetch");
        myCookie = getCookies(res);
    })
    it('Cookie with appropriate name is returned', function() {
        assert.include(myCookie, 'TourSid');
    });
    describe('Login Sequence', function() {
        before(async function() {
            res = await fetch(urlBase + 'login', {
                method: "post",
                body: JSON.stringify({
                    "email": "stedhorses1903@yahoo.com",
                    "password": "nMQs)5Vi"
                }),
                headers: {
                    "Content-Type": "application/json",
                    cookie: myCookie
                },
            });
        });
        it('Login Good', function() {
            assert.equal(res.status, 200);
        });
        it('User returned', async function() {
            let user = await res.json();
            assert.containsAllKeys(user, ['firstName', 'lastName', 'role']);
        });
        it('Cookie session ID changed', function() {
            let cookie = getCookies(res);
            assert.notEmpty(cookie);
            assert.notEqual(cookie, myCookie);
            console.log(cookie, myCookie);
        });
    });
    describe('Bad Logins', function() {
        it('Bad Email', async function() {
            res = await fetch(urlBase + 'login', {
                method: "post",
                body: JSON.stringify({
                    "email": "Bstedhorses1903@yahoo.com",
                    "password": "nMQs)5Vi"
                }),
                headers: {
                    "Content-Type": "application/json",
                    cookie: myCookie
                },
            });
            assert.equal(res.status, 401);
        });
        it('Bad Password', async function() {
            before(async function() {
                res = await fetch(urlBase + 'login', {
                    method: "post",
                    body: JSON.stringify({
                        "email": "stedhorses1903@yahoo.com",
                        "password": "BnMQs)5Vi"
                    }),
                    headers: {
                        "Content-Type": "application/json",
                        cookie: myCookie
                    },
                });
                assert.equal(res.status, 401);
            });
        })
    })
})

Login Test Output

$ ./node_modules/mocha/bin/mocha test/loginTest.js

  Login Tests
    √ Cookie with appropriate name is returned
    Login Sequence
      √ Login Good
      √ User returned
      √ Cookie session ID changed
    Bad Logins
      √ Bad Email
      √ Bad Password (83ms)


  6 passing (238ms)

Testing Support in IDEs

IDE

VSC Plugins 1

Mocha Specific Plugin

VSC Mocha Specific

VSC Plugins 2

General Test Plugin

VSC General Test

VSC Mocha Test and ES6 Modules

Extension had trouble finding tests with *.mjs files. My fix:

Use the menu item “File/Preferences/Settings”. In the “Settings” panel choose the Workspace tab (otherwise this won’t work). Scroll down to “Mocha Explorer: Files”, click on “Edit in settings.json” this will open up a local settings.json file for your workspace. Add the line: "mochaExplorer.files": "test/**/*.mjs" where the path on the right should be to where your test files are kept relative to the VSC project root. For my homework solution project it is "mochaExplorer.files":"clubServer/test/**/*.mjs".

Testing Sever APIs

  • Have separate test files for testing different portions of the API
  • tourTest.js, loginTest.js, etc in test/ directory
  • When debugging throw lots of console.log statements into both testing side and server side!
  • Can create helper functions to initialize the databases

Takeaways

  • Testing frameworks greatly increased ease of development of server APIs
  • Many editors and IDEs support Mocha testing
// reveal.js plugins