JavaScript Testing

Dr. Greg Bernstein

April 5th, 2020

JavaScript Testing

Readings

References

  • Mocha. JavaScript testing framework.
  • Chai. Assertion library
  • SuperTest. A library that greatly helps in testing HTTP interfaces.

Why Test?

  • To make sure the software does what it is supposed to do.
  • To see if you broke the software when you changed it.
  • Because testing is a lot of fun!

Types of Tests

  • Unit Tests: Tests of individual functions or classes.
  • Integration Tests: Tests of processes, components, or interfaces.
  • UI Testing: Testing the user interface, frequently a GUI.

Why Automate Testing?

  • Elaborate “setup procedures” may be required for testing some part of code functionality. We want to make these tests easy to repeat. Particularly during development!
  • We may have a large amount of tests that are needed to be run to ensure the system is functioning correctly.
  • We may want to report the results of the tests in a nice manner every time we run them.

Example: Web Server Development

Testing even a fairly simple web server API requires:

  • Initialization of server databases for a test run
  • Running the server at a particular IP address and port
  • Creating a test script to make HTTP requests for each interface and feature on that interface.
  • Configuring and running the test script to make calls to the server on the correct IP address and port.
  • Noting the results, debugging issues, and repeating

Outline and Tools

  • Software Development Processes

  • Assertions and assertion libraries, We’ll use Chai

  • Test Frameworks two of the most popular are Mocha and Jest. We’ll be using Mocha.

  • Server Testing Tools for Node.js. We’ll use Supertest to ease server testing.

  • Example Zips:MochaTesting.zip SessionJSONExample.zip

Software Process and Testing

Software Development Processes

Successful software projects:

  • Figure out the minimum viable project (MVP) to build
  • Identify and attack the riskiest parts of the project early (unknowns count as risks)
  • Show incremental functionality via iterations sooner rather than later.

Testing, Requirements, and Progress

  • Tests should be linked to requirements in some way otherwise they are unnecessary.
  • Tests can also be seen as a direct realization of detailed requirements
  • Writing tests and having them run successfully is one sign of progress on a software project.

Software Processes 1

From Wikipedia Software development process

In software engineering, a software development process is the process of dividing software development work into distinct phases to improve design, product management, and project management.

Software Processes 2

  • There are many flavors of “software development process”
  • Some are very formal with specific procedures and terminology
  • All include testing. Some emphasize testing such as Test Driven Development (TDD) and Behavior Driven Development (BDD)
  • Both TDD and BDD have influenced the design of test frameworks, but you don’t need to know much about either to use frameworks such as Mocha or Jest.

Tests as Documentation

A few hints when working with open source projects

  • Requirements and testing are closely linked. So looking at the tests for a project can fill in gaps in documentation.

  • Tests can be a good source of “simple” usage examples when documentation is lacking.

  • All but the smallest projects should include tests. Use them as a partial indicator of project quality.

Assertions

Assertions

From Wikipedia Assertion

In computer programming, an assertion is a statement that a predicate (Boolean-valued function, i.e. a true–false expression) is always true at that point in code execution. It can help a programmer read the code, help a compiler compile it, or help the program detect its own defects.

Where are Assertions used?

  • In some software development methodologies such as “design by contract”, “test driven development” (TDD), “behavior driven development” (BDD)

  • In development or runtime correctness checks

  • In software testing systems such as Mocha, Jest, etc. to make tests easier to write and understand.

Are these exceptions?

  • JavaScript assertion libraries generally use JavaScript exception mechanisms such as throw and try/catch.

  • Many build off the standard Error object

  • There isn’t a standard assertion interface for the browser, Node.js does have an assert module.

The Chai assertion library

From the Chai website

Chai is a BDD / TDD assertion library for node and the browser that can be delightfully paired with any javascript testing framework.

My take: Chai makes writing test cases a lot easier!

Chai TDD style assertions

Examples from the Chai website

var assert = require('chai').assert
  , foo = 'bar'
  , beverages = { tea: [ 'chai', 'matcha', 'oolong' ] };

assert.typeOf(foo, 'string'); // without optional message
assert.typeOf(foo, 'string', 'foo is a string'); // with optional message
assert.equal(foo, 'bar', 'foo equal `bar`');
assert.lengthOf(foo, 3, 'foo`s value has a length of 3');
assert.lengthOf(beverages.tea, 3, 'beverages has 3 types of tea');

What good is this?

  • If the condition of the assertion is true nothing happens
  • If the condition is false an exception will be thrown with useful info
  • Chai provides a ton of different kinds of checks. See Chai assert. There are over 120 different checks!
  • A test framework can catch the exceptions and report on them.

Chai BDD Style Assertions 1

Chai Expect assertions

var expect = require('chai').expect
  , foo = 'bar'
  , beverages = { tea: [ 'chai', 'matcha', 'oolong' ] };

expect(foo).to.be.a('string');
expect(foo).to.equal('bar');
expect(foo).to.have.lengthOf(3);
expect(beverages).to.have.property('tea').with.lengthOf(3);

Chai BDD Style Assertions 2

Chai Should assertions

var should = require('chai').should() //actually call the function
  , foo = 'bar'
  , beverages = { tea: [ 'chai', 'matcha', 'oolong' ] };

foo.should.be.a('string');
foo.should.equal('bar');
foo.should.have.lengthOf(3);
beverages.should.have.property('tea').with.lengthOf(3);

Test Frameworks

Test Framework Features

What do we need beyond assertions?

  • A way of running multiple tests, i.e., a test runner
  • A way to setup and clean up after tests or groups of tests
  • Mechanisms for grouping tests together and describing them

Example Test Framework Mocha

Mocha Installation

Global or Local Installation

  1. Global npm install --global mocha
    • Then use mocha in the console
  2. Local npm install --save-dev mocha
    • Then use ./node_modules/mocha/bin/mocha in the console

Mocha Test Runner Usage

  • Command Line Reference
  • Basic usage: mocha spec where spec is One or more files, directories, or globs to test and defaults to test directory.
  • As a script in package.json: "scripts": {"test": "mocha"},

Mocha Test Grouping

  • By directory, By default looks for test/ directory
  • Separate files for different groups of tests
  • The describe() function for grouping and documenting similar tests within a file.
  • The it() function for individual tests.

Mocha Example 1

From Mocha getting started, also test.js

var assert = require('assert'); // Node's assertion library
describe('Array', function() {
  describe('#indexOf()', function() { // can nest these for more grouping
    it('should return -1 when the value is not present', function() {
      assert.equal([1,2,3].indexOf(4), -1);
    });
  });
});

Important Points

  • Tests are JavaScript code
  • Assertions of some type need to be used
  • We don’t import/require Mocha, we run the test files with Mocha and not Node.js.

Mocha Example 1 Output

From Mocha getting started

$ ./node_modules/mocha/bin/mocha

  Array
    #indexOf()
      ✓ should return -1 when the value is not present

  1 passing (9ms)

Mocha Test with Chai Assertions

test/testChai.js example

const assert = require('chai').assert;
const expect = require('chai').expect;
const should = require('chai').should();

// Chai assert
describe('Array via Assert Style', function() {
    const numbers = [1, 2, 3, 4, 5];
    it('is array of numbers', function() {
        assert.isArray(numbers, 'is array of numbers');
    });
    it('array contains 2', function() {
        assert.include(numbers, 2, 'array contains 2');
    });
    it('array contains 5 numbers', function() {
        assert.lengthOf(numbers, 5, 'array contains 5 numbers');
    });
});

// Expect style from Chai
describe('Array tests via Expect style', function() {
    const numbers = [1, 2, 3, 4, 5];
    it('A test with multiple assertions', function() {
        expect(numbers).to.be.an('array').that.includes(2);
        expect(numbers).to.have.lengthOf(5);
    });
});

// Should style from Chai
describe('Array tests via Should style', function(){
    const numbers = [1, 2, 3, 4, 5];
    it('Includes test', function() {
        numbers.should.be.an('array').that.includes(2);
    });
    it('Length test', function() {
        numbers.should.have.lengthOf(5);
    });
});

Example Output

Test Setup and Cleanup

Within each describe() function:

  • Use before() to set up things before all tests
  • Use after() to clean up things after all tests
  • Use beforeEach() to setup something before each test
  • Use afterEach() to cleanup something after each test

Test Setup and Cleanup Outline Code

From Mocha hooks

describe('A bunch of tests', function() {

  before(function() {
    // runs before all tests in this block
  });

  after(function() {
    // runs after all tests in this block
  });

  beforeEach(function() {
    // runs before each test in this block
  });

  afterEach(function() {
    // runs after each test in this block
  });

  // test cases here using it()
});

Testing a function

An Algorithm for Peer Review

Requirements drive tests

A set of N students have submitted homework assignments, we want each student to review M < N other students assignments (but not their own). A reviewer cannot review another students paper more than once. We must make sure that each students assignment gets M reviews.

Algorithm Code

From peerAlg4.js:

function shuffle(array) {
    for (let i = array.length - 1; i > 0; i--) {
        const j = Math.floor(Math.random() * i)
        const temp = array[i];
        array[i] = array[j];
        array[j] = temp;
    }
}

function reviewAssignment(numStudents, numReviews, randomize=true) {
    if ((numStudents <= 0) || (numReviews <= 0)) {
        throw Error('number of students and reviews must be positive');
    }
    if (numReviews >= numStudents) {
        throw Error('number of students must be greater than number of reviews');
    }
    // Array of student order, as if students were in a circle
    let ordering = [];
    for (let i = 0; i < numStudents; i++) {
        ordering[i] = i;
    }
    if(randomize){
        shuffle(ordering);
    }

    // Keep track of who is reviewing each students assignment
    let assignments = [];
    for (let i = 0; i < numStudents; i++) {
        let assignInfo = {
            student: ordering[i],
            reviewers: new Set()
        };
        assignments.push(assignInfo);
    }
    // Keep track of the assignments each student is reviewing
    let reviews = [];
    for (let i = 0; i < numStudents; i++) {
        let reviewInfo = {
            student: ordering[i],
            reviewees: new Set()
        };
        reviews.push(reviewInfo);
    }

    // Fixed mapping of reviewers to assignments based on
    // a circular pass the papers around notion.
    for (let i = 0; i < numStudents; i++) {
        let assignment = assignments[i];
        let increment = 1;
        while (assignment.reviewers.size < numReviews) {
            let trial = (i + increment) % numStudents;
            if (reviews[trial].reviewees.size >= numReviews) continue;
            assignment.reviewers.add(ordering[trial]);
            reviews[trial].reviewees.add(ordering[i]);
            increment++;
        }
    }
    let sortFunc = (a,b)=>a.student-b.student;
    assignments.sort(sortFunc);
    reviews.sort(sortFunc);
    return [assignments, reviews];
}

module.exports = reviewAssignment;

Example Algorithm Output

{ student: 0, reviewers: Set { 2, 10, 6, 7, 1 } }
{ student: 0, reviewees: Set { 12, 14, 9, 5, 11 } }

{ student: 1, reviewers: Set { 13, 8, 3, 4, 11 } }
{ student: 1, reviewees: Set { 0, 2, 10, 6, 7 } }

{ student: 2, reviewers: Set { 10, 6, 7, 1, 13 } }
{ student: 2, reviewees: Set { 12, 14, 9, 5, 0 } }

{ student: 3, reviewers: Set { 4, 11, 12, 14, 9 } }
{ student: 3, reviewees: Set { 6, 7, 1, 13, 8 } }

{ student: 4, reviewers: Set { 11, 12, 14, 9, 5 } }
{ student: 4, reviewees: Set { 7, 1, 13, 8, 3 } }

{ student: 5, reviewers: Set { 0, 2, 10, 6, 7 } }
{ student: 5, reviewees: Set { 12, 14, 9, 4, 11 } }

{ student: 6, reviewers: Set { 7, 1, 13, 8, 3 } }
{ student: 6, reviewees: Set { 9, 5, 0, 2, 10 } }

{ student: 7, reviewers: Set { 1, 13, 8, 3, 4 } }
{ student: 7, reviewees: Set { 5, 0, 2, 10, 6 } }

{ student: 8, reviewers: Set { 3, 4, 11, 12, 14 } }
{ student: 8, reviewees: Set { 10, 6, 7, 1, 13 } }

{ student: 9, reviewers: Set { 5, 0, 2, 10, 6 } }
{ student: 9, reviewees: Set { 12, 14, 3, 4, 11 } }

{ student: 10, reviewers: Set { 6, 7, 1, 13, 8 } }
{ student: 10, reviewees: Set { 14, 9, 5, 0, 2 } }

{ student: 11, reviewers: Set { 12, 14, 9, 5, 0 } }
{ student: 11, reviewees: Set { 1, 13, 8, 3, 4 } }

{ student: 12, reviewers: Set { 14, 9, 5, 0, 2 } }
{ student: 12, reviewees: Set { 13, 8, 3, 4, 11 } }

{ student: 13, reviewers: Set { 8, 3, 4, 11, 12 } }
{ student: 13, reviewees: Set { 2, 10, 6, 7, 1 } }

{ student: 14, reviewers: Set { 9, 5, 0, 2, 10 } }
{ student: 14, reviewees: Set { 12, 8, 3, 4, 11 } }

Algorithm Tests

const reviewAssignment = require('../peerAlg4');
const assert = require('chai').assert;

describe('Peer Assignment Algorithm Tests', function () {
    let numStudents = 15,
        numReviews = 5,
        randomize = true,
        assignments, reviews;
    beforeEach(function(){
        [assignments, reviews] = reviewAssignment(numStudents, numReviews, randomize);
    });

    describe('Array and Set Checks', function(){
        it('Length reviews and assignments', function(){
            assert.isArray(reviews);
            assert.isArray(assignments);
            assert.lengthOf(reviews, numStudents, 'Every student reviews');
            assert.lengthOf(assignments, numStudents, 'Every assignment is reviewed');
        });
        it('Reviewers and Reviewees', function(){
            reviews.forEach(function(r){
                assert.lengthOf(r.reviewees, numReviews, 'Each student must perform M reviews');
            });
            assignments.forEach(function(a){
                assert.lengthOf(a.reviewers, numReviews, 'Each assignment must have M reviewers');
            });
        });
    });

    describe('Cannot review yourself checks', function(){
        it('Assignment cannot be reviewed by author', function(){
            assignments.forEach(function(a){
                assert(!a.reviewers.has(a.student));
            })
        });
        it('Reviewer cannot review themself', function(){
            reviews.forEach(function(r){
                assert(!r.reviewees.has(r.student));
            })
        });
    });

    describe('Bad Input Checks', function(){
        it('Zero or negative parameters', function(){
            assert.throws(reviewAssignment.bind(null, 0, numReviews, randomize));
            assert.throws(reviewAssignment.bind(null, numStudents, 0, randomize));
            assert.throws(reviewAssignment.bind(null, -3, -5, randomize));
        });
        it('numReviews > numStudents', function(){
            assert.throws(reviewAssignment.bind(null, 10, 15));
        });
    });

});

Algorithm Test Output

Testing Asynchronous Things

Mocha Support for Asynchronous Testing

See Mocha Asynchronous

  • Callback style via done() function parameter
  • Promises
  • async/await style Will demonstrate this here

Asynchronous Example: Testing a Server

A subset of our JSON tour server

  • /tours, GET: Returns an array of tour objects with name, date, and type fields. Sets a cookie.

  • login, POST: Takes an object with email and password fields. If successful updates session ID in cookie.

  • logout, GET: Ends session, removes cookie.

Setup

From testWEbAPI.js uses request-promise-native, and saves cookies for inspection

const assert = require('chai').assert;
const rp = require('request-promise-native');
const cookieJar = rp.jar();

let baseURL = 'https://cs651.grotto-networking.com';
// let baseURL = 'http://127.0.0.11:1711';

let loginCust = {
    uri: baseURL + '/login',
    json: true,
    method: "POST",
    body: {
        "email": "sylvan2059@live.com",
        "password": "1wQX_lYt"
    },
    jar: cookieJar
};

let tourSite = {
    uri: baseURL + '/tours',
    json: true,
    jar: cookieJar
};
let logout = {
    uri: baseURL + '/logout',
    json: true,
    method: "GET",
    jar: cookieJar
}

Testing Portion of Code

From testWebAPI.js uses async function in setup and in tests

describe('Get Tour Tests', function () {
    let result;
    before(async function () {
        result = await rp(tourSite);
    });

    it('Check for Tour Array', async function () {
        console.log("Check for Tour array?");
        assert.isArray(result);
    });

    it('Check for appropriate fields in tours', function () {
        result.forEach(function (tour) {
            assert.containsAllKeys(tour, ['name', 'date', 'type']);
        })
    });
});

describe('Good Login Tests', function () {
    let result;
    let cookieValue;
    before(async function () { // Note async function!
        result = await rp(tourSite);
    });
    it('Check for TourSID cookie', function () {
        const cookies = cookieJar.getCookies(baseURL);
        let myCookies = cookies.filter((c) => c.key === 'QD7373toursid');
        assert.notEmpty(myCookies);
        cookieValue = myCookies[0].value;
        console.log(`cookie value = ${cookieValue}`);
    });
    it('Good Login, Changed Cookie', async function () {
        result = await rp(loginCust);
        const cookies = cookieJar.getCookies(baseURL);
        let myCookies = cookies.filter((c) => c.key === 'QD7373toursid');
        assert.notEmpty(myCookies);
        assert.notEqual(myCookies[0].value, cookieValue);
    });
    it('Logout, check no cookie', async function () {
        result = await rp(logout);
        const cookies = cookieJar.getCookies(baseURL);
        let myCookies = cookies.filter((c) => c.key === 'QD7373toursid');
        assert.empty(myCookies);
    });
})

Example Output

Testing https://cs651.grotto-networking.com/Tours

Testing a Server API

Client/Server Debugging?

Seems like server debugging still involves turning on and off the server.

Example Zip: SessionJSONExample.zip

SuperTest to the Rescue

SuperTest home page

  • “The motivation with this module is to provide a high-level abstraction for testing HTTP, while still allowing you to drop down to the lower-level API provided by superagent.”
  • SuperAgent an HTTP request interface.

SuperTest Use, Versus Server Use

  • The final step in running a Node.js based server (Express or otherwise) is calling app.listen(port, address).

  • This starts the server listening on a specific IP address and TCP destination port.

  • With SuperTest we don’t start the server, we give the app to SuperTest to use in testing.

  • SuperTest commands are similar but different from what we saw in the request-promise-native library.

Spliting the Server Code

Session Server JSON example, taking out app.listen

const express = require('express');
let app = express(); // Can't use const if exporting
const session = require('express-session');
// Lots more stuff
app.use(setUpSessionMiddleware);

// Available to all visitors
app.get('/tours', function (req, res) {
    res.json(tours.virtTours);
});

app.post('/login',  express.json(), function (req, res) {
    // Lots of processing
});

module.exports = app; // Don't start app, export it

Server Runner Script

File serverRun.js, run server with node serverRun.js

const app = require('./tourServer'); // Import server
const host = '127.0.0.1';
const port = '3434';
app.listen(port, host, function () {
    console.log("Tour JSON session server listening on IPv4: " + host +
        ":" + port);
});

Testing Tours Interface

From test/tourTest.js

const app = require('../tourServer'); // Imports our server!!!
const assert = require('chai').assert;
const request = require('supertest');
const cookie = require('cookie'); // To help look at cookies

describe('Get Tour Tests', function () {
    let response;
    let tours = null;
    before(async function(){
        response = await request(app).get('/tours'); // app as a parameter
    })
    it('Everything is OK', async function(){
        assert.equal(response.status, 200);
    });
    it('Returns an array', function(){
        tours = JSON.parse(response.text);
        assert.isArray(tours);
    });
    it('All tour elements have name and date', function(){
        tours.forEach(function(tour){
            assert.containsAllKeys(tour, ['name', 'date']);
        });
    });
    it('Cookie with appropriate name is returned', function(){
        let cookies = response.header['set-cookie'].map(cookie.parse);
        let mycookie = cookies.filter(c => c.hasOwnProperty('TourSid'));
        assert.notEmpty(mycookie);
    });
})

Tour Test Results

$ ./node_modules/mocha/bin/mocha test/tourTest.js

  Get Tour Tests
    √ Everything is OK
    √ Returns an array
    √ All tour elements have name and date
    √ Cookie with appropriate name is returned


  4 passing (36ms)

The Response Object

The response object returned by supertest/superagent

{
    "req": {
        "method": "GET",
        "url": "http://127.0.0.1:50856/tours",
        "headers": {
            "user-agent": "node-superagent/3.8.3"
        }
    },
    "header": {
        "x-powered-by": "Express",
        "content-type": "application/json; charset=utf-8",
        "content-length": "208",
        "etag": "W/\"d0-rPgz52j8LBkh9nkobBntZRhkuhE\"",
        "set-cookie": ["TourSid=s%3Ajdducu9gSobtiAvnLJRCNsFPRYnD2Rl6.JRlh7ODZ0bEdiQrYuIC9x0g%2FbuCKEu1uG3NpdpPI6zA; Path=/; HttpOnly"],
        "date": "Tue, 12 Nov 2019 23:54:39 GMT",
        "connection": "close"
    },
    "status": 200,
    "text": "[{\"name\":\"Kiting Neptune\",\"date\":\"Starting June 2022\"},{\"name\":\"Windsurfing the Methane Lakes of Titan\",\"date\":\"Starting December 2022\"},{\"name\":\"Kiting Jupiter's Great Red Spot\",\"date\":\"Starting June 2023\"}]"
}

Dealing with Sessions

  • Supertest keeps requests separate, doesn’t remember cookies. We get a SuperTest instance via request(app)

  • SuperAgent remembers cookies. We get a SuperAgent instance via request.agent(app) then use that same instance across multiple requests.

Testing Login

From test/loginTest.js

const app = require('../tourServer');
const assert = require('chai').assert;
const request = require('supertest');
const cookie = require('cookie');

describe('Login Tests', function () {
    let response;
    let tours = null;
    let myCookie = null;
    let agent = request.agent(app); //Use across many requests

    before(async function(){
        response = await agent.get('/tours');
    })
    it('Cookie with appropriate name is returned', function(){
        let cookies = response.header['set-cookie'].map(cookie.parse);
        cookies= cookies.filter(c => c.hasOwnProperty('TourSid'));
        assert.notEmpty(cookies);
        myCookie = cookies[0];
    });
    describe('Login Sequence', function() {
        before(async function(){
            response = await agent.post('/login')
                .send({"email": "stedhorses1903@yahoo.com", "password": "nMQs)5Vi"});
        });
        it('Login Good', function(){
            assert.equal(response.status, 200);
        });
        it('User returned', function(){
            let user = JSON.parse(response.text);
            assert.containsAllKeys(user, ['firstName', 'lastName', 'role']);
        });
        it('Cookie session ID changed', function () {
            let cookies = response.header['set-cookie'].map(cookie.parse);
            cookies = cookies.filter(c => c.hasOwnProperty('TourSid'));
            assert.notEmpty(cookies);
            assert.notEqual(cookies[0]['TourSid'], myCookie['TourSid']);
        });
    });
    describe('Bad Logins', function(){
        it('Bad Email', async function(){
            response = await agent.post('/login')
                .send({"email": "Bstedhorses1903@yahoo.com",    "password": "nMQs)5Vi"});
            assert.equal(response.status, 401);
        });
        it('Bad Password', async function(){
            response = await agent.post('/login')
                .send({"email": "stedhorses1903@yahoo.com", "password": "BnMQs)5Vi"});
            assert.equal(response.status, 401);
        });
    })
})

Login Test Output

$ ./node_modules/mocha/bin/mocha test/loginTest.js

  Login Tests
    √ Cookie with appropriate name is returned
    Login Sequence
      √ Login Good
      √ User returned
      √ Cookie session ID changed
    Bad Logins
      √ Bad Email
      √ Bad Password (83ms)


  6 passing (238ms)

Testing Support in IDEs

Testing Sever APIs

  • Have separate test files for testing different portions of the API
  • tourTest.js, loginTest.js, etc in test/ directory
  • When debugging throw lots of console.log statements into both testing side and server side!
  • Can create helper functions to initialize the databases

Takeaways

  • Testing frameworks greatly increased ease of development of server APIs
  • No more starting/stopping server and client test scripts
  • Many editors and IDEs support Mocha testing