Node and NPM

Dr. Greg Bernstein

Updated September 15th, 2021

Node.js

Learning Objectives

  • What is Node.js, origin, capabilities
  • Basic usage: running programs, reading and writing files
  • Basics of NPM: package manager, package.json, open source repository, script runner
  • Advanced understanding: asynchronous operation

What is it?

A way of running JavaScript outside of a browser.

In Depth

nodejs.org

  • Node.js® is a JavaScript runtime built on Chrome’s V8 JavaScript engine.

  • Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.

  • Node.js’ package ecosystem, npm, is the largest ecosystem of open source libraries in the world.

Where do I get it?

Node.js is available for free from https://nodejs.org/en/download/.

Linux, Mac, and Windows operating systems are supported.

Note that you do not need the extra build tools! These take 3GB of space on windows.

How do I run it?

  • For a console-like JavaScript shell type node in a windows, Mac, or Linux command window. Use .exit to exit.

  • To run a JavaScript file type node Filename.js from a command shell.

Documentation

  • For JavaScript in the Browser we have the DOM
  • For JavaScript on the server we have the Node API
  • You need the API documentation corresponding to your version of Node
  • On 2/21/2021 the latest LTS release was 14.15.5, Node 14.x Documentation

Expanded Access

  • JavaScript operations in browsers are heavily restricted from touching the host computer, particularly the file system.

  • Via the Node.js API JavaScript programs have access to the file system, network, and some OS services.

Node Hands On and I/O!

Running JavaScript Programs

  • For a console like JavaScript shell type node in a windows, Mac, or Linux command window. Use .exit to exit.

  • To run a JavaScript file type node Filename.js from a command shell.

Example

File hiNode.js, run with node hiNode.js

// File: hiNode.js
let a = [1, 3, 5, 7, 9, 11];
function cube(x) {
    return x*x*x;
}

console.log("Hello from Node.js");
for (let x of a) {
    console.log(`x = ${x} and x cubed = ${cube(x)}`);
}

Node.js API Use

Node.js now supports ES6 modules

  • Use ES6 import syntax
  • Use ES6 export syntax
  • Don’t need paths for either built in modules or those install with NPM

Example: Filesystem Access

Getting a list of all files in a directory fileList.mjs

// List all the files in the current directory
// ES6 module syntax
import fs from 'fs'; // File system module
let allFiles = fs.readdirSync(".");
console.log("Files in the current directory: ");
allFiles.forEach(function(f) { console.log('\t' + f); });

Synchronous File Read

Simple File Reading syncRead.mjs

// Simple Synchronous file reading
// Using ES6 Modules.
import fs from 'fs'; // File system module
let fname = './whoAmI.js';
let fdata = fs.readFileSync(fname, 'utf8');
console.log("Contents of file whoAmI.js:");
console.log(fdata);

Synchronous File Write

File syncWrite.mjs

// Simple Synchronous file Write
// ES6 modules
import fs from 'fs'; // File system module
let fname = './tempWrite.txt';
let data = "Just a little text. \n Did this write?";
let fdata = fs.writeFileSync(fname, data);
console.log("Wrote test file: tempWrite.txt");

Network Access

Getting information about network interfaces netIf.mjs

// A simple program to look at IPv4 addresses
// for network interface via Node.js and ES6 modules
import os from 'os';
let networkInterfaces = os.networkInterfaces();
// console.log(networkInterfaces); // Shows everything
for (let intf in networkInterfaces) {
    console.log(intf);
    // Only interested in IPv4 interfaces
    let addresses = networkInterfaces[intf]
        .filter(a => a.family === 'IPv4');
    console.log(addresses);
}

JSON Read/Parse

Read and Parse JSON quickJSON.mjs

// JSON reading and parsing with Node.js ES6 modules
import { readFile } from 'fs/promises'; // promise based file reading 
const clubEvents = JSON.parse(await readFile(new URL('./eventData.json',
    import.meta.url))); // URL for relative file location, regular JSON parsing

clubEvents.forEach(function(event) { //Look at data
    console.log(event);
});

Node and CommonJS Modules

Useful Global Stuff

In the Globals section of the API documents you will find

  • Important file related variables: _dirname, _filename CommonJS only
  • Import module function: require() CommonJS only
  • The console for simple output and debugging always available

Node Knows about files/dirs’

When running a JavaScript file Node under CommonJS provides some global variables:

  • __filename the full file name of this JavaScript file
  • __dirname the directory this file is in

Example file and dir name

The following program is from the file whoAmI.js and is run with node whoAmI.js.

console.log("Hello from Node.js");
console.log("The name of this file is: " + __filename);
console.log("This file is in the directory: " + __dirname);

Node.js CommonJS API Use

Node.js has a module system that predates the ES6 standard

  • To “import” an API module for use syntax such as:
const varName = require('moduleName');
  • Where varName is just the name of a variable that you choose and moduleName is the name of the API module.

Example: CommonJS Filesystem Access

Getting a list of all files in a directory fileList.js

// List all the files in the current directory
const fs = require('fs'); // Import file system module
let allFiles = fs.readdirSync(__dirname);
console.log("Files in the current directory: " + __dirname);
allFiles.forEach(function(f){console.log('\t' + f);});

CommonJS Synchronous File Read

Simple File Reading syncRead.js

// syncRead.js
const fs = require('fs'); // File system module
let fname = __dirname + '/whoAmI.js';
let fdata = fs.readFileSync(fname, 'utf8');
console.log("Contents of file whoAmI.js:");
console.log(fdata);

CommonJS Synchronous File Write

File syncWrite.js

// syncWrite.js
const fs = require('fs'); // File system module
let fname = __dirname + '/tempWrite.txt';
let data = "Just a little text. \n Did this write?";
let fdata = fs.writeFileSync(fname, data);
console.log("Wrote test file: tempWrite.txt");

CommonJS Network Access

Getting information about network interfaces netIf.js

// netIf.js
// Let's look at your machines IPv4 addresses
const os=require('os');
let networkInterfaces = os.networkInterfaces();
// console.log(networkInterfaces); // Shows everything
for (let intf in networkInterfaces){
    console.log(intf);
    // Only interested in IPv4 interfaces
    let addresses = networkInterfaces[intf]
      .filter(a => a.family === 'IPv4');
    console.log(addresses);
}

NPM

What is it

From npmjs.com

npm is the package manager for JavaScript and the world’s largest software registry.

It also offers a simple script running system.

All the development tools and JavaScript libraries we will be using will be obtained via npm. npm gets installed with Node.js. npm can take some time to install packages.

Installing Packages Locally

See npm docs

  • Packages can be installed “locally” to your working directory
    • npm install <package_name> See local install
    • We’ll use this option for libraries and some development tools

Installing Packages Globally

See npm docs

  • Packages can be install “globally” on your machine *npm install -g <package_name> See global install
    • This option is generally used for general development tools.

Example: Static Webserver I

  • A great server for static site development http-server
  • npm install -g http-server
  • From the command line go to the root directory for your site and type: http-server
  • Use serve --help to see options

Example: Static Webserver II

  • “A global install doesn’t work for me…”

  • Do a local install: npm install http-server

  • In the directory where you installed it, Run it with:

node_modules/.bin/http-server

Example Static Webserver III

What if there is already a server at localhost:5000?

  • Specify another port with http-server -p 5001 or any other unused port number
  • Can also specify an IP address to be used serve -l tcp://127.0.0.22:2112 where you can put in any suitable IP address for 127.0.0.22 and port for 2112

Example Static Webserver IV

Externally Visible Server

  • Need to use an externally visible IPv4 address, Use the netIf.js program to find yours.

  • Use the -c-1 option to turn off caching if desred

  • Start serve with:

http-server -a yourIPv4Address -p YourPortChoice

node_modules directory

  • npm puts locally installed packages into a node_modules directory within your project directory.
  • The node_modules directory can get very large.
    • Don’t check this directory into version control!!!
    • You don’t need to share this directory since it can be re-created from a package.json file (to be discussed)

package.json

Why

From Using package.json

  • It serves as documentation for what packages your project depends on.
  • It allows you to specify the versions of a package that your project can use using semantic versioning rules.
  • Makes your build reproducible which means that its way easier to share with other developers.

Minimal package.json

You can build it by hand:

{
    "name": "your-project-name",
    "version": "0.0.1"
}

Creation Script

  • Run npm init and npm will ask you questions to build you package.json file.

  • Run npm init --yes will create a package.json you can later edit without all the questions. Try it!

Specifying Packages

From using package.json

To specify the packages your project depends on, you need to list the packages you’d like to use in your package.json file. There are 2 types of packages you can list:

  • “dependencies”: these packages are required by your application in production
  • “devDependencies”: these packages are only needed for development and testing

Maintaining package.json

  • You can manually add and delete your projects dependencies, but its much easier to let npm help you.

  • For a regular package dependency use npm install package_name --save when you install a package.

  • For a development dependency use npm install package_name --save-dev when you install a development package.

Re-creating node_modules

  • You should version control your package.json file.
  • With a package.json file in a directory, just run npm install to re-create your node_modules directory.

Node: Event Driven, Non-Blocking IO

Node Advanced Performance Concept

Required for CS651, Optional for CS351

  • How can a Node.js based server keep up with servers based on close to the “metal” languages such as C++ and Go

  • Browser vendors highly optimize JavaScript Execution

  • Most of the time servers are waiting on file servers and database servers

Node uses an Event Driven Approach

  • A different approach to concurrency

  • No threads or processes (for the programmer)

  • Instead a model built around events and callbacks

Ex: Filesystem Access

Asynchronous file reading simpleRead.js

// Simple asynchronous file reading
const fs = require('fs'); // File system module
let fname = __dirname + '/hiNode.js';
// Callback for readFile
function processFile(err, data) {
    if (err) {
        console.log("Some kind of error");
        return;
    } 
    console.log(data);
}
// This function takes a callback function as an argument.
fs.readFile(fname, 'utf8', processFile);

A Callback on ReadFile?

  • Yes, this allows for “non-blocking” operation. See the blocking/non-blockking guide
  • Recall: Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.
  • JavaScript execution in Node.js is single threaded, so Node.js uses an event driven model for I/O to increase performance and scalability.

How do they do it?

With a some help from libuv

  • Full-featured event loop backed by epoll, kqueue, IOCP, event ports.

  • Asynchronous TCP and UDP sockets

  • Asynchronous DNS resolution

  • Asynchronous file and file system operations

Is this “new”

Kind of…

  • In the past blocking operations, separate threads, and/or processes where used per TCP/UDP socket or so…

  • The NGINX web server (comercial and open source) used this approach to get greatly improved performance over Apache.

  • The Apache Webserver moved to an event based approach in version 2.4

  • Python libraries such as eventlet take a similar approach to achieve high performance.

// reveal.js plugins