Node and NPM

Dr. Greg Bernstein

Updated February 18th, 2020

Node.js

What is it?

A way of running JavaScript outside of a browser.

In Depth

nodejs.org

  • Node.js® is a JavaScript runtime built on Chrome’s V8 JavaScript engine.

  • Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.

  • Node.js’ package ecosystem, npm, is the largest ecosystem of open source libraries in the world.

Where do I get it?

Node.js is available for free from https://nodejs.org/en/download/.

Linux, Mac, and Windows operating systems are supported.

Note that you do not need the extra build tools! These take 3GB of space on windows.

How do I run it?

  • For a console-like JavaScript shell type node in a windows, Mac, or Linux command window. Use .exit to exit.

  • To run a JavaScript file type node Filename.js from a command shell.

Node Knows about files/dirs

When running a JavaScript file Node provides some global variables:

  • __filename the full file name of this JavaScript file
  • __dirname the directory this file is in

Example

Put the following code into a file named whoAmI.js and run it with node whoAmI.js.

console.log("Hello from Node.js");
console.log("The name of this file is: " + __filename);
console.log("This file is in the directory: " + __dirname);

Documentation

  • For JavaScript in the Browser we have the DOM
  • For JavaScript on the server we have the Node API
  • You need the API documentation corresponding to your version of Node
  • On 2/18/2020 the latest LTS release was 12.16.1, Node 12.x Documentation

Expanded Access

  • JavaScript operations in browsers are heavily restricted from touching the host computer, particularly the file system.

  • Via the Node.js API JavaScript programs have access to the file system, network, and some OS services.

Node Hands On and I/O!

Running JavaScript Programs

  • For a console like JavaScript shell type node in a windows, Mac, or Linux command window. Use .exit to exit.

  • To run a JavaScript file type node Filename.js from a command shell.

Example

File hiNode.js:

// run me with: node hiNode.js
let a = [1, 3, 5, 7, 9, 11];
function cube(x) {
    return x*x*x;
}

console.log("Hello from Node.js");
for (let x of a) {
    console.log(`x = ${x} and x cubed = ${cube(x)}`);
}

Important Global Stuff

In the Globals section of the API documents you will find

  • Important file related variables: _dirname, _filename
  • Import module function: require()
  • The console for simple output and debugging

Node Knows about files/dirs

When running a JavaScript file Node provides some global variables:

  • __filename the full file name of this JavaScript file
  • __dirname the directory this file is in

Example current file/dir

Put the following code into a file named whoAmI.js and run it with node whoAmI.js.

console.log("Hello from Node.js");
console.log("The name of this file is: " + __filename);
console.log("This file is in the directory: " + __dirname);

Node.js API Use

Node.js has a module system that predates the ES6 standard

  • To “import” an API module for use syntax such as:
const varName = require('moduleName');
  • Where varName is just the name of a variable that you choose and moduleName is the name of the API module.

Ex: Filesystem Access

Getting a list of all files in a directory

const fs = require('fs'); // File system module
// List all the files in the current directory
const fs = require('fs'); // File system module
let allFiles = fs.readdirSync(__dirname);
console.log("Files in the current directory: " + __dirname);
allFiles.forEach(function(f){console.log('\t' + f);});

Synchronous File Read

Simple File Reading

// syncRead.js
const fs = require('fs'); // File system module
let fname = __dirname + '/whoAmI.js';
let fdata = fs.readFileSync(fname, 'utf8');
console.log("Contents of file whoAmI.js:");
console.log(fdata);

Synchronous File Write

// syncWrite.js
const fs = require('fs'); // File system module
let fname = __dirname + '/tempWrite.txt';
let data = "Just a little text. \n Did this write?";
let fdata = fs.writeFileSync(fname, data);
console.log("Wrote test file: tempWrite.txt");

Network Access

Getting information about network interfaces

// netIf.js
// Let's look at your machines IPv4 addresses
const os=require('os');
let networkInterfaces = os.networkInterfaces();
// console.log(networkInterfaces); // Shows everything
for (let intf in networkInterfaces){
    console.log(intf);
    // Only interested in IPv4 interfaces
    let addresses = networkInterfaces[intf]
      .filter(a => a.family === 'IPv4');
    console.log(addresses);
}

NPM

What is it

From npmjs.com

npm is the package manager for JavaScript and the world’s largest software registry.

It also offers a simple script running system.

All the development tools and JavaScript libraries we will be using will be obtained via npm. npm gets installed with Node.js. npm can take some time to install packages.

Installing Packages Locally

See npm docs

  • Packages can be installed “locally” to your working directory
    • npm install <package_name> See local install
    • We’ll use this option for libraries and some development tools

Installing Packages Globally

See npm docs

  • Packages can be install “globally” on your machine
    • npm install -g <package_name> See global install
    • This option is generally used for general development tools.

Example: Static Webserver I

  • A great server for static site development Serve
  • npm install -g serve
  • From the command line go to the root directory for your site and type: serve
  • Use serve --help to see options

Example: Static Webserver II

  • “A global install doesn’t work for me…”

  • Do a local install: npm install serve

  • In the directory where you installed it, Run it with:
node_modules/serve/bin/serve.js

Example Static Webserver III

What if there is already a server at localhost:5000?

  • Specify another port with serve -l 5001 or any other unused port number
  • Can also specify an IP address to be used serve -l tcp://127.0.0.22:2112 where you can put in any suitable IP address for 127.0.0.22 and port for 2112

Example Static Webserver IV

Externally Visible Server

  • Need to use an externally visible IPv4 address, Use the netIf.js program to find yours.

  • Start serve with:
serve -l tcp:yourIPv4Address:YourPortChoice

node_modules directory

  • npm puts locally installed packages into a node_modules directory within your project directory.
  • The node_modules directory can get very large.
    • Don’t check this directory into version control.
    • You don’t need to share this directory since it can be re-created from a package.json file (to be discussed)

package.json

Why

From Using package.json

  • It serves as documentation for what packages your project depends on.
  • It allows you to specify the versions of a package that your project can use using semantic versioning rules.
  • Makes your build reproducible which means that its way easier to share with other developers.

Minimal package.json

You can build it by hand:

{
    "name": "your-project-name",
    "version": "0.0.1"
}

Creation Script

  • Run npm init and npm will ask you questions to build you package.json file.

  • Run npm init --yes will create a package.json you can later edit without all the questions. Try it!

Specifying Packages

From using package.json

To specify the packages your project depends on, you need to list the packages you’d like to use in your package.json file. There are 2 types of packages you can list:

  • “dependencies”: these packages are required by your application in production
  • “devDependencies”: these packages are only needed for development and testing

Maintaining package.json

  • You can manually add and delete your projects dependencies, but its much easier to let npm help you.

  • For a regular package dependency use npm install package_name --save when you install a package.
  • For a development dependency use npm install package_name --save-dev when you install a development package.

Re-creating node_modules

  • You should version control your package.json file.
  • With a package.json file in a directory, just run npm install to re-create your node_modules directory.

Node: Event Driven, Non-Blocking IO

Node Advanced Performance Concept

Required for CS651, Optional for CS351

  • How can a Node.js based server keep up with servers based on close to the “metal” languages such as C++ and Go

  • Browser vendors highly optimize JavaScript Execution

  • Most of the time servers are waiting on file servers and database servers

Node use an Event Driven Approach

  • A different approach to concurrency

  • No threads or processes (for the programmer)

  • Instead a model built around events and callbacks

Ex: Filesystem Access

Reading from a particular file:

const fs = require('fs'); // File system module
let fname = __dirname + '/corpText.txt';
function processFile(err, data) {
    if (err) {
        console.log("Some kind of error");
        return;
    }
    console.log(data);
}

fs.readFile(fname, 'utf8', processFile);

A Callback on ReadFile?

  • Yes, this allows for “non-blocking” operation. See the blocking/non-blockking guide
  • Recall: Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.
  • JavaScript execution in Node.js is single threaded, so Node.js uses an event driven model for I/O to increase performance and scalability.

How do they do it?

With a some help from libuv

  • Full-featured event loop backed by epoll, kqueue, IOCP, event ports.

  • Asynchronous TCP and UDP sockets

  • Asynchronous DNS resolution

  • Asynchronous file and file system operations

Is this “new”

Kind of…

  • In the past blocking operations, separate threads, and/or processes where used per TCP/UDP socket or so…

  • The NGINX web server (comercial and open source) used this approach to get greatly improved performance over Apache.

  • The Apache Webserver moved to an event based approach in version 2.4

  • Python libraries such as eventlet take a similar approach to achieve high performance.