Node and NPM

Dr. Greg Bernstein

Updated September 11th, 2019

Node.js

What is it?

A way of running JavaScript outside of a browser.

In Depth

nodejs.org

  • Node.js® is a JavaScript runtime built on Chrome’s V8 JavaScript engine.

  • Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.

  • Node.js’ package ecosystem, npm, is the largest ecosystem of open source libraries in the world.

Where do I get it?

Node.js is available for free from https://nodejs.org/en/download/.

Linux, Mac, and Windows operating systems are supported.

How do I run it?

  • For a console-like JavaScript shell type node in a windows, Mac, or Linux command window. Use .exit to exit.

  • To run a JavaScript file type node Filename.js from a command shell.

Node Knows about files/dirs

When running a JavaScript file Node provides some global variables:

  • __filename the full file name of this JavaScript file
  • __dirname the directory this file is in

Example

Put the following code into a file named whoAmI.js and run it with node whoAmI.js.

console.log("Hello from Node.js");
console.log("The name of this file is: " + __filename);
console.log("This file is in the directory: " + __dirname);

Documentation

  • For JavaScript in the Browser we have the DOM
  • For JavaScript on the server we have the Node API
  • You need the API documentation corresponding to your version of Node
  • On 9/11/2019 the latest LTS release was 10.16.3, [https://nodejs.org/dist/latest-v10.x/docs/api/)

Expanded Access

  • JavaScript operations in browsers are heavily restricted from touching the host computer, particularly the file system.

  • Via the Node.js API JavaScript programs have access to the file system, network, and some OS services.

Node.js API Use

Node.js has a module system that predates the ES6 standard

  • To “import” an API module for use syntax such as:
const varName = require('moduleName');
  • Where varName is just the name of a variable that you choose and moduleName is the name of the API module.

Ex: Filesystem Access

Getting a list of all files in a directory

const fs = require('fs'); // File system module
// List all the files in the current directory
let allFiles = fs.readdirSync(__dirname);
console.log("Files in the current directory:")
for (let file of allFiles) {
    console.log('\t' + file);
}

Ex: Filesystem Access

Reading from a particular file:

const fs = require('fs'); // File system module
let fname = __dirname + '/corpText.txt';
function processFile(err, data) {
    if (err) {
        console.log("Some kind of error");
        return;
    }
    console.log(data);
}

fs.readFile(fname, 'utf8', processFile);

A Callback on ReadFile?

  • Yes, this allows for “non-blocking” operation. See the blocking/non-blockking guide
  • Recall: Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.
  • JavaScript execution in Node.js is single threaded, so Node.js uses an event driven model for I/O to increase performance and scalability.

Important Global Stuff

In the Globals section of the API documents you will find

  • Important file related variables: _dirname, _filename
  • Import module function: require()
  • The console for simple output and debugging

How do they do it?

With a some help from libuv

  • Full-featured event loop backed by epoll, kqueue, IOCP, event ports.

  • Asynchronous TCP and UDP sockets

  • Asynchronous DNS resolution

  • Asynchronous file and file system operations

Is this “new”

Kind of…

  • In the past blocking operations, separate threads, and/or processes where used per TCP/UDP socket or so…

  • The NGINX web server (comercial and open source) used this approach to get greatly improved performance over Apache.

  • The Apache Webserver moved to an event based approach in version 2.4

  • Python libraries such as eventlet take a similar approach to achieve high performance.

Node Hands On!

Running JavaScript Programs

  • For a console like JavaScript shell type node in a windows, Mac, or Linux command window. Use .exit to exit.

  • To run a JavaScript file type node Filename.js from a command shell.

Example

File hiNode.js:

// run me with: node hiNode.js
let a = [1, 3, 5, 7, 9, 11];
function cube(x) {
    return x*x*x;
}

console.log("Hello from Node.js");
for (let x of a) {
    console.log(`x = ${x} and x cubed = ${cube(x)}`);
}

Node Knows about files/dirs

When running a JavaScript file Node provides some global variables:

  • __filename the full file name of this JavaScript file
  • __dirname the directory this file is in

Example current file/dir

Put the following code into a file named whoAmI.js and run it with node whoAmI.js.

console.log("Hello from Node.js");
console.log("The name of this file is: " + __filename);
console.log("This file is in the directory: " + __dirname);

Node.js API Use

Node.js has a module system that predates the ES6 standard

  • To “import” an API module for use syntax such as:
const varName = require('moduleName');
  • Where varName is just the name of a variable that you choose and moduleName is the name of the API module.

Example: List of Files

Getting a list of all files in a directory

const fs = require('fs'); // File system module
// List all the files in the current directory
let allFiles = fs.readdirSync(__dirname);
console.log("Files in the current directory:")
for (let file of allFiles) {
    console.log('\t' + file);
}

Example: Reading a File

Reading from a particular file:

const fs = require('fs'); // File system module
let fname = __dirname + '/hiNode.js';
// Callback for readFile
function processFile(err, data) {
    if (err) {
        console.log("Some kind of error");
        return;
    }
    console.log(data);
}
// This function takes a callback function as an argument.
fs.readFile(fname, 'utf8', processFile);

A Callback on ReadFile?

  • Yes, this allows for “non-blocking” operation
  • Recall: Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.
  • But JavaScript execution in Node.js is single threaded, so Node.js uses an event driven model for I/O to increase performance and scalability.

Synchronous File Read

Alternative synchronous call. Simpler but will block. But we won’t worry about that in this course.

const fs = require('fs'); // File system module
let fname = __dirname + '/whoAmI.js';
let fdata = fs.readFileSync(fname, 'utf8');
console.log("Contents of file whoAmI.js:");
console.log(fdata);

Synchronous File Write

const fs = require('fs'); // File system module
let fname = __dirname + '/tempWrite.txt';
let data = "Just a little text. \n Did this write?";
let fdata = fs.writeFileSync(fname, data);
console.log("Wrote test file: tempWrite.txt");

NPM

What is it

From npmjs.com

npm is the package manager for JavaScript and the world’s largest software registry.

It also offers a simple script running system.

All the development tools and JavaScript libraries we will be using will be obtained via npm. npm gets installed with Node.js. npm can take some time to install packages.

Installing Packages

See npm docs

  • Packages can be installed “locally” to your working directory
    • npm install <package_name> See local install
    • We’ll use this option for libraries and some development tools

Installing Packages Globally

See npm docs

  • Packages can be install “globally” on your machine
    • npm install -g <package_name> See global install
    • This option is generally used for general development tools.

Example: Static Webserver I

  • A great server for static site development Serve
  • npm install -g serve
  • From the command line go to the root directory for your site and type: serve
  • Use serve --help to see options

Example Static Webserver II

What if there is already a server at localhost:5000?

  • Specify another port with serve -l 5001 or any other unused port number
  • Can also specify an IP address to be used serve -l tcp://127.0.0.22:2112 where you can put in any suitable IP address for 127.0.0.22 and port for 2112

Example: Global

  • Global install of the “hot new” JavaScript code formatter called prettier.

  • npm install -g prettier

  • Can now run prettier via the command line with:
    • prettier filename.js

Example: Local

The request library

  • npm install request
  • Command line session:
const request = require('request');
var myErr, myResp, myBody;
request('http://www.grotto-networking.com',
    function (error, response, body) {
        myErr = error;
        myResp = response;
        myBody = body;});
// A bunch of stuff gets spit out...
myResp.statusCode; // What was the code
myResp.headers  // What was the server, date,...

node_modules directory

  • npm puts locally installed packages into a node_modules directory within your project directory.
  • The node_modules directory can get very large.
    • Don’t check this directory into version control.
    • You don’t need to share this directory since it can be re-created from a package.json file (to be discussed)

package.json

Why

From Using package.json

  • It serves as documentation for what packages your project depends on.
  • It allows you to specify the versions of a package that your project can use using semantic versioning rules.
  • Makes your build reproducible which means that its way easier to share with other developers.

Minimal package.json

You can build it by hand:

{
    "name": "your-project-name",
    "version": "0.0.1"
}

Creation Script

  • Run npm init and npm will ask you questions to build you package.json file.

  • Run npm init --yes will create a package.json you can later edit without all the questions. Try it!

Specifying Packages

From using package.json

To specify the packages your project depends on, you need to list the packages you’d like to use in your package.json file. There are 2 types of packages you can list:

  • “dependencies”: these packages are required by your application in production
  • “devDependencies”: these packages are only needed for development and testing

Maintaining package.json

  • You can manually add and delete your projects dependencies, but its much easier to let npm help you.

  • For a regular package dependency use npm install package_name --save when you install a package.
  • For a development dependency use npm install package_name --save-dev when you install a development package.

Re-creating node_modules

  • You should version control your package.json file.
  • With a package.json file in a directory, just run npm install to re-create your node_modules directory.