Page Contents

Web Development Topics

Here is a categorized list of topics that must or maybe covered during the class. We will not necessarily cover these in linear order. For example picking up a new programming language takes time even for those already familiar with a programming language or two. Hence we will start getting comfortable with JavaScript as a programming language right away before while we are learning HTML and CSS and before we start applying it to web development.

There will be a separate lesson/lecture plan that I'll update each week to indicate which of the readings from the long list below need to be read. The goad is to be enable students to be much more productive sooner than a dry linear traversal of all the technologies involved in web development. In particular we'll be striving to get "hands on" with topics as early as possible.

Web technology and hence web development has evoled substantially over the years. The following blog post provides a very good web development/technology history lesson.

Some Resources

Markup Languages and Such

HTML

  1. Getting Started with HTML. Topics covered:

    • Elements, Nesting Elements, Block versus inline elements, empty elements;
    • Attributes, boolean attributes, single or double quotes;
    • Anatomy of an HTML document; Whitespace in HTML; Entity references/special characters; HTML Comments;

  2. What’s in the head? Metadata in HTM. Topics covered:

    • Title, Medtadata, character encoding, description and SEO
    • Custom icons, adding CSS and JavaScript to the page, document language

  3. HTML text fundamentals. Topics covered:

    • Structure in web pages. Headings and Paragraphs
    • Lists: Unordered, Ordered, (they don't cover description lists), nested lists.
    • Emphasis and Importance, but not style...

  4. Creating hyperlinks. Topics covered:

    • Anatomy of a link, the title attribute, block level links.
    • URLs and paths, block fragments and ids, absolute and relative URLs.
    • Link best practices, download attribute, email links.

  5. Advanced Text Formating. Read this, will only skim in class. Topics covered:

    • Description lists, Blockquotes, Inline quotations
    • Citations, Abbreviations, Address, Super/sub-script
    • Computer code markup, times and dates.

  6. HTML Character Entities, escaping, Unicode and such

  7. Document and website structure. Topics include:

    • Basic sections of a web page. Structuring a page.
    • <main>, <article>, <section>, <aside>, <header>, <nav>, <footer>.
    • Non-semantic wrappers: <span>, <div>.

  8. HTML Forms

    1. Your first form. Basics of the <form> element, action, and submit button.

    2. How to structure a form. How to use fieldsets, legends, labels, and such.

    3. The native form widgets. Includes: all sorts of text input fields, select boxes, checkable items, buttons, and more. Use this more as a reference. Did you notice that things like tabbed panes, dialog boxes, drop down menus are missing?

    4. Sending form data

  9. Multimedia and Embedding

    1. Images in HTML
    2. Video and audio content.
    3. Other embedding techniques. We'll probably just mention this...
    4. Adding vector graphics.
    5. Responsive images

  10. Custom data attributes. Defines the format of data-* attributes and how to use them from JavaScript and CSS.

  11. Should check your HTML against the W3C Markup Validation Service. It's free!

SVG

Scalable Vector Graphics (SVG) gives us an example of another markup language that can play a complementary role to HTML.

  • MDN SVG page.

  • MDN SVG Tutorial.

  • The Inkscape free drawing program uses SVG as its native format.

  • The SVG optimizers missing GUI. SVG files produced by programs frequently contain lots of extra stuff making them much larger than needed. SVGO can optimize these files, but may also break things, hence use the GUI (Browser) version and watch the effects of the various optimizations.

  • Data Driven Documents D3 is the premier open source JavaScript library for data visualization and allows you to do amazing things with SVG. See this example on the Health/Wealth of Nations.

  • A huge Compendium of SVG Information.

Development Tools/Environment

Recomended Other good free editors include Atom, Notepad++ (windows), Vim, Geany.

  1. Node.js for server side (non-browser), and development environment.

    1. Good overview: Why the H-ll Would You Use Node.js.

  2. NPM -- package management and scripting

    1. Beginners guide to npm.

  3. JavaScript Linter ESLint. Modern and well documented.

  4. Webpack: "webpack is a bundler for modules. The main purpose is to bundle JavaScript files for usage in a browser, yet it is also capable of transforming, bundling, or packaging just about any resource or asset."

  5. For syntax highlighting for assignments and such we will be using Prism.js.

CSS

MDN CSS is a really good introduction and reference.

Introduction to CSS

Utilizing MDN Intro to CSS.

  1. How CSS works. Topics covered:

    • General concepts of documents and stylesheets, rendering
    • External, Internal, and Inline stylesheets

  2. CSS Syntax. Topics covered:

    • CSS declarations, CSS declaration blocks
    • CSS selectors and rules, CSS statements, "At-rules"
    • White space, comments, and shorthand

  3. Selectors Introduction.

  4. Simple Selectors. Topics covered:

    • Element selectors, Class selectors, ID selectors
    • Multiple classes on elements

  5. Attribute Selectors. Read. Know these exist and can be used to solve problems.

  6. Pseudo-classes and pseudo-elements. Read and practice. These are two important CSS mechanism used in a variety of situations including UIs.

  7. Combinators. Important! This is how we combine selectors for more refined selection. These are used all the time. However, I tend to forget the syntax hence the table at the beginning of the article is great!

  8. CSS values and units. Important but you are not expected to remember everything here. Some fun code examples.

  9. Cascade and inheritance. Important. This explains the notion of the cascade and which of multiple rules gets applied to an element. Discusses specificity. Key to debugging your CSS!

  10. The Box Model. Fundamental. Topics covered:

    • Box properties: content, padding, border, and margin
    • Advanced properties: overflow (scrolling), clip, outline
    • Types of boxes: block, inline, inline-block

Styling Text

Styling Text Overview

  1. Fundamental text and Font styling. Topics covered include:

    • Fonts: color, families, default, stacks, size, style, weight, decoration, transform.
    • Text layout: alignment, line height, spacing, etc...

  2. Styling Lists. Topics include: list spacing, bullet styles, position, custom bullets, list counting.

  3. Styling Links. Topics covered: link states, psuedo-selectors, styling links as buttons. Note: you need to know about this stuff but I never remember all the details.

  4. Web fonts.

Styling Boxes

Styling Boxes Overview

  1. Box Model Recap. Nice review also hits some more advanced options such as max/min width/height.

  2. Backgrounds. Important since a lot of very nice styling can be done via backgrounds. Topics covered: color, image, position, repeats, gradients, size.

    • Assignment idea: Go to one of the Websites to Generate SVG Patterns and use one of these to generate background and or border patterns. Make sure you understand what you are getting!

  3. Borders. A more in depth look at borders.

  4. Styling Tables. Tables have lots of "knobs and dials" for styling. This is a good tutorial, however we may not explicitly cover this in class.

CSS Layout

CSS Layout Overview

  1. Introduction to CSS layout. Good overview. Topics include: floats, positioning, tables, flex box, and grid.

  2. Floats. Important. This is the foundation for most layouts today.

  3. Positioning. Important. Topics include: static, relative, absolute, fixed, top, bottom, left, right, positioning contexts, z-index, margin-collapse.

  4. Practical Positioning Examples. Good examples of how tabbed panels and slide out panels are implemented. Requires the following JavaScript knowledge:

    • DOM API: document.querySelectorAll(), GlobalEventHandlers.onclick, Element.getAttribute(), Element.setAttribute(), Element.removeAttribute()
    • Functions, anonymous functions, passing a function as a value, shows a closure being used.

  5. A Complete Guide to Flexbox. Nice new layout technique. Available in current Firefox and Chrome.

  6. A Complete Guide to Grid

    • Grids. May or may not cover depending on browser support status.
    • Possible guide to CSS Grids
    • A pretty good introduction: Getting to know CSS Grid Layout. Note for class we would not be concerned with their "legacy grid" discussion.

CSS Transitions & Animations

Optional topic. Note that CSS transitions are well supported now.

  1. Using CSS transitions
  2. [All you need to know about CSS Transitions](All you need to know about CSS Transitions)
  3. Using CSS animations

CSS Practice

  1. To check whether you can use a CSS/JavaScript/DOM feature head over to Can I Use?.

  2. Hands on Practice with CSS selectors via either document.querySelectorAll(...). querySelector/querySelectorAll are new features of the document interface and available in current versions of IE, edge, Firefox, and Chrome. Can use the HTMLElement.style DOM attribute to look at and set the style.

  3. Style frameworks: Bootstrap, Foundation, Pure, and many others.

    1. All provide grid systems, these systems are implemented via classes and underneath are CSS floats.
    2. All provide a nice uniform look and feel.
    3. One usually starts with one of these rather than doing all your styling from scratch, unless your primary role is page layout/design.

  4. CSS processors: Sass (note bootstrap 4 is moving to sass), Less. These "languages" compile into regular CSS but add programming capabilities such as variables, containment, inheritance, etc... to help reduce code size and maintain consistency across large CSS projects.

  5. Invisible CSS. Good review and in depth look at important CSS concepts.

  6. CSS in the Large. One thing we didn't have time to cover is techniques for dealing with CSS when the amount of styling work becomes large.

    1. CSS Evolution: From CSS, SASS, BEM, CSS Modules to Styled Components

    2. BEM "BEM is a highly useful, powerful, and simple naming convention that makes your front-end code easier to read and understand, easier to work with, easier to scale, more robust and explicit, and a lot more strict." How we use BEM to modularise our CSS

JavaScript

We'll be using two environments to run JavaScript: (1) Web Browser, (2) Node.js for the server side of things.

Javascript Language

JavaScript History from a standards perspective. We'll be using the version known as ES6, aka ES2015.

ES6 In Depth Articles. A lot of good articles on niffty ES6 features.

We are going to use MDN Javascript Guide as our JavaScript textbook.

  1. Introduction. Covers JavaScipt history (short), relation to Java, JavaScript/ECMAScript, Web Console.

  2. Grammar and types. Topics include:

    • Case sensitive, Unicode,
    • Variables, declarations, scope, globals, hoisting, function hoisting, constants.
    • Data types: Primitives- Boolean, null, undefined, Number, String, Symbol; Object.
    • Type conversion, strings to numbers.
    • Literals: Arrays, Boolean, Floating point, Integers, Object, RegExp, String.

  3. Control flow and error handling. Topics include:

  • ";" as a statement separator, block statements
  • Conditional statements, "falsy values", Switch statements
  • Exception handling statements, exception types, throw statement, try/catch statement.
  • Promises (we'll cover these a bit later in the course)

  1. Loops and iteration. Topics include:

    • for, do/while, and while statements
    • break and continue statements
    • for...in statement, for...of statement

  2. Functions. Topics include:

    • function declarations and expressions, primitives pass by value, objects pass by reference.
    • function scope, nested functions and closures
    • arguments object, parameters, default parameters, rest parameters.
    • arrow functions, predefined functions

  3. Dealing with this

  4. Expressions and operators. Topics include:

    • assignment, comparison, arithmetic, bitwise, logical, and string operators.
    • conditional (ternary) operator, comma, unary (delete, typeof), relational operators (in, instanceof), operator precedence.
    • Primary expressions (this, grouping, comprehensions), left hand-side expressions (new, super, spread)

  5. Numbers and dates. Topics include:

    • decimal, binary, octal, hexadecimal, exponentiation (scientific notation)
    • Number object: Lots of good stuff including min/max limits, NaN, machine epsilon,
    • Methods of Number: parseFloat(), parseInt(), isFinite(), isInteger(), isNaN(), isSafeInteger()
    • String conversion/formating: toExponential(), toFixed(), toPrecision().
    • The Math object: here is where you will find all the functions!
    • Date object

  6. Text formatting. Topics include:

    • A survey of the very helpful string functions that you shouldn't be re-writing!
    • Multi-line template literals (new)

  7. Regular expressions. May or may not cover.

  8. Indexed collections. Topics include:

    • Basic usage and iteration
    • Array methods: search like, combination, stack like
    • Array methods functional: forEach, map, filter, every, some

  9. Keyed collections. Maps and Sets. Don't write your own! Will probably not cover weak maps and sets.

JavaScript Objects, prototypes, and inheritance

  1. Understanding JavaScript Constructors.

  2. Understanding "Prototypes" in JavaScript. Written in 2011 and covers ECMAScript 5.1 (JavaScript 5.1). Very well written and organized.

  3. Working with objects. Important! Topics include:

    • Objects and properties, enumeration of properties
    • Creating new objects: initializers, constructor function, Object.create()
    • Inheritance, prototype, defining properties, defining methods
    • Using this for object references, getters, setters
    • Object comparison

  4. MDN JavaScript Object. Reference.

  5. Some details on Functions that we will use. In particular we will have a need to sometimes explicitly set the this value. We'll use the Function.prototype.bind() method that all functions support. Related methods are apply() and call()

  6. Details of the object model. Discusses the differences between a class based and prototype based languages. Compare Java and JavaScript in a inheritance hierarchy example.

Promises

The Document Object Model (DOM)

  1. Introduction to the DOM. Fundamental. Topics covered:

    • What is the DOM?, DOM and JavaScript
    • Accessing the DOM, important Datatypes
    • DOM interfaces and objects, Core interfaces.

  2. Make sure we go over key features of the following interfaces:

  3. The Basics of DOM Manipulation in Vanilla JavaScript. Covers (without jQuery):

    • querying and modifying the DOM,
    • modifying classes and attributes,
    • listening to events, and animation.

  4. A much better Introduction to DOM Events from Smashing Magazine. Hits key issues. Topics covered:

    • Listening for DOM events; add and remove listeners
    • Maintaining Callback Context: the perils of this and how to fix it.
    • The Event object and what's in it.
    • Event phases: capture, target, and bubbling.
    • Stopping propagation, and default browser behavior
    • Custom Events, Delegate event listeners, Some useful events.

  5. Events and the DOM. Fundamental, but a bit too short. Topics covered:

  • Registering event listeners: addEventListener method, HTML attribute, DOM element property.
  • Accessing event interfaces

  1. DOM Reference.

  2. Special Purpose APIs (optional material)

    1. Intro to Web Audio

JavaScript Style and Idioms

  1. Clean Code JavaScript

Networks, HTTP, and Servers

Networking

  1. My introductory networking slides.
  2. localhost and loopback addresses
    • IPv4 loopback address range: 127.0.0.1 to 127.255.255.254. We can listen on any of these.
    • localhost gets mapped to 127.0.0.1
  3. Note that these local/loopback IPv4 addresses should not be confused with private IP addresses.

HTTP/HTTPS/1 and 2, Websockets

  1. Overview of HTTP. Topics include:

    • Client (user-agent), Server, and Proxies
    • Basics of HTTP as a protocol; stateless but not sessionless
    • Controlling some aspects of HTTP: Cache, CORS, Authentication,...

  2. More information on proxy types. Forward and Reverse.

  3. HTTP Messages. Good overview of HTTP messages (both 1.x and 2). Goes over general structure of both Request and Response messages including start/status line, headers, and body.

  4. HTTP cookies. Lots of good info. More than we'll need for the class.

  5. Identifying Resources on the Web. Fundamental. Topics include:

    • URI, URL, URN and all that...
    • Syntax of URIs: protocol, authority, port, path, query, fragment

  6. MIME Types. Used to inform the other side as to the type of content we are sending or wish to receive. See also their handy MIME reference list. We will frequently use the application/json type for data exchange.

  7. JSON is the the most popular general data exchange format used in modern web programming. Although the term AJAX for Asynchronous JavaScript And XML most modern web applications use JSON for these types of data exchanges.

  8. Websocket (Wikipedia), WebSockets (MDN). Note that Can I use? shows WebSockets supported by most modern browsers and is no longer "experimental".

    • A pragmatic approach based on Socket.io
    • Their discussion of the benefits of websockets and the realities is quite good. They use a "long-polling" approach and they try to set up a better channel over Websockets or other mechanisms.

  9. HTTPS (wikipedia) runs the HTTP protocol over Transport Layer Security (TLS). It is important to use HTTPS for any web site that stores user data. In addition, you can only be sure of the legitimacy of a website if it uses HTTPS.

  10. Debugging applications at the network. Modern browsers (Firefox and Chrome) provide built in tools to see the messages that are sent between client and server. If one needs to actually verify what is being sent a sniffer/protocol analyzer such as the Wireshark can be used.

    • A good filter to use with wireshark is tcp.port == 8080 && http This selects all HTTP trafic associated with a particular port.

  11. The IETF HTTP Standards

    • Original HTTP 1.1 RFC2616 obsolete
    • Up to date HTTP 1.1 suite RFCs 7230, 7231, 7232, 7233, 7234, and 7235.
    • HTTP 2 RFC7540.

The Server Side of Things

  1. From the recent Netcraft surveys the two most popular web servers for active and top million busiest sites were Apache and nginx.

    • Wikipedia has a reasonably short Apache Overview. Basic Apache functionality is extended via Apache Modules (Wikipedia). These provide security enhancements, ability to interface with languages such as PHP, Python, Ruby, etc...
    • Apache 2.2 uses a process/thread model, making it significantly slower than nginx. Apache 2.4 has changed this somewhat. Apache probably has the steepest learning curve when it comes to configuration and use.
    • Note that there are OpenSource and Comercial versions of nginx.

  2. Server Side Web Frameworks

    • One "popularity" based listing — skim, but this includes some frontend frameworks as well. Another listing somewhat broken down by languages is available at Web Frameworks (Wikipedia) — skim.

    • For learning purposes we will use a minimalistic web framework, also known as a microframework — read. It should be noted however that such systems are also used underneath much larger frameworks. Since we are restricting ourselves to a single language, JavaScript, in this course we will be using the Expressjs.

      • A microframework will give us near full access to the HTTP protocol (request, response messages) in a nice way without having to deal with TCP and IP layers.

      • Frequently in deployment server side frameworks sit behind a primary server. Recall the proxy stuff from the HTTP introduction.

      • A bit more fully featured but still relatively small framework built on express is Feathers. Fairly extensive documentation. I have not tried this yet.

      • Good article on how to scale up a Node.js application based on number of users. Optional.

Server Practice with Express.js

  1. The "Hello World" of websites.

  2. Making HTTP requests without a browser. We will use the very popular request library for Node.js to poke and probe our server side code. Note that this library is similar in spirit to the popular Requests Python package.

    • Use this to make a GET request to your website in part 1.
    • Use this to GET http://www.grotto-networking.com, look at the HTTP headers and tell me what server my web hosting company is using for my static site.

  3. Application layer routing on the Server.

    • Basic routing. This is a common approach used by many different frameworks in many different languages.
    • Looking at the request information including headers. Example code.
    • We need to be able to serve static files such as HTML pages, CSS, and JavaScript files. Most microframeworks have some provision for this, however, in production systems this could be handled more effectively by the front end server.
    • More advanced Routing can include matching for paths, and parameters extracted from the URL. This section also summarizes response methods.

Templates

  • Template engines are used to create HTML from user inputs, database queries, etc on the server. In addition some Front-end frameworks also use them. Different computing languages can have different template languages, some work across multiple languages. These are also used to create static site generators. A comparison of some JavaScript Templating Engines shows quite a diversity and some rather unusual looking syntax for some of these.

  • We will use Mustache since it requires one of the smaller learning curves. The particular version we'll use is mustache.js which we'll install via npm: npm install -g mustache.

Markdown

Using raw HTML or template languages can be quite tedious for everyday types of writing or code documentation. In addition, taking the time to insert HTML tags can make the already painful task of writing even more horrible. An alternative approach is to use a simplified text format. One very popular format is markdown. As they say on the Commonmark home page:

It’s a plain text format for writing structured documents, based on formatting conventions from email and usenet.

Pandoc understands a number of useful markdown syntax extensions, including document metadata (title, author, date); footnotes; tables; definition lists; superscript and subscript; strikeout; enhanced ordered lists, ...

  • The Brackets and Atom editors have plugins for rendering markdown previews. In fact I use markdown and Pandoc to create these notes.

  1. Server Side Rendering via Templates & Markdown

    • Forms and POST request example. Students will make a form that can be retreived as "static" content, then post to a particular path and have the server issue a simple reply. Complication: Express.js doesn't do any parsing of the request body by default. We will use the body-parser middleware library to handle this for us. Note the content-type sent in the request header to pick the appropriate parser from this library.

    • Mustach-express NPM module to ease integration of mustach with express. This works out of the box except for the missing information that your templates need a *.mustache extension.

    • Other uses of template engines: static site generators. Example Node.js/JavaScript static site generators: Metalsmith and Wintersmith. Note that I have not tried either of these.

    • Templates from another realm the game of Madlibs!

  2. Sending and receiving JSON (no Browser)

    • Use request to send a JSON encoded request to the server.
    • Use express with body-parser to receive and respond with JSON.

  3. There is a websockets module for express. We may or may not try something with this.

Sessions Between Browser and Servers

  1. HTTP cookies (MDN). Read section onf Creating Cookies, Skim sections on Security and Tracking and privacy.

  2. Simple preferences and cookies example with express.js Will use templates so we can show information from the cookie.

    • See express examples.

  3. Web Token example (Will not cover)

Webpage/Webapp and server Interactions

Under Construction, subject to change

  1. AJAX and its progeny (we'll use fetch), Promises
    • What could be sent from a browser? Forms, Files, ...
    • Ability to send and receive from JavaScript running on a page.
  2. Web APIs: REST, GraphQL, etc...

Notes on Node.js

  • My Slides on Node.js and NPM.

  • Node.js homepage.

  • Node.js ES2015 (ES6) support.

  • Node.js has its own simple Module system based on a function called require(...). Currently standard JavaScript modules import/export are not supported by any browsers we'll use them via a transpiler but not on the server side.

  • After installation you can start a command line JavaScript interface with the command node.

  • Exit the interface with the command process.exit()

  • You can use the require function to import JSON files into objects. For example var test1 = require('./first.json').

  • Example quick session with request library to check it out:

var request = require('request') // Pre-standard module system
var rError, rResponse, rBody;
request('http://www.grotto-networking.com',
    function(error, response, body) {
        rError = error;
        rResponse = response;
        rBody = body;
        }
    );
// Then can look at things with the rResponse, and rBody variables.
console.log(rResponse.statusCode);

Web Applications, Front End Libraries and Frameworks

Under Construction, subject to change

From documents — web pages — to applications and GUIs. Was HTML, CSS, and JavaScript designed for this? Not originally... but they have adapted well.

Single Page Web Applications

Under Construction, subject to change

  1. A modern approach to web applications

    • What is MVC? Anatomy of a JavaScript MV* Framework. Although written in 2014 the key concepts are sound, don't bother with the details of any particular framework mentioned. Goes over: routing, data binding, templates/views, models, and data access.
    • More insights into Client-Side Routing. Just the introductory and comparison material, not the design discussion.

  2. Front end frameworks in general: Angular, Ember, Vue. we'll most likely use React this semester, but Vue is coming on strong and seems very well documented.

    • What parts of MVC do they provide? What else do they provide? Routing, Rendering, State management, ....
    • React. "A JavaScript library for building user interfaces". Read this page, don't worry if you don't understand all the details yet.
    • Vue.js. The Vue.js folks provide a very good comparison of Vue.js to other frameworks.

Basic React

  1. Good, simple, comparison to native DOM API article illustrating the "virtual DOM" Why react is taking over.

  2. When Does a Project Need React?

We will use the React "Quick Start" series to get up to speed with the fundamentals of React.

  1. Elements and Rendering

  2. Components

  3. Putting React to Work (more power!) *Conditional Rendering

  4. Basic Design Patterns

  5. Advanced Functionality

More Functionality via the React Eco-system

  1. AJAX and React: React AJAX Best Practices. Like this article we will be using the standardized fetch() JavaScript API in this class.

  2. Routing in React via the React-Router.

  3. An more natural approach to bringing in a CSS/UI framework: React-Bootstrap. Bootstrap styling with a collection of React components for Bootstrap look and feel.

    1. Best UI Frameworks for your new React.js App.

  4. My favorite React table library: react-bootstrap-table. Has bootstrap styling, but offers an incredible amount of functionality as a React component.

  5. Growing bigger. State Management with Redux.

  6. If we have time we'll also look at Immutable.js and what these immutable data structures do for us. When I first heard the term I wasn't sure that if it was a joke like a "Write Only Memory". It's better to think in terms of it like version control. In addition for state management in complex applications I've used it to create undo/redo functionality with minimal effort, i.e., the library does the "hard parts".

Miscellaneous Possible Topics

Under Construction, subject to change

  1. D3 and SVG

  2. Animation

  3. WebGL/Three.js

  4. Audio API

  5. Performance. For tools see Lighthouse, and listen to this JavaScript Jabber podcast. Also Webpack-bundle-analyzer.

  6. NoSQL Databases and similar things: examples MongoDB, Redis.