Software
Tools in
JavaScript

Good programming is not learned from generalities, but by seeing how significant programs can be made clean, easy to read, easy to maintain and modify, human-engineered, efficient, and reliable, by the application of common sense and good programming practices. Careful study and imitation of good programs leads to better writing.

— Brian Kernighan and P.J. Plauger, Software Tools in Pascal

The best way to learn design in any field is to study examples, and some of the best examples of software design come from the tools programmers use in their own work. In these lessons we build small versions of things like file backup systems, testing frameworks, regular expression matchers, and browser layout engines both to demystify them and to give some insights into how experienced programmers think. We draw inspiration from Brown2011,Brown2012,Brown2016, Mary Rose Cook's Gitlet, and the trilogy of books that introduced the Unix philosophy to an entire generation of programmers Kernighan1979,Kernighan1988,Kernighan1983.

All of the written material in this project can be freely reused under the terms of the Creative Commons - Attribution license, while all of the software is made available under the terms of the Hippocratic License; see for details.

All proceeds from this project will go to support the Red Door Family Shelter.

Who is our audience?

Every lesson should be written with specific learners in mind. These three personas are ours:

Like these three personas, readers should:

What tools do we cover?

Programmers have invented a lot of tools to make their lives easier. This volume focuses on a few that individual developers use while writing software; we hope that the second volume will explore those used in the applications that programmers build.

TitleContent
Systems ProgrammingUsing callbacks to manipulate files and directories
Asynchronous ProgrammingUsing promises to manage delayed computation
Unit TestingTesting software piece by piece
File BackupArchiving files with directory structure
Data TablesLoading, saving, and manipulating tables efficiently
Pattern MatchingUsing patterns to find things in data
Parsing ExpressionsTurning text into code
Page TemplatesGenerating HTML pages from templates
Build ManagerUpdating files that depend on other files
Layout EngineFiguring out what goes where in a web page
File InterpolatorManaging source files that have been broken into pieces
Module LoaderLoading source files as modules
Style CheckerChecking that code conforms to style guidelines
Code GeneratorModifying code to track coverage and execution times
Documentation GeneratorGenerating documentation from comments embedded in code
Module BundlerTurning many files into one
Package ManagerGetting and installing packages
Virtual MachineAssembling and running low-level code
DebuggerRunning programs under the control of a breakpointing debugger
ConclusionWhere we have been and where you could go next

What ideas do we cover?

The glossary in defines the terms we introduce in these lessons. Together, these define the scope of our lessons:

How are these lessons laid out?

We display JavaScript source code like this:

for (const thing in collection) {
  console.log(thing)
}

and Unix shell commands like this:

for filename in *.dat
do
    cut -d , -f 10 $filename
done

Data and output are shown in italics:

Package,Releases
0,1
0-0,0
0-0-1,1
00print-lol,2
00smalinux,0
01changer,0

We occasionally wrap lines in source code in unnatural ways to make listings fit the printed page. Where we need to break lines of output for the same reason, we end all but the last line with a single backslash \. We also sometimes use ... to show where lines have been omitted to shorten listings. The full listings are all available in this project's Git repository and on our website.

Finally, we write functions as functionName rather than functionName(); the latter is more common, but people don't use objectName{} for objects or arrayName[] for arrays, and the empty parentheses makes it impossible to tell whether we're talking about "the function itself" or "a call to the function with no parameters", which is confusing when we're passing functions as arguments.

How did we get here?

In the early 2000s, the University of Toronto asked Greg Wilson to teach an undergraduate course on software architecture. After delivering the course three times he told the university they should cancel it because of a lack of useful material: between them, the dozen textbooks he had purchased with the phrase "software architecture" in their titles devoted a total of less than 30 pages to describing the designs of actual systems.

Frustrated by that, he and Andy Oram persuaded some well-known programmers to contribute a chapter each to a collection called Beautiful Code Oram2007, which went on to win the Jolt Award in 2007. Entries in the book described everything from figuring out whether three points are on a line to core components of Linux and the software for the Mars Rover, but the breadth that made them fun to read also meant they weren't particularly useful for teaching.

To fix that, Greg Wilson, Amy Brown, Tavish Armstrong, and Mike DiBernardo edited a four-book series between 2011 and 2016 called The Architecture of Open Source Applications. In the first two volumes, the creators of 50 open source projects described their systems' designs; the third book explored the performance of those systems, while in the fourth and final volume, contributors built scale models of common tools as a way of demonstrating how those tools worked. These books were closer to what an instructor would need for an undergraduate class on software design, but still not quite right: the intended audience would probably not be familiar with many of the problem domains, and since each author used the programming language of their choice, much of the code would be hard to understand.

Software Tools in JavaScript is meant to address these shortcomings: all of the code is written in one language, and the examples are all tools that programmers use daily. Most of the programs are less than 60 lines long and the longest is just over 300; we believe each chapter can be covered in class in 1-2 hours, while the exercises range in difficulty from a few minutes to a couple of days.

How can people contribute?

If you would like to improve what we have or add new material, please see the Code of Conduct in and the contributor guidelines in . If you have questions or would like to use this material in a course, please send us email.

Who helped us and inspired us?

This book is dedicated to Brian Kernighan, who taught us all how to write about software.

We are very grateful for feedback from Darren McElligott and Evan Schultz, and to the creators of EJS, Emacs, ESLint, Glosario, GNU Make, LaTeX, Node, NPM, Standard JS, WAVE, and all the other open source tools we used in creating these lessons. If we all give a little, we all get a lot.