Installing and running Node.js bin scripts

This content originally appeared on 2ality – JavaScript and more and was authored by Dr. Axel Rauschmayer

The package.json property "bin" lets an npm package specify which shell scripts it provides (for more information, see “Creating ESM-based shell scripts for Unix and Windows with Node.js”). If we install such a package, Node.js ensures that we can access these shell scripts (so-called bin scripts) from a command line. In this blog post, we explore two ways of installing packages with bin scripts:

• Locally installing a package with bin scripts means installing it as a dependency inside a package. The scripts are only accessible within that package.

• Globally installing a package with bin scripts means installing it in a “global location” so that the scripts are accessible everywhere – for either the current user or all users of a system (depending on how npm is set up).

We explore what all of that means and how we can run bin scripts after installing them.

Installing npm registry packages globally  #

Package cowsay has the following package.json property:

"bin": {
  "cowsay": "./cli.js",
  "cowthink": "./cli.js"
},

To install this package globally, we use npm install -g:

npm install -g cowsay

Caveat: On Unix, we may have to use sudo (we’ll learn soon how to avoid that):

sudo npm install -g cowsay

After that, we can use the commands cowsay and cowthink in our command lines.

Note that only the bin scripts are available globally. The packages are ignored when Node.js looks up bare module specifiers in node_modules directories.

Which packages are installed globally? npm ls -g  #

We can check which packages are installed globally and where:

% npm ls -g
/usr/local/lib
├── corepack@0.12.1
├── cowsay@1.5.0
└── npm@8.15.0

On Windows, the installation path is %AppData%\npm, e.g.:

>echo %AppData%\npm
C:\Users\jane\AppData\Roaming\npm

Where are packages installed globally? npm root -g  #

Result on macOS:

% npm root -g
/usr/local/lib/node_modules

Result on Windows:

>npm root -g
C:\Users\jane\AppData\Roaming\npm\node_modules

Where are shell scripts installed globally? npm bin -g  #

npm bin -g tells us where npm installs shell scripts globally. It also ensures that that directory is available in the shell PATH.

Result on macOS:

% npm bin -g
/usr/local/bin

% which cowsay
/usr/local/bin/cowsay

Result on the Windows Command shell:

>npm bin -g
C:\Users\jane\AppData\Roaming\npm

>where cowsay
C:\Users\jane\AppData\Roaming\npm\cowsay
C:\Users\jane\AppData\Roaming\npm\cowsay.cmd

The executable cowsay without a filename extension is for Unix-based Windows environments such as Cygwin, MinGW, and MSYS.

Windows PowerShell returns this path for gcm cowsay:

C:\Users\jane\AppData\Roaming\npm\cowsay.ps1

Where are packages installed globally? The npm installation prefix  #

npm’s installation prefix determines where packages and bin scripts are installed globally.

This is the installation prefix on macOS:

% npm config get prefix
/usr/local

Accordingly:

• Packages are installed in /usr/local/lib/node_modules
• Bin scripts are installed in /usr/local/bin

This is the installation prefix on Windows:

>npm config get prefix
C:\Users\jane\AppData\Roaming\npm

Accordingly:

• Packages are installed in C:\Users\jane\AppData\Roaming\npm\node_modules
• Bin scripts are installed in C:\Users\jane\AppData\Roaming\npm

Changing where packages are installed globally  #

In this section, we examine two ways of changing where packages are installed globally:

• Changing the npm installation prefix
• Using a Node.js version manager

Changing the npm installation prefix  #

One way of changing where packages are installed globally is to change the npm installation prefix.

Unix:

mkdir ~/npm-global
npm config set prefix '~/npm-global'

Windows Command shell:

mkdir "%UserProfile%\npm-global"
npm config set prefix "%UserProfile%\npm-global"

Windows PowerShell:

mkdir "$env:UserProfile\npm-global"
npm config set prefix "$env:UserProfile\npm-global"

The configuration data is saved to a file .npmrc in the home directory.

From now on, global installs will be added to the directory we have just specified.

Afterward, we still have to add the npm bin -g directory to our shell PATH so that our shell finds bin scripts we install globally.

A downside of changing the npm prefix: npm will now also be installed at the new location if we tell it to upgrade itself.

Using a Node.js version manager  #

Node.js version managers let us install multiple versions of Node.js at the same time and switch between them. Popular ones include:

• Unix: nvm
• Cross-platform: Volta

Installing npm registry packages locally  #

To install an npm registry package such as cowsay locally (into a package), we do the following:

cd my-package/
npm install cowsay

This adds the following data to package.json:

"dependencies": {
  "cowsay": "^1.5.0",
  ···
}

Additionally, the package is downloaded into the following directory:

my-package/node_modules/cowsay/

On Unix, npm adds these symbolic links for the bin scripts:

my-package/node_modules/.bin/cowsay -> ../cowsay/cli.js
my-package/node_modules/.bin/cowthink -> ../cowsay/cli.js

On Windows, npm adds these files to my-package\node_modules\.bin\:

cowsay
cowsay.cmd
cowsay.ps1
cowthink
cowthink.cmd
cowthink.ps1

The files without extensions are scripts for Unix-based Windows environments such as Cygwin, MinGW, and MSYS.

npm bin tells us where locally installed bin scripts are located – for example:

% npm bin
/Users/john/my-package/node_modules/.bin

Note: Locally, packages are always installed in a directory node_modules next to a package.json file. If the latter doesn’t exist in the current directory, npm searches for it in an ancestor directory and installs the package there. To check where npm would install packages locally, we can use the command npm root – for example (Unix):

% cd $HOME
% npm root
/Users/john/node_modules

There is no package.json in John’s home directory, but npm can’t install anything in an ancestor directory, which is why npm root shows this directory. Installing a package locally at the current location will lead to package.json being created and installation progressing as usual.

Running locally installed bin scripts  #

(All commands in this subsection are executed inside directory my-package.)

Running bin scripts directly  #

We can run cowsay as follows from a shell:

./node_modules/.bin/cowsay Hello

On Unix, we can set up a helper:

alias npm-exec='PATH=$(npm bin):$PATH'

Then the following command works:

npm-exec cowsay Hello

Running bin scripts via package scripts  #

We can also add a package script to package.json:

{
  ···
  "scripts": {
    "cowsay": "cowsay"
  },
  ···
}

Now we can execute this command in a shell:

npm run cowsay Hello

That works because npm temporarily adds the following entries to $PATH on Unix:

/Users/john/my-package/node_modules/.bin
/Users/john/node_modules/.bin
/Users/node_modules/.bin
/node_modules/.bin

On Windows, similar entries are added to %Path% or $env:Path:

C:\Users\jane\my-package\node_modules\.bin
C:\Users\jane\node_modules\.bin
C:\Users\node_modules\.bin
C:\node_modules\.bin

The following command lists the environment variables and their values that exist while a package script runs:

npm run env

Running bin scripts via npx  #

Inside a package, npx can be used to access bin scripts:

npx cowsay Hello
npx cowthink Hello

More on npx later.

Installing unpublished packages  #

Sometimes, we have a package that we either haven’t published yet or won’t ever publish and would like to install it.

npm link: installing an unpublished package globally  #

Let’s assume we have an unpublished package whose name is @my-scope/unpublished-package that is stored in a directory /tmp/unpublished-package/. We can make it available globally as follows:

cd /tmp/unpublished-package/
npm link

If we do that:

• npm adds a symbolic link to the global node_modules (as returned by npm root -g) – for example:

  /usr/local/lib/node_modules/@my-scope/unpublished-package
  -> ../../../../../tmp/unpublished-package
  

• On Unix, npm also adds one symbol link from the global bin directory (as returned by npm bin -g) to each bin script. That link is not direct, it goes through the global node_modules directory:

  /usr/local/bin/my-command
  -> ../lib/node_modules/@my-scope/unpublished-package/src/my-command.js
  

• On Windows, it adds the usual 3 scripts (which refer to the linked package via relative paths into the global node_modules):

  C:\Users\jane\AppData\Roaming\npm\my-command
  C:\Users\jane\AppData\Roaming\npm\my-command.cmd
  C:\Users\jane\AppData\Roaming\npm\my-command.ps1
  

Due to how the linked package is referred to, any changes in it will take effect immediately. There is no need to re-link it when it changes.

To check if the global installation worked, we can use npm ls -g to list all globally installed packages.

npm link: installing a globally linked package locally  #

After we have installed our upublished package globally (see previous subsection), we have the option to install it locally in one of our packages (which can be published or unpublished):

cd /tmp/other-package/
npm link @my-scope/unpublished-package

That creates the following link:

/tmp/other-package/node_modules/@my-scope/unpublished-package
-> ../../../unpublished-package

By default, the unpublished package is not added as a dependency to package.json. The rationale behind that is that npm link is often used to temporarily work with an unpublished version of a registry package – which shouldn’t show up in the dependencies.

npm link: undoing linking  #

Undoing the local link:

cd /tmp/other-package/
npm uninstall @my-scope/unpublished-package

Undoing the global link:

cd /tmp/unpublished-package/
npm uninstall -g

Installing unpublished packages via local paths  #

Another way of installing an unpublished package locally, is to use npm install and refer to it via a local path (and not via its package name):

cd /tmp/other-package/
npm install ../unpublished-package

That has two effects.

First, the following symbolic link is created:

/tmp/other-package/node_modules/@my-scope/unpublished-package
-> ../../../unpublished-package

Second, a dependency is added to package.json:

"dependencies": {
  "@my-scope/unpublished-package": "file:../unpublished-package",
  ···
}

This way of installing unpublished packages also works globally:

cd /tmp/unpublished-package/
npm install -g .

Other ways of installing unpublished packages  #

• Yalc lets us publish packages to a local “Yalc repository” (think local registry). From that repository, we can install packages as dependencies for, e.g., a package my-package/. They are copied into the directory my-package/.yalc and file: or link: dependencies are added to package.json.

• relative-deps supports "relativeDependencies" in package.json which (if they exist) override normal dependencies. In contrast to npm link and local path installations:

  • Normal dependencies don’t have to be changed.
  • Relative dependencies are installed as if they came from the npm registry (not via symbolic links).

  relative-deps also helps with keeping locally installed relative dependencies and their originals in sync.

• npx link is a safer version of npm link which doesn’t require a global install, among other benefits.

npx: running bin scripts in npm packages without installing them  #

npx is a shell command for running bin scripts that is bundled with npm.

Its most common usage is:

npx <package-name> arg1 arg2 ...

This command installs the package whose name is package-name in the npx cache and runs the bin script that has the same name as the package – for example:

npx cowsay Hello

That means we can run bin scripts without installing them first. npx is most useful for one-off invocations of bin scripts – for example, many frameworks provide bin scripts for setting up new projects and these are often run via npx.

After npx has used a package for the first time, it is available in its cache and subsequent invocations are much faster. However, we can’t be sure how long a package stays in the cache. Therefore, npx isn’t a substitute for installing bin scripts globally or locally.

If a package comes with bin scripts whose names are different from its package name, we can access them like this:

npx --package=<package-name> <bin-script> arg1 arg2 ...

For example:

npx --package=cowsay cowthink Hello

The npx cache  #

Where is npx’s cache located?

On Unix, we can find that out via the following command:

npx --package=cowsay node -p \
  "process.env.PATH.split(':').find(p => p.includes('_npx'))"

That returns a path similar to this one:

/Users/john/.npm/_npx/8f497369b2d6166e/node_modules/.bin

On Windows, we can use (one line broken up into two):

npx --package=cowsay node -p
  "process.env.Path.split(';').find(p => p.includes('_npx'))"

That returns a path similar to this one (single path broken up into two lines):

C:\Users\jane\AppData\Local\npm-cache\_npx\
  8f497369b2d6166e\node_modules\.bin

Note that npx’s cache is different from the cache that npm uses for the modules it installs:

• Unix:

  • npm cache: $HOME/.npm/_cacache/
  • npx cache: $HOME/.npm/_npx/

• Windows (PowerShell):

  • npm cache: $env:UserProfile\AppData\Local\npm-cache\_npx\
  • npx cache: $env:UserProfile\AppData\Local\npm-cache\_cacache\

The parent directory of both caches can be determined via:

npm config get cache

For more information on the npm cache, see the npm documentation.

In contrast to the npx cache, data is never removed from the npm cache, only added. We can check its size as follows on Unix:

du -sh $(npm config get cache)/_cacache/

And on Windows PowerShell:

DiskUsage /d:0 "$(npm config get cache)\_cacache"

Further reading  #

This blog post is part of a series on Node.js shell scripting:

• Using web streams on Node.js
• Working with the file system on Node.js
• Executing shell commands from Node.js
• Node.js: checking if an ESM module is “main”
• Working with file system paths on Node.js
• Node.js: creating ESM-based shell scripts for Unix and Windows
• Parsing command line arguments with util.parseArgs() on Node.js
• Installing and running Node.js bin scripts

You may also be interested in my upcoming book “Shell scripting with Node.js”.

This content originally appeared on 2ality – JavaScript and more and was authored by Dr. Axel Rauschmayer