Yarn: `npm prune` equivalent behavior

56

Do you want to request a _feature_ or report a _bug_?

Feature

What is the current behavior?

Currently yarn doesn't seem to be removing any packages that aren't needed. With npm you can run npm prune, but yarn doesn't have that and doesn't do the same operation in other cases.

What is the expected behavior?

yarn init
npm install react
yarn clean
yarn install # I expect one of these two commands to remove node_modules/react

Please mention your node.js, yarn and operating system version.

Node v5.7.0
Yarn v0.15.1
macOS 10.12

bouk picture bouk  ·  11 Oct 2016

Most helpful comment

51

On our CI platform, we currently do this:

npm install
tests
npm prune --production
deploy

i.e. after testing, but before deploying, we remove devDependencies. They get added back again at the start of the next build/test. Is there an equivalent way with yarn that doesn't bypass the cache or require npm?

rarkins picture rarkins  ·  27 Oct 2016

All comments

9

Thanks for the report! yarn install --force should do what you're looking for. When generating integrity hashes (what we use to check validity of the current install) we should also do an fs.readdir of all module folders.

sebmck picture sebmck  ·  11 Oct 2016
0

yarn remove <package-name> removes the package... @kittens can we also add instruction for yarn prune for the user as we are adding the instruction for the yarn -i?

akshat-ilen picture akshat-ilen  ·  11 Oct 2016
-6

@kittens running yarn prune says:

error The prune command isn't necessary. `yarn install` will prune extraneous packages.
torifat picture torifat  ·  11 Oct 2016
0

@torifat Yea that looks to be working, even when you cut out devDependencies by using --production

jljorgenson18 picture jljorgenson18  ·  12 Oct 2016
1

When I run yarn, then npm prune --production, then have to run yarn again, It doesn't reinstall the dev dependencies. Force seems like overkill. How else would you get around this?

thomaswmanion picture thomaswmanion  ·  14 Oct 2016
0

My expectation that yarn will not require additional and explicit prune or dedupe. And my node_modules will be always clean and small. Currently, removing dependency from package.json manually (e.g. @types/semver) and then yarn, yarn doesn't remove files. npm prune is still required :(

develar picture develar  ·  18 Oct 2016
51

On our CI platform, we currently do this:

npm install
tests
npm prune --production
deploy

i.e. after testing, but before deploying, we remove devDependencies. They get added back again at the start of the next build/test. Is there an equivalent way with yarn that doesn't bypass the cache or require npm?

rarkins picture rarkins  ·  27 Oct 2016
0

I agree with @rarkins. Do y'all have an example of how you produce an app suitable for deployment that removes all of the devDependencies? But still allows for the lock file to make sure that all the deps are solid? This is the biggest issue that we face with npm's shrinkwrap. We want production deps to be locked but devDependencies to be free floating so that they are updated more easily as they tend to change more rapidly.

davglass picture davglass  ·  31 Oct 2016
48

yarn install --production is close, but it does a few things prune doesn't. It's a little annoying to have to build native modules twice in a CI pipeline just to prune. You can prevent rebuilding packages that already exist in cache with the --ignore-scripts flag. Additionally, prune doesn't check npm registry, so add --prefer-offline.

So, yarn install --production --ignore-scripts --prefer-offline == npm prune --production

erulabs picture erulabs  ·  4 Nov 2016
4

@erulabs what order would you run those things in?

$ yarn install
$ <run tests>
$ <remove node_modules?>
$ yarn install --production --ignore-scripts --prefer-offline
$ deploy?

I'm having trouble getting the equivalent to npm prune --production. The use-case is ripping out any build time node_modules so they don't get deployed out with the running service that only needs the runtime dependencies.

chetanddesai picture chetanddesai  ·  9 Dec 2016
8

@chetanddesai You don't need to wipe out the node_modules directory (yarn more or less does that already) - just:

$ yarn
$ <build phase and/or test phase>
$ yarn install --production --ignore-scripts --prefer-offline

Remember that if you're trying to do this stuff in a container - yarn keeps its cache outside the node_modules dir, so you'll need to yarn cache clean if you want to really cleanup everything in the environment.

erulabs picture erulabs  ·  4 Jan 2017
7
yarn install --production

It is not very intuitive. I do not want to (re)install production packages, I want to remove dev packages. Is it hard to implement an alias yarn prune --production?

Diokuz picture Diokuz  ·  9 Jan 2017
2

Does Yarn not remove files when using --force. For the prune suggestions above, I went ahead and installed some extraneous dependencies. Then ran yarn install --force, everything was fine in both package.json and my yarn.lock, but unfortunately the files in node_modules were still present. This could potentially be an issue if you're using a mono-repo and your extraneous dependency is an incorrect parent repository dependency.

abhiaiyer91 picture abhiaiyer91  ·  10 Jan 2017
0

As @abhiaiyer91 says yarn install --production --ignore-scripts --prefer-offline still leaves the node_module packages. I'm just using npm prune --production --loglevel=warn

sdbondi picture sdbondi  ·  11 Jan 2017
5

Running this script gets me close to npm prune --production for docker builds.

const exec = require('child_process').exec;
const devDependencies = Object.keys(require('./package.json').devDependencies).join(' ');
const command = 'yarn remove ' + devDependencies;

const child = exec(command, (err, stdout, stderr) => {
  if (err) throw err;
  console.log(`stdout: \n${stdout}`);
  console.log(`stderr: \n${stderr}`);
});
pxwise picture pxwise  ·  20 Jan 2017
37

For me yarn install --production --ignore-scripts --prefer-offline doesn't clean up all the dependencies. Is there a particular reason that you don't want to add a prune command, or do you just need someone to step up and add it?

tech4him1 picture tech4him1  ·  24 Mar 2017
0

@tech4him1 I found the same, try adding --force and it does clean them up on 0.22.0.

rockymadden picture rockymadden  ·  11 Apr 2017
0

I can't reproduce this bug. If you still experience it on the latest Yarn release, please update the script below to make it a viable reproduction, and reopen the issue (or ping me if you're not the OP).

#!/usr/bin/env bash

YARN=${YARN:-yarn}

TEMP="$(mktemp -d)"
trap 'rm -rf "$TEMP"' EXIT

echo $TEMP
cd "$TEMP"

printf '{ "name": "main", "version": "1.0.0", "license": "MIT", "dependencies": {} }' > package.json

$YARN add --dev babel-core
$YARN add left-pad

$YARN install
$YARN install --production

ls -l node_modules node_modules/*
arcanis picture arcanis  ·  28 Apr 2017
6

Another issue here: some packages are private, so docker cannot install them. I want to copy all the stuff to docker context, then just _remove_ devDeps. No reinstall – it fails.

Diokuz picture Diokuz  ·  13 Jun 2017
0

@Diokuz Docker can install private packages if you provide NPM_TOKEN environment variable in Dockerfile

vladgolubev picture vladgolubev  ·  16 Sep 2017
0

@vladgolubev in my case it was not possible because our private npm server (not an npm.org) was physically unreachable from docker.

Diokuz picture Diokuz  ·  22 Oct 2017
0

No news here ?

HelloEdit picture HelloEdit  ·  16 Oct 2018
3

Running this script gets me close to npm prune --production for docker builds.

const exec = require('child_process').exec;
const devDependencies = Object.keys(require('./package.json').devDependencies).join(' ');
const command = 'yarn remove ' + devDependencies;

const child = exec(command, (err, stdout, stderr) => {
  if (err) throw err;
  console.log(`stdout: \n${stdout}`);
  console.log(`stderr: \n${stderr}`);
});

Of course this becomes more complicated when you use yarn workspaces.
One needs to get a list of package.json files e.g. via some sort of glob, then run the "delete devDependencies" trick over them.

Here's a helper for that. It does have a dependency on globby and needs a recent node because it uses async/await.
But chances are you already have it in your project, as many popular packages depend on globby (e.g create-react-app, lerna, eslint, webpack-dev-server - just try yarn why globby).
And since we're talking Docker cleanup, a recent Node.js version can be assumed.

https://gist.github.com/loopmode/318e881454dc0498874a4e764d3dce55

loopmode picture loopmode  ·  29 Jan 2019
7

My jq usage solution is,

$ yarn remove $(cat package.json | jq -r '.devDependencies | keys | join(" ")')
b6pzeusbc54tvhw5jgpyw8pwz2x6gs picture b6pzeusbc54tvhw5jgpyw8pwz2x6gs  ·  9 Feb 2019
4

For anyone still desiring this feature, all of the current workarounds require the yarn command to work online. In my case I have private packages from GitHub, and don't want the access credentials to be accessible in the final image.

With npm prune this can be done, as I can use a multi-stage Docker build. The first image has access to the credentials and installs dependencies and devDependencies (for compilation). Then in the second image which doesn't have access to the credentials, I can copy the node_modules folder from the first image and prune it.

None of the current solutions work with the --offline flag (which could be a separate bug, as the dependencies are up to date according to yarn install), so I can't use yarn for this.

lucaspiller picture lucaspiller  ·  14 Oct 2019
0

My solution, using jq for Docker is

COPY package.json yarn.lock ./
RUN echo $(cat package.json | jq 'del(.devDependencies)') > package.json
RUN yarn --frozen-lockfile
patarapolw picture patarapolw  ·  3 Jul 2020
0

@patarapolw isn't that equivalent to yarn install --production --frozen-lockfile? What's the benefit?

loopmode picture loopmode  ·  3 Jul 2020