Yarn: [feat] yarn audit

396

Do you want to request a feature or report a bug?

feature

What is the current behavior?

npm added audit to warn about packages with known security issues. There was some conversation about this previously and one of the core npm folks said the API was likely to be open/public to pull this info. Therefore, yarn should be able to add this feature.

What is the expected behavior?

  • Add a yarn audit command that mimics npm audit
  • Add warnings when adding/installing packages with known issues.

Please mention your node.js, yarn and operating system version.

This would be a minor version bump, so likely target yarn v1.7.0 or v1.8.0 depending on timing.
This is probably too important to wait for v2.0.

rally25rs picture rally25rs  ยท  11 May 2018

Most helpful comment

108

Just a quick update. I actually found some time to get started on this feature.

~/Projects/yarn-test ๐Ÿ’   yarn audit
yarn audit v1.8.0-0
2 vulnerabilities found - Packages audited: 5
Severity: 1 Low | 1 High
โœจ  Done in 0.70s.

There is still a _ton_ of work to do for the more in-depth reporting, but it prints a summary now at least. That's all, just wanted to let people know that this is at least in progress.

rally25rs picture rally25rs  ยท  7 Jun 2018

All comments

0

One thing that yarn could do, is improve on the concept. npm audit's output and configurability is somewhat basic right now.

Perhaps yarn could add some interactivity, allowing the user to automatically update packages, as long as the suggested version is just a semver patch update.

Also definitely useful would be different types of output (html, json, csv, or similar console outputs to test or lint runners etc.)

I'm sure there are more ways yarn can add a bunch of value here.

batjko picture batjko  ยท  13 May 2018
13

Would _love_ yarn to add this. We're currently using snyk.io for vulnerability management but I've never been 100% happy with it as it always felt like something that should be more integrated.

Was super excited when I heard that npm added the audit command. But also super disappointed when I realized that it only works with a corresponding package-lock.json, which would basically mean for us to either drop yarn completely and go back to npm. Or use both tools, which already sounds like a recipe for horrible merge-conflicts and syncing-hell between the two lock-files.

chapati23 picture chapati23  ยท  16 May 2018
0

horrible merge-conflicts

Right now both yarn and npm are able to autoresolve the lockfile conflicts

syncing-hell

Yeah, that's the real problem

Hypnosphi picture Hypnosphi  ยท  16 May 2018
56

yarn is upward compatibility to npm.
I hope yarn should support audit option officially!

shinriyo picture shinriyo  ยท  17 May 2018
0

Unfortunately, any new command addition is a breaking change for yarn, because of yarn scriptName shortcut

Hypnosphi picture Hypnosphi  ยท  17 May 2018
29

Unfortunately, any new command addition is a breaking change for yarn, because of yarn scriptName shortcut

I don't see how.
npm start, for example, has a default behavior which kicks in if no explicit script for it is defined in package.json.
So if someone has yarn audit already defined, it will simply override the default which would be npm's audit. E.g.:

"scripts": {
  "start": "echo 'this will override the default behavior of npm start'",
  "audit": "echo 'my custom audit script will now be used'"
}
batjko picture batjko  ยท  17 May 2018
0

Ok I see

Hypnosphi picture Hypnosphi  ยท  17 May 2018
2

I haven't had time to actually implement anything here yet, so if anyone has free time, help is appreciated. Here are some notes from capturing the network traffic from npm:


After all manifests metadata is loaded and before any .tgz packages are downloaded, npm makes a new request:

POST https://registry.npmjs.org/-/npm/v1/security/audits/quick

headers:

connection: keep-alive
user-agent: npm/6.0.1 node/v8.11.2 win32 x64
npm-in-ci: false
npm-scope: 
npm-session: 4e017f155c654f93
referer: undefined
content-type: application/json
content-type: application/octet-stream
authorization: Bearer {removed}
accept: */*
content-length: 654
accept-encoding: gzip,deflate
Host: registry.npmjs.org

request body:

{
  "name": "demo-npm",
  "version": "1.0.0",
  "requires": {
    "moment": "^2.10.5",
    "minimatch": "^1.0.0"
  },
  "dependencies": {
    "lru-cache": {
      "version": "2.7.3",
      "integrity": "sha1-bUUk6LlV+V1PW1iFHOId1y+06VI="
    },
    "minimatch": {
      "version": "1.0.0",
      "integrity": "sha1-4N0hILSeG3JM6NcUxSCCKpQ4V20=",
      "requires": {
        "lru-cache": "2",
        "sigmund": "~1.0.0"
      }
    },
    "moment": {
      "version": "2.22.1",
      "integrity": "sha512-shJkRTSebXvsVqk56I+lkb2latjBs8I+pc2TzWc545y2iFnSjm7Wg0QMh+ZWcdSLQyGEau5jI8ocnmkyTgr9YQ=="
    },
    "sigmund": {
      "version": "1.0.1",
      "integrity": "sha1-P\/IfGYytIXX587eBhT\/ZTQ0ZtZA="
    }
  },
  "install": [
    "[email protected]"
  ],
  "remove": [

  ],
  "metadata": {
    "npm_version": "6.0.1",
    "node_version": "v8.11.2",
    "platform": "win32"
  }
}

response headers:

content-type: application/json; charset=utf-8
access-control-allow-origin: *
access-control-allow-credentials: true
Cache-Control: max-age=300
Accept-Ranges: bytes
Date: Sun, 20 May 2018 21:25:49 GMT
Via: 1.1 varnish
Connection: keep-alive
X-Served-By: cache-mdw17321-MDW
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1526851550.558067,VS0,VE246
Vary: Accept-Encoding, Accept
Content-Length: 1605

response body:

{
  "actions": [

  ],
  "advisories": {
    "118": {
      "findings": [
        {
          "version": "1.0.0",
          "paths": [
            "minimatch"
          ],
          "dev": false,
          "optional": false,
          "bundled": false
        }
      ],
      "id": 118,
      "created": "2016-05-25T16:37:20.000Z",
      "updated": "2018-03-01T21:58:01.072Z",
      "deleted": null,
      "title": "Regular Expression Denial of Service",
      "found_by": {
        "name": "Nick Starke"
      },
      "reported_by": {
        "name": "Nick Starke"
      },
      "module_name": "minimatch",
      "cves": [
        "CVE-2016-10540"
      ],
      "vulnerable_versions": "<=3.0.1",
      "patched_versions": ">=3.0.2",
      "overview": "Affected versions of `minimatch` are vulnerable to regular expression denial of service attacks when user input is passed into the `pattern` argument of `minimatch(path, pattern)`.\n\n\n## Proof of Concept\n```\nvar minimatch = require(\u201cminimatch\u201d);\n\n\/\/ utility function for generating long strings\nvar genstr = function (len, chr) {\n  var result = \u201c\u201d;\n  for (i=0; i<=len; i++) {\n    result = result + chr;\n  }\n  return result;\n}\n\nvar exploit = \u201c[!\u201d + genstr(1000000, \u201c\\\\\u201d) + \u201cA\u201d;\n\n\/\/ minimatch exploit.\nconsole.log(\u201cstarting minimatch\u201d);\nminimatch(\u201cfoo\u201d, exploit);\nconsole.log(\u201cfinishing minimatch\u201d);\n```",
      "recommendation": "Update to version 3.0.2 or later.",
      "references": "",
      "access": "public",
      "severity": "high",
      "cwe": "CWE-400",
      "metadata": {
        "module_type": "Multi.Library",
        "exploitability": 4,
        "affected_components": "Internal::Code::Function::minimatch({type:'args', key:0, vector:{type:'string'}})"
      }
    }
  },
  "muted": [

  ],
  "metadata": {
    "vulnerabilities": {
      "info": 0,
      "low": 0,
      "moderate": 0,
      "high": 1,
      "critical": 0
    },
    "dependencies": 4,
    "devDependencies": 0,
    "optionalDependencies": 0,
    "totalDependencies": 4
  }
}

The request includes the integrity for each package, but I'm not sure how that is generated or if yarn already figures that out somewhere.


In addition to that, manually running the yarn audit command does:

request headers:

POST https://registry.npmjs.org/-/npm/v1/security/audits

connection: keep-alive
user-agent: npm/6.0.1 node/v8.11.2 win32 x64
npm-in-ci: false
npm-scope: 
npm-session: 686c3e1b85b7d5d2
referer: undefined
content-encoding: gzip
content-type: application/json
content-type: application/octet-stream
authorization: Bearer {removed}
accept: */*
content-length: 401
accept-encoding: gzip,deflate
Host: registry.npmjs.org

request body:

{
  "name": "demo-npm",
  "version": "1.0.0",
  "requires": {
    "minimatch": "^1.0.0",
    "moment": "^2.10.5"
  },
  "dependencies": {
    "lru-cache": {
      "version": "2.7.3",
      "integrity": "sha1-bUUk6LlV+V1PW1iFHOId1y+06VI="
    },
    "minimatch": {
      "version": "1.0.0",
      "integrity": "sha1-4N0hILSeG3JM6NcUxSCCKpQ4V20=",
      "requires": {
        "lru-cache": "2",
        "sigmund": "~1.0.0"
      }
    },
    "moment": {
      "version": "2.22.1",
      "integrity": "sha512-shJkRTSebXvsVqk56I+lkb2latjBs8I+pc2TzWc545y2iFnSjm7Wg0QMh+ZWcdSLQyGEau5jI8ocnmkyTgr9YQ=="
    },
    "sigmund": {
      "version": "1.0.1",
      "integrity": "sha1-P\/IfGYytIXX587eBhT\/ZTQ0ZtZA="
    }
  },
  "install": [

  ],
  "remove": [

  ],
  "metadata": {
    "npm_version": "6.0.1",
    "node_version": "v8.11.2",
    "platform": "win32"
  }
}

response headers:

content-type: application/json; charset=utf-8
access-control-allow-origin: *
access-control-allow-credentials: true
Cache-Control: max-age=300
Accept-Ranges: bytes
Date: Sun, 20 May 2018 21:26:06 GMT
Via: 1.1 varnish
Connection: keep-alive
X-Served-By: cache-mdw17381-MDW
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1526851566.039865,VS0,VE325
Vary: Accept-Encoding, Accept
Content-Length: 1766

response body:

{
  "actions": [
    {
      "action": "install",
      "module": "minimatch",
      "target": "3.0.4",
      "isMajor": true,
      "resolves": [
        {
          "id": 118,
          "path": "minimatch",
          "dev": false,
          "optional": false,
          "bundled": false
        }
      ]
    }
  ],
  "advisories": {
    "118": {
      "findings": [
        {
          "version": "1.0.0",
          "paths": [
            "minimatch"
          ],
          "dev": false,
          "optional": false,
          "bundled": false
        }
      ],
      "id": 118,
      "created": "2016-05-25T16:37:20.000Z",
      "updated": "2018-03-01T21:58:01.072Z",
      "deleted": null,
      "title": "Regular Expression Denial of Service",
      "found_by": {
        "name": "Nick Starke"
      },
      "reported_by": {
        "name": "Nick Starke"
      },
      "module_name": "minimatch",
      "cves": [
        "CVE-2016-10540"
      ],
      "vulnerable_versions": "<=3.0.1",
      "patched_versions": ">=3.0.2",
      "overview": "Affected versions of `minimatch` are vulnerable to regular expression denial of service attacks when user input is passed into the `pattern` argument of `minimatch(path, pattern)`.\n\n\n## Proof of Concept\n```\nvar minimatch = require(\u201cminimatch\u201d);\n\n\/\/ utility function for generating long strings\nvar genstr = function (len, chr) {\n  var result = \u201c\u201d;\n  for (i=0; i<=len; i++) {\n    result = result + chr;\n  }\n  return result;\n}\n\nvar exploit = \u201c[!\u201d + genstr(1000000, \u201c\\\\\u201d) + \u201cA\u201d;\n\n\/\/ minimatch exploit.\nconsole.log(\u201cstarting minimatch\u201d);\nminimatch(\u201cfoo\u201d, exploit);\nconsole.log(\u201cfinishing minimatch\u201d);\n```",
      "recommendation": "Update to version 3.0.2 or later.",
      "references": "",
      "access": "public",
      "severity": "high",
      "cwe": "CWE-400",
      "metadata": {
        "module_type": "Multi.Library",
        "exploitability": 4,
        "affected_components": "Internal::Code::Function::minimatch({type:'args', key:0, vector:{type:'string'}})"
      }
    }
  },
  "muted": [

  ],
  "metadata": {
    "vulnerabilities": {
      "info": 0,
      "low": 0,
      "moderate": 0,
      "high": 1,
      "critical": 0
    },
    "dependencies": 4,
    "devDependencies": 0,
    "optionalDependencies": 0,
    "totalDependencies": 4
  }
}
rally25rs picture rally25rs  ยท  20 May 2018
0

Discovered that the integrity _may_ be returned by the registry for newer packages in the metadata under versions[version_number].dist.integrity

When that does not exist, the npm lockfile docs says "the sha1 in shasum" but I haven't quite figured out how that is generated.

https://github.com/zkat/pacote/blob/ccc6e9094c2e872f09cc12ae966a0cbc1a570eed/lib/finalize-manifest.js#L100 leads me to believe that npm generates the missing integrity from the tarball, but watching network traffic, it has not yet downloaded the tarball ๐Ÿค”

@zkat @imsnif can you provide any details as to how this works? (just asking is probably a lot easier than me hunting through the code ๐Ÿ˜„I appreciate any help!)

rally25rs picture rally25rs  ยท  21 May 2018
0

The manifest json (eg. https://registry.npmjs.org/left-pad) should also have a shasum entry under dist - versions[version_number].dist.shasum
Was that the question? I don't have full context here, but can read back if I misunderstood. :)

imsnif picture imsnif  ยท  21 May 2018
0

@imsnif sorry for the lack of clarity on my part :) If you look at the registry response for minimatch v1.0.0, you get:

            "dist": {
                "shasum": "e0dd2120b49e1b724ce8d714c520822a9438576d",
                "tarball": "https://registry.npmjs.org/minimatch/-/minimatch-1.0.0.tgz"
            },

and the audit request sends:

    "minimatch": {
      "version": "1.0.0",
      "integrity": "sha1-4N0hILSeG3JM6NcUxSCCKpQ4V20=",

so I'm wondering how it takes e0dd2120b49e1b724ce8d714c520822a9438576d and turns it into sha1-4N0hILSeG3JM6NcUxSCCKpQ4V20= (or is there something else that it s generated from?)

rally25rs picture rally25rs  ยท  21 May 2018
0

Ah, I think you're looking for this: https://github.com/zkat/ssri/blob/latest/index.js#L163-L173
Specifically (in regards to minimatch):

Buffer.from('e0dd2120b49e1b724ce8d714c520822a9438576d', 'hex').toString('base64')
// 4N0hILSeG3JM6NcUxSCCKpQ4V20=
imsnif picture imsnif  ยท  21 May 2018
0

Also - reading back now - if we need to talk to an npm api and send it the integrity of packages in the form of an sri, I'd very much recommend waiting for this to be merged: https://github.com/yarnpkg/yarn/pull/5042
In short, it adds an integrity field to yarn.lock, handled by the same ssri module created and used by npm. I think that will solve this part?

imsnif picture imsnif  ยท  21 May 2018
1

Oh cool, yes that's probably something we want. Thanks for pointing it out ๐Ÿ‘

rally25rs picture rally25rs  ยท  21 May 2018
0

https://github.com/zkat/pacote/blob/latest/lib/fetchers/registry/manifest.js#L129

This is where we fill this in for shasum-only packages

zkat picture zkat  ยท  21 May 2018
0

And if a registry doesn't give either a shasum or an integrity field, then we calculate it off the tarball (ditto for other types that don't have this data, like remote tarball deps)

zkat picture zkat  ยท  21 May 2018
0

But also super disappointed when I realized that it only works with a corresponding package-lock.json, which would basically mean for us to either drop yarn completely and go back to npm.

I do think that was quite intentional on the npm team's part.

batjko picture batjko  ยท  22 May 2018
10

@batjko Yeah uh, yarn.lock isn't our format. We'll eventually have support for importing yarn.lock, and that should enable us to then do this, but we don't have that functionality right now. :) Were someone to provide a PR that grants npm the ability to parse yarn.lock and build a tree out of it (much like #5745 does, but on our end), that would speed up that part of the process a lot. It's already on our roadmap, but it's not exactly the highest-priority thing there.

An interesting note is that we require a package-lock.json right now because it was the fastest way to implement this feature effectively. It was a bit more complex and involved to calculate a full tree _first_, and _then_ post it. It's coming eventually, though, and that would allow projects using yarn.lock even before we're able to parse the file, so long as they had an existing node_modules we could read concrete trees from.

But tbh, in the end, you want this in Yarn itself. It's not gonna be as nice to be switching back and forth to npm just for the one feature and it won't have as good integration. [email protected] can already auto-fix vulnerabilities, to boot, and that's not something we'd do for Yarn projects at all (because lockfile).

zkat picture zkat  ยท  22 May 2018
41

(and it should tell you something that I'm showing up in this issue tracker and helping out, if you're concerned about Competition Issues)

zkat picture zkat  ยท  22 May 2018
108

Just a quick update. I actually found some time to get started on this feature.

~/Projects/yarn-test ๐Ÿ’   yarn audit
yarn audit v1.8.0-0
2 vulnerabilities found - Packages audited: 5
Severity: 1 Low | 1 High
โœจ  Done in 0.70s.

There is still a _ton_ of work to do for the more in-depth reporting, but it prints a summary now at least. That's all, just wanted to let people know that this is at least in progress.

rally25rs picture rally25rs  ยท  7 Jun 2018
0

@rally25rs excellent progress, I can not wait to test this new feature.

rodrigooler picture rodrigooler  ยท  7 Jun 2018
11

@zkat To be clear, are you saying that even if Yarn adds an audit command, it would not add an audit fix command? The latter is super useful. I can't speak for anyone but me, but I'd switch back to NPM for improved security if Yarn doesn't plan to add an audit fix command.

openjck picture openjck  ยท  13 Jun 2018
2

@openjck she's saying that doing audit fix with the npm CLI is a bad idea because of lockfile incompatibilities; not that yarn can't implement that feature itself.

bencooper222 picture bencooper222  ยท  23 Jun 2018
0

Note that npm have a separate module for handling the security API response, npm-audit-report, if this is something that can be reused.

robations picture robations  ยท  13 Jul 2018
3

Just another idea, maybe not fully inline with the vision of yarn...

As far as I know, the source of all the vulnerabilities is https://github.com/nodejs/security-wg/ , e.g. for minimatch https://github.com/nodejs/security-wg/blob/master/vuln/npm/118.json

What about using the security-wg repo as source instead and do yarn audit really fast by using local cache?

bufferoverflow picture bufferoverflow  ยท  31 Jul 2018
6

hey @bufferoverflow, @rally25rs, great progress!
Liran here from the Node.js Security WG team; I'll be happy to further explore how we can work together and what can we do to improve your experience and usability of the vuln db.

@zkat are you indeed relying on the securtiy-wg/ repo for vulnerabilities audited with npm audit?

lirantal picture lirantal  ยท  1 Aug 2018
3

@rally25rs Thanks for working on this! Any update on when it will be available? Guessing that it will be before Sept 30, when NSP shuts down? Trying to determine if we need to move back to npm

katiepeters picture katiepeters  ยท  1 Aug 2018
10

An option to exit with a failure based on severity level would be very nice for CI and test runners:

yarn audit --fail-at <severity> #low/moderate/high/critical

npm audit lacks this, so I previously had to write a script that parses npm audit --json output.

ostrgard picture ostrgard  ยท  3 Aug 2018
4
zkat picture zkat  ยท  3 Aug 2018
13

Hi guys, is it possible to provide a guide for when to expect this feature?

My team is getting more and more concerned about the approaching deadline for NSP removal, and we either have to move back to NPM or wait for the yarn audit feature. Cheers

sam-warren-finnair picture sam-warren-finnair  ยท  14 Aug 2018
41

Sorry for the delay on this. I was waiting for #5042 to be merged because I need to send the integrity field to the audit API. That PR got merged, but then I was on vacation the last couple weeks. I'll get back to work on this...

I have it to the point where the audit report prints on yarn install and yarn audit but there is a lot of work to do with setting the exit code, figuring out a way to unit test this, adding it to the JSON reporter...

If anyone wants to beta test the audit command, I'll probably have a working branch for hte basic functionality by next week... stay tuned...

rally25rs picture rally25rs  ยท  21 Aug 2018
0

@rally25rs you're a boss <3

csvan picture csvan  ยท  21 Aug 2018
10

If anyone wants to start _alpha_ testing this, check out my branch: https://github.com/rally25rs/yarn/tree/audit

This will print an audit summary after yarn install and yarn add, and also adds the yarn audit command.

This is very rough and I'm sure will fail for some cases.

If you run into problems, please run with --verbose and find the log output for "Audit Request:" and provide the JSON that is logged, along with a package.json to reproduce the issue.

Known issues and work to still be done:

  • No new tests added yet. I haven't quite figured out how to test most of this stuff.
  • Existing unit tests fail massively. This is because most tests install packages, which now tries to hit npm registry to audit those packages. This will need to somehow be mocked out for all tests.
  • yarn list command should be refactored to use new hoisted-tree-builder.
  • I think optionalDependencies might be an issue, including the --ignore-optional flag. Some testing needs to be done here (do we audit a package that was resolved but wasn't installed?)
  • JSON reporter will not print anything yet (so you won't get audit output if you use the --json flag).
rally25rs picture rally25rs  ยท  27 Aug 2018
0

@rally25rs Thanks for the work on this. I tried yarn audit on the Yarn repo, and on my day job: mozilla/normandy. It worked fine on Yarn, but failed with this error on Normandy:

verbose 15.325 Performing "POST" request to "https://registry.yarnpkg.com/-/npm/v1/security/audits".
verbose 15.461 Request "https://registry.yarnpkg.com/-/npm/v1/security/audits" finished with status code 400.
verbose 15.461 Audit Response: false

Full output with --verbose and the package.json file used here: https://gist.github.com/mythmon/63a6141de0016e148be424fad73506c0

mythmon picture mythmon  ยท  27 Aug 2018
0

@mythmon Thanks for reporting this! I also need to figure out how to get the actual response from npm back to what I log... unfortunately the existing request-manager has a conditional that returns false as the response if the API returns a 400.

Interestingly, the error reported by the npm audit API is:

child "version" fails because ["version" is not allowed to be empty],

which seems to be because your package.json doesn't specify a version for the `"normandy" project. If you add one, it fixes the issue with the audit command.

I'll investigate what normal npm does in this case and mimic that...

Edit:

I pushed a fix for this issue. version is no longer passed if it isn't in package.json.

rally25rs picture rally25rs  ยท  27 Aug 2018
0

@rally25rs I think in the case of optionalDependencies I would still want yarn audit to check those packages.

But it's great to see this progress being made ๐ŸŽ‰

Fleker picture Fleker  ยท  28 Aug 2018
1

Adding OWASP as an output format (not just plain old XML) would be amazing and make it possible to include the yarn audit report in SonarQube and other tools. Here is a NPM RFC for the same feature which offers a mapping in pseudo code from npm audit to OWASP.
How do you think about it @rally25rs ?

crunchtime-ali picture crunchtime-ali  ยท  12 Sep 2018
0

@dj-hedgehog Can you please provide an example of the OWASP output format?

Phylu picture Phylu  ยท  12 Sep 2018
1

@Phylu No problem. NSP (which became npm audit supported custom reporters. There is a owasp reporter. Sadly NSP is shutting down on the 30th of September so I won't be able to use it much longer.

Here is a quick example:
I just executed this in a Node.js project:

npm i -g nsp nsp-reporter-owasp
nsp check --reporter owasp

The output is:

<?xml version="1.0" encoding="utf-8"?>
<analysis xmlns="https://jeremylong.github.io/DependencyCheck/dependency-check.1.6.xsd">
  <scanInfo>
    <engineVersion>3.0.1</engineVersion>
  </scanInfo>
  <projectInfo>
    <name>[email protected]</name>
    <reportDate>2018-9-12 22:31:38</reportDate>
    <credits>nsp</credits>
  </projectInfo>
  <dependencies>
    <dependency isVirtual="false">
      <fileName>[email protected]</fileName>
      <filePath>[email protected]</filePath>
      <md5/>
      <sha1/>
      <description>Prototype Pollution</description>
      <identifiers>
        <identifier type="nsp">
          <name>lodash</name>
        </identifier>
      </identifiers>
      <vulnerabilities>
        <vulnerability source="NSP">
          <name>https://nodesecurity.io/advisories/577</name>
          <cvssScore>2</cvssScore>
          <cvssAccessVector/>
          <cvssAccessComplexity>LOW</cvssAccessComplexity>
          <cvssAuthenticationr>NONE</cvssAuthenticationr>
          <cvssConfidentialImpact>NONE</cvssConfidentialImpact>
          <cvssIntegrityImpact>NONE</cvssIntegrityImpact>
          <cvssAvailabilityImpact>PARTIAL</cvssAvailabilityImpact>
          <severity>Low</severity>
          <description>Versions of `lodash` before 4.17.5 are vulnerable to prototype pollution.

The vulnerable functions are &amp;#39;defaultsDeep&amp;#39;, &amp;#39;merge&amp;#39;, and &amp;#39;mergeWith&amp;#39; which allow a malicious user to modify the prototype of `Object` via `__proto__` causing the addition or modification of an existing property that will exist on all objects.

||Update to version 4.17.5 or later. vulnerable versions: &amp;lt;4.17.5 patched_versions: &amp;gt;=4.17.5</description>
        </vulnerability>
      </vulnerabilities>
    </dependency>
  </dependencies>
</analysis>
crunchtime-ali picture crunchtime-ali  ยท  12 Sep 2018
11

Currently I use this hack to run npm audit using yarn. In your package.json:

"scripts": {
   "audit-deps": "synp --source-file yarn.lock; npm audit && rm -f package-lock.json"
},
"devDependencies": {
   "synp": "^1.3.1"
}

Then

yarn run audit-deps

I don't know how good it is but it works fine for me ๐Ÿ˜„

Waiting for yarn audit ๐ŸŽ‰

P.S. synp converts yarn.lock to package-lock.json and vise versa

probil picture probil  ยท  20 Sep 2018
7

npm audit && rm -f package-lock.json should probably be npm audit; rm -f package-lock.json so that package-lock is removed regardless of audit success or failure

Hypnosphi picture Hypnosphi  ยท  20 Sep 2018
1

Thanks @Hypnosphi
I've updated the line :)

Updated: I've decided to change your approach because CI marks failed job as succesful

probil picture probil  ยท  21 Sep 2018
0

the approach I've described above works fine but fails in CI
I don't know why

npm ERR! code E400
npm ERR! 400 Bad Request - POST https://registry.npmjs.org/-/npm/v1/security/audits
probil picture probil  ยท  21 Sep 2018
0

Ok, probably it should be this, to ensure that package-lock correctly represents current yarn.lock:

rm -f package-lock.json && synp --source-file yarn.lock && npm audit
Hypnosphi picture Hypnosphi  ยท  21 Sep 2018
1

i've been using @rally25rs' fork and it works great. just missing the --exceptions option - using a manual solution for CI to allow programmatic vulnerability exceptions at the moment. here's the relevant code from our Dockerfile building on linux (note it requires another version of yarn to be installed in a prior step):

RUN git clone --single-branch -b audit https://github.com/rally25rs/yarn.git /yarn && \
  cd /yarn && \
  yarn install && \
  yarn build && \
  chmod +x /yarn/lib/cli/index.js && \
  echo "\
#\!/bin/bash \n\
PREFIX='/usr/local' NPM_CONFIG_PYTHON='/usr/bin/python' \
exec node '/yarn/lib/cli/index.js' \"\[email protected]\" " > /usr/bin/yarn && \
  cd -

definitely will help bridge the gap between NSP and the main yarn build officially supporting audit. thanks for your hard work @rally25rs!

jakeonfire picture jakeonfire  ยท  23 Sep 2018
1

Thank you for the great work on this @rally25rs :)

The NSP service is shutting down on 28th September according to this article: https://blog.npmjs.org/post/175511531085/the-node-security-platform-service-is-shutting

Would you recommend we move back to npm for now until yarn has been updated?

Appreciate all your effort on this!

sam-warren-finnair picture sam-warren-finnair  ยท  24 Sep 2018
15

I feel like it's time for another status update on this...

My PR branch has master merged into it and a couple conflicts resolved, so the new Plug-n-Play stuff in in there now.

I also started gzipping the request sent to npm to reduce the payload size. npm client does this too, so it seemed like a good idea.

I'm just handling a couple PR comments/suggestions and trying to compare yarn results vs npm.

Anyway, still in progress! Sorry I haven't had more freetime to get this done by now...

rally25rs picture rally25rs  ยท  27 Sep 2018
0

When the NSP service is shut down, is a CVE checking service available for yarn to use for its audit?

mileslane picture mileslane  ยท  28 Sep 2018
0

@mileslane the API/backend that npm audit uses is public and available to other package managers like yarn.

rarkins picture rarkins  ยท  28 Sep 2018
0

Awesome, so there is progress. Do we have any schedule for a specific release which will ship this?

DanielRuf picture DanielRuf  ยท  30 Sep 2018
0

@rally25rs it currently says "1 vulnerabilities found" but should be probably "1 vulnerability found" using some ternary check vulns_num === 1 ? 'vulnerability' : 'vulnerabilities'

DanielRuf picture DanielRuf  ยท  30 Sep 2018
0

@mileslane currently just using the npm registry's API. There is not a "pluggable" CVE checking service. Might be an interesting addition in the future.

@DanielRuf no specific version.


I noticed that one of the differences I was seeing was that npm can report something like:

# Run  npm update url-parse --depth 4  to resolve 1 vulnerability
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ High          โ”‚ Open Redirect                                                โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ Package       โ”‚ url-parse                                                    โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ Dependency of โ”‚ 5869564e2d5b507b081c73d06fa7969c764f414f57458eb75b02e90f292โ€ฆ โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ Path          โ”‚ 5869564e2d5b507b081c73d06fa7969c764f414f57458eb75b02e90f292โ€ฆ โ”‚
โ”‚               โ”‚ > webpack-dev-server > sockjs-client > url-parse             โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ More info     โ”‚ https://nodesecurity.io/advisories/678                       โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

so apparently npm can upgrade a dependency at a depth which isn't a feature that yarn has. This makes the suggested resolution returned from npm unusable by yarn.

Not sure how to solve this, and building a new resolution suggestion system would be quite the project...

For now I'll probably just remove that line of output. Better to get this feature out there in a usable, albeit manual, form then add features iteratively.

rally25rs picture rally25rs  ยท  30 Sep 2018
0

@rally25rs you can suggest removing this particular entry from yarn.lock and re-generating it

Hypnosphi picture Hypnosphi  ยท  30 Sep 2018
0

From yarn.lock? Which line do yo mean?

DanielRuf picture DanielRuf  ยท  1 Oct 2018
0

By "entry", I mean this (in my case it's already resolved to a non-vulnerable version):

[email protected]^1.1.8, [email protected]^1.4.3:
  version "1.4.3"
  resolved "https://registry.yarnpkg.com/url-parse/-/url-parse-1.4.3.tgz#bfaee455c889023219d757e045fa6a684ec36c15"
  integrity sha512-rh+KuAW36YKo0vClhQzLLveoj8FwPJNu65xLb7Mrt+eZht0IPT0IXgSv8gcMegZ6NvjJUALf6Mf25POlMwD1Fw==
  dependencies:
    querystringify "^2.0.0"
    requires-port "^1.0.0"
Hypnosphi picture Hypnosphi  ยท  1 Oct 2018
1

Well, in the general it makes no sense to edit the lockfile (see the first line), as it is generated.

Otherwise yarn upgrade and so on are the right way to go.
Your approach might have unwanted side effects.

DanielRuf picture DanielRuf  ยท  1 Oct 2018
16

@aracarie hooray! When it will be released?

ai picture ai  ยท  2 Oct 2018
0

@DanielRuf can yarn upgrade the nested dependencies?

Hypnosphi picture Hypnosphi  ยท  2 Oct 2018
60

For anyone who might stumble upon this issue, yarn audit has been released in version v1.12.3.

jure picture jure  ยท  27 Nov 2018
20

I'm happy to see this new command released, but I still miss a vulnerability report for installed package(s) on running yarn add as npm install does. That behavior catches vulnerabilities that you might miss if you don't manually run the audit command.

alejandroiglesias picture alejandroiglesias  ยท  27 Nov 2018
0

Great! Happy to see the yarn version being available in the 8.14.0 Docker image, too. Are there plans already to include an ignore file like .nsprc exceptions used to be for NSP?

jverhoelen picture jverhoelen  ยท  11 Dec 2018
0

@jverhoelen nsp was bought by npm and shut down. Also the Node.js version is not related to yarn audit.

DanielRuf picture DanielRuf  ยท  11 Dec 2018
1

You might want to try Snyk.

DanielRuf picture DanielRuf  ยท  11 Dec 2018
1

@DanielRuf Sorry, I didn't express myself well. I was referring to Yarn 1.12.3 being available in the Docker image of Node 8.14.0 โ€“ off topic ;-)
I know that NSP was bought and just today I saw some red pipelines due to the final shutdown.
But I could imagine that people maybe want to maintain an exception/ignore file for vulnerabilities that yarn audit discovers because of several reasons.

jverhoelen picture jverhoelen  ยท  11 Dec 2018
0
timja picture timja  ยท  11 Dec 2018
0

@alejandroiglesias you can use the --audit flag on both yarn add and yarn install to run an audit with those commands. I personally think it is better for those to run by default as it makes it easier to know there is a vulnerability without needing to remember to either add the flag or run yarn audit hopefully the default will change in time.

fedorareis picture fedorareis  ยท  11 Jan 2019
6

You can make it a default by adding the following into your yarnrc:

--install.audit true
--add.audit true
arcanis picture arcanis  ยท  11 Jan 2019
0

@arcanis the .yarnrc in my home directory says that I shouldn't edit that file directly? Is there where I should add the flags you suggested?

alejandroiglesias picture alejandroiglesias  ยท  18 Jan 2019
0

Because the yarnrc files and the lockfiles share the same format, all yarn-generated configuration file have this header, but it's perfectly safe to edit the rc files (and even the lockfile, if you know what you're doing). It'll be fixed in the next major.

arcanis picture arcanis  ยท  18 Jan 2019
0

@arcanis thanks!

alejandroiglesias picture alejandroiglesias  ยท  18 Jan 2019
0

@arcanis one more question: is that alternatively achievable by using yarn config? I never used that command, but was wondering about that being easier than locating the .yarnrc file in my Meteor Node env (+ re-adding the entries after each Meteor upgrade since it resets the env and I even have to re-install Yarn).

alejandroiglesias picture alejandroiglesias  ยท  19 Jan 2019
2

@alejandroiglesias it is achievable with yarn config that's how I did it. The commands would be

yarn config set add.audit true
yarn config set install.audit true

for add and install respectively

fedorareis picture fedorareis  ยท  19 Jan 2019
0

@arcanis I added both CLI arguments in my project's yarnrc file:

--install.audit true
--add.audit true

It now runs yarn audit automatically during yarn install but I am having trouble getting yarn audit to run for yarn add.

My yarn version is 1.13.0

KinTsang picture KinTsang  ยท  21 Jan 2019
0

Am I right in saying that there's no plan to add the fix option for yarn audit?

Berkmann18 picture Berkmann18  ยท  30 Jan 2019
35

Is there any chance we're able to use yarn audit fix just like we would with npm audit fix? Or is this not supported and not planned to support with yarn?

mesqueeb picture mesqueeb  ยท  22 Feb 2019
0

Yarn v1.3.2 not allowing yarn audit
$ yarn audit
yarn run v1.3.2
error Command "audit" not found.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

If you guys are literally just mocking NPM why not mock the damned parts that work

shopglobal picture shopglobal  ยท  14 Mar 2019
0

YARN is a shit FORK

shopglobal picture shopglobal  ยท  14 Mar 2019
2

@shopglobal the latest stable version is v1.13.0 -- 1.3.2 is rather old at this point

MatrixFrog picture MatrixFrog  ยท  14 Mar 2019
1
DanielRuf picture DanielRuf  ยท  14 Mar 2019
-6

Current solution for fixiing audit results is like so:

  1. delete node_modules directory.
  2. delete yarn.lock file
  3. use npm i
  4. run npm audit fix --force
  5. delete node_modules directory
  6. run yarn
  7. run again yarn audit to see results.
  8. delete package-lock.json file

why it works? Yarn respects package-lock.json file when installing packages

FDiskas picture FDiskas  ยท  6 Jan 2020
0

why it works? Yarn respects package-lock.json file when installing packages

Not anymore afaik. Also this is not really a solution. I think you have a different issue.

DanielRuf picture DanielRuf  ยท  6 Jan 2020
2

Current solution for fixiing audit results is like so:

  1. delete node_modules directory.
  2. delete yarn.lock file
  3. use npm i
  4. run npm audit fix --force
  5. delete node_modules directory
  6. run yarn
  7. run again yarn audit to see results.
  8. delete package-lock.json file

why it works? Yarn respects package-lock.json file when installing packages

That's really a smart and fast and convinient solution.

fengerzh picture fengerzh  ยท  7 Jan 2020
4

why it works? Yarn respects package-lock.json file when installing packages

Not anymore afaik.

That's never been the case - yarn.lock predates package-lock.json so we've never had to do this (even the old shrinkwrap was never used afaik).

My guess is that by removing the lockfile OP simply forced Yarn to upgrade all their packages, hereby "fixing" the versions. You could do the same thing without calling npm and have the same result.

arcanis picture arcanis  ยท  7 Jan 2020
3

the idea is using resolutions: {} into package.json works! I've wrote post how it workd in my case:
How to fix security vulnerabilities in NPM/Yarn dependencies

greybax picture greybax  ยท  14 May 2020
0

Or try snyk or use dependabot which can help here too (if you use a lockfile) =)

DanielRuf picture DanielRuf  ยท  14 May 2020
0

Or sometime you can just remove node_modules and yarn.lock then reinstall to fix vulnerabilities.

I use this method to fix vulnerabilities on my project many times. ๐Ÿ˜

gluons picture gluons  ยท  15 May 2020
0

@gluons in this case all packages will be updated and this is probably not what you want. Audit should update only effected packages or sub packages.

FDiskas picture FDiskas  ยท  15 May 2020
0

Or sometime you can just remove node_modules and yarn.lock

Why not use yarn upgrade-interactive? Also see what dependabot on GitHub and snyk CLI do.

DanielRuf picture DanielRuf  ยท  15 May 2020
0

Why not use yarn upgrade-interactive?

From my experience, some deep dependencies still haven't been updated. (Some vulnerabilities are from deep nested dependencies.)

If I remove yarn.lock, every dependencies are definitely updated.
So some vulnerabilities are fixed by the latest dependencies.

gluons picture gluons  ยท  15 May 2020