"Incorrect integrity when fetching from the cache"

[xclidongbo]

yarn version: 1.19.0
Do you want to request a feature or report a bug?

bug:
error Incorrect integrity when fetching from the cache

What is the current behavior?
When I install thie lib, the error is show:
for example: yarn add react-native-zip-archive

yarn add v1.19.0
info No lockfile found.
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
[1/4] 🔍  Resolving packages...
warning react-native > create-react-class > fbjs > core-js@1.2.7: core-js@<2.6.8 is no longer maintained. Please, upgrade to core-js@3 or at least to actual version of core-js@2.
info There appears to be trouble with your network connection. Retrying...
warning jest > jest-cli > jest-config > jest-environment-jsdom > jsdom > left-pad@1.3.0: use String.prototype.padStart()
[2/4] 🚚  Fetching packages...
error Incorrect integrity when fetching from the cache
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.
error Incorrect integrity when fetching from the cache

If the current behavior is a bug, please provide the steps to reproduce.

What is the expected behavior?
success install.
Please mention your node.js, yarn and operating system version.
node: v12.10.0
yarn: 1.19.0
macos: 10.14.6

[chuan6]

Strengthening of integrity check introduced in #7582 seems to surface the issue. As a temporary fix, I commented out the code doing integrity check.

My guess is that for some packages, while local yarn.lock file may contain integrity, the record on remote registry may not. In my case, it's underscore@1.7.0.

Hope that @arcanis can help share some advice on resolving the issue @~@

[chuan6]

@xclidongbo Please re-title the issue to reflect the error being thrown, so that people encountering it can all arrive here. Thanks :)

[arcanis]

Some context: old lockfile entries (more than a year ago?) don't have the integrity field present. Back when it was added, we decided not to automatically migrate them, as it was generating a lot of changes on unsuspecting projects.

So what happens is that when installing from such a no-integrity entry, the integrity doesn't get stored in the cache, and then when later installing a project with a modern lockfile with an integrity field, we report that the cache got stored without integrity and refuse to use it.

The solution is to first enable the migration by adding the following in ~/.yarnrc:

unsafe-disable-integrity-migration false

Then cleaning the cache and reinstalling:

yarn cache clean
yarn install -f

I think I'll just make a patch release that will force-migrate the lockfiles... It was more than a year ago, hopefully the changes will be less disruptive now 🤔 Or I can wait for the next major which is scheduled to be released in a few months.

[Blasz]

The validation fix seems to be a breaking change since cache now cannot be shared across entries with and without integrity fields.

Any dependencies installed from a yarn.lock with unsafe-disable-integrity-migration true that aren't previously cached will now cause this error whenever another repo without that setting tries to install one of those dependencies...which is going to be pretty common given that there's a very high probability of different projects sharing at least one transitive dependency.

Force migrating the lockfiles also seems like a breaking change, since versions before/after the change won't be compatible anymore.

From my limited perspective, I would:

1. Update the error message, it doesn't provide any information making it incredibly hard to figure out what to do. I had to debug yarn code directly to figure out what the problem was.
2. Support both kinds of lockfile entries by either:
   a) Always adding the integrity field to the cache, regardless of whether lockfile entries have the integrity field or not, so that both kinds of yarn.lock entries can utilise the cache
   b) Storing separate cached versions for entries with and without integrity fields

If neither options for 2) work then I guess force migrating lockfiles is better than yarn not working at all.

[arcanis]

Any dependencies installed from a yarn.lock with unsafe-disable-integrity-migration true that aren't previously cached will now cause this error whenever another repo without that setting tries to install one of those dependencies...which is going to be pretty common given that there's a very high probability of different projects sharing at least one transitive dependency.

Note quite - only old lockfile entries. For example, if you create a new project now, you'll see the integrity field is properly populated. Similarly, if you take an old lockfile and add a new dependency the integrity field will also be there (but the existing entries won't have received a new one).

Update the error message, it doesn't provide any information making it incredibly hard to figure out what to do. I had to debug yarn code directly to figure out what the problem was.

Sounds reasonable.

b) Storing separate cached versions for entries with and without integrity fields

Hm 🤔 That sounds possible. I investigated using the integrity as cache entry before, and it proved quite challenging due to the timing of the integrity check. But if we were to just add a marker to disambiguate whether there's an integrity field or not ... yeah, it might work.

[vilicvane]

@arcanis We have experienced this issue with up-to-date yarn.lock files multiple times. And after cleaning the cache, the issue surface again under certain circumstances. Would corrupted cache due to unexpected termination of yarn or disconnection of the network cause this problem?

[arcanis]

Can someone use our latest nightly and tell me whether you still get this error?

[saimonmoore]

We are experiencing issues related to failing integrity checksums with yarn 1.19.0. Looks very much like this. I tested the nightly and it didn't seem to help much.

[vilicvane]

@arcanis Seems to work for my case.

[Blasz]

@arcanis I'm no longer experiencing the error after using latest nightly, thanks for the quick fix!

[Blasz]

Actually, I spoke too soon. I just experienced the same error but caused for a different reason:

The integrity between the cached and non-cached versions don't match.

From what I can see, the cached remote integrity is sha-1 whereas the integrity it is comparing to is sha-512, which I assume is causing a mismatch.