Do not suggest deprecated versions by default

[mohd-akram]

I noticed that in version 17 the default was changed to include deprecated versions, but it doesn't seem to be mentioned in the changelog. The reason seems to be higher bandwidth, but IMO saving bandwidth at the cost of safety and correctness is not recommended.

[raineorshine]

Hi, thanks for reporting. Please provide an example package and version and I'd be happy to look into it.

[mohd-akram]

This can be seen with the totp-generator package, version 1.1.0, which was deprecated. If a user had 1.0.0 in their package.json and ran ncu -t minor, they would be recommended 1.1.0, which was deprecated because it inadvertently had a breaking change.

[raineorshine]

Thanks for bringing this to my attention. I did some investigation, and it was indeed changed in v17 but not added to the changelog. It was originally changed in a37d193 and d254d98.

Since the deprecation status is only returned in the larger versions query to the npm registry, checking it by default adds a significant performance overhead to npm-check-updates. It runs about 20–25% slower by my estimates.

Usually when a version is deprecated, a fix is published to latest. So with the default target, npm-check-updates will upgrade to the newer, non-deprecated package anyway. The problem you discovered only occurs when using the minor target, and you can opt-in to deprecated checks by adding deprecated: false to your ncurc.json.

While I would normally favor the safe approach, the added performance cost is not something I'm willing to set as a default. The fact that nobody has reported this before attests to how unlikely it is to happen, though I apologize that it impacted you and wasn't documented properly.

Hope that makes sense.

[mohd-akram]

Usually when a version is deprecated, a fix is published to latest.

I believe npm does not return deprecated versions in latest, so that shouldn't require an expensive query I think?

The problem you discovered only occurs when using the minor target

When using the minor target, do you not also fetch all versions by default? How is the latest minor determined if I may ask?

While I would normally favor the safe approach, the added performance cost is not something I'm willing to set as a default.

For sure the performance cost is high, although I'm curious if we can work around it. Especially with what's happening with supply chain attacks and security issues, I think we really don't want to recommend deprecated versions.

Note that npm outdated also does not recommend deprecated versions, but I don't know how they do that.

[raineorshine]

Usually when a version is deprecated, a fix is published to latest.

I believe npm does not return deprecated versions in latest, so that shouldn't require an expensive query I think?

If the latest version is deprecated, it's still published to latest. It is marked as deprecated, but that field is still only accessible in the versions section of the packument. It's not included in dist-tags, e.g. npm view ncu-test-deprecated dist-tags.

Some background information:

The npm registry exposes the deprecation status in the versions section of the packument. When you query the npm registry, this is what you get back:

https://registry.npmjs.org/totp-generator

The versions can get quite long, accounting for the major bulk of the data transfer when fetching packuments. Usually we just need to know the version published to latest. So npm-check-updates actually streams the packument line by line, and, in theory, can return immediately once the latest version is read, without having to read all the versions. In the default case, this significantly improves performance and benefits the majority of users.

When using the minor target, do you not also fetch all versions by default? How is the latest minor determined if I may ask?

Yes, you're correct. Non-tag targets like minor we do fetch all versions, so you can check the deprecation status "for free." However, you still have to opt in. This is simply to ensure consistent defaults regardless of target.

While I would normally favor the safe approach, the added performance cost is not something I'm willing to set as a default.

For sure the performance cost is high, although I'm curious if we can work around it. Especially with what's happening with supply chain attacks and security issues, I think we really don't want to recommend deprecated versions.

You make a good point. I don't see a way around it without significantly impacting performance though.

Note that npm outdated also does not recommend deprecated versions, but I don't know how they do that.

Yeah, I'm not sure either. I didn't see anything in outdated.js that specifically filters out deprecated packages, so maybe it's higher up. It's possible that they fetch the full packument with the deprecation status.

[mohd-akram]

Okay, that makes sense, I see how it's working now. It seems there might be a way to work around this by calling:

https://registry.npmjs.org/ncu-test-deprecated/latest

instead. That would tell you the latest version. Then only if it's deprecated, you'd have to do the more expensive call.

[raineorshine]

Yes, good find! That looks doable. Are you interested in helping with a PR?

[mohd-akram]

I'll try to look into it soon.