Improve the risk calculation algorithm

[scovetta]

The risk calculation algorithm (in oss-risk-calculator) is pretty basic, combining project health (from oss-health) and the number of high-risk characteristics (from oss-characteristic), then normalizing to a range of 0 to 1.

I think this could be improved significantly, but would require a good amount of "think" time. If anyone is interested in contributing, it'd be very much appreciated.

[stevep-arm]

Would it make sense to use the scorecard metrics?

[Sverre-rt]

Will this risk calculation algorithm always return high risk (1)? I have tested the 18 most popular packages and they all return 1. After inspecting the code it seems like the Characteristics check is always above 1.

An example from testing Newtonsoft.json:
Aggregate score risk after Health Metrics: 0,648734693877551
Aggregate risk after Characteristics: 1,1541142857142856
This is more than 1 which leads to  Aggregate risk: 1

One idea might be to lower the aggregate risk multiplier from 0.7:

[gfs]

@Sverre-rt Thanks for the report. I don't think this is intentional. A stop gap might be to reduce the 0.7 to 0.5 to keep the 0-1 range.

I think I'll also double check that the high risk tag score is being calculated appropriately first. If it's consistently above 1 then the scaling may need to be adjusted there as well.

[gfs]

OSSGadget/src/oss-risk-calculator/RiskCalculatorTool.cs

Lines 155 to 159 in 1730841

	double highRiskTagRisk = (
	0.4 * highRiskTagsSeen.Keys.Count() +
	0.6 * Math.Min(highRiskTagsSeen.Values.Sum(), 5)
	);
	highRiskTagRisk = highRiskTagRisk > 1.0 ? 1.0 : highRiskTagRisk;

I think the issue may lie here. I'll have to investigate more about this calculation here. 0.6 * the sum of counts of tags with a max of 5, as this would mean 3 is a common value here before adding 0.4 times the number of unique tags, but the intention is to derive a value between 0-1.

[scovetta]

I think we should deprecate this tool (oss-risk-calculator) and direct folks to OpenSSF Security Scorecard instead, which gives IMHO better information.