Fix(stats): Correct p-value calculation in permutation test

[Geeks-Sid]

Description

This pull request addresses a bug in the perform_permutation_test method where the p-value was calculated using an incorrect one-sided test. This resulted in unreliable significance testing, potentially masking real differences between methods.

The Bug: Incorrect One-Sided Test Logic

The previous implementation calculated the observed difference in summed ranks as diff_ranks = arr_i.sum() - arr_j.sum() and then counted how many permuted differences were smaller (diff_ranks_rand < diff_ranks).

This logic only worked correctly if method_i was superior to method_j (resulting in a large negative diff_ranks). If method_j was superior to method_i, diff_ranks would be a large positive number. The condition diff_ranks_rand < diff_ranks would then be met by almost all permutations, leading to an erroneously high p-value (e.g., > 0.5) and a failure to detect a statistically significant difference.

The Fix: Implementing a Standard Two-Sided Test

The permutation test has been corrected to use a standard two-sided test, which is the correct approach for determining if two methods are significantly different, regardless of the direction of the difference.

The changes are as follows:

1. Use Absolute Differences: The test now compares the absolute magnitude of the observed difference with the absolute magnitude of the permuted differences.
   • Before the loop: observed_diff = abs(arr_i.sum() - arr_j.sum())
   • Inside the loop: if abs(permuted_diff) >= observed_diff:
2. Symmetric P-value Matrix: The p-value matrix (self.pvals) is now correctly populated to be symmetric (self.pvals[i, j] = self.pvals[j, i]), as the significance of the difference between method A and B is identical to that between B and A.
3. Improved Initialization: The self.pvals matrix is now initialized with np.ones(), which is more semantically correct, as the default state (including the diagonal) represents no significant difference (p=1.0).

[Geeks-Sid]

This code should be good to go and can be tested under normal tests.

[Copilot]

Pull Request Overview

This PR fixes a critical bug in the perform_permutation_test method where the p-value calculation used an incorrect one-sided test that could fail to detect statistically significant differences when the second method outperformed the first.

• Corrects the permutation test to use a proper two-sided test by comparing absolute differences
• Ensures the p-value matrix is symmetric as it should be for pairwise comparisons
• Improves code efficiency with boolean indexing and better variable management