adding --only-allowlisted flag to scan for inline ignores

[domanchi]

Summary

Adds --only-allowlisted to extract all allowlisted secrets from a codebase. In doing so, it ports #346 to the new architecture. For example:

detect-secrets scan --only-allowlisted test_data/config.yaml
{
  "generated_at": "2020-11-12T15:49:24Z",
  "version": "1.0.0",
  "plugins_used": [
    {
      "name": "ArtifactoryDetector"
    },
    {
      "name": "AWSKeyDetector"
    },
    {
      "name": "BasicAuthDetector"
    },
    {
      "name": "CloudantDetector"
    },
    {
      "name": "Base64HighEntropyString"
    },
    {
      "name": "HexHighEntropyString"
    },
    {
      "name": "IbmCloudIamDetector"
    },
    {
      "name": "IbmCosHmacDetector"
    },
    {
      "name": "JwtTokenDetector"
    },
    {
      "name": "KeywordDetector"
    },
    {
      "name": "MailchimpDetector"
    },
    {
      "name": "PrivateKeyDetector"
    },
    {
      "name": "SlackDetector"
    },
    {
      "name": "SoftlayerDetector"
    },
    {
      "name": "StripeDetector"
    },
    {
      "name": "TwilioKeyDetector"
    }
  ],
  "filters_used": [
    {
      "path": "detect_secrets.filters.heuristic.is_potential_uuid"
    },
    {
      "path": "detect_secrets.filters.heuristic.is_sequential_string"
    },
    {
      "path": "detect_secrets.filters.heuristic.is_likely_id_string"
    }
  ],
  "results": {
    "test_data/config.yaml": [
      {
        "type": "Secret Keyword",
        "filename": "test_data/config.yaml",
        "hashed_secret": "7225a6c4ec89562503785ad5332182cd30e8a52a",
        "is_verified": false,
        "line_number": 12
      },
      {
        "type": "Base64 High Entropy String",
        "filename": "test_data/config.yaml",
        "hashed_secret": "7225a6c4ec89562503785ad5332182cd30e8a52a",
        "is_verified": false,
        "line_number": 12
      }
    ]
  }
}

I decided to use the baseline format to be consistent and compatible with all other functionality in this tool. This way, you'll be able to feed this baseline into the audit tool for an interactive view of the results.

The main difference between this and grep -rE '(allow|white)list secret' is that this includes all the other filters that are configured with the tool. Hence, if you pass in a baseline argument with this, it will respect all configured plugins / filters, and not output a secret if it:

1. Isn't flagged by a configured plugin, OR
2. Is ignored by a filter

In doing so, it paves the way for identifying redundant pragma comments. However, this PR will not include that feature (left as an exercise for the reader -- PRs welcome!)

Technical Details

This works by essentially making detect_secrets.filters.allowlist.is_line_allowlisted an optional plugin, but a default one at that. This means that you can disable it, but it will come with preconfigured for all new baselines (and upgraded baselines).

This change allows us to use the standard scanning architecture (making it compatible with audit), as well as keeping the configuration around in the baseline itself (for manual review).

With regards to the initial scanning (to create the list of secrets), we invert the filter and only scan lines that this filter would have ignored. This way, we know that we're only focusing on the lines that matter.

Testing Done

make test

passes.

[OiCMudkips]

Looks good to me.