Skip to content

AnandSundar/ccm-engine

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 

Repository files navigation

CCM Engine — Continuous Control Monitoring

A real-time AWS compliance engine that continuously monitors security controls, scores your posture, and alerts your team before auditors do.

Python AWS License CI Coverage

image
┌─────────────────────────────────────────────────┐
│         CCM ENGINE — COMPLIANCE SCORECARD        │
├─────────────────────────────────────────────────┤
│  Score:  78.5 / 100      Grade: C               │
│  ████████████████░░░░░░░  78%                   │
├──────────────────┬──────────────┬───────────────┤
│ Control          │ Status       │ Severity      │
├──────────────────┼──────────────┼───────────────┤
│ IAM MFA          │ ✅ PASS      │ CRITICAL      │
│ S3 Public Access │ ❌ FAIL      │ HIGH          │
│ RDS Encryption   │ ✅ PASS      │ HIGH          │
│ CloudTrail       │ ✅ PASS      │ HIGH          │
│ EC2 Open SSH     │ ❌ FAIL      │ CRITICAL      │
│ Root Access Keys │ ✅ PASS      │ CRITICAL      │
└──────────────────┴──────────────┴───────────────┘
  ⚠ 2 controls failing | Last evaluated: 2026-03-06T12:00:00Z

🎯 Why I Built This

Most companies still run compliance audits once a year. By the time they find a misconfigured S3 bucket, it's been public for 11 months. I built this because I watched compliance teams drown in spreadsheets, manually checking AWS console settings, filling out audit questionnaires by hand, and praying nothing had broken since the last audit.

The turning point was realizing that platforms like Vanta and Drata charge $50,000–$200,000 per year to do exactly what a well-designed Python service can do at infrastructure cost. These tools are excellent, but the price tag puts them out of reach for startups, mid-market companies, and anyone who wants to understand how continuous compliance actually works under the hood.

I wanted to work in GRC (Governance, Risk, and Compliance) because I understand that compliance isn't just a checkbox exercise — it's continuous risk management. But I didn't want to just use the tools; I wanted to build them. I built this to understand the mechanics behind continuous compliance — not just how to use the tools, but how to build them.


🔥 The Problem This Solves

Without CCM Engine With CCM Engine
Compliance reviewed once a year Controls checked every 15 minutes
Spreadsheets updated manually Real-time scorecard via REST API
Misconfigured resources discovered during audits Instant SNS alert on first failure
No audit trail Immutable DynamoDB history log
$50K–$200K/yr for Vanta/Drata Runs on ~$5/month AWS infrastructure
"We think we're compliant" "We scored 94/100 at 09:00 this morning"

For a business, this means shifting from reactive compliance (finding problems after they've existed for months) to proactive security posture management. Instead of scrambling before an audit, you have continuous evidence of your controls working. Instead of guessing whether you're compliant, you have a number and a grade you can defend.


Try it yourself:

git clone https://github.com/yourname/ccm-engine
cd ccm-engine
cp .env.example .env
docker-compose up
# Visit http://localhost:8000/scorecard

🏗️ How It Works — Architecture

Every 15 minutes, the scheduler triggers six security checks against your AWS account. Each check asks a simple yes/no question about your infrastructure: "Does this S3 bucket allow public access?" "Is MFA enabled on every IAM user?" The results get stored in DynamoDB for the audit trail, trigger SNS alerts on state changes, and power a real-time scorecard API.

┌─────────────────────────────────────────────────────────────┐
│                        CCM ENGINE                           │
│                                                             │
│  ┌──────────────┐    ┌─────────────────────────────────┐   │
│  │  Scheduler   │───▶│         Control Checks          │   │
│  │ (every 15min)│    │  iam_mfa  │ s3_public │ ec2_ssh │   │
│  └──────────────┘    │  rds_enc  │ cloudtrail│ root_key│   │
│                       └────────────────┬────────────────┘   │
│                                        │ CheckResult        │
│                       ┌────────────────▼────────────────┐   │
│                       │         State Store             │   │
│                       │    DynamoDB (audit log +        │   │
│                       │     latest state per control)   │   │
│                       └────────────┬────────────────────┘   │
│                                    │                        │
│              ┌─────────────────────┼──────────────┐        │
│              ▼                     ▼              ▼        │
│      ┌───────────────┐   ┌──────────────┐  ┌──────────┐   │
│      │  Notifier     │   │  FastAPI     │  │Scorecard │   │
│      │  SNS Alert    │   │  REST API    │  │Weighted  │   │
│      │  (PASS→FAIL   │   │  /scorecard  │  │Score +   │   │
│      │  only)        │   │  /controls   │  │Grade     │   │
│      └───────────────┘   └──────────────┘  └──────────┘   │
└─────────────────────────────────────────────────────────────┘
  1. Scheduler runs every 15 minutes (configurable via CHECK_INTERVAL_MINUTES)
  2. Control Checks execute independently against AWS APIs via boto3
  3. State Store persists results to DynamoDB — immutable audit log + latest state
  4. Notifier sends SNS alerts only on PASS→FAIL transitions (no alert fatigue)
  5. API exposes scorecard, control details, and history via FastAPI

📋 Security Controls Reference

Control ID What It Checks AWS Service Used Severity Frameworks Covered
iam_mfa_all_users Every IAM user has MFA device enabled IAM: ListUsers, ListMFADevices CRITICAL SOC 2, ISO 27001, NIST CSF
s3_no_public_buckets No bucket allows public read/write S3: GetBucketAcl, GetBucketPolicyStatus HIGH SOC 2, PCI DSS, HIPAA
rds_encryption_at_rest All RDS instances use encrypted storage RDS: DescribeDBInstances HIGH HIPAA, SOC 2, PCI DSS
cloudtrail_all_regions CloudTrail logging active in every region CloudTrail: DescribeTrails HIGH SOC 2, NIST CSF, CIS Benchmark
ec2_no_open_ssh No security group exposes SSH to the world EC2: DescribeSecurityGroups CRITICAL CIS Benchmark, NIST CSF
iam_no_root_access_keys AWS root account has no active access keys IAM: GetAccountSummary CRITICAL CIS Benchmark, SOC 2

Each control maps to real-world compliance frameworks. This means you're not just passing checks — you're building evidence for audits.


📊 Compliance Scoring — How It Works

image

Not all security failures are equal. Leaving SSH open to the internet is worse than a missing cost allocation tag. CCM Engine weights failures by severity, so a CRITICAL failure drags your score down more than a HIGH severity one.

Score = (Σ weights of PASS controls / Σ all possible weights) × 100

Weights: CRITICAL = 4 pts | HIGH = 3 pts | MEDIUM = 2 pts | LOW = 1 pt

Grade Scale:
  A90Audit-ready posture
  B75Minor gaps, low risk
  C60Moderate risk, action needed
  D45Significant exposure
  F  < 45Critical failures present

Worked Example:

  • 2 CRITICAL PASS = 2 × 4 = 8 points
  • 2 HIGH PASS = 2 × 3 = 6 points
  • 2 CRITICAL FAIL = 2 × 4 = 0 points (from denominator)
  • Score = (8 + 6) / (8 + 6 + 8) × 100 = 14 / 22 × 100 = 63.6 → Grade C

🚀 API Reference

GET /scorecard

Returns the current compliance scorecard with weighted score and grade.

curl -X GET http://localhost:8000/scorecard
{
  "score": 61.5,
  "grade": "C",
  "total_controls": 6,
  "passed": 4,
  "failed": 2,
  "critical_failures": [
    "iam_mfa_all_users",
    "ec2_no_open_ssh"
  ],
  "last_evaluated": "2026-03-06T12:00:00Z",
  "controls": [
    {
      "control_id": "iam_mfa_all_users",
      "control_name": "All IAM users have MFA enabled",
      "category": "Identity",
      "severity": "CRITICAL",
      "status": "FAIL",
      "affected_resources": [
        "arn:aws:iam::123456789012:user/dev-user"
      ]
    },
    {
      "control_id": "s3_no_public_buckets",
      "control_name": "No S3 buckets are publicly accessible",
      "category": "Data Protection",
      "severity": "HIGH",
      "status": "PASS",
      "affected_resources": []
    },
    {
      "control_id": "rds_encryption_at_rest",
      "control_name": "All RDS instances encrypted at rest",
      "category": "Data Protection",
      "severity": "HIGH",
      "status": "PASS",
      "affected_resources": []
    },
    {
      "control_id": "cloudtrail_all_regions",
      "control_name": "CloudTrail enabled in all regions",
      "category": "Audit & Logging",
      "severity": "HIGH",
      "status": "PASS",
      "affected_resources": []
    },
    {
      "control_id": "ec2_no_open_ssh",
      "control_name": "No security groups allow 0.0.0.0/0 on port 22",
      "category": "Network",
      "severity": "CRITICAL",
      "status": "FAIL",
      "affected_resources": [
        "sg-0123456789abcdef"
      ]
    },
    {
      "control_id": "iam_no_root_access_keys",
      "control_name": "Root account has no active access keys",
      "category": "Identity",
      "severity": "CRITICAL",
      "status": "PASS",
      "affected_records": []
    }
  ]
}

GET /controls

Lists all registered compliance controls.

curl -X GET http://localhost:8000/controls

GET /controls/{id}

Gets details for a specific control.

curl -X GET http://localhost:8000/controls/iam_mfa_all_users

POST /controls/{id}/run

Manually triggers a specific control check.

curl -X POST http://localhost:8000/controls/iam_mfa_all_users/run

GET /health

Health check endpoint for monitoring.

curl -X GET http://localhost:8000/health

💻 Tech Stack

Component Technology Why I Chose It
Language Python 3.11 Industry standard for AWS automation and GRC tooling
AWS Integration boto3 Native AWS SDK, used by every major security tool
API Layer FastAPI Auto-generated OpenAPI docs, async support, type safety
State Store DynamoDB Serverless, infinitely scalable, no schema migration headaches
Alerting AWS SNS Native AWS, integrates with PagerDuty/Slack/email out of the box
Testing pytest + moto Zero real AWS credentials needed in CI — no accidental costs
Infrastructure Terraform Infrastructure-as-code, reproducible, audit-friendly
CI/CD GitHub Actions Runs lint + type check + full test suite on every push
Local Dev Docker + LocalStack Full AWS stack locally, demo without an AWS account

🔐 Security-First Design

Read-only IAM Policy: The engine only ever needs read permissions. If my tool is compromised, an attacker gains zero write access to your infrastructure.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "iam:ListUsers",
        "iam:ListMFADevices",
        "iam:GetAccountSummary",
        "s3:GetBucketAcl",
        "s3:GetBucketPolicyStatus",
        "rds:DescribeDBInstances",
        "cloudtrail:DescribeTrails",
        "ec2:DescribeSecurityGroups"
      ],
      "Resource": "*"
    }
  ]
}

No credentials in code: All config via environment variables, .env.example provided, .env gitignored.

Immutable audit log: DynamoDB records are append-only. Historical results are never overwritten — this is intentional for compliance evidence chains.

Alert fatigue prevention: SNS fires only on PASS→FAIL transitions, not on every repeated failure. This prevents getting 1440 alerts a day for a check that failed once.

Error isolation: One failing check never crashes the full run — each check is wrapped in independent error handling and returns status: "ERROR".


📁 Project Structure

ccm-engine/
├── checks/
│   ├── base.py              # Abstract BaseCheck class — plugin interface
│   ├── iam_mfa.py          # Control: MFA on all IAM users
│   ├── s3_public.py        # Control: No public S3 buckets
│   ├── rds_encryption.py   # Control: RDS encryption at rest
│   ├── cloudtrail.py       # Control: CloudTrail in all regions
│   ├── ec2_ssh.py          # Control: No open SSH security groups
│   └── iam_root_keys.py    # Control: No root account access keys
├── core/
│   ├── models.py           # Pydantic models: CheckResult, ScorecardResponse
│   ├── state_store.py      # DynamoDB read/write abstraction
│   ├── notifier.py         # SNS alerts on state transitions
│   ├── scheduler.py        # APScheduler: run all checks on cron
│   └── registry.py         # Auto-discovers and registers all checks
├── api/
│   └── main.py             # FastAPI app: all endpoints
├── infra/
│   └── main.tf             # Terraform: DynamoDB, SNS, IAM role
├── tests/
│   ├── test_checks.py      # moto-mocked unit tests per control
│   └── test_api.py         # TestClient integration tests
├── scripts/
│   └── seed_localstack.py  # Creates failing resources for local demo
├── .github/workflows/
│   └── ci.yml              # GitHub Actions: lint → typecheck → test
├── Dockerfile
├── docker-compose.yml
├── .env.example
└── README.md

➕ How to Add a New Control

Adding a new control requires touching exactly one file. The registry auto-discovers all checks.

Step 1: Create the check file in checks/

# checks/iam_password_policy.py
from checks.base import BaseCheck
from core.models import CheckResult, CheckStatus
import boto3


class IAMPasswordPolicyCheck(BaseCheck):
    def __init__(self):
        super().__init__(
            check_id="iam_password_policy",
            check_name="Account password policy meets requirements",
            description="Password policy requires uppercase, lowercase, numbers, and 14+ characters"
        )

    async def run(self) -> CheckResult:
        iam = boto3.client("iam")
        try:
            policy = iam.get_account_password_policy()
            requirements = policy["PasswordPolicy"]
            
            required = (
                requirements.get("RequireUppercaseCharacters") and
                requirements.get("RequireLowercaseCharacters") and
                requirements.get("RequireNumbers") and
                requirements.get("MinimumPasswordLength", 0) >= 14
            )
            
            if required:
                return self.create_result(CheckStatus.PASS, "Password policy compliant")
            else:
                return self.create_result(
                    CheckStatus.FAIL, 
                    "Password policy does not meet requirements"
                )
        except iam.exceptions.NoSuchEntityException:
            return self.create_result(
                CheckStatus.FAIL,
                "No account password policy configured"
            )

Step 2: The registry automatically picks it up — no registration needed.

Step 3: Run pytest to verify it works.

Step 4: Deploy. The API will automatically surface the new control.


🗺️ Roadmap

Feature Status Business Value
6 core AWS controls ✅ Complete Covers most common SOC 2 / CIS findings
Severity-weighted scoring + grades ✅ Complete Gives instant posture summary to leadership
DynamoDB audit history ✅ Complete Evidence trail for auditors
SNS alerts on state transition ✅ Complete Replaces manual monitoring
Slack webhook integration 🔄 In Progress Direct team notification without email
NIST CSF / SOC 2 control mapping 📋 Planned Maps findings to framework requirements automatically
Multi-account support 📋 Planned Enterprise-grade: monitor entire AWS Organization
PDF compliance report export 📋 Planned Audit-ready reports without a GRC platform subscription
Web dashboard (React) 📋 Planned Visual scorecard for non-technical stakeholders
Azure + GCP support 💡 Future Multi-cloud compliance from a single engine

📈 Why This Matters for GRC

Most GRC tools are black boxes. Analysts use them without understanding how they actually check controls. Building CCM Engine from scratch means I understand the exact API calls, the data shapes, the failure modes, and the alert logic — the same mechanics that power Vanta, Drata, and AWS Security Hub. When an auditor asks "how does this check work?", I can explain not just the result, but the implementation.

For a GRC Analyst, this matters because the day-to-day work involves gathering evidence, maintaining control inventories, and responding to audit findings. Automating this work means a GRC team can focus on risk assessment and framework alignment instead of manual screenshot collection. Instead of being the person who clicks through the AWS console checking settings, you're the person who built the system that does it automatically — and can debug it when it breaks.


🤝 Connect With Me

LinkedIn: linkedin.com/in/anandsundar96

GitHub: github.com/anandsundar

If you're building a compliance program that moves faster than your auditors, I'd like to be part of that team.

About

Continuous Control Monitoring Engine - Real-time AWS compliance monitoring

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors