kunabi

A CloudTrail log analyzer built in Gerbil Scheme. Downloads AWS CloudTrail logs from S3, stores them in LevelDB with multi-index support, and provides querying, security detection, and billing impact analysis.

Prerequisites

System Libraries

Library	Purpose
libyaml	YAML configuration parsing
libleveldb	Key-value database storage
libstdc++	Required by LevelDB
libssl, libcrypto	TLS/SSL for S3 access
libz	Gzip decompression of CloudTrail logs
patchelf	Post-build RPATH patching (build-time only)

On Debian/Ubuntu:

apt install libyaml-dev libleveldb-dev libssl-dev zlib1g-dev patchelf

Gerbil Scheme

Requires Gerbil v0.19+ with the following packages:

gxpkg install github.com/mighty-gerbils/gerbil-leveldb
gxpkg install github.com/mighty-gerbils/gerbil-libyaml

AWS Credentials

S3 access uses the standard AWS credential chain via :std/net/s3. Configure credentials through any of the standard methods:

• ~/.aws/credentials file
• AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables
• IAM instance roles (EC2/ECS)

Building

make build       # compile and patch RPATH
make clean       # remove build artifacts
make install     # copy binary to /usr/local/bin

Or directly:

gerbil build

The Makefile patches the binary's RPATH to find OpenSSL 3 libraries. Edit OPENSSL_RPATH in the Makefile if your OpenSSL is installed elsewhere:

OPENSSL_RPATH = /home/linuxbrew/.linuxbrew/opt/openssl@3/lib

The build links against: -lyaml -lleveldb -lstdc++ -lssl -lcrypto -lz -lm -lutil

Configuration

Config File: ~/.kunabi.yaml

All fields are optional. CLI flags override config file values.

# Path to LevelDB database directory
db: "./cloudtrail.db"

# S3 bucket containing CloudTrail logs
bucket: "my-cloudtrail-bucket"

# S3 key prefix (path within the bucket)
prefix: "AWSLogs/123456789012/CloudTrail"

# AWS regions to scan (iterates prefix/{region}/... for each)
regions:
  - us-east-1
  - us-west-2
  - eu-west-1

# Date range for S3 path generation
# Values: "current_month", "previous_month", "current_day", or "" (disabled)
# Generates YYYY/MM/DD path suffixes appended to the prefix
duration: "current_month"

# Event names to exclude from security detection
omit_events:
  - "DescribeInstances"
  - "ListBuckets"

# User + event combinations to exclude from security detection
# Each entry is a list of [username, event_name]
omit_filters:
  - ["automation-user", "CreateLogGroup"]
  - ["deploy-role", "PutBucketPolicy"]

The config file path is hardcoded to ~/.kunabi.yaml. A custom path can be passed programmatically to load-config but there is no CLI flag for it.

Environment Variables

Kunabi itself does not read any environment variables. All configuration is file-based (~/.kunabi.yaml) or passed via CLI flags.

Standard Gerbil environment variables apply to the build process:

Variable	Purpose
GERBIL_PATH	Gerbil workspace root (default: ~/.gerbil)
GERBIL_LOADPATH	Additional module search paths (colon-separated)
GERBIL_BUILD_CORES	Parallel compilation cores

AWS credentials are resolved by the :std/net/s3 module through the standard AWS credential chain (~/.aws/credentials, AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY, IAM roles).

Usage

kunabi <command> [options]

Commands

load -- Download CloudTrail logs from S3

kunabi load [OPTIONS]

Flag	Long	Default	Description
	--db	from config or ./cloudtrail.db	LevelDB database path
-b	--bucket	from config	S3 bucket name (required)
-p	--prefix	from config	S3 key prefix
-r	--region	us-east-1	AWS region for S3 API calls
-w	--workers	4	Number of concurrent download threads
-d	--duration	from config	Date range: current_month, previous_month, current_day
-f	--full	off	Keep ResponseElements (stripped by default to save space)

When regions is set in the config, the loader iterates over each region as a path component. When duration is set, it generates YYYY/MM/DD date paths. Both can combine: {prefix}/{region}/{YYYY}/{MM}/{DD}/.

Already-processed S3 objects are tracked and skipped on subsequent runs.

report -- Query CloudTrail events

kunabi report [OPTIONS]

Flag	Long	Default	Description
	--db	from config	LevelDB database path
-u	--username		Filter by IAM username
-e	--event		Filter by event name
-E	--error		Filter by error code
-r	--filter-region		Filter by AWS region
-s	--start		Start date inclusive (YYYY-MM-DD)
-n	--end		End date inclusive (YYYY-MM-DD, covers full day)
-S	--summary	off	Show aggregated summary instead of individual events
-w	--warn	off	Show summary with source IP warnings

The query engine uses a 3-pass strategy: index scan, batch fetch, post-filter. Index selection priority: user-event composite > user > event > error > region > date.

list -- List unique values

kunabi list [--db PATH] <type>

<type> is one of: users, events, dates, regions

detect -- Security detection

kunabi detect [OPTIONS]

Flag	Long	Default	Description
	--db	from config	LevelDB database path
-s	--severity	all	Minimum severity: critical, high, medium, low
-S	--summary	off	Show summary counts only

Scans 65 rules across 7 categories:

Category	Examples
Persistence/Backdoor	CreateUser, CreateAccessKey, CreateLoginProfile
Covering Tracks	StopLogging, DeleteTrail, DeleteLogGroup
Data Exfiltration	S3 policy changes, snapshot sharing
Resource Destruction	Instance termination, volume deletion
Network/DNS Hijacking	Route53, Security Group, VPC changes
Crypto Mining	EC2 launch, Lambda creation
Privilege Escalation	Policy attachment, role assumption

Respects omit_events and omit_filters from the config file.

billing -- Billing impact analysis

kunabi billing [OPTIONS]

Flag	Long	Default	Description
	--db	from config	LevelDB database path
-i	--impact	all	Filter by impact: increase, decrease, change
-s	--service	all	Filter by AWS service name
-S	--summary	off	Show summary counts only

Tracks 150+ rules across 19 AWS services: EC2, EBS, RDS, S3, Lambda, ECS, EKS, ElastiCache, DynamoDB, Redshift, VPC, ELB, CloudFront, OpenSearch, Kinesis, Firehose, SageMaker, CloudWatch, SecretsManager.

search -- Full-text search

kunabi search [OPTIONS] <term>

Flag	Long	Default	Description
	--db	from config	LevelDB database path
-i	--case-insensitive	off	Case-insensitive matching
-l	--limit	0 (unlimited)	Maximum results to return

Scans all stored event JSON for substring matches.

get -- Fetch single event

kunabi get [--db PATH] <event-id>

purge -- Delete old events

kunabi purge [OPTIONS]

Flag	Long	Default	Description
	--db	from config	LevelDB database path
-d	--days	required	Keep events from the last N days
-n	--dry-run	off	Preview deletions without deleting
-c	--leveldb-compact	off	Run LevelDB compaction after purge

Deletes events and all associated index entries in batches of 1000.

compact -- Strip ResponseElements

kunabi compact [OPTIONS]

Flag	Long	Default	Description
	--db	from config	LevelDB database path
-n	--dry-run	off	Preview without modifying
-c	--leveldb-compact	off	Run LevelDB compaction after

Removes responseElements from stored events to reduce database size.

leveldb-compact -- LevelDB compaction

kunabi leveldb-compact [--db PATH]

Triggers LevelDB's built-in SST file compaction to reclaim disk space.

reindex -- Build composite indices

kunabi reindex [--db PATH]

Scans all stored events and creates user-event composite index entries. Run this after loading data if composite indices are missing.

Database Schema

Kunabi uses LevelDB with the following key prefixes:

Prefix	Key Format	Value
event:	event:{eventID}	Full event JSON
idx:user:	idx:user:{username}:{time}:{eventID}	Event ID
idx:event:	idx:event:{eventName}:{time}:{eventID}	Event ID
idx:user-event:	idx:user-event:{username}:{eventName}:{time}:{eventID}	Event ID
idx:error:	idx:error:{errorCode}:{time}:{eventID}	Event ID
idx:region:	idx:region:{region}:{time}:{eventID}	Event ID
idx:date:	idx:date:{YYYY-MM-DD}:{eventID}	Event ID
idx:instance:	idx:instance:{instanceId}	Instance metadata JSON
processed:	processed:{bucket}:{key}	"done"

LevelDB Tuning

Parameter	Value
Bloom filter	10 bits per key
Write buffer	64 MB
LRU cache	32 MB
Compression	Enabled (Snappy)
Batch delete size	1000 events per write batch

These values are hardcoded in kunabi/storage.ss.

Project Structure

kunabi/
  gerbil.pkg              # Package: ober, dependencies
  build.ss                # Build script with linker flags
  Makefile                # Build, clean, install targets
  manifest.ss             # Version manifest (auto-generated)
  kunabi.ss               # Module re-exports
  kunabi/
    main.ss               # CLI entry point and command dispatch
    config.ss             # ~/.kunabi.yaml loader
    parser.ss             # CloudTrail JSON parser and field extractors
    storage.ss            # LevelDB storage engine and indices
    loader.ss             # S3 download with concurrent worker pool
    query.ss              # Query engine with index selection
    detection.ss          # Security detection rules (65 rules)
    billing.ss            # Billing impact rules (150+ rules)