Skip to content

Zwerd/ziochub

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

53 Commits
__pycache__
__pycache__
 
 
data
data
 
 
docs
docs
 
 
packages
packages
 
 
routes
routes
 
 
scripts
scripts
 
 
static
static
 
 
templates
templates
 
 
utils
utils
 
 
README.md
README.md
 
 
README_HE.md
README_HE.md
 
 
app.py
app.py
 
 
backup_threatgate.sh
backup_threatgate.sh
 
 
backup_ziochub.sh
backup_ziochub.sh
 
 
cleaner.py
cleaner.py
 
 
config.py
config.py
 
 
constants.py
constants.py
 
 
create_lab_users.py
create_lab_users.py
 
 
extensions.py
extensions.py
 
 
misp_settings.py
misp_settings.py
 
 
misp_sync_job.py
misp_sync_job.py
 
 
models.py
models.py
 
 
requirements-offline.txt
requirements-offline.txt
 
 
requirements.txt
requirements.txt
 
 
reset_data.py
reset_data.py
 
 
setup.sh
setup.sh
 
 
start.sh
start.sh
 
 
test_imports.sh
test_imports.sh
 
 
uninstall.sh
uninstall.sh
 
 
ziochub-backup.service
ziochub-backup.service
 
 
ziochub-backup.timer
ziochub-backup.timer
 
 
ziochub-cleaner.service
ziochub-cleaner.service
 
 
ziochub-cleaner.timer
ziochub-cleaner.timer
 
 
ziochub-misp-sync.service
ziochub-misp-sync.service
 
 
ziochub-misp-sync.timer
ziochub-misp-sync.timer
 
 
ziochub.service
ziochub.service
 
 
ziochub_2.0_Beta_installer.zip
ziochub_2.0_Beta_installer.zip
 
 
מדריך מערכת מקוצר - ziochub.pdf
מדריך מערכת מקוצר - ziochub.pdf
 
 

Repository files navigation

ZIoCHub v2.0 Beta - IOC & YARA Management Platform

ZIoCHub is a modern IOC & YARA Management Platform built for SOC operations. Analysts submit indicators, ZIoCHub stores them in a SQLite database, and security devices ingest plain-text feeds for enforcement. Designed for air-gapped / offline environments.

Recent highlights

  • YARA Manager is organized like Submit IOCs: Upload · Write · Status with an info card per mode. Write supports typing or pasting a rule, a live Prism syntax preview (same C-like highlighting as View), and Check syntax - server-side compilation via yara-python (official YARA / libyara engine). Install yara-python in the same Python environment as the app (pip install -r requirements.txt). For offline deployments, include a matching wheel in packages/ or install system libyara dev packages when building from source; see OFFLINE.md.
  • Cisco Secure Email Gateway (ESA): optional content dictionary sync from Admin > Integrations > Cisco ESA - push IOC values after login (jwttoken), batched POST add and DELETE remove per dictionary (Cisco AsyncOS API v2.0 shapes). Mappings are simple rows: dictionary name + one of Email / Domain / IP / URL. Supports standalone / cluster / group / machine query modes per the Secure Email API Guide. cleaner.py can remove words from ESA before deleting expired IOCs (toggle in UI).
  • Submit IOCs: bulk preview/staging improvements (e.g. TXT/paste deduplication, CSV parsing and URL/domain extraction).
  • Champs Analysis: monthly team goals count activity from the 1st of the current calendar month through today (not a rolling 30-day window), so the HUD meter resets on the 1st. Team-goal API cache keys are scoped by calendar week/month to avoid stale bars after a period change. Ticker banner scroll direction (RTL / LTR) is configurable from admin Ticker Message Settings.
  • Offline installer: ships built-in default avatars only under static/avatars/default/ (lab-specific images under static/avatars/ are not bundled).

Open Source Projects Used

ZIoCHub is built on the following open source projects:

# Project Purpose
1 Flask Web framework
2 Flask-Login User session & authentication
3 Flask-SQLAlchemy ORM and database integration
4 Werkzeug WSGI utilities
5 SQLite Embedded database
6 geoip2 / MaxMind GeoIP lookups (optional)
7 maxminddb MaxMind DB reader (optional, used with geoip2)
8 ldap3 LDAP/AD authentication (optional)
9 PyMISP MISP API integration (optional)
10 dxlclient / dxltieclient McAfee DXL/TIE – ePO hash reputation (optional)
11 yara-python Python bindings for YARA; server-side rule compile / Check syntax in YARA Manager → Write (requires libyara at install time; see requirements.txt, OFFLINE.md)
12 Tailwind CSS UI styling (build step; local tailwind-built.css / tailwind.min.js)
13 Chart.js Charts and dashboards
14 vis-network (vis.js) Campaign & IOC graph visualization
15 marked Markdown parsing (Reports, Playbook)
16 turndown HTML-to-Markdown conversion
17 jsPDF PDF export (Reports)
18 html2canvas Screenshot for PDF export
19 Prism Syntax highlighting (YARA as C-like in View modal and Write preview)
20 Flag Icons Country flags in UI

Development-only (see requirements-dev.txt, not required at runtime): pytest, pytest-cov, Ruff, Black.

Features

  • Authentication & User Management: Local accounts, optional LDAP/AD integration, admin roles, profile (display name, avatar), change password, optional "must change password" on first login
  • MISP Integration: Automatic IOC pull from a local MISP instance with configurable intervals
  • Cisco ESA dictionary sync (optional): After analyst submit/revoke (and on expiry via cleaner), sync IOC values to AsyncOS content dictionaries over HTTPS; configured under Admin > Integrations
  • Champs Analysis: Analyst leaderboard, multiple scoring methods (Weighted, Flat, By Type, Campaign Focus, Time Decay, Quality, Goal-Based, Smart/Effort), streak bonuses, team goals (weekly uses ISO weeks; monthly uses calendar month), rank tracking, activity spotlight, news ticker (banner direction RTL/LTR configurable in admin)
  • Feed Pulse: Real-time incoming/outgoing IOC monitoring with anomaly detection and analyst exclusions
  • Campaign Management: Visual graph (vis.js) of campaigns and associated IOCs
  • YARA Rule Management: Upload · Write · Status modes (aligned with Submit IOCs UX); Write with live Prism preview and server-side Check syntax (yara-python); upload, approval workflow, quality scoring (10-50 pts), campaign linking
  • Intelligence Reports: Period-based reports (day/week/month) with KPIs, type distribution, feed health, analyst activity, export to PDF
  • Multi-vendor Feeds: Standard, Palo Alto (EDL), Checkpoint (CSV) feed formats
  • TAXII 2.1 / STIX 2.1: TAXII 2.1 server for STIX 2.1 threat intelligence; active IOCs exposed as a TAXII collection for clients (e.g. Cisco IronPort ESA)
  • SSL/TLS & HTTPS: Certificate upload via Admin UI, automatic HTTP-to-HTTPS redirect
  • IOC History: Full lifecycle tracking per IOC (created, edited, deleted, expired, excluded, unexcluded)
  • IOC Notes: Analyst notes per IOC (type+value) for knowledge sharing; notes survive IOC deletion cycles
  • Allowlist / Safety Net: Admin-managed allowlist (raw text); prevents blocking of critical infrastructure
  • Sanity Checks: Automatic anomaly detection (local IPs, short domains, critical infra)
  • GeoIP Intelligence: Country, TLD, and email domain analytics from active IOCs; Rare Find badges for first-ever country/TLD/email domain
  • CEF / Syslog: Optional CEF-format audit logging with 48-hour local rotation and UDP syslog forwarding (Admin Settings)
  • Multi-language: English and Hebrew (i18n)
  • 100% Offline: No external network calls. All assets served locally

Table of Contents

Installation

Option 1: Online (Linux with Internet)

# Copy project to server
scp -r ZIoCHub/ user@server:/tmp/

# Install
cd /tmp/ZIoCHub
sudo ./setup.sh

Option 2: Offline (Air-Gapped)

# On dev machine (with internet):
./package_offline.sh
# Transfer ziochub_installer.zip to server

# On target server:
unzip ziochub_installer.zip -d ziochub_install
cd ziochub_install
sudo ./setup.sh --offline

Option 3: Upgrade Existing

sudo ./setup.sh --upgrade
# or
sudo ./setup.sh --upgrade --offline

Local Development

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt   # includes yara-python for YARA "Check syntax"
python app.py
# Open http://127.0.0.1:5000

If Check syntax fails with a compiler / No module named 'yara' message, ensure yara-python is installed in this venv (same interpreter as python app.py). On Linux, if only a source tarball is available, you may need libyara-dev (or equivalent) to build. See OFFLINE.md for air-gapped wheel placement.

Default credentials: admin / admin

Ports & Network

Port Protocol Service Description
8443 HTTPS ZIoCHub Main application (gunicorn + SSL)
8080 HTTP Redirect 301 redirect to HTTPS on 8443
5000 HTTP Dev only Flask development server (python app.py)
  • If SSL certificates are configured (Admin > Certificate), port 8443 serves HTTPS automatically
  • If no certificates exist, port 8443 serves plain HTTP
  • The HTTP redirect server on port 8080 runs alongside the main service

Systemd Services

Unit Type Description
ziochub.service Main Gunicorn application server (port 8443)
ziochub-redirect.service Main HTTP-to-HTTPS redirect (port 8080)
ziochub-cleaner.timer Timer Expired IOC cleanup (daily)
ziochub-backup.timer Timer Database + SSL + YARA backup (daily)
ziochub-misp-sync.timer Timer MISP IOC pull (interval set by admin) 
# Common commands
sudo systemctl status ziochub
sudo systemctl restart ziochub
sudo journalctl -u ziochub -f

UI Screens Overview

Live Stats

Real-time dashboard with IOC counts, Top Countries / TLDs / Email Domains leaderboards, and live IOC feed.

Search & Investigate

Full-text search across IOCs with filters (value, type, ticket, user, date, expiration status). Edit, delete, view history.

Submit IOCs

Single and bulk submission: auto-type detection, input cleaning (refanger), TTL, campaign assignment, allowlist validation. Bulk: CSV and TXT import with preview/staging, auto-detection, metadata extraction, and conflict handling.

Feed Pulse

Real-time feed health monitoring: incoming IOCs, outgoing (expired), deleted, sanity anomalies with exclude/un-exclude.

YARA Manager

Upload (file drop), Write (rule in browser with Prism live preview and Check syntax via yara-python), Status (pending + active repository). Preview/edit modals with Prism highlighting; approve/reject; quality scoring; campaign linking.

Champs Analysis

Analyst leaderboard with weighted scoring, streak bonuses, rank trends, team goals, activity spotlight, news ticker.

Campaign Graph

Interactive vis.js graph of campaigns linked to IOCs and YARA rules. Create, link, export to CSV.

Hunter's Playbook

Customizable quick-links panel for external investigation tools.

Intelligence Reports

Period-based reports (day/week/month): executive KPIs, type distribution, feed health score, analyst activity, comparisons vs. previous period. Export to PDF (html2canvas + jsPDF).

Profile & Change Password

User profile (display name, avatar, role description, email) and change-password flow; admins can enforce "must change password" for users.

Admin Panel

  • Users: Create, edit, deactivate users; avatar management; system users marked separately
  • Settings / Integrations: Auth mode (local/LDAP), LDAP, MISP, Syslog/CEF, DXL, outbound pushes, Cisco ESA, feeds-related options (see Settings page for feed/TAXII/cache)
  • Allowlist: Edit raw allowlist file (known-good / critical assets)
  • Certificate: SSL/TLS certificate upload for HTTPS
  • Scoring: Champs scoring method (Weighted, Flat, By Type, Campaign Focus, Time Decay, Quality, Goal-Based, Smart)

Feed Endpoints

Standard Feeds

Endpoint Content
/feed/ip IP addresses
/feed/domain Domains
/feed/url URLs (with protocol)
/feed/hash All hashes
/feed/md5 MD5 only
/feed/sha1 SHA1 only
/feed/sha256 SHA256 only

Palo Alto (EDL)

Endpoint Note
/feed/pa/ip Standard
/feed/pa/domain Standard
/feed/pa/url URLs without protocol
/feed/pa/md5, /sha1, /sha256 Standard

Checkpoint (CSV)

Endpoint Format
/feed/cp/ip, /feed/cp/domain, /feed/cp/url CSV with observe numbers
/feed/cp/hash All hashes (MD5, SHA-1, SHA-256, SHA-512)
/feed/cp/md5, /feed/cp/sha1, /feed/cp/sha256 Hash only for that algorithm; /feed/cp/sha2 is an alias for SHA-256 (same as /sha256)

YARA Feeds

Endpoint Description
/feed/yara-list List of approved YARA filenames
/feed/yara-content/<filename> Raw YARA rule content

TAXII 2.1 / STIX 2.1

ZIoCHub exposes active IOCs as a TAXII 2.1 server so TAXII clients (e.g. Cisco IronPort ESA) can pull STIX 2.1 indicators.

Endpoint Description
GET /taxii2/ TAXII 2.1 Discovery (server info, API roots)
GET /taxii2/ziochub/ API Root (collections list)
GET /taxii2/ziochub/collections/ Collections (single collection: indicators)
GET /taxii2/ziochub/collections/indicators/ Collection metadata
GET /taxii2/ziochub/collections/indicators/objects/ STIX 2.1 objects (paginated)
GET /taxii2/ziochub/collections/indicators/objects/<id>/ Single STIX object
GET /taxii2/ziochub/collections/indicators/manifests/ Manifests (paginated)
GET /taxii2/ziochub/collections/indicators/objects/<id>/versions/ Object versions

Requests must include Accept: application/taxii+json;version=2.1. Only active (non-expired) IOCs are included.

All plain-text and YARA feeds return only active (non-expired) IOCs. Content-Type: text/plain.

MISP Integration

ZIoCHub can pull IOC attributes from a local MISP instance automatically.

Configuration (Admin > Settings > MISP Integration)

Setting Description
MISP Sync Enabled Enable/disable automatic sync
MISP URL URL of the MISP instance
API Key MISP API authentication key
Verify SSL Verify MISP server certificate
Lookback (days) How far back to fetch attributes
Auto-sync interval (min) Pull frequency (minimum 5 minutes)
Filter by Tags Comma-separated MISP tags to filter
Filter Attribute Types Comma-separated MISP types (e.g. ip-src, domain, sha256)
Published Events Only Only fetch from published MISP events
Default TTL Expiration for imported IOCs (days, or permanent)
Sync Analyst Username Username recorded as analyst (default: misp_sync)
Exclude from Champs Hide MISP sync user from Champs leaderboard

Supported MISP Attribute Types

MISP Type ZIoCHub Type
ip-src, ip-dst, ip-src|port, ip-dst|port IP
domain, hostname Domain
url, uri, link URL
md5, sha1, sha256, sha512, ssdeep, imphash Hash
email-src, email-dst, email Email

How It Works

  1. Systemd timer triggers misp_sync_job.py every 5 minutes
  2. The job checks admin-configured interval and skips if not enough time has passed
  3. Fetches attributes from MISP via pymisp
  4. Validates each IOC (same regex validation as manual submission)
  5. Inserts new IOCs with deduplication, logs to ioc_history
  6. A DB-based lock prevents concurrent syncs (auto-expires after 10 minutes)

The misp_sync user is created as source='system' and cannot log in.

Cisco ESA dictionary sync

Optional integration to push Email, Domain, IP, and URL IOC values into Cisco Secure Email Gateway (AsyncOS) content dictionaries using the REST API (v2.0 style, as in the Cisco Secure Email API Guide).

Configuration (Admin > Integrations > Cisco ESA)

Area Description
Enable Turn sync on/off
API base URL No trailing slash; must include the API root (e.g. https://esa.example.com:6080/esa/api/v2.0)
Username / passphrase Plain credentials; the app Base64-encodes them for POST .../login and uses the returned JWT as the jwttoken header on later calls
Verify TLS Disable only if you use a private CA and accept the risk
API deployment level Standalone (device_type=esa only), Cluster (mode=cluster), Group (mode=group + Group name), or Machine (mode=machine + Host name). Must match your deployment.
Dictionary mappings One or more rows: exact dictionary name on the appliance + one IOC type (Email, Domain, IP, URL)
Skip MISP sync user Recommended: do not push IOCs whose analyst is the MISP sync user
Remove on expire If enabled, cleaner.py calls ESA DELETE before removing expired rows locally (same DB as the app)

Test connection calls login and GET .../config/dictionaries with the same query string as dictionary operations.

Behaviour (implemented in code)

  • Add: POST .../config/dictionaries/<name>/words?... with body {"data":{"words":[["term1"],["term2"],...]}} (Cisco add shape).
  • Remove: DELETE to the same path with body {"data":{"words":["term1","term2",...]}} (Cisco delete shape: flat string list).
  • Batching: Multiple IOCs in one bulk operation are grouped per dictionary; one login per push, then one POST (or DELETE) per dictionary with all relevant terms.
  • Triggers: Successful IOC create (including bulk/staging paths), manual revoke in Search, and expired IOC cleanup when the setting is on.

Settings are stored in system_settings (esa_* keys). There is no per-admin JSON templating; paths and bodies follow the documented API.

Data Model

SQLite database: data/ziochub.db

Table Description
users User accounts (username, password_hash, source, is_admin, is_active, must_change_password, last_login_at)
user_profiles Display name, avatar_path, role_description, email
user_sessions Login/logout tracking (IP, login_at, logout_at)
iocs IOC records (type, value, analyst, ticket_id, comment, expiration, campaign_id, tags, user_id, submission_method, country_code, tld, email_domain, rare_find_type)
ioc_history Lifecycle events per IOC (created, edited, deleted, expired, excluded, unexcluded); payload JSON
ioc_notes Analyst notes per IOC (ioc_type, ioc_value, user_id, content); keyed by type+value, survive deletion
campaigns Campaign metadata (name, description, dir ltr/rtl)
yara_rules YARA rule metadata, quality_points, status (pending/approved/rejected)
sanity_exclusions Analyst-excluded Feed Pulse anomalies (value, ioc_type, anomaly_type)
system_settings Key-value store (auth, LDAP, MISP, Cisco ESA esa_*, Champs, syslog UDP, IOC push, etc.)
activity_events Champs activity log (ioc_submit, yara_upload, rank_change, goal_progress, deletion)
team_goals Champs team goals (target, current, period, goal_type)
champ_rank_snapshots Daily rank snapshots for trend tracking

Configuration

Environment Variables

Variable Default Description
FLASK_PORT 5000 Dev server port
FLASK_DEBUG false Debug mode
FLASK_ENV (empty) Set to production in production; used with DEV_MODE for startup security warning
ZIOCHUB_ENV (empty) Alternative to FLASK_ENV; set to production in production
DEV_MODE 0 Do not enable in production (dev auto-login, LDAP mock). App logs warning when on, and error when on with production env
SECRET_KEY random Flask secret key (set in production)
ZIOCHUB_DATA_DIR <app>/data Data directory override
ZIOCHUB_MAX_CONTENT_MB 16 Max upload size
ADMIN_DEFAULT_PASSWORD admin Initial admin password
ZIOCHUB_PORT 8443 Production gunicorn port
ZIOCHUB_WORKERS 3 Gunicorn worker count
REDIRECT_HTTP_PORT 8080 HTTP redirect listen port
REDIRECT_HTTPS_PORT 8443 HTTPS redirect target port

Admin Settings (Web UI)

  • Auth Mode: local_only, ldap_only, ldap_with_local_fallback
  • LDAP: URL, Base DN, Bind DN, User Filter
  • MISP: URL, API key, filters, sync interval, TTL, Champs exclusion
  • Integrations (/admin/integrations): LDAP, MISP, Syslog, DXL, YARA push, IOC push, Cisco ESA dictionary sync
  • Syslog / CEF: Optional UDP syslog (host, port) for CEF audit events (also reachable from Integrations in some installs)
  • Champs: Scoring method (Admin > Scoring), ticker messages, team goals (set from Champs Analysis UI; monthly progress is calendar month)

Maintenance

Backup

Automated via ziochub-backup.timer (daily). Backs up:

  • ziochub.db (database)
  • data/ssl/*.pem (SSL certificates)
  • data/YARA/*.yar (YARA rules)
  • data/allowlist.txt

Retention: 30 days. Manual run:

sudo -u ziochub /opt/ziochub/backup_ziochub.sh

Expired IOC Cleanup

Automated via ziochub-cleaner.timer. Removes expired IOCs and logs each deletion to ioc_history with event_type='expired'. If Cisco ESA “remove on expire” is enabled, the cleaner removes matching dictionary words on the appliance before deleting rows (uses the same SQLite system_settings as the app).

Data Reset

cd /opt/ziochub
sudo systemctl stop ziochub

# Interactive (asks per category)
python reset_data.py

# Full wipe (IOCs, YARA, campaigns, history, champs, sessions, MISP settings, users)
python reset_data.py --all --yes

# Selective
python reset_data.py --iocs --yara --history --yes
python reset_data.py --settings   # Reset MISP settings only

sudo systemctl start ziochub

Lab User Setup

# Development (app on port 5000): run from project root, or use --env dev
python create_lab_users.py
python create_lab_users.py --env dev

# Production (app on port 8443): use --env prod (reads /opt/ziochub/users/users.json)
cd /opt/ziochub
sudo -u ziochub /opt/ziochub/venv/bin/python create_lab_users.py --env prod
# Or with password non-interactive:
sudo -u ziochub /opt/ziochub/venv/bin/python create_lab_users.py --env prod --password 'YourPassword'

Prompts for a common password (or use --password) and creates/updates users from users/users.json. For production, ensure users/users.json exists under /opt/ziochub/users/ (copy from dev or create there).

Reset Admin (or Any User) Password

If an admin or user is locked out or the password is forgotten, use the script under scripts/ (installed with the app):

cd /opt/ziochub
sudo -u ziochub python scripts/reset_admin_password.py --username admin --password 'NewSecurePassword'
# Optional: set source to local so the user can log in with the new password
sudo -u ziochub python scripts/reset_admin_password.py --username admin --password 'NewSecurePassword' --source local

Admin Scripts

When using a full deployment package (e.g. offline installer), the following scripts may be provided:

Script Description
setup.sh Production installer (online/offline/upgrade)
uninstall.sh Full removal (services, files, user)
package_offline.sh Build offline installer ZIP
backup_ziochub.sh Manual backup (DB, SSL, YARA, allowlist)
reset_data.py Wipe operational data (granular or full)
create_lab_users.py Create lab analyst accounts
cleaner.py Remove expired IOCs (runs via systemd timer)
misp_sync_job.py MISP sync job (runs via systemd timer)
http_redirect.py HTTP-to-HTTPS redirect server
start.sh Gunicorn launcher with auto SSL detection

Scripts in scripts/ (installed under /opt/ziochub/scripts/):

Script Description
scripts/reset_admin_password.py Reset any user's password and optionally set source=local (e.g. after LDAP lockout)

For development, only the application is required: python app.py (see Local Development).

Security

  • Authentication: All pages and API endpoints require login (Flask-Login)
  • Passwords: Scrypt hashing via werkzeug
  • LDAP: Optional AD/LDAP authentication with local fallback
  • SSL/TLS: Certificate upload via admin UI, gunicorn serves HTTPS directly
  • HTTP Redirect: Automatic 301 redirect from HTTP to HTTPS
  • System Users: MISP sync user has source='system', password_hash=None (cannot log in)
  • Input Validation: Regex validation on all IOC types, refanger for obfuscated input
  • SQL Injection: SQLAlchemy ORM with parameterized queries
  • Allowlist: Prevents blocking critical infrastructure assets
  • Audit Log: CEF format; local file with 48-hour rotation; optional UDP syslog (Admin > Settings)
  • Feed Endpoints: Public (no auth). Includes plain-text feeds and TAXII 2.1; restrict access via firewall rules
  • Offline: No external network calls. All assets local
  • DEV_MODE: Must not be enabled in production (enables dev auto-login and LDAP mock). In production set DEV_MODE=0 or unset; the app logs a startup warning when DEV_MODE is on, and an error when both DEV_MODE and production environment (FLASK_ENV=production or ZIOCHUB_ENV=production) are detected.

Project Architecture

Backend

app.py              Main Flask application
models.py           SQLAlchemy models
extensions.py       Flask extensions (db)
constants.py        Application constants (VERSION, IOC_FILES, limits)
config.py           Configuration (optional)

routes/
  admin.py          Admin API (users, settings, certificate, MISP, ESA test, allowlist)
  auth.py           Login, logout, profile, change password, LDAP health
  champs.py         Champs leaderboard, team goals (calendar month when monthly), ticker
  campaigns.py      Campaign CRUD and graph API
  feeds.py          Feed generation (standard, PA, CP, YARA; STIX helpers)
  ioc.py            IOC submit (single/bulk) API; schedules optional ESA dictionary add (background)
  reports.py        Intelligence reports (period-based stats, PDF export)
  search.py         Search, edit, delete, history API; schedules optional ESA remove on revoke
  stats.py          Live stats counts, geo/TLD/email intelligence
  taxii_server.py   TAXII 2.1 server (STIX 2.1 indicators collection)
  yara.py           YARA rule management API

utils/
  validation.py       IOC regex validation and type detection
  refanger.py         Input cleaning (defang reversal)
  allowlist.py        Allowlist loading and checking
  feed_helpers.py     Feed formatting helpers
  yara_utils.py       YARA file path safety, in-memory syntax validation (yara-python)
  validation_warnings.py   IOC submission warnings
  validation_messages.py   Error message constants
  sanity_checks.py    Feed Pulse anomaly detection
  auth.py             Password hashing (scrypt)
  decorators.py       @login_required, @admin_required
  ldap_auth.py        LDAP/AD authentication
  champs.py           Analyst scoring, ranking, badges, XP
  misp_sync.py        MISP fetch, validate, import, lock
  ioc_decode.py       Text extraction for bulk IOC parsing
  esa_dictionary.py   Cisco ESA AsyncOS dictionary sync (login, batched add/remove)
  cef_logger.py       CEF audit logging (local file + optional UDP syslog)
  cache.py            In-memory TTL cache (`get_cached` / `set_cached` / `delete_cached` / `delete_cached_prefix`)
  mentorship.py       SOC Mentorship Insights Engine (behavioral analysis, 45 rules)

Frontend

Single-Page Application in templates/index.html with lazy-loaded JS modules:

Module Purpose
static/js/api.js Centralized API client
static/js/utils.js HTML escaping, clipboard
static/js/app.js Tab routing, i18n, theme
static/js/live-stats.js Dashboard, charts, intelligence
static/js/search.js Search, edit, delete, history
static/js/submit.js Single/bulk IOC submission UI
static/js/champs.js Leaderboard, spotlight, ticker
static/js/feed-pulse.js Feed health, anomalies, exclusions
static/js/yara.js YARA management
static/js/campaigns.js Campaign graph (vis.js)
static/js/playbook.js / playbook-edit.js Playbook view and site management
static/js/reports.js Intelligence reports (period picker, charts, PDF export)
static/js/profile.js User profile and avatar

Vendor libraries (all local, no CDN): Tailwind, Chart.js, vis.js, marked, turndown, Prism, html2canvas, jsPDF. Python runtime also uses yara-python (see Open Source Projects Used).

Templates

templates/
  index.html          Main SPA (IOC & YARA Mgmt)
  login.html          Login page
  change_password.html  Forced password change
  profile.html        User profile (display name, avatar, role, email)
  base_app.html       (if used)
  admin/
    base.html         Admin layout
    users.html        User management
    settings.html     System settings (feeds, cache, search options, etc.)
    integrations.html LDAP, MISP, Syslog, DXL, pushes, **Cisco ESA**
    allowlist.html    Allowlist editor
    certificate.html SSL/TLS certificate
    scoring.html      Champs scoring method
    403.html          Forbidden

Offline Deployment

ZIoCHub is designed for air-gapped environments and is 100% offline: no CDN, no external scripts, no external stylesheets.

100% Offline - No CDN

  • No <script src="https://cdn.tailwindcss.com"> or any CDN. Tailwind is loaded from static/js/tailwind.min.js (local).
  • All <script> and <link> tags use url_for('static', filename='...') - everything is served from your server (e.g. tailwind.min.js, chart.min.js, vis-network.min.js, marked.min.js, prism.min.js, style.css, flag-icons.min.css, etc.).
  • i18n: Translation JSON files are loaded from static/i18n/ (e.g. en.json, he.json) via relative URLs.
  • Lazy-loaded tab scripts (champs, search, yara, feed-pulse, campaigns, reports) are all under static/js/ - no external URLs.
  • No fetch() or XHR to external domains - only same-origin calls to /api/... and /static/....
  • Hunter's Playbook default entries contain URLs (e.g. VirusTotal, OTX) as link data only; the app does not fetch them. If the user clicks a link, the browser may try to open it (in air-gap, that will fail unless you have a proxy).
  • GeoIP database is local: data/GeoLite2-City.mmdb.
  • LDAP is optional; works with local_only auth.
  • MISP integration talks only to a local MISP instance (same server / internal network).
  • No telemetry, analytics, or external API calls.

Building an Offline Package

# On a machine with internet:
./package_offline.sh

# Output: ziochub_installer.zip
# Contains: all code (app, utils/, routes/ incl. TAXII 2.1, scripts/), templates, static assets, Python wheels, systemd units

Installing on Air-Gapped Server

unzip ziochub_installer.zip -d ziochub_install
cd ziochub_install
sudo ./setup.sh --offline

Prerequisites on target server:

  • Python 3.8+ with python3-venv package
  • SQLite3
  • systemd

Troubleshooting

Where to find logs

Where What
systemd journal All app output (gunicorn, Flask, errors). Main place to debug.
CEF audit file Optional audit events (logins, avatar upload, etc.).

Commands:

# Last 100 lines (errors, tracebacks, access)
journalctl -u ziochub -n 100 --no-pager

# Follow live
journalctl -u ziochub -f

# Today only
journalctl -u ziochub --since today --no-pager

Audit log (if enabled): /opt/ziochub/data/audit_cef.log (48-hour rotation).

Avatar upload / "Network error"

The UI shows "Network error" when the request to upload an avatar fails (e.g. server error, timeout, or no response). Check the server side:

  1. Logs: See Where to find logs above. After a failed upload, run:

    journalctl -u ziochub -n 50 --no-pager

    Look for api_profile_avatar_upload failed or api_admin_user_avatar_upload failed and the Python traceback below.

  2. Permissions: The app (user ziochub) must be able to write to static/avatars/:

    ls -la /opt/ziochub/static/avatars
sudo chown -R ziochub:ziochub /opt/ziochub/static/avatars
sudo chmod 755 /opt/ziochub/static/avatars

  3. File size: Max upload is 16 MB by default (MAX_CONTENT_LENGTH). Very large images may be rejected with 413.

  4. Lab users’ avatars: If you used create_lab_users.py, avatars are copied from users/ only when image files exist there (e.g. users/alice.jpg). To add/update avatars after creation, either run the script again with images in users/, or use the UI (Profile or Admin > Users) to upload; if the UI shows "Network error", use the logs and permissions steps above.

Service won't start

journalctl -u ziochub -n 50 --no-pager

Database locked

  • Ensure only one instance is running
  • Restart: sudo systemctl restart ziochub
  • The app retries commits automatically (3 attempts with backoff)

MISP sync not running

systemctl status ziochub-misp-sync.timer
journalctl -u ziochub-misp-sync -n 20 --no-pager

Check Admin > Settings > MISP: ensure Enabled = Yes, URL and API key configured.

HTTP redirect not working

systemctl status ziochub-redirect

Ensure SSL certificates are uploaded via Admin > Certificate, then restart:

sudo systemctl restart ziochub ziochub-redirect

Reset all data

sudo systemctl stop ziochub
cd /opt/ziochub
python reset_data.py --all --yes
sudo systemctl start ziochub

Full reinstall

sudo ./uninstall.sh --backup   # Saves data to /opt/ziochub_backup_*
sudo ./setup.sh --offline      # Fresh install

Version

ZIoCHub v2.0 Beta - IOC & YARA Mgmt
Single source of version: constants.py → VERSION (used in UI and docs).
Last updated: April 2026

About

ziochub - An open-source IOC & YARA Management Platform

zwerd.github.io/posts/BuildingZIoCHub/

Topics

platform ioc yara-rules threat-intelligence cyber-threat-intelligence cybersecurity-tools ioc-analysis ioc-management

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages