Java SQL Engine

A foundational project to build a lightweight, relational database engine from scratch in Java. The primary goal is to gain a deep, systems-level understanding of database internals, including storage, indexing, concurrency, and recovery.

This repository also contains a detailed developer log that chronicles the design decisions, challenges, and learning process throughout the project's development.

🏛️ Architecture Overview

The engine is designed with a classic, layered database architecture. Each layer is responsible for a specific set of tasks, abstracting the complexity of the layers below it.

• View the High-Level System Architecture
• See the SELECT Query Execution Flow
• Explore the Low-Level Slotted Page Design

Client -> Parser -> Planner -> Executor -> Transaction & Storage Manager

• SQL Parser: Converts SQL strings into an Abstract Syntax Tree (using ANTLR).
• Query Planner: Translates the AST into a logical plan of operators.
• Execution Engine: Executes the plan (e.g., SeqScan, IndexScan).
• Transaction Manager: Ensures ACID properties using locking and logging.
• Storage Manager: Manages data on disk through a buffer pool.
  • Buffer Manager: Caches disk pages in memory using an LRU policy.
  • Disk Manager: Handles the physical I/O of reading/writing pages to table files.
• Index Manager: Manages B+-Tree indexes for fast lookups.

✅ Project Roadmap

• Phase 1: The Storage Layer

  • Page Class and On-Disk Representation
  • DiskManager for Raw Page I/O
  • BufferPoolManager with LRU Caching
  • SlottedPage and Tuple Storage
  • HeapFile Table Abstraction

• Phase 2: Basic Query Execution

  • SQL Parser (ANTLR) Setup
  • SeqScan (Sequential Scan) Executor
  • The Volcano/Iterator Execution Model

• Phase 3: Core Query Processing Engine

  • Expand SQL Grammar (Projections, Filters, Insert, Delete)
  • ProjectionExecutor (SELECT column1, ...)
  • FilterExecutor (WHERE ...)
  • InsertExecutor (INSERT INTO ...)
  • DeleteExecutor (DELETE FROM ...)
  • Simple Query Planner (AST to Executor Tree)

• Phase 4: Indexing for Performance

  • B+-Tree Implementation
  • IndexScanExecutor
  • Planner Integration for Index Selection

• Phase 5: Concurrency & Recovery

  • Transaction Manager
  • Write-Ahead Logging (WAL)
  • Strict Two-Phase Locking (2PL)

🚀 Getting Started

This project is built using Java 21 and Maven.

# Clone the repository
git clone https://github.com/lokiT26/java-sql-engine.git

# Navigate into the project directory
cd java-sql-engine

# Build the project and run tests
mvn clean install