Implement async GenAI module with socketpair-based non-blocking architecture

genai_prototype/.gitignore

@@ -0,0 +1,26 @@
+# Build artifacts
+genai_demo
+genai_demo_event
+*.o
+*.oo
+
+# Debug files
+*.dSYM/
+*.su
+*.idb
+*.pdb
+
+# Editor files
+*~
+.*.swp
+.vscode/
+.idea/
+
+# Core dumps
+core
+core.*
+
+# Temporary files
+*.tmp
+*.temp
+*.log

genai_prototype/Makefile

@@ -0,0 +1,83 @@
+# Makefile for GenAI Prototype
+# Standalone prototype for testing GenAI module architecture
+
+CXX = g++
+CXXFLAGS = -std=c++17 -Wall -Wextra -O2 -g
+LDFLAGS = -lpthread -lcurl
+CURL_CFLAGS = $(shell curl-config --cflags)
+CURL_LDFLAGS = $(shell curl-config --libs)
+
+# Target executables
+TARGET_THREAD = genai_demo
+TARGET_EVENT = genai_demo_event
+TARGETS = $(TARGET_THREAD) $(TARGET_EVENT)
+
+# Source files
+SOURCES_THREAD = genai_demo.cpp
+SOURCES_EVENT = genai_demo_event.cpp
+
+# Object files
+OBJECTS_THREAD = $(SOURCES_THREAD:.cpp=.o)
+OBJECTS_EVENT = $(SOURCES_EVENT:.cpp=.o)
+
+# Default target (build both demos)
+all: $(TARGETS)
+
+# Individual demo targets
+genai_demo: genai_demo.o
+	@echo "Linking genai_demo..."
+	$(CXX) genai_demo.o $(LDFLAGS) -o genai_demo
+	@echo "Build complete: genai_demo"
+
+genai_demo_event: genai_demo_event.o
+	@echo "Linking genai_demo_event..."
+	$(CXX) genai_demo_event.o $(CURL_LDFLAGS) $(LDFLAGS) -o genai_demo_event
+	@echo "Build complete: genai_demo_event"
+
+# Compile source files
+genai_demo.o: genai_demo.cpp
+	@echo "Compiling $<..."
+	$(CXX) $(CXXFLAGS) -c $< -o $@
+
+genai_demo_event.o: genai_demo_event.cpp
+	@echo "Compiling $<..."
+	$(CXX) $(CXXFLAGS) $(CURL_CFLAGS) -c $< -o $@
+
+# Run the demos
+run: $(TARGET_THREAD)
+	@echo "Running thread-based GenAI demo..."
+	./$(TARGET_THREAD)
+
+run-event: $(TARGET_EVENT)
+	@echo "Running event-based GenAI demo..."
+	./$(TARGET_EVENT)
+
+# Clean build artifacts
+clean:
+	@echo "Cleaning..."
+	rm -f $(OBJECTS_THREAD) $(OBJECTS_EVENT) $(TARGETS)
+	@echo "Clean complete"
+
+# Rebuild
+rebuild: clean all
+
+# Debug build with more warnings
+debug: CXXFLAGS += -DDEBUG -Wpedantic
+debug: clean all
+
+# Help target
+help:
+	@echo "GenAI Prototype Makefile"
+	@echo ""
+	@echo "Targets:"
+	@echo "  all          - Build both demos (default)"
+	@echo "  genai_demo   - Build thread-based demo"
+	@echo "  genai_demo_event - Build event-based demo"
+	@echo "  run          - Build and run thread-based demo"
+	@echo "  run-event    - Build and run event-based demo"
+	@echo "  clean        - Remove build artifacts"
+	@echo "  rebuild      - Clean and build all"
+	@echo "  debug        - Build with debug flags and extra warnings"
+	@echo "  help         - Show this help message"
+
+.PHONY: all run run-event clean rebuild debug help

genai_prototype/README.md

@@ -0,0 +1,139 @@
+# GenAI Module Prototype
+
+Standalone prototype demonstrating the GenAI module architecture for ProxySQL.
+
+## Architecture Overview
+
+This prototype demonstrates a thread-pool based GenAI module that:
+
+1. **Receives requests** from multiple clients (MySQL/PgSQL threads) via socket pairs
+2. **Queues requests** internally with a fixed-size worker thread pool
+3. **Processes requests asynchronously** without blocking the clients
+4. **Returns responses** to clients via the same socket connections
+
+### Components
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                   GenAI Module                          │
+│                                                          │
+│  ┌────────────────────────────────────────────────┐    │
+│  │  Listener Thread (epoll-based)                 │    │
+│  │  - Monitors all client file descriptors        │    │
+│  │  - Reads incoming requests                     │    │
+│  │  - Pushes to request queue                     │    │
+│  └──────────────────┬─────────────────────────────┘    │
+│                     │                                  │
+│                     ▼                                  │
+│  ┌────────────────────────────────────────────────┐    │
+│  │  Request Queue                                 │    │
+│  │  - Thread-safe queue                           │    │
+│  │  - Condition variable for worker notification  │    │
+│  └──────────────────┬─────────────────────────────┘    │
+│                     │                                  │
+│                     ▼                                  │
+│  ┌────────────────────────────────────────────────┐    │
+│  │  Thread Pool (configurable number of workers)  │    │
+│  │  ┌──────┐  ┌──────┐  ┌──────┐  ┌──────┐      │    │
+│  │  │Worker│  │Worker│  │Worker│  │Worker│  ...  │    │
+│  │  └───┬──┘  └───┬──┘  └───┬──┘  └───┬──┘        │    │
+│  │      └──────────┴──────────┴──────────┘         │    │
+│  └────────────────────────────────────────────────┘    │
+└─────────────────────────────────────────────────────────┘
+         ▲                    │                    ▲
+         │                    │                    │
+    socketpair()         Responses           socketpair()
+    from clients          to clients          from clients
+```
+
+### Communication Protocol
+
+**Client → GenAI (Request)**:
+```cpp
+struct RequestHeader {
+    uint64_t request_id;     // Client's correlation ID
+    uint32_t operation;      // 0=embedding, 1=completion, 2=rag
+    uint32_t input_size;     // Size of following data
+    uint32_t flags;          // Reserved
+};
+// Followed by input_size bytes of input data
+```
+
+**GenAI → Client (Response)**:
+```cpp
+struct ResponseHeader {
+    uint64_t request_id;     // Echo client's ID
+    uint32_t status_code;    // 0=success, >0=error
+    uint32_t output_size;    // Size of following data
+    uint32_t processing_time_ms;  // Time taken to process
+};
+// Followed by output_size bytes of output data
+```
+
+## Building and Running
+
+```bash
+# Build
+make
+
+# Run
+make run
+
+# Clean
+make clean
+
+# Debug build
+make debug
+
+# Show help
+make help
+```
+
+## Current Status
+
+**Implemented:**
+- ✅ Thread pool with configurable workers
+- ✅ epoll-based listener thread
+- ✅ Thread-safe request queue
+- ✅ socketpair communication
+- ✅ Multiple concurrent clients
+- ✅ Non-blocking async operation
+- ✅ Simulated processing (random sleep)
+
+**TODO (Enhancement Phase):**
+- ⬜ Real LLM API integration (OpenAI, local models)
+- ⬜ Request batching for efficiency
+- ⬜ Priority queue for urgent requests
+- ⬜ Timeout and cancellation
+- ⬜ Backpressure handling (queue limits)
+- ⬜ Metrics and monitoring
+- ⬜ Error handling and retry logic
+- ⬜ Configuration file support
+- ⬜ Unit tests
+- ⬜ Performance benchmarking
+
+## Integration Plan
+
+Phase 1: **Prototype Enhancement** (Current)
+- Complete TODO items above
+- Test with real LLM APIs
+- Performance testing
+
+Phase 2: **ProxySQL Integration**
+- Integrate into ProxySQL build system
+- Add to existing MySQL/PgSQL thread logic
+- Implement GenAI variable system
+
+Phase 3: **Production Features**
+- Connection pooling
+- Request multiplexing
+- Caching layer
+- Fallback strategies
+
+## Design Principles
+
+1. **Zero Coupling**: GenAI module doesn't know about client types
+2. **Non-Blocking**: Clients never wait on GenAI responses
+3. **Scalable**: Fixed resource usage (bounded thread pool)
+4. **Observable**: Easy to monitor and debug
+5. **Testable**: Standalone, independent testing