docs(eldritch): add standard library summaries and support UI tooltips

docs/_docs/user-guide/eldritch.md

@@ -348,6 +348,8 @@ for user_home_dir in file.list("/home/"):
 
 ## Agent
 
+The `agent` library provides functions for meta-style interactions with the agent itself. It allows you to inspect its configuration, check transport details, or list and manage background tasks.
+
 ### agent._terminate_this_process_clowntown
 
 `agent._terminate_this_process_clowntown() -> None`
@@ -418,6 +420,8 @@ assumptions on `Transport` requirements no checks are applied to the passed stri
 
 ## Assets
 
+The `assets` library is used to interact with embedded files stored natively within the agent. It provides capabilities to list, read, or extract these files to disk for further execution or deployment.
+
 ### assets.copy
 
 `assets.copy(src: str, dst: str) -> None`
@@ -457,6 +461,8 @@ The **assets.read** method returns a UTF-8 string representation of the asset fi
 
 ## Crypto
 
+The `crypto` library offers functionalities to encrypt, decrypt, and hash data. It includes support for algorithms like AES, MD5, SHA1, and SHA256, as well as helpers for base64 encoding and JSON parsing.
+
 ### crypto.aes_decrypt
 
 `crypto.aes_decrypt(key: Bytes, iv: Bytes, data: Bytes) -> Bytes`
@@ -580,6 +586,8 @@ The **crypto.sha256** method calculates the SHA256 hash of the provided data.
 
 ## File
 
+The `file` library gives you comprehensive control to interact with files and directories on the host system. It includes methods for reading, writing, moving, copying, and compressing files, as well as searching and timestomping.
+
 ### file.append
 
 `file.append(path: str, content: str) -> None`
@@ -820,6 +828,8 @@ The **file.find** method finds all files matching the used parameters. Returns f
 
 ## HTTP
 
+The `http` library allows the agent to send HTTP and HTTPS requests over the network. You can download files, submit form data, or interact with external REST APIs directly from the agent.
+
 The HTTP library also allows the user to allow the http client to ignore TLS validation via the `allow_insecure` optional parameter (defaults to `false`).
 
 ### http.download
@@ -844,6 +854,8 @@ The **http.post** method sends an HTTP POST request to the URI specified in `uri
 
 ## Pivot
 
+The `pivot` library provides tools to identify and move laterally between systems on a network. It includes functionalities like port scanning, reverse shells, port forwarding, and executing commands remotely over SMB or SSH.
+
 ### pivot.arp_scan
 
 `pivot.arp_scan(target_cidrs: List<str>) -> List<str>`
@@ -987,6 +999,8 @@ Status will be equal to the code returned by the command being run and -1 in the
 
 ## Process
 
+The `process` library is used to interact with running processes on the local system. It provides functionalities to list processes, gather detailed information, enumerate network sockets, or terminate specific processes.
+
 ### process.info
 
 `process.info(pid: Optional<int>) -> Dict`
@@ -1089,7 +1103,7 @@ _Currently only shows LISTENING TCP connections_
 
 ## Random
 
-The random library is designed to enable generation of cryptographically secure random values. None of these functions will be blocking.
+The `random` library is designed to enable generation of cryptographically secure random values without blocking execution. It allows you to create random booleans, integers, bytes, strings, and UUIDs.
 
 ### random.bool
 
@@ -1124,8 +1138,7 @@ The **random.uuid** method returns a randomly generated UUID (v4).
 
 ## Regex
 
-The regex library is designed to enable basic regex operations on strings. Be aware as the underlying implementation is written
-in Rust we rely on the Rust Regex Syntax as talked about [here](https://rust-lang-nursery.github.io/rust-cookbook/text/regex.html). Further, we only support a single capture group currently, defining more/less than one will cause the tome to error.
+The `regex` library provides regular expression capabilities for operating on strings. Using Rust's regex syntax, you can match, extract, or replace substrings within larger text blocks. Be aware as the underlying implementation is written in Rust we rely on the Rust Regex Syntax as talked about [here](https://rust-lang-nursery.github.io/rust-cookbook/text/regex.html). Further, we only support a single capture group currently, defining more/less than one will cause the tome to error.
 
 ### regex.match_all
 
@@ -1156,7 +1169,7 @@ The **regex.replace** method returns the given haystack with the first capture g
 
 ## Report
 
-The report library is designed to enable reporting structured data to Tavern. Its API is still in the active development phase, so **future versions of Eldritch may break tomes that rely on this API**.
+The `report` library is designed to enable reporting structured data to Tavern. You can use it to securely exfiltrate files, process lists, captured credentials, or screenshots from the host. Its API is still in the active development phase, so **future versions of Eldritch may break tomes that rely on this API**.
 
 ### report.file
 
@@ -1198,6 +1211,8 @@ Reports a screenshot of all screens to Tavern.
 
 ## Sys
 
+The `sys` library offers general system capabilities to retrieve context about the host environment. It provides functionalities to check the operating system, retrieve environment variables, query the registry, and run native shell commands.
+
 ### sys.dll_inject
 
 `sys.dll_inject(dll_path: str, pid: int) -> None`
@@ -1518,6 +1533,8 @@ True
 
 ## Time
 
+The `time` library contains general functions for obtaining and formatting the current system time. It allows you to convert between timestamps and readable strings, or introduce execution delays using sleep.
+
 ### time.format_to_epoch
 
 `time.format_to_epoch(input: str, format: str) -> int`

fix_test.js

@@ -0,0 +1,15 @@
+const fs = require('fs');
+
+const path = 'tavern/internal/www/src/pages/shellv2/hooks/shellUtils.ts';
+let content = fs.readFileSync(path, 'utf8');
+
+// The issue is that the builtins are joined to form a regex: `(\b(?:sys|sys\.shell|...)\b)`
+// And because it's a pipe without sorting by length descending, `sys` matches before `sys.shell`!
+// We need to sort `builtins` by length descending before joining.
+
+content = content.replace(
+    'const builtins = Object.keys(docsData).map(k => k.replace(/\\./g, "\\\\."));',
+    'const builtins = Object.keys(docsData).sort((a, b) => b.length - a.length).map(k => k.replace(/\\./g, "\\\\."));'
+);
+
+fs.writeFileSync(path, content);

tavern/internal/www/scripts/generate-docs.js

@@ -17,21 +17,67 @@ try {
     const content = fs.readFileSync(inputPath, 'utf-8');
     const docs = {};
 
-    // Simple parsing logic
-    // We look for "### function_name"
-    // Then capture the next code block as signature
-    // Then capture the text until the next header or end of file as description
-
     const lines = content.split('\n');
     let currentFunction = null;
+    let currentLibrary = null;
     let currentSignature = '';
     let currentDescription = [];
     let capturingDescription = false;
+    let inLibrarySection = false;
 
     for (let i = 0; i < lines.length; i++) {
         const line = lines[i].trim();
 
+        if (line.startsWith('## ')) {
+            // New library section
+            const match = line.match(/^##\s+(\w+)/);
+            if (match) {
+                const libName = match[1].toLowerCase();
+
+                const ignoreHeaders = ['examples', 'data', 'error', 'built-ins', 'standard'];
+                if (!ignoreHeaders.includes(libName)) {
+                    // Save previous function if any
+                    if (currentFunction) {
+                        docs[currentFunction] = {
+                            signature: currentSignature,
+                            description: currentDescription.join('\n').trim()
+                        };
+                        currentFunction = null;
+                    }
+
+                    currentLibrary = libName;
+                    inLibrarySection = true;
+                    currentDescription = [];
+                    capturingDescription = true;
+                } else {
+                    inLibrarySection = false;
+                    currentLibrary = null;
+                }
+            } else {
+                inLibrarySection = false;
+                currentLibrary = null;
+            }
+            continue;
+        }
+
+        if (inLibrarySection && !line.startsWith('### ')) {
+            if (line.length > 0) {
+                currentDescription.push(line);
+            }
+            continue;
+        }
+
         if (line.startsWith('### ')) {
+            // If we were capturing a library description, save it
+            if (inLibrarySection && currentLibrary) {
+                docs[currentLibrary] = {
+                    signature: currentLibrary, // Use a dummy signature or just name
+                    description: currentDescription.join('\n').trim()
+                };
+                inLibrarySection = false;
+                currentLibrary = null;
+            }
+
             // New function found, save previous one if exists
             if (currentFunction) {
                 docs[currentFunction] = {
@@ -41,59 +87,48 @@ try {
             }
 
             // Start new function
-            // format: ### agent.get_config
             const match = line.match(/^###\s+([\w\.]+)/);
             if (match) {
                 currentFunction = match[1];
                 currentSignature = '';
                 currentDescription = [];
                 capturingDescription = false;
             } else {
-                currentFunction = null; // invalid header format or not a function header we care about
+                currentFunction = null;
             }
             continue;
         }
 
         if (currentFunction) {
-            // Try to capture signature
-            // It might be in inline code `...` or block ```python ... ```
-            // The markdown seems to use inline code `function() -> type` right after header mostly
-
             if (!currentSignature && line.startsWith('`') && line.endsWith('`')) {
                 currentSignature = line.slice(1, -1);
                 capturingDescription = true;
                 continue;
             }
 
-            // Handle multiline code block for signature if present (rare in the snippet I saw but possible)
-            // The snippet shows `function` so I will stick to that first.
-            // If the signature is missing, maybe it is in the description?
-
-            // If we have signature, subsequent text is description
             if (capturingDescription) {
-                 // Stop capturing if we hit another header (handled by the loop start)
-                 // Just append lines
                  if (line.length > 0) {
                      currentDescription.push(line);
                  }
-            } else if (!currentSignature && line.length > 0) {
-                 // If we haven't found a signature yet but found text, assume signature is missing or in a different format
-                 // For now, let's just treat it as description start if it doesn't look like code
-                 // But looking at the file, signature is almost always immediately after header in backticks
             }
         }
     }
 
-    // Add the last function
-    if (currentFunction) {
+    // Add the last function or library
+    if (inLibrarySection && currentLibrary) {
+        docs[currentLibrary] = {
+            signature: currentLibrary,
+            description: currentDescription.join('\n').trim()
+        };
+    } else if (currentFunction) {
         docs[currentFunction] = {
             signature: currentSignature,
             description: currentDescription.join('\n').trim()
         };
     }
 
     fs.writeFileSync(outputPath, JSON.stringify(docs, null, 2));
-    console.log(`Successfully generated docs for ${Object.keys(docs).length} functions.`);
+    console.log(`Successfully generated docs for ${Object.keys(docs).length} items.`);
 
 } catch (error) {
     console.error('Error generating docs:', error);