feat: add tos evaluator skill

docs/skills/README.md

@@ -38,6 +38,7 @@ Enforces privacy, guardrails, and secure handling of sensitive data before it re
 | :--- | :--- | :--- |
 | **[PII Masker](pii_masker.md)** | `compliance/pii_masker` | High-precision, local PII (Personally Identifiable Information) detection and redaction using the micro-f1-mask model. |
 | **[MiCA Module](mica_module.md)** | `compliance/mica_module` | Self-contained local Policy Enforcement and RAG engine strictly adhering to MiCA crypto-asset regulation. |
+| **[Terms of Service Evaluator](tos_evaluator.md)** | `compliance/tos_evaluator` | Local-first evaluation of robots.txt and website legal pages to decide whether an intended automated action appears permissible. |
 
 ---
 

docs/skills/tos_evaluator.md

@@ -0,0 +1,89 @@
+# Terms of Service Evaluator
+
+**Domain:** `compliance`
+**Skill ID:** `compliance/tos_evaluator`
+
+A local-first compliance guardrail that checks whether an intended automated action appears permissible on a target website. It evaluates `robots.txt`, discovers candidate legal pages, extracts relevant clauses, and can optionally use a low-cost LLM to interpret ambiguous policy language.
+
+## What It Checks
+
+1. `robots.txt` rules for the exact target URL and user-agent.
+2. Likely Terms, Legal, Acceptable Use, and API policy pages on the same site.
+3. Clauses related to scraping, crawling, indexing, monitoring, downloading, and API-only access.
+4. Optional LLM-backed clause review when local heuristics cannot confidently classify the policy language.
+
+## Manifest Details
+
+**Parameters Schema:**
+* `target_url` (string): Full URL the agent intends to access.
+* `intended_action` (string): Natural-language action such as `scrape pricing data` or `index docs`.
+* `user_agent` (string, optional): User-agent used for `robots.txt` checks.
+* `fetch_mode` (string, optional): `lightweight` or `deep`.
+* `use_llm_evaluator` (boolean, optional): Enables optional clause interpretation for low-confidence cases.
+* `llm_provider` (string, optional): Provider name for the optional evaluator.
+* `llm_model` (string, optional): Model name such as `gemini-2.5-flash-lite`.
+* `assume_authenticated_session` (boolean, optional): Helps represent paid or logged-in usage contexts.
+* `max_terms_pages` (integer, optional): Caps discovery breadth.
+
+**Outputs Schema:**
+* `is_safe_to_proceed` (boolean): Whether the action was approved.
+* `confidence_score` (number): Confidence in the verdict.
+* `verdict` (string): `SAFE`, `UNSAFE`, `CAUTION`, or `INSUFFICIENT_EVIDENCE`.
+* `reason` (string): Short explanation of the verdict.
+* `robots_assessment` (object): Structured `robots.txt` result.
+* `tos_assessment` (object): Structured policy discovery and clause result.
+* `llm_assessment` (object): Optional evaluator result.
+* `evidence` (array): Supporting snippets and sources.
+
+## Verdict Semantics
+
+* `SAFE`: strong evidence suggests the requested action is allowed, and `robots.txt` does not block it.
+* `UNSAFE`: `robots.txt` blocks the path or discovered policy text explicitly restricts the automation.
+* `CAUTION`: the site may allow access, but only with conditions such as API usage, permission, or strict rate limits.
+* `INSUFFICIENT_EVIDENCE`: the evaluator could not find enough trustworthy evidence to safely approve the action.
+
+## Example Usage (Direct)
+
+```python
+from skillware.core.loader import SkillLoader
+
+bundle = SkillLoader.load_skill("compliance/tos_evaluator")
+TOSEvaluatorSkill = bundle["module"].TOSEvaluatorSkill
+skill = TOSEvaluatorSkill()
+
+result = skill.execute(
+    {
+        "target_url": "https://hackernoon.com/tagged/ai",
+        "intended_action": "crawl tagged article pages for research indexing",
+        "use_llm_evaluator": True,
+        "llm_provider": "gemini",
+        "llm_model": "gemini-2.5-flash-lite",
+    }
+)
+
+print(result["verdict"])
+print(result["reason"])
+```
+
+## Gemini Example
+
+Use the skill through `SkillLoader.to_gemini_tool(...)` and pass the skill instructions as the `system_instruction`. See `examples/gemini_tos_evaluator.py`.
+
+## Claude Example
+
+Use the skill through `SkillLoader.to_claude_tool(...)` and return the structured result back to Claude as a tool result. See `examples/claude_tos_evaluator.py`.
+
+## Ollama Example
+
+Use the text-based prompt adapter from `SkillLoader.to_ollama_prompt(...)` and execute the skill locally when the model emits a JSON tool block. See `examples/ollama_tos_evaluator.py`.
+
+## Notes
+
+This skill is a practical operational safeguard, not legal counsel. If the result is `CAUTION` or `INSUFFICIENT_EVIDENCE`, the safe default is manual review or an official API/developer integration path.
+
+To run tests specifically for this skill:
+
+```bash
+pytest tests/skills/compliance/test_tos_evaluator.py
+pytest skills/compliance/tos_evaluator/test_skill.py
+```

examples/claude_tos_evaluator.py

@@ -0,0 +1,68 @@
+import json
+import os
+
+import anthropic
+
+from skillware.core.env import load_env_file
+from skillware.core.loader import SkillLoader
+
+load_env_file()
+
+bundle = SkillLoader.load_skill("compliance/tos_evaluator")
+print(f"Loaded Skill: {bundle['manifest']['name']}")
+
+TOSEvaluatorSkill = bundle["module"].TOSEvaluatorSkill
+tos_skill = TOSEvaluatorSkill()
+
+client = anthropic.Anthropic(api_key=os.environ.get("ANTHROPIC_API_KEY"))
+tools = [SkillLoader.to_claude_tool(bundle)]
+
+user_query = (
+    "Can I use an automated crawler against https://hackernoon.com/tagged/devops "
+    "for research indexing? Check first."
+)
+print(f"User: {user_query}")
+
+message = client.messages.create(
+    model="claude-3-5-sonnet-latest",
+    max_tokens=1024,
+    system=bundle["instructions"],
+    messages=[{"role": "user", "content": user_query}],
+    tools=tools,
+)
+
+if message.stop_reason == "tool_use":
+    tool_use = next(block for block in message.content if block.type == "tool_use")
+    tool_name = tool_use.name
+    tool_input = tool_use.input
+
+    print(f"Claude requested tool: {tool_name}")
+    print(f"Input: {tool_input}")
+
+    if tool_name == "compliance/tos_evaluator":
+        result = tos_skill.execute(tool_input)
+        print(json.dumps(result, indent=2))
+
+        response = client.messages.create(
+            model="claude-3-5-sonnet-latest",
+            max_tokens=1024,
+            system=bundle["instructions"],
+            tools=tools,
+            messages=[
+                {"role": "user", "content": user_query},
+                {"role": "assistant", "content": message.content},
+                {
+                    "role": "user",
+                    "content": [
+                        {
+                            "type": "tool_result",
+                            "tool_use_id": tool_use.id,
+                            "content": json.dumps(result),
+                        }
+                    ],
+                },
+            ],
+        )
+
+        print("\nFinal Response:")
+        print(response.content[0].text)

examples/gemini_tos_evaluator.py

@@ -0,0 +1,61 @@
+import json
+import os
+
+import google.generativeai as genai
+
+from skillware.core.env import load_env_file
+from skillware.core.loader import SkillLoader
+
+load_env_file()
+
+bundle = SkillLoader.load_skill("compliance/tos_evaluator")
+print(f"Loaded Skill: {bundle['manifest']['name']}")
+
+TOSEvaluatorSkill = bundle["module"].TOSEvaluatorSkill
+tos_skill = TOSEvaluatorSkill()
+
+genai.configure(api_key=os.environ.get("GOOGLE_API_KEY"))
+tools = [SkillLoader.to_gemini_tool(bundle)]
+
+model = genai.GenerativeModel(
+    "gemini-2.5-flash-lite",
+    tools=tools,
+    system_instruction=bundle["instructions"],
+)
+
+chat = model.start_chat(enable_automatic_function_calling=True)
+user_query = (
+    "Before scraping Hackernoon tagged AI pages, check whether automated crawling "
+    "appears allowed for https://hackernoon.com/tagged/ai."
+)
+print(f"User: {user_query}")
+
+response = chat.send_message(user_query)
+while response.candidates and response.candidates[0].content.parts:
+    part = response.candidates[0].content.parts[0]
+    if not part.function_call:
+        break
+
+    fn_name = part.function_call.name
+    fn_args = dict(part.function_call.args)
+    print(f"Gemini requested tool: {fn_name}")
+    print(f"Input: {fn_args}")
+
+    if fn_name == "compliance/tos_evaluator":
+        result = tos_skill.execute(fn_args)
+        print(json.dumps(result, indent=2))
+        response = chat.send_message(
+            [
+                {
+                    "function_response": {
+                        "name": fn_name,
+                        "response": {"result": result},
+                    }
+                }
+            ]
+        )
+    else:
+        break
+
+print("\nFinal Response:")
+print(response.text)

examples/ollama_tos_evaluator.py

@@ -0,0 +1,71 @@
+import json
+import re
+
+import ollama
+
+from skillware.core.env import load_env_file
+from skillware.core.loader import SkillLoader
+
+load_env_file()
+
+bundle = SkillLoader.load_skill("compliance/tos_evaluator")
+TOSEvaluatorSkill = bundle["module"].TOSEvaluatorSkill
+tos_skill = TOSEvaluatorSkill()
+
+tool_description = SkillLoader.to_ollama_prompt(bundle)
+tool_description += f"\n**Cognitive Instructions:**\n{bundle['instructions']}\n"
+
+system_prompt = f"""You are an intelligent agent equipped with a local website policy evaluation skill.
+To use a skill, output exactly one JSON code block:
+```json
+{{
+  "tool": "the_tool_name",
+  "arguments": {{
+    "param_name": "value"
+  }}
+}}
+```
+Wait for the system response containing the tool result before continuing.
+
+Available skill:
+{tool_description}
+"""
+
+messages = [
+    {"role": "system", "content": system_prompt},
+    {
+        "role": "user",
+        "content": (
+            "Before I crawl https://hackernoon.com/tagged/startups with a bot for research, "
+            "check whether that seems allowed."
+        ),
+    },
+]
+
+model_name = "llama3"
+response = ollama.chat(model=model_name, messages=messages)
+message_content = response.get("message", {}).get("content", "")
+print(message_content)
+
+tool_match = re.search(r"```json\s*({.*?})\s*```", message_content, re.DOTALL)
+if tool_match:
+    tool_call = json.loads(tool_match.group(1))
+    fn_name = tool_call.get("tool")
+    fn_args = tool_call.get("arguments", {})
+
+    if fn_name == "compliance/tos_evaluator":
+        result = tos_skill.execute(fn_args)
+        print(json.dumps(result, indent=2))
+        messages.append({"role": "assistant", "content": message_content})
+        messages.append(
+            {
+                "role": "user",
+                "content": (
+                    "SYSTEM RESPONSE:\n"
+                    f"```json\n{json.dumps(result)}\n```\n"
+                    "Please provide the final answer."
+                ),
+            }
+        )
+        final_response = ollama.chat(model=model_name, messages=messages)
+        print(final_response.get("message", {}).get("content", ""))

pyproject.toml

@@ -23,6 +23,7 @@ dependencies = [
     "anthropic",
     "google-generativeai",
     "pymupdf",
+    "beautifulsoup4",
 ]
 requires-python = ">=3.10"
 

requirements.txt

@@ -2,6 +2,7 @@
 pyyaml
 python-dotenv
 requests
+beautifulsoup4
 
 # LLM SDKs (Optional but recommended for examples)
 google-generativeai

skills/compliance/tos_evaluator/__init__.py

@@ -0,0 +1,3 @@
+from .skill import TOSEvaluatorSkill
+
+__all__ = ["TOSEvaluatorSkill"]

skills/compliance/tos_evaluator/card.json

@@ -0,0 +1,23 @@
+{
+    "name": "TOS Evaluator",
+    "description": "Checks robots.txt and legal pages before automated website access.",
+    "icon": "shield",
+    "color": "green",
+    "ui_schema": {
+        "type": "card",
+        "fields": [
+            {
+                "key": "verdict",
+                "label": "Verdict"
+            },
+            {
+                "key": "confidence_score",
+                "label": "Confidence"
+            },
+            {
+                "key": "reason",
+                "label": "Reason"
+            }
+        ]
+    }
+}

skills/compliance/tos_evaluator/instructions.md

@@ -0,0 +1,34 @@
+# Operational Instructions: TOS Evaluator
+
+You are an agent equipped with the `compliance/tos_evaluator` skill.
+
+## When to use this skill
+- Use this skill before scraping, crawling, indexing, bulk downloading, automated monitoring, or other programmatic access to a website.
+- Use it when the user asks whether a website permits bots, scraping, or non-browser automation.
+- Use it when you need a conservative compliance check before a downstream agent or script touches a domain.
+
+## What this skill actually checks
+The skill evaluates:
+1. `robots.txt` access rules for the requested URL and user-agent.
+2. Candidate Terms of Service, legal, acceptable-use, and API policy pages discovered on the same site.
+3. Optional low-cost LLM review for ambiguous legal clauses when enabled.
+
+## How to interpret the output
+- `SAFE`: strong evidence suggests the requested action is allowed, and `robots.txt` does not block it.
+- `UNSAFE`: `robots.txt` blocks the path or the discovered policy text explicitly restricts the requested automation.
+- `CAUTION`: the site may allow some access, but the policy text contains conditions, ambiguities, or API-only restrictions.
+- `INSUFFICIENT_EVIDENCE`: the skill could not find enough trustworthy evidence to safely approve the action.
+
+Always explain the result conservatively. This tool is an operational guardrail, not legal counsel.
+
+## Important behavior rules
+- Do not claim legal certainty.
+- If the result is `CAUTION` or `INSUFFICIENT_EVIDENCE`, recommend manual review or an official API path.
+- If the result is `UNSAFE`, clearly tell the user not to proceed without explicit permission.
+- If the result relied on `llm_assessment`, mention that it was an auxiliary interpretation layer on top of fetched evidence.
+
+## Example uses
+- User: "Can I scrape pricing from this page?" -> call the tool with `target_url` and `intended_action=\"scrape pricing data\"`
+- User: "Can I index these docs into search?" -> call the tool with `intended_action=\"index documentation pages\"`
+- User: "Can I use a bot to monitor changes daily?" -> call the tool with `intended_action=\"monitor content with automated bot\"`
+- If legal language is vague and the user enabled fallback review, set `use_llm_evaluator=true`.