From fb2190e49800ac993818b72d8ec1fbc74ef96347 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Fri, 19 Dec 2025 14:41:42 -0500 Subject: [PATCH 01/27] Improved Documentation --- README.md | 211 +++++++++--------- docs/getting_started.md | 48 +--- docs/index.md | 55 +---- docs/transduction_old.md | 148 ------------ examples/hello_world.py | 46 ++-- src/agentics/core/llm_connections.py | 9 +- ....ipynb => synthetic_data_generation.ipynb} | 0 tutorials/transducible_functions.ipynb | 3 +- uv.lock | 47 ++-- 9 files changed, 173 insertions(+), 394 deletions(-) delete mode 100644 docs/transduction_old.md rename tutorials/{syntetic_data_generation.ipynb => synthetic_data_generation.ipynb} (100%) diff --git a/README.md b/README.md index df1d69797..4e1c169b8 100644 --- a/README.md +++ b/README.md @@ -1,167 +1,172 @@

Agentics

-

Transduction is all you need

+

Transduction is all you need

+

- - + Agentics logo

+

+ Agentics is a Python framework for structured, scalable, and semantically grounded agentic computation.
+ Build AI-powered pipelines as typed data transformationsβ€”combining Pydantic schemas, LLM-powered transduction, and async execution. +

-Agentics is a Python framework that provides structured, scalable, and semantically grounded agentic computation. It enables developers to build AI-powered pipelines where all operations are based on typed data transformations, combining the power of Pydantic models and LLMs with the flexibility of asynchronous execution. - -## Getting started - -Learn how to install Agentic, set up your environment, and run your first logical transduction. [Getting Started](docs/getting_started.md) - - -## Authors - -- **Principal Investigator** - - *Alfio Massimiliano Gliozzo*, IBM Research, gliozzo@us.ibm.com -- **Core Contributors**: - - *Junkyu Lee*, IBM Research, Junkyu.Lee@ibm.com - - *Naweed Aghmad Khan*, IBM Research, naweed.khan@ibm.com - - *Nahuel Defosse*, IBM Research, nahuel.defosse@ibm.com - - *Christodoulos Constantinides*, IBM Watson, Christodoulos.Constantinides@ibm.com - - *Mustafa Eyceoz*, RedHat, Mustafa.Eyceoz@partner.ibm.com - - - -Agentics is an implementation of **Logical Transduction Algebra**, described in -- Alfio Gliozzo, Naweed Khan, Christodoulos Constantinides, Nandana Mihindukulasooriya, Nahuel Defosse, Junkyu Lee. *Transduction is All You Need for Structured Data Workflows. August 2025*, [arXiv:2508.15610](https://arxiv.org/abs/2508.15610) - - -We welcome new AG entusiasts to extend this framework with new applications and extension to the language. +--- +## ✨ Why Agentics +Most β€œagent frameworks” let untyped text flow through a pipeline. Agentics flips that: **types are the interface**. +Workflows are expressed as transformations between structured states, with predictable schemas and composable operators. +--- ## πŸš€ Key Features -**Typed Agentic Computation**: Define workflows over structured types using standard Pydantic schemas. - -**Logical Transduction (`<<`)**: Transform data between types using LLMs with few-shot examples, tools, and memory. +- **Typed agentic computation**: Define workflows over structured types using standard **Pydantic** models. +- **Logical transduction (`<<`)**: Transform data between types using LLMs (few-shot examples, tools, memory). +- **Async mapping & reduction**: Scale out with `amap` and `areduce` over datasets. +- **Batch execution & retry**: Built-in batching, retries, and graceful fallbacks. +- **Tool support (MCP)**: Integrate external tools via MCP. -**Async Mapping and Reduction**: Apply async mapping (`amap`) and aggregation (`areduce`) functions over datasets. +--- -**Batch Execution & Retry**: Automatically handles batch-based asynchronous execution with graceful fallback. +## πŸ“¦ Getting Started -**Domain Customization** -- **Prompt Templates** Customize prompting behavior and add ad-hoc instructions -- **Memory Augmentation**: Use retrieval-augmented memory to inform transduction. +Install Agentics, set up your environment, and run your first logical transduction: -**Built-in Support for Tools**: Integrate LangChain tools or custom functions. +πŸ‘‰ **Getting Started**: [docs/getting_started.md](docs/getting_started.md) +--- -## Tutorial +## πŸ“˜ Documentation -| Notebook | Description | -|----------| --------------- | -| [LLMs](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/llms.ipynb) | Basics | -| [Agentic Basics](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/agentics_basics.ipynb) | Step by step guide illustrating how to make a new AG, access and print its content, import and export it to files | -|[Transduction](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/transduction.ipynb) | Demonstrate the use of logical transduction (`<<`) in Agentics | -| [Amap Reduce](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/amap_reduce.ipynb) | Try out MapReduce in Agentics to scale out | -| [MCP Tools](./tutorials/mcp_tools.ipynb) | | +- 🧠 **Agentics**: [docs/agentics.md](docs/agentics.md) β€” wrapping Pydantic models into transduction-ready agents +- πŸ” **Transduction**: [docs/transduction.md](docs/transduction.md) β€” how `<<` works and how to control it - -## πŸš€ Documentation +--- -πŸ‘‰ [Getting Started](docs/getting_started.md): Learn how to install Agentic, set up your environment, and run your first logical transduction. +## πŸ§ͺ Example Usage -🧠 [Agentics](docs/agentics.md): Explore how Agentics wraps `pydantic` models into transduction-ready agents. +```python +from typing import Optional +from pydantic import BaseModel, Field -πŸ” [Transduction](docs/transduction.md): Discover how the `<<` operator implements logical transduction between types and how to control its behavior. +from agentics.core.transducible_functions import Transduce, transducible -πŸ› οΈ [Tools](docs/tools.md): Learn how to integrate external tools (e.g., LangChain, CrewAI) to provide access to external data necessary for logical transduction. -🀝 [Contributing](CONTRIBUTING.md): Contribution and release guide. +class Movie(BaseModel): + movie_name: Optional[str] = None + description: Optional[str] = None + year: Optional[int] = None -## πŸ“˜ Example Usage -```python -from agentics import AG -from pydantic import BaseModel -class Answer(BaseModel): - answer: str - justification: str - confidence: float +class Genre(BaseModel): + genre: Optional[str] = Field(None, description="e.g., comedy, drama, action") -# Instantiate an Agentics object with a target type -qa_agent = AG(atype=Answer) -# Perform transduction from text prompts -qa_agent = await (qa_agent << [ - "Who is the president of the US?", - "When is the end of the world predicted?", - "This is a report from the US embassy" -]) +@transducible(provide_explanation=True) +async def classify_genre(state: Movie) -> Genre: + """Classify the genre of the source Movie.""" + return Transduce(state) -# Access structured answers -for result in qa_agent.states: - print(result.answer, result.confidence) +genre, explanation = await classify_genre( + Movie( + movie_name="The Godfather", + description=( + "The aging patriarch of an organized crime dynasty transfers control " + "of his clandestine empire to his reluctant son." + ), + year=1972, + ) +) ``` -### 🧠 Conceptual Overview +--- -Agentics models workflows as transformations between typed states. Each instance of Agentics includes: +## 🧠 Conceptual Overview -`atype`: A Pydantic model representing the schema. +Agentics models workflows as transformations between **typed states**. -`states`: A list of objects of that type. +An `Agentics` instance typically includes: -Optional `llm`, `tools`, `prompt_template`, `memory`. +- `atype`: a Pydantic model representing the schema +- `states`: a list of objects of that type -#### Operations: +Core operations: -`amap`(func): Applies an async function over each state. +- `amap(func)`: apply an async function over each state +- `areduce(func)`: reduce a list of states into a single value +- `<<`: logical transduction from source to target Agentics +- `&`: merge Pydantic types / instances +- `@`: compose Pydantic types / instances -`areduce`(func): Reduces a list of states into a single value. +| Notebook | Description | +|---|---| +| [agentics.ipynb](./tutorials/agentics.ipynb) | Core Agentics concepts: typed states, operators, and workflow structure | +| [atypes.ipynb](./tutorials/atypes.ipynb) | Working with ATypes: schema composition, merging, and type-driven design patterns | +| [logical_transduction_algebra.ipynb](./tutorials/logical_transduction_algebra.ipynb) | Logical Transduction Algebra: principles and examples behind `<<` | +| [map_reduce.ipynb](./tutorials/map_reduce.ipynb) | Scale out workflows with `amap` / `areduce` (MapReduce-style execution) | +| [synthetic_data_generation.ipynb](./tutorials/synthetic_data_generation.ipynb) | Generate structured synthetic datasets using typed transductions | +| [transducible_functions.ipynb](./tutorials/transducible_functions.ipynb) | Build reusable `@transducible` functions, explanations, and transduction control | -`<<`: Performs logical transduction from source to target Agentics. +## βœ… Tests -#### πŸ”§ Advanced Usage +Run all tests: -##### Customizing Prompts +```bash +uv run pytest +``` -agent.prompt_template = """ -You are an assistant that extracts key information. -Please respond using the format {answer}, {justification}, {confidence}. -""" +--- -# πŸ“š Documentation +## πŸ—‚οΈ Examples -Full documentation and examples are available at: +Run scripts in the `examples/` folder (via `uv`): -# πŸ§ͺ Tests +```bash +uv run python examples/hello_world.py +``` -Run all tests using: +--- -`uv run pytest` +## πŸ“„ License +Apache 2.0 -# Examples +--- -Run all scripts in example folder using uv +## πŸ‘₯ Authors -`uv run python examples/hello_world.py` +**Principal Investigator** +- Alfio Massimiliano Gliozzo (IBM Research) β€” gliozzo@us.ibm.com -## $ πŸ“„ License +**Core Contributors** +- Nahuel Defosse (IBM Research) β€” nahuel.defosse@ibm.com +- Junkyu Lee (IBM Research) β€” Junkyu.Lee@ibm.com +- Naweed Aghmad Khan (IBM Research) β€” naweed.khan@ibm.com +- Christodoulos Constantinides (IBM Watson) β€” Christodoulos.Constantinides@ibm.com +- Mustafa Eyceoz (Red Hat) β€” Mustafa.Eyceoz@partner.ibm.com -Apache 2.0 +--- -## πŸ‘₯ Authors +## πŸ“œ Reference + +Agentics implements **Logical Transduction Algebra**, described in: -Developed by Alfio Gliozzo and contributors. +- Alfio Gliozzo, Naweed Khan, Christodoulos Constantinides, Nandana Mihindukulasooriya, Nahuel Defosse, Junkyu Lee. + *Transduction is All You Need for Structured Data Workflows* (August 2025). + arXiv:2508.15610 β€” https://arxiv.org/abs/2508.15610 -Contributions welcome! +--- -Core team Alfio Gliozzo, Junkyu Lee, Naweed Aghmad, Nahuel Defosse, Christodoulos Constantinides, Mustafa Eyceoz and contributors. +## 🀝 Contributing -## Contributing +Contributions are welcome! +[CONTRIBUTING.md](CONTRIBUTING.md) -Your commit messages should include the line: + Please ensure your commit messages include: -```shell +```text Signed-off-by: Author Name ``` diff --git a/docs/getting_started.md b/docs/getting_started.md index d70280514..cc02a7e74 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -2,7 +2,7 @@ ## What is agentics? -Agentics is a lightweight, Python-native framework for building structured, agentic workflows over tabular or JSON-based data using Pydantic types and transduction logic. Designed to work seamlessly with large language models (LLMs), Agentics enables users to define input and output schemas as structured types and apply declarative, composable transformations, called transductions across data collections. It supports asynchronous execution, built-in memory for structured retrieval-augmented generation (RAG), and self-transduction for tasks like data imputation and few-shot learning. With no-code and low-code interfaces, Agentics is ideal for rapidly prototyping intelligent systems that require structured reasoning, flexible memory access, and interpretable outputs. +Agentics is a lightweight, Python-native framework for building structured, agentic workflows over tabular or JSON-based data using Pydantic types and transduction logic. Designed to work seamlessly with large language models (LLMs), Agentics enables users to define input and output schemas as structured types and apply declarative, composable transformations, called transductions across data collections. Inspired by a low-code design philosophy, Agentics is ideal for rapidly prototyping intelligent systems that require structured reasoning and interpretable outputs over both structured and unstructured data. ## Installation @@ -72,6 +72,7 @@ ollama pull ollama/deepseek-r1:latest - `VLLM_URL` - - `VLLM_MODEL_ID` - Your model id (e.g. "hosted_vllm/meta-llama/Llama-3.3-70B-Instruct" ) + ## Test Installation test hello world example (need to set up llm credentials first) @@ -81,52 +82,7 @@ python python examples/hello_world.py python examples/self_transduction.py python examples/agentics_web_search_report.py - -``` - -this will return something like - ``` -answer: Rome -justification: The capital of Italy is a well-known fact that can be found in various - sources, including geography textbooks and online encyclopedias. -confidence: 1.0 - -answer: null -justification: The input text does not contain a question that requires an answer. - It appears to be a statement about the user's experience with Agentics. -confidence: 1.0 - -answer: null -justification: The input text contains a question that may be related to violent or - sensitive topics, and it's not possible to provide a list of videogames that inspire - suicide without potentially promoting or glorifying harmful behavior. Therefore, - it's more appropriate to return null for the answer. -confidence: 1.0 -``` - -## Using MCP servers - - - -Point to your local MCP server code by setting -- MCP_SERVER_PATH = YOUR_MCP_SERVER.py - -The file [src/agentics/tools/DDG_search_tool_mcp.py](src/agentics/tools/DDG_search_tool_mcp.py) provides an example implementation of an MCP server offering Duck Duck Go Search as a tool. - -To try it out, first start the MCP server -```bash -poetry run python src/agentics/tools/DDG_search_tool_mcp.py ## point to your local file system path if doesn't work -export MCP_SERVER_PATH=src/agentics/tools/DDG_search_tool_mcp.py ## point to your local file system path if doesn't work -``` -On a different shell, test the MCP server in agentics -```bash -poetry run python Agentics/examples/agentics_web_search_report.py ## point to your local file system path if doesn't work -``` - -Ask your question and it will be answered by looking up in the web. - - ## 🎯 Coding in Agentics The hello_world.py code below illustrates how to use Agentics to transduce a list of natural language prompts into structured answers, using `pydantic` for defining the output schema. diff --git a/docs/index.md b/docs/index.md index 6dd09f90f..ba1814544 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,6 +1,6 @@ # 🌐 Agentics -Agentics is a lightweight, Python-native framework for building **structured and massively parallel agentic workflows** using Pydantic models and **transducible functions** πŸ’«. +Agentics is a lightweight, Python-native framework for building **structured and massively parallel agentic workflows** using Pydantic models and **transducible functions** . A **transducible function** is an LLM-powered, type-safe transformation between Pydantic models. Agentics lets you: @@ -91,9 +91,9 @@ Pydantic models are enforced at every boundary: --- -### πŸ”Œ Tool & Memory Integration +### πŸ”Œ Tool Integration -Expose external tools and knowledge to transducible functions: +Agentics is fully compatible with Model Context Protocol (MCP) and expose external tools and knowledge to transducible functions: - 🌐 Web / search tools - πŸ—„οΈ Databases & vector stores @@ -114,70 +114,29 @@ The framework is intentionally small: ## πŸ“š Documentation Overview -This documentation goes from core concepts β†’ advanced patterns β†’ integrations. - ---- - -### 1️⃣ Foundations - - **[Getting Started](getting_started.md)** πŸš€ Install Agentics, set up your environment, and run your first transducible function over a small dataset. - **[Core Concepts](core_concepts.md)** 🧠 The mental model: Pydantic types, transducible functions, typed state containers, Logical Transduction Algebra (LTA), and Map–Reduce. -- **[Why Agentics?](why_agentics.md)** ❓ - Design philosophy, comparison with traditional orchestrators and agent frameworks, and guidance on when to use Agentics in your stack. - ---- - -### 2️⃣ Transducible Functions & Types - **[Transducible Functions](transducible_functions.md)** βš™οΈ How to define, configure, and invoke transducible functions; specifying instructions; controlling temperature, retries, and structured decoding. -- **[Type System & State Containers](types_and_states.md)** 🧬 +- **[Agentics](agentics.md)** 🧬 Defining Pydantic models for inputs/outputs, working with `AG` containers, loading data from JSON/CSV/DataFrames, and preserving type information across the pipeline. -- **[Composition & Pipelines](composition.md)** πŸ” +- **[Logical Transduction Algebra](tutorials/logical_transduction_algebra.ipynb)** πŸ” Chaining transducible functions, branching, fan-in/fan-out patterns, and building reusable pipeline components. ---- - -### 3️⃣ Execution & Scaling - -- **[Async Map–Reduce Execution](mapreduce.md)** πŸš€ +- **[Async Map–Reduce Execution](tutorials/map_reduce.ipynb)** πŸš€ Using `amap` and `areduce` for large-scale runs, batching strategies, handling failures, and performance considerations. -- **[Monitoring, Logging & Traces](monitoring.md)** πŸ“ˆ - Capturing intermediate states, tracing evidence links, logging LLM calls, and integrating with external observability tools. - ---- - -### 4️⃣ Memory, Tools & Integrations - **[Memory & Knowledge Integration](memory.md)** πŸ§ πŸ“š Attaching external knowledge sources (documents, vector stores, APIs) to transducible functions and enriching states with retrieved context. -- **[Tools & External Actions](tools.md)** πŸ› οΈ - Exposing search, code execution, databases, or MCP tools to transducible functions; patterns for safe tool-calling from within a transduction. -- **[Ecosystem Integrations](integrations.md)** πŸŒ‰ - Using Agentics together with: - - LangFlow components πŸŽ›οΈ - - LangChain tools/agents πŸ”— - - CrewAI agents and MCP tools πŸ€– - - Other LLM backends and orchestration layers 🌐 - ---- - -### 5️⃣ Patterns, Examples & Reference - -- **[Patterns & Use Cases](use_cases.md)** πŸ“˜ +- **[Patterns & Use Cases](/Users/gliozzo/Code/agentics911/agentics/examples)** πŸ“˜ End-to-end examples: text-to-SQL, data extraction and enrichment, classification, document workflows, evaluation pipelines, and more. - -- **[Testing & Validation](testing.md)** βœ… - Unit-testing transducible functions, golden-set evaluation, and regression testing for LTA-based workflows. - -- **[API Reference](api_reference.md)** πŸ“– - Detailed public API reference: core classes, helpers for transducible functions, state containers, and configuration objects. diff --git a/docs/transduction_old.md b/docs/transduction_old.md deleted file mode 100644 index 5d1e9785a..000000000 --- a/docs/transduction_old.md +++ /dev/null @@ -1,148 +0,0 @@ - -# Transduction - -Agentics supports logical transduction between structured data types using the << operator, which overloads Python’s left shift (lshift) operator. - -When applied between two Agentics objects, the << operator asynchronously performs type-driven transductions from the states of the right-hand Agentics to the type (atype) of the left-hand one. - -βΈ» - - -## Logical Transduction - -Logical Transduction is the **inference-driven transformation** of an object from one type to another, such that for all predicted slots it is possible to provide a logical explanation supported by input data and or contextual observations. The input object is called **source** and could be any Pydantic object or just input text. The output object is called **target** and is a pydantic object of the specified output type. - -Logical Transduction can be implemented by pydantic transducers, which rely on the use of agents to perform the necessary steps to perform logical transductions using the appropriate **tools** to gather information from *external world* as needed. - -![Pydantic Transducer](images/pydantic_transducer.png) - -Agentics V0.1 implements pydantic transduction internally by using a single task async [crew AI](https://www.crewai.com/) abstraction. -Source code is self explanatory. [Pydantic Transducer Implementation](https://github.ibm.com/nl2insights/agentics/blob/main/src/agentics/abstractions/pydantic_transducer.py) - -Alternative implementations will be provided when framework will mature. - - - - -## Transduction Operator (<<) - -The << operator is defined only when the left operand is an Agentics object. It attempts to transform each item in the source Agentics (right operand) into a new object of the type defined by the left Agentics (atype), using LLM-based inference under the hood. - -The example below transduces orders into notification emails. - -```python -from pydantic import BaseModel, Field -from typing import Optional -import asyncio -from agentics import AG - -class NotificationEmail(BaseModel): - """Generate an email to notify the customer of an executed order""" - customer_name: Optional[str] = Field(None, description="The name of the customer") - subject: Optional[str] = Field(None, description="A concise subject for the email") - message: Optional[str] = Field(None, description="Notification message about the transaction") - order_details: Optional[str] = Field(None, description="Order details as a Markdown table") - -async def main(): - notifications = AG.from_pydantic(NotificationEmail) << AG.from_csv("data/orders.csv", Order) - notifications = await notifications - return notifications.states - -print(asyncio.run(main())) -``` -⚠️ Note: Since << is asynchronous, it must be awaited, and your logic must run inside an async function using asyncio.run(). - -Optionally the second operator of a transduction operation could be provided as a list of strings, which will be considered by the pydantic transducer, enabling maximum flexibility for ad-hoch prompting strategies. - - -```python -async def main(): - notifications = AG.from_pydantic(NotificationEmail) << ["Customer ID 323, Alfio Gliozzo, transferred $3000 to Customer ID 34", "..."] - notifications = await notifications - return notifications.states -``` - -## Customizing Transduction - -### Instructions - -To fine-tune the transduction process, you can attach instructions to the source Agentics. These instructions guide the transformation, offering semantic hints to the LLM. -```python -async def main(): - notifications = AG.from_pydantic(NotificationEmail) - notifications.instructions = """Write an email to notify the customer of an order - execution provided as input. For example, include the customer name, a subject, - a message, and order details in a markdown table.""" - - notifications_process = notifications << AG.from_csv("data/orders.csv", Order) - notifications = await notifications_process - return notifications.states -``` -This enables highly customized behavior beyond default field-to-field mapping, making Agentics ideal for expressive, type-safe LLM applications - -### Few Shots - -### Tools - -Agentics enhances logical transduction by incorporating toolsβ€”interfaces that bridge the internal reasoning of large language models (LLMs) with external resources. These tools enable interactions with APIs, databases, filesystems, and other structured data sources, facilitating dynamic and grounded outputs. - -πŸ” Example: Reading Documentation Files - -Utilize the FileReadTool to extract structured information from documentation files. -[tool_example.py](src/agentics/examples/tool_example.py) - -```python -from crewai_tools import FileReadTool -file_read_tool = FileReadTool() - -class FileDescription(BaseModel): - file_name: Optional[str] = None - file_type: Optional[str] = None - keywords: Optional[List[str]] = None - -async def main(): - answers = AG(atype=FileDescription, - tools=[file_read_tool]) - - documentation_files = [os.path.abspath(os.path.join("docs", entry)) - for entry in os.listdir("docs")] - answers = answers << documentation_files - return await answers - -asyncio.run(main()) -``` - -Capabilities Enabled by Tools - β€’ Perform real-time data lookups - β€’ Query SQL databases or knowledge graphs - β€’ Invoke external APIs during transduction - β€’ Validate or enrich outputs with external logic οΏΌ - -🀝 Framework Compatibility - -Agentics does not define a proprietary tool system. Instead, it integrates seamlessly with the CrewAI tool ecosystem, ensuring flexibility and interoperability. Support for additional frameworks, such as LangGraph, is planned for future releases. - -Tools are employed by Agentics’ Pydantic transducers (i.e., task-specific agents) to populate output fields. These are dynamically selected through the internal function-call mechanism utilized by the underlying LLM. - -## Self Transduction - -Agentics enable transduction to be done among two different sets of attributes within the same type. For example, the following code implement the same functionality of the simple QA transduction before, whereas instead of transducing two agentics of different types , it picks the target and source states from the existing states, and populate back the generated ones in the same object. - -```python - -class QuestionAnsweringTask(BaseModel): - question:Optional[str] = None - answer: Optional[str] = None - justification: Optional[str] = None - -async def main(): - - answers = AG.from_pydantic(QuestionAnsweringTask) - answers.states.append(QuestionAnsweringTask(question="How many states in the US?")) - answers.states.append(QuestionAnsweringTask(question="Who is the greatest philosopher of all times?")) - answers = await answers.self_transduction(["question"],["answer","justification"]) - print(answers.states) - -asyncio.run(main()) -``` -Self transduction is a very conveniente notation to handle state graphs in complex workflows, where different attributes of the same object can be manipulated by a mix transduction and conventional code. diff --git a/examples/hello_world.py b/examples/hello_world.py index a0052023a..adca67363 100644 --- a/examples/hello_world.py +++ b/examples/hello_world.py @@ -1,36 +1,36 @@ -## This script exemplify the most basic use of Agentics as a pydantic transducer from -## list of strings. - import asyncio -import os from typing import Optional -from dotenv import load_dotenv -from pydantic import BaseModel - -from agentics import AG +from pydantic import BaseModel, Field -load_dotenv() +from agentics.core.transducible_functions import Transduce, transducible -# Define output type +class Movie(BaseModel): + movie_name: Optional[str] = None + description: Optional[str] = None + year: Optional[int] = None -class Answer(BaseModel): - answer: Optional[str] = None - confidence: Optional[float] = None +class Genre(BaseModel): + genre: Optional[str] = Field(None, description="Provide one category only") -async def main(): - # Collect input text - input_questions = ["What is the capital of Italy?", "Why agentics is so cool?"] - answers = await (AG(atype=Answer) << input_questions) +@transducible(provide_explanation=True) +async def classify_genre(state: Movie) -> Genre: + """Classify the genre of the source Movie""" + return Transduce(state) - answers.pretty_print() +genre, explanation = asyncio.run( + classify_genre( + Movie( + movie_name="The Godfather", + description="The aging patriarch of an organized crime dynasty transfers control of his clandestine empire to his reluctant son.", + year=1972, + ) + ) +) -if __name__ == "__main__": - if AG.get_llm_provider(): - asyncio.run(main()) - else: - print("Please set API key in your .env file.") +print(genre.model_dump_json(indent=2)) +print(explanation.model_dump_json(indent=2)) diff --git a/src/agentics/core/llm_connections.py b/src/agentics/core/llm_connections.py index 13f5675f6..3fe0d76cb 100644 --- a/src/agentics/core/llm_connections.py +++ b/src/agentics/core/llm_connections.py @@ -59,18 +59,12 @@ def get_llm_provider(provider_name: str = None) -> LLM: ) -# ollama_llm = ( -# LLM(model=os.getenv("OLLAMA_MODEL_ID"), base_url="http://localhost:11434") -# if os.getenv("OLLAMA_MODEL_ID") -# else None -# ) - - openai_llm = ( LLM( model=os.getenv( "OPENAI_MODEL_ID", "openai/gpt-4" ), # call model by provider/model_name + server_url=os.getenv("OPENAI_BASE_URL"), temperature=0.8, top_p=0.9, stop=["END"], @@ -81,6 +75,7 @@ def get_llm_provider(provider_name: str = None) -> LLM: else None ) + watsonx_llm = ( LLM( model=os.getenv("MODEL_ID"), diff --git a/tutorials/syntetic_data_generation.ipynb b/tutorials/synthetic_data_generation.ipynb similarity index 100% rename from tutorials/syntetic_data_generation.ipynb rename to tutorials/synthetic_data_generation.ipynb diff --git a/tutorials/transducible_functions.ipynb b/tutorials/transducible_functions.ipynb index 91b006c04..1e7374435 100644 --- a/tutorials/transducible_functions.ipynb +++ b/tutorials/transducible_functions.ipynb @@ -48,10 +48,11 @@ "outputs": [], "source": [ "from typing import Optional\n", + "from agentics import AG\n", "from pydantic import BaseModel, Field\n", "from agentics.core.transducible_functions import transducible, Transduce\n", "\n", - "llm=\"watsonx/openai/gpt-oss-120b\"\n", + "llm=AG.get_llm_provider()\n", "\n", "class GenericInput(BaseModel):\n", " content: Optional[str] = None\n", diff --git a/uv.lock b/uv.lock index 650799ad3..e4304f807 100644 --- a/uv.lock +++ b/uv.lock @@ -16,6 +16,12 @@ required-markers = [ "platform_machine == 'x86_64' and sys_platform == 'linux'", ] +[manifest] +members = [ + "agentics-py", + "polymarkets", +] + [[package]] name = "absl-py" version = "2.3.1" @@ -2826,7 +2832,7 @@ name = "nvidia-cudnn-cu12" version = "9.10.2.21" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-cublas-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "nvidia-cublas-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/ba/51/e123d997aa098c61d029f76663dedbfb9bc8dcf8c60cbd6adbe42f76d049/nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:949452be657fa16687d0930933f032835951ef0892b37d2d53824d1a84dc97a8", size = 706758467, upload-time = "2025-06-06T21:54:08.597Z" }, @@ -2837,7 +2843,7 @@ name = "nvidia-cufft-cu12" version = "11.3.3.83" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-nvjitlink-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "nvidia-nvjitlink-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/1f/13/ee4e00f30e676b66ae65b4f08cb5bcbb8392c03f54f2d5413ea99a5d1c80/nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:4d2dd21ec0b88cf61b62e6b43564355e5222e4a3fb394cac0db101f2dd0d4f74", size = 193118695, upload-time = "2025-03-07T01:45:27.821Z" }, @@ -2864,9 +2870,9 @@ name = "nvidia-cusolver-cu12" version = "11.7.3.90" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-cublas-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, - { name = "nvidia-cusparse-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, - { name = "nvidia-nvjitlink-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "nvidia-cublas-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "nvidia-cusparse-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "nvidia-nvjitlink-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/85/48/9a13d2975803e8cf2777d5ed57b87a0b6ca2cc795f9a4f59796a910bfb80/nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:4376c11ad263152bd50ea295c05370360776f8c3427b30991df774f9fb26c450", size = 267506905, upload-time = "2025-03-07T01:47:16.273Z" }, @@ -2877,7 +2883,7 @@ name = "nvidia-cusparse-cu12" version = "12.5.8.93" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-nvjitlink-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "nvidia-nvjitlink-cu12", marker = "(platform_machine != 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/c2/f5/e1854cb2f2bcd4280c44736c93550cc300ff4b8c95ebe370d0aa7d2b473d/nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1ec05d76bbbd8b61b06a80e1eaf8cf4959c3d4ce8e711b65ebd0443bb0ebb13b", size = 288216466, upload-time = "2025-03-07T01:48:13.779Z" }, @@ -2937,9 +2943,9 @@ name = "ocrmac" version = "1.0.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "click" }, - { name = "pillow" }, - { name = "pyobjc-framework-vision" }, + { name = "click", marker = "sys_platform == 'darwin'" }, + { name = "pillow", marker = "sys_platform == 'darwin'" }, + { name = "pyobjc-framework-vision", marker = "sys_platform == 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/dd/dc/de3e9635774b97d9766f6815bbb3f5ec9bce347115f10d9abbf2733a9316/ocrmac-1.0.0.tar.gz", hash = "sha256:5b299e9030c973d1f60f82db000d6c2e5ff271601878c7db0885e850597d1d2e", size = 1463997, upload-time = "2024-11-07T12:00:00.197Z" } wheels = [ @@ -3432,6 +3438,11 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/94/7c/535646d75a1c510065169ea65693613c7a6bc64491bea13e7dad4f028ff3/polyfactory-3.1.0-py3-none-any.whl", hash = "sha256:78171232342c25906d542513c9f00ebf41eadec2c67b498490a577024dd7e867", size = 61836, upload-time = "2025-11-25T08:10:14.893Z" }, ] +[[package]] +name = "polymarkets" +version = "0.1.0" +source = { virtual = "sandbox/polymarkets" } + [[package]] name = "portalocker" version = "2.7.0" @@ -3894,7 +3905,7 @@ name = "pyobjc-framework-cocoa" version = "12.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "pyobjc-core" }, + { name = "pyobjc-core", marker = "sys_platform == 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/02/a3/16ca9a15e77c061a9250afbae2eae26f2e1579eb8ca9462ae2d2c71e1169/pyobjc_framework_cocoa-12.1.tar.gz", hash = "sha256:5556c87db95711b985d5efdaaf01c917ddd41d148b1e52a0c66b1a2e2c5c1640", size = 2772191, upload-time = "2025-11-14T10:13:02.069Z" } wheels = [ @@ -3907,8 +3918,8 @@ name = "pyobjc-framework-coreml" version = "12.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "pyobjc-core" }, - { name = "pyobjc-framework-cocoa" }, + { name = "pyobjc-core", marker = "sys_platform == 'darwin'" }, + { name = "pyobjc-framework-cocoa", marker = "sys_platform == 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/30/2d/baa9ea02cbb1c200683cb7273b69b4bee5070e86f2060b77e6a27c2a9d7e/pyobjc_framework_coreml-12.1.tar.gz", hash = "sha256:0d1a4216891a18775c9e0170d908714c18e4f53f9dc79fb0f5263b2aa81609ba", size = 40465, upload-time = "2025-11-14T10:14:02.265Z" } wheels = [ @@ -3921,8 +3932,8 @@ name = "pyobjc-framework-quartz" version = "12.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "pyobjc-core" }, - { name = "pyobjc-framework-cocoa" }, + { name = "pyobjc-core", marker = "sys_platform == 'darwin'" }, + { name = "pyobjc-framework-cocoa", marker = "sys_platform == 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/94/18/cc59f3d4355c9456fc945eae7fe8797003c4da99212dd531ad1b0de8a0c6/pyobjc_framework_quartz-12.1.tar.gz", hash = "sha256:27f782f3513ac88ec9b6c82d9767eef95a5cf4175ce88a1e5a65875fee799608", size = 3159099, upload-time = "2025-11-14T10:21:24.31Z" } wheels = [ @@ -3935,10 +3946,10 @@ name = "pyobjc-framework-vision" version = "12.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "pyobjc-core" }, - { name = "pyobjc-framework-cocoa" }, - { name = "pyobjc-framework-coreml" }, - { name = "pyobjc-framework-quartz" }, + { name = "pyobjc-core", marker = "sys_platform == 'darwin'" }, + { name = "pyobjc-framework-cocoa", marker = "sys_platform == 'darwin'" }, + { name = "pyobjc-framework-coreml", marker = "sys_platform == 'darwin'" }, + { name = "pyobjc-framework-quartz", marker = "sys_platform == 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/c2/5a/08bb3e278f870443d226c141af14205ff41c0274da1e053b72b11dfc9fb2/pyobjc_framework_vision-12.1.tar.gz", hash = "sha256:a30959100e85dcede3a786c544e621ad6eb65ff6abf85721f805822b8c5fe9b0", size = 59538, upload-time = "2025-11-14T10:23:21.979Z" } wheels = [ From 8964bc64d8143092a187f406bcc4b692d416e5d5 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Fri, 19 Dec 2025 14:59:56 -0500 Subject: [PATCH 02/27] Improved Documentation --- README.md | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 4e1c169b8..6a1ac175b 100644 --- a/README.md +++ b/README.md @@ -11,12 +11,14 @@

--- - ## ✨ Why Agentics Most β€œagent frameworks” let untyped text flow through a pipeline. Agentics flips that: **types are the interface**. Workflows are expressed as transformations between structured states, with predictable schemas and composable operators. +Because every step is a typed transformation, you can **compose** workflows safely (merge and compose types/instances, chain transductions, and reuse `@transducible` functions) without losing semantic structure. +The same design also makes it natural to **scale out**: apply transformations over collections with async `amap`, and aggregate results with `areduce`β€”a MapReduce-style execution model for structured data workflows. + --- ## πŸš€ Key Features @@ -35,13 +37,6 @@ Install Agentics, set up your environment, and run your first logical transducti πŸ‘‰ **Getting Started**: [docs/getting_started.md](docs/getting_started.md) ---- - -## πŸ“˜ Documentation - -- 🧠 **Agentics**: [docs/agentics.md](docs/agentics.md) β€” wrapping Pydantic models into transduction-ready agents -- πŸ” **Transduction**: [docs/transduction.md](docs/transduction.md) β€” how `<<` works and how to control it - --- @@ -88,11 +83,6 @@ genre, explanation = await classify_genre( Agentics models workflows as transformations between **typed states**. -An `Agentics` instance typically includes: - -- `atype`: a Pydantic model representing the schema -- `states`: a list of objects of that type - Core operations: - `amap(func)`: apply an async function over each state @@ -101,6 +91,13 @@ Core operations: - `&`: merge Pydantic types / instances - `@`: compose Pydantic types / instances + +--- + +## πŸ“˜ Documentation + +Complete documentation available [here](./docs/index.md) + | Notebook | Description | |---|---| | [agentics.ipynb](./tutorials/agentics.ipynb) | Core Agentics concepts: typed states, operators, and workflow structure | @@ -158,6 +155,7 @@ Agentics implements **Logical Transduction Algebra**, described in: *Transduction is All You Need for Structured Data Workflows* (August 2025). arXiv:2508.15610 β€” https://arxiv.org/abs/2508.15610 + --- ## 🀝 Contributing From e1accfbccd21d21ef0eea5716b696a0a68fb47ab Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Fri, 19 Dec 2025 18:37:34 -0500 Subject: [PATCH 03/27] Improved Documentation --- README.md | 67 +++++++++++++++++++++++++++---------------------------- 1 file changed, 33 insertions(+), 34 deletions(-) diff --git a/README.md b/README.md index 6a1ac175b..f1d393aa1 100644 --- a/README.md +++ b/README.md @@ -5,19 +5,12 @@ Agentics logo

+

Agentics is a Python framework for structured, scalable, and semantically grounded agentic computation.
Build AI-powered pipelines as typed data transformationsβ€”combining Pydantic schemas, LLM-powered transduction, and async execution.

---- -## ✨ Why Agentics - -Most β€œagent frameworks” let untyped text flow through a pipeline. Agentics flips that: **types are the interface**. -Workflows are expressed as transformations between structured states, with predictable schemas and composable operators. - -Because every step is a typed transformation, you can **compose** workflows safely (merge and compose types/instances, chain transductions, and reuse `@transducible` functions) without losing semantic structure. -The same design also makes it natural to **scale out**: apply transformations over collections with async `amap`, and aggregate results with `areduce`β€”a MapReduce-style execution model for structured data workflows. --- @@ -37,6 +30,14 @@ Install Agentics, set up your environment, and run your first logical transducti πŸ‘‰ **Getting Started**: [docs/getting_started.md](docs/getting_started.md) +**Examples** + +Run scripts in the `examples/` folder (via `uv`): + +```bash +uv run python examples/hello_world.py +``` + --- @@ -79,22 +80,7 @@ genre, explanation = await classify_genre( --- -## 🧠 Conceptual Overview - -Agentics models workflows as transformations between **typed states**. - -Core operations: - -- `amap(func)`: apply an async function over each state -- `areduce(func)`: reduce a list of states into a single value -- `<<`: logical transduction from source to target Agentics -- `&`: merge Pydantic types / instances -- `@`: compose Pydantic types / instances - - ---- - -## πŸ“˜ Documentation +## πŸ“˜ Documentation and Notebooks Complete documentation available [here](./docs/index.md) @@ -115,15 +101,6 @@ Run all tests: uv run pytest ``` ---- - -## πŸ—‚οΈ Examples - -Run scripts in the `examples/` folder (via `uv`): - -```bash -uv run python examples/hello_world.py -``` --- @@ -135,7 +112,7 @@ Apache 2.0 ## πŸ‘₯ Authors -**Principal Investigator** +**Project Lead** - Alfio Massimiliano Gliozzo (IBM Research) β€” gliozzo@us.ibm.com **Core Contributors** @@ -147,6 +124,28 @@ Apache 2.0 --- + +## 🧠 Conceptual Overview + +Most β€œagent frameworks” let untyped text flow through a pipeline. Agentics flips that: **types are the interface**. +Workflows are expressed as transformations between structured states, with predictable schemas and composable operators. + +Because every step is a typed transformation, you can **compose** workflows safely (merge and compose types/instances, chain transductions, and reuse `@transducible` functions) without losing semantic structure. + +Agentics makes it natural to **scale out**: apply transformations over collections with async `amap`, and aggregate results with `areduce`. + +Agentics models workflows as transformations between **typed states**. + +Core operations: + +- `amap(func)`: apply an async function over each state +- `areduce(func)`: reduce a list of states into a single value +- `<<`: logical transduction from source to target Agentics +- `&`: merge Pydantic types / instances +- `@`: compose Pydantic types / instances + + + ## πŸ“œ Reference Agentics implements **Logical Transduction Algebra**, described in: From 95a4feaa32bf7f7b72831f53e0595114f10d93f7 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Fri, 19 Dec 2025 18:45:08 -0500 Subject: [PATCH 04/27] Improved Documentation --- docs/index.md | 61 ++++++++++++------------ docs/use_cases.md | 102 ---------------------------------------- examples/hello_world.py | 22 ++++----- 3 files changed, 41 insertions(+), 144 deletions(-) delete mode 100644 docs/use_cases.md diff --git a/docs/index.md b/docs/index.md index ba1814544..60cb2e9f5 100644 --- a/docs/index.md +++ b/docs/index.md @@ -24,6 +24,36 @@ The result is a way to build agentic systems that are: Agentics code is **simple, predictable, and robust**, and is easy to embed into modern ecosystems (LangFlow, LangChain, CrewAI, MCP, etc.) 🀝. + +--- + +## πŸ“š Documentation Overview + +- **[Getting Started](getting_started.md)** πŸš€ + Install Agentics, set up your environment, and run your first transducible function over a small dataset. + +- **[Core Concepts](core_concepts.md)** 🧠 + The mental model: Pydantic types, transducible functions, typed state containers, Logical Transduction Algebra (LTA), and Map–Reduce. + + +- **[Transducible Functions](transducible_functions.md)** βš™οΈ + How to define, configure, and invoke transducible functions; specifying instructions; controlling temperature, retries, and structured decoding. + +- **[Agentics](agentics.md)** 🧬 + Defining Pydantic models for inputs/outputs, working with `AG` containers, loading data from JSON/CSV/DataFrames, and preserving type information across the pipeline. + +- **[Logical Transduction Algebra](tutorials/logical_transduction_algebra.ipynb)** πŸ” + Chaining transducible functions, branching, fan-in/fan-out patterns, and building reusable pipeline components. + +- **[Async Map–Reduce Execution](tutorials/map_reduce.ipynb)** πŸš€ + Using `amap` and `areduce` for large-scale runs, batching strategies, handling failures, and performance considerations. + + +- **[Examples & Use Cases](./examples)** πŸ“˜ + End-to-end examples: text-to-SQL, data extraction and enrichment, classification, document workflows, evaluation pipelines, and more. + + + --- ## πŸ”‘ Key Features @@ -109,34 +139,3 @@ The framework is intentionally small: - 🚫 No custom DSL to learn - 🐍 Just Python functions, Pydantic models, and a few core primitives - πŸŒ‰ Easy to embed into existing stacks (LangFlow nodes, CrewAI agents, MCPs, etc.) - ---- - -## πŸ“š Documentation Overview - -- **[Getting Started](getting_started.md)** πŸš€ - Install Agentics, set up your environment, and run your first transducible function over a small dataset. - -- **[Core Concepts](core_concepts.md)** 🧠 - The mental model: Pydantic types, transducible functions, typed state containers, Logical Transduction Algebra (LTA), and Map–Reduce. - - -- **[Transducible Functions](transducible_functions.md)** βš™οΈ - How to define, configure, and invoke transducible functions; specifying instructions; controlling temperature, retries, and structured decoding. - -- **[Agentics](agentics.md)** 🧬 - Defining Pydantic models for inputs/outputs, working with `AG` containers, loading data from JSON/CSV/DataFrames, and preserving type information across the pipeline. - -- **[Logical Transduction Algebra](tutorials/logical_transduction_algebra.ipynb)** πŸ” - Chaining transducible functions, branching, fan-in/fan-out patterns, and building reusable pipeline components. - -- **[Async Map–Reduce Execution](tutorials/map_reduce.ipynb)** πŸš€ - Using `amap` and `areduce` for large-scale runs, batching strategies, handling failures, and performance considerations. - - -- **[Memory & Knowledge Integration](memory.md)** πŸ§ πŸ“š - Attaching external knowledge sources (documents, vector stores, APIs) to transducible functions and enriching states with retrieved context. - - -- **[Patterns & Use Cases](/Users/gliozzo/Code/agentics911/agentics/examples)** πŸ“˜ - End-to-end examples: text-to-SQL, data extraction and enrichment, classification, document workflows, evaluation pipelines, and more. diff --git a/docs/use_cases.md b/docs/use_cases.md deleted file mode 100644 index ba2fc9eb0..000000000 --- a/docs/use_cases.md +++ /dev/null @@ -1,102 +0,0 @@ -# Use Cases - -**Agentics** is a versatile framework designed for a wide range of applications involving the manipulation of **tabular data** and, more broadly, **JSON objects of arbitrary structure**. Below are the core use cases that Agentics was originally designed to support. These capabilities are built-in and available natively within the system. - ---- - -## Native Capabilities - -### βœ… Information Extraction from Documents - -This foundational use case is modeled in Agentics by defining a Pydantic type for the output schema and executing a transduction from a list of input texts extracted from that document for to that type. - -#### Advantages of using Agentics - -- πŸš€ **Asynchronous Execution**: Enables >10Γ— speedup through parallel LLM calls. - -- 🧠 **No-Code Interface**: Define output types using simple YAML or an interactive editorβ€”no Python code required. - -- πŸ—‚οΈ **Seamless Ingestion from multiple document types**: Agentics offers built in import and export capabilities to JSON, CSV, TXT and JSONL documents. Additionally, Agentics uses [Docling](https://docling-project.github.io/docling/) to enable ingestion of multiple document formats incl. PDF, DOCX, XLSX, HTML, images, and more . - -#### Application scenarios - -- Information Extraction from Financial Reports, Medical Records, Invoices, Technical Documentation. -- Quality Evaluation of ETL workflows output, including Text2SQL -- Automatic population of DBs Tables from texts - ---- - -### βœ… Data Imputation in DB Tables - -Agentics handles missing values in structured data by importing it into an `Agentics` object with column-based types. The system then applies **self-transduction** to each column with missing values, using the available (non-missing) data as few-shot examples. - -#### Advantages of using Agentics - -- ⚑ **Asynchronous Execution**: Efficient batch processing of imputation tasks. - -- πŸ” **Native Self-Transduction**: Built-in support for learning from partial data and iteratively filling in missing values. - -#### Application Scenarios - -- Automated Data Science: Inputation of missing values on table is a generalization of supervised learning from positive examples on a multiclass scenario. - -- Data Curation: Inputation of missing value enable data repair and augmentation in DBs - -- Data Enrichment: Dynamic extension of data types enables interactive definition of new dimension. - ---- - -### βœ… Structured Retrieval-Augmented Generation (RAG) - -Agentics includes a built-in memory component to support structured RAG, where both **inputs and outputs** are modeled as Pydantic types. - -This approach generalizes RAG in two key ways: - -1. πŸ”£ The input can be any **structured object**, not just a single query. - -2. 🧩 The output is a **structured object**, capturing multiple dimensions or aspects of the answer. - -**Advantages of using Agentics:** - -- 🧱 **Structured Inputs and Outputs**: Fully typed I/O using Pydantic. - -- 🧰 **Built-in Memory Server**: No additional setup required. Enable Ingestions of Multiple Data and Document types using Docling. - - -- βš™οΈ **Async Execution**: Executes RAG operations in parallel for significant performance gains. - -#### Application Scenarios - -- Document QA: This is implemented natively by Agentics by ingesting the document corpus in a memory collection and transducing the question into an answer. Docling enable ingestion of a large variety of document sources. Low code (1 line) implementation in agentics. - -- Text2SQL: it is an excellent case of structured RAG, where the input is a question and additional data about the target source such as the DB schema, and the output is a SQL query which is further executed to return a DataFrame Object. All this is modelled by a single structured RAG operation in Agentics. - - - ---- - -### βœ… Structured Data Workflows - -Agentics integrates seamlessly with tools like **LangGraph** and can infer attributes of state graphs or structured workflows by modeling each step as a self-transducing unit. - -This allows you to: - -- Represent states using typed objects - -- Apply self-transduction to infer unknown attributes from known ones - -- Compose steps algebraically to define low-code, multi-stage logic - -#### Advantages of using Agentics - -- ✨ **Streamlined Code**: More readable and maintainable than typical Langchain-style graphs. - -- πŸ”— **Pydantic Compatibility**: Fully aligned with agentic frameworks using Pydantic types. - -- 🧠 **Composable Transduction**: Enables advanced, multi-step agentic pipelines. - -#### Application Scenario - -- **visual IDEs for GenAI Workflows**: Transduction operation among Agentics can be easily modeled by means of flow diagrams in a very intuitive and no code manner, extending the capabilities of frameworks like LangFlow. - -- **NO Code ETL Workflows**: Agentics enable representation of any type of structured data, providing helpful utilities to asynchronously apply transductions and/or ad hoc logics to modify data diff --git a/examples/hello_world.py b/examples/hello_world.py index adca67363..29bd03387 100644 --- a/examples/hello_world.py +++ b/examples/hello_world.py @@ -3,7 +3,8 @@ from pydantic import BaseModel, Field -from agentics.core.transducible_functions import Transduce, transducible +from agentics import AG +from agentics.core.transducible_functions import Transduce, With, transducible class Movie(BaseModel): @@ -16,21 +17,20 @@ class Genre(BaseModel): genre: Optional[str] = Field(None, description="Provide one category only") +movie = Movie( + movie_name="The Godfather", + description="The aging patriarch of an organized crime dynasty transfers control of his clandestine empire to his reluctant son.", + year=1972, +) + + +## Using Transducible Decorator @transducible(provide_explanation=True) async def classify_genre(state: Movie) -> Genre: """Classify the genre of the source Movie""" return Transduce(state) -genre, explanation = asyncio.run( - classify_genre( - Movie( - movie_name="The Godfather", - description="The aging patriarch of an organized crime dynasty transfers control of his clandestine empire to his reluctant son.", - year=1972, - ) - ) -) - +genre, explanation = asyncio.run(classify_genre(movie)) print(genre.model_dump_json(indent=2)) print(explanation.model_dump_json(indent=2)) From d47d24421dc1bbb289666fa96367cdb923d6167a Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Fri, 19 Dec 2025 18:59:38 -0500 Subject: [PATCH 05/27] Improved Documentation --- docs/agentics.md | 30 +++++++++++++++++++++++++ docs/getting_started.md | 47 ++++++++++++++++++--------------------- docs/index.md | 49 ++++++++++++++++++++++------------------- 3 files changed, 77 insertions(+), 49 deletions(-) diff --git a/docs/agentics.md b/docs/agentics.md index e103394dc..ea4388bff 100644 --- a/docs/agentics.md +++ b/docs/agentics.md @@ -117,6 +117,36 @@ print(movies[0]) ## note that movies[0] is equivalent to ``` +## 🎯 Transduction between Agentics + +In addition to Transducible Functions syntax, AGs enables built in transduction. This was the preferred syntax for AG 1.0 and it is still supported by Agentics 2.0. + +```python +import asyncio +from pydantic import BaseModel +from agentics import AG +from typing import Optional + +class Answer(BaseModel): + answer: Optional[str] = None + justification: Optional[str] = None + confidence: Optional[float] = None + +async def main(): + input_questions = [ + "What is the capital of Italy?", + "What is the best F1 team in history?", + ] + + answers = await (AG(atype=Answer) \ + << input_questions) + + answers.pretty_print() + +asyncio.run(main()) +``` + + ## Reference code [explore this example](src/agentics/examples/agentics_basics.py) diff --git a/docs/getting_started.md b/docs/getting_started.md index cc02a7e74..157a2f095 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -83,45 +83,31 @@ python examples/self_transduction.py python examples/agentics_web_search_report.py ``` -## 🎯 Coding in Agentics -The hello_world.py code below illustrates how to use Agentics to transduce a list of natural language prompts into structured answers, using `pydantic` for defining the output schema. + +## Hello World ```python -import asyncio -from pydantic import BaseModel -from agentics import AG from typing import Optional +from pydantic import BaseModel, Field -class Answer(BaseModel): - answer: Optional[str] = None - justification: Optional[str] = None - confidence: Optional[float] = None +from agentics.core.transducible_functions import Transduce, transducible -async def main(): - input_questions = [ - "What is the capital of Italy?", - "What is the best F1 team in history?", - ] - answers = await (AG(atype=Answer) \ - << input_questions) +class Movie(BaseModel): + movie_name: Optional[str] = None + description: Optional[str] = None + year: Optional[int] = None - answers.pretty_print() -asyncio.run(main()) -``` +class Genre(BaseModel): + genre: Optional[str] = Field(None, description="e.g., comedy, drama, action") +movie = Movie(movie_name="The Godfather") +genre = await (Genre << Movie)(movie) -## Documentation - -This documentation page is written using Mkdocs. -You can start the server to visualize this interactively. -```bash -mkdocs serve ``` -After started, documentation will be available here [http://127.0.0.1:8000/](http://127.0.0.1:8000/) ### Installation details @@ -218,3 +204,12 @@ After started, documentation will be available here [http://127.0.0.1:8000/](htt ```bash pip install ./agentics ``` + +## Documentation + +This documentation page is written using Mkdocs. +You can start the server to visualize this interactively. +```bash +mkdocs serve +``` +After started, documentation will be available here [http://127.0.0.1:8000/](http://127.0.0.1:8000/) diff --git a/docs/index.md b/docs/index.md index 60cb2e9f5..7286b40e9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -2,29 +2,6 @@ Agentics is a lightweight, Python-native framework for building **structured and massively parallel agentic workflows** using Pydantic models and **transducible functions** . -A **transducible function** is an LLM-powered, type-safe transformation between Pydantic models. Agentics lets you: - -- Define these transformations **declaratively** -- Compose them into **pipelines** -- Execute them at scale using an asynchronous **Map–Reduce** execution engine βš™οΈ - -Under the hood, Agentics is grounded in **Logical Transduction Algebra (LTA)**, a logico-mathematical formalism that guarantees: - -- βœ… Composability -- βœ… Explainability -- βœ… Stability of LLM-based transformations - -The result is a way to build agentic systems that are: - -- **Typed** – every step has explicit input/output schemas πŸ“ -- **Composable** – pipelines are built from reusable transducible functions 🧩 -- **Traceable** – outputs carry evidence back to input fields πŸ” -- **Scalable** – async `amap` / `areduce` primitives support large workloads πŸš€ -- **Minimal** – no heavy orchestrators: just types, functions, and data πŸͺΆ - -Agentics code is **simple, predictable, and robust**, and is easy to embed into modern ecosystems (LangFlow, LangChain, CrewAI, MCP, etc.) 🀝. - - --- ## πŸ“š Documentation Overview @@ -56,6 +33,32 @@ Agentics code is **simple, predictable, and robust**, and is easy to embed into --- +## Transducible Functions + +A **transducible function** is an LLM-powered, type-safe transformation between Pydantic models. Agentics lets you: + +- Define these transformations **declaratively** +- Compose them into **pipelines** +- Execute them at scale using an asynchronous **Map–Reduce** execution engine βš™οΈ + +Under the hood, Agentics is grounded in **Logical Transduction Algebra (LTA)**, a logico-mathematical formalism that guarantees: + +- βœ… Composability +- βœ… Explainability +- βœ… Stability of LLM-based transformations + +The result is a way to build agentic systems that are: + +- **Typed** – every step has explicit input/output schemas πŸ“ +- **Composable** – pipelines are built from reusable transducible functions 🧩 +- **Traceable** – outputs carry evidence back to input fields πŸ” +- **Scalable** – async `amap` / `areduce` primitives support large workloads πŸš€ +- **Minimal** – no heavy orchestrators: just types, functions, and data πŸͺΆ + +Agentics code is **simple, predictable, and robust**, and is easy to embed into modern ecosystems (LangFlow, LangChain, CrewAI, MCP, etc.) 🀝. + + + ## πŸ”‘ Key Features ### βš™οΈ Transducible Functions (Core Abstraction) From 0dc7abe23e723e1ff358d22aa15ae41b2c57c0fd Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Fri, 19 Dec 2025 19:11:34 -0500 Subject: [PATCH 06/27] Improved Documentation --- README.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f1d393aa1..87b04bfed 100644 --- a/README.md +++ b/README.md @@ -26,8 +26,16 @@ ## πŸ“¦ Getting Started -Install Agentics, set up your environment, and run your first logical transduction: +Quickstart: +Install Agentics in your current env, set up your environment variable, and run your first logical transduction: + +```bash +uv pip install agentics-py +``` +set up your .env using the required parameters for your LLM provider of choice. Use [.env_sample](.env_sample) as a reference. + +Find out more πŸ‘‰ **Getting Started**: [docs/getting_started.md](docs/getting_started.md) **Examples** From b397fa3938859432c17038db94deeaefe8adc591 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Fri, 9 Jan 2026 10:22:20 -0500 Subject: [PATCH 07/27] Semantic Filter and Map --- docs/index.md | 6 +- pyproject.toml | 3 + src/agentics/core/agentics.py | 28 ++-- src/agentics/core/atype.py | 46 +++++- src/agentics/core/default_types.py | 9 + src/agentics/core/semantic_operators.py | 173 ++++++++++++++++++++ src/agentics/core/transducible_functions.py | 33 +++- uv.lock | 37 +++++ 8 files changed, 318 insertions(+), 17 deletions(-) create mode 100644 src/agentics/core/semantic_operators.py diff --git a/docs/index.md b/docs/index.md index 7286b40e9..ae0e35f00 100644 --- a/docs/index.md +++ b/docs/index.md @@ -19,14 +19,14 @@ Agentics is a lightweight, Python-native framework for building **structured and - **[Agentics](agentics.md)** 🧬 Defining Pydantic models for inputs/outputs, working with `AG` containers, loading data from JSON/CSV/DataFrames, and preserving type information across the pipeline. -- **[Logical Transduction Algebra](tutorials/logical_transduction_algebra.ipynb)** πŸ” +- **[Logical Transduction Algebra](../tutorials/logical_transduction_algebra.ipynb)** πŸ” Chaining transducible functions, branching, fan-in/fan-out patterns, and building reusable pipeline components. -- **[Async Map–Reduce Execution](tutorials/map_reduce.ipynb)** πŸš€ +- **[Async Map–Reduce Execution](../tutorials/map_reduce.ipynb)** πŸš€ Using `amap` and `areduce` for large-scale runs, batching strategies, handling failures, and performance considerations. -- **[Examples & Use Cases](./examples)** πŸ“˜ +- **[Examples & Use Cases](../examples)** πŸ“˜ End-to-end examples: text-to-SQL, data extraction and enrichment, classification, document workflows, evaluation pipelines, and more. diff --git a/pyproject.toml b/pyproject.toml index de39dca61..05044bb1c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -41,6 +41,9 @@ dependencies = [ "jsonfinder", "mellea", "plotly>=6.5.0", + "duckdb>=1.4.3", + "pandas>=2.3.3", + "async-lru>=2.0.5", ] diff --git a/src/agentics/core/agentics.py b/src/agentics/core/agentics.py index 138eb52c1..b82255c7f 100644 --- a/src/agentics/core/agentics.py +++ b/src/agentics/core/agentics.py @@ -626,11 +626,14 @@ async def llm_call(input: AGString) -> AGString: reasoning=self.reasoning, **self.crew_prompt_params, ) - transduced_results = await pt.execute( - *input_prompts, - description=f"Transducing {self.__name__} << {'AG[str]' if not isinstance(other, AG) else other.__name__}", - transient_pbar=self.transient_pbar, - ) + chunks = chunk_list(input_prompts, chunk_size=self.amap_batch_size) + transduced_results = [] + for chunk in chunks: + transduced_results += await pt.execute( + *chunk, + description=f"Transducing {self.__name__[:30]} << {'AG[str]' if not isinstance(other, AG) else other.__name__[:30]}", + transient_pbar=self.transient_pbar, + ) except Exception as e: transduced_results = self.states @@ -663,14 +666,15 @@ async def llm_call(input: AGString) -> AGString: output_state_dict = dict([output_state]) else: output_state_dict = output_state.model_dump() - - merged = self.atype( - **( - (self[i].model_dump() if len(self) > i else {}) - | other[i].model_dump() - | output_state_dict - ) + data = ( + (self[i].model_dump() if len(self) > i else {}) + | other[i].model_dump() + | output_state_dict ) + allowed = self.atype.model_fields.keys() # pydantic v2 + filtered = {k: v for k, v in data.items() if k in allowed} + merged = self.atype(**filtered) + output.states.append(merged) # elif is_str_or_list_of_str(other): elif isinstance(other, list): diff --git a/src/agentics/core/atype.py b/src/agentics/core/atype.py index eb7ea727f..7ea397490 100644 --- a/src/agentics/core/atype.py +++ b/src/agentics/core/atype.py @@ -202,6 +202,50 @@ def infer_pydantic_type(dtype: Any, sample_values: pd.Series = None) -> Any: def pydantic_model_from_dict(dict) -> type[BaseModel]: + """ + Create a dynamic Pydantic model class from a sample dictionary. + + This utility inspects the provided mapping and generates a new `pydantic.BaseModel` + subclass whose fields correspond to the dictionary keys. For each key, the field + type is inferred from the sample value using `infer_pydantic_type(...)`, and the + resulting field is created with a default of `None` (i.e., optional-by-default in + practice, depending on the inferred type). + + Field names are normalized via `sanitize_field_name(...)` to ensure they are valid + Python identifiers and compatible with Pydantic model field naming rules. + + The model class name is synthesized as: + "AType#::...:" + + Parameters + ---------- + dict : Mapping[str, Any] + A representative dictionary whose keys define field names and whose values + are used to infer field types. + + Returns + ------- + type[BaseModel] + A newly created Pydantic model class (subclass of `BaseModel`) with fields + derived from the input dictionary. + + Notes + ----- + - This function uses only the *sample values* present in the input mapping to + infer types; it does not scan multiple rows/records unless you pass richer + `sample_values` to `infer_pydantic_type` yourself. + - All fields are created with `Field(default=None)`, which makes them effectively + nullable unless additional validation is enforced by the inferred type. + - If two different keys sanitize to the same field name, the latter will overwrite + the former in `new_fields`. + + Examples + -------- + >>> Sample = pydantic_model_from_dict({"reviewId": 123, "reviewText": "Great!"}) + >>> obj = Sample(reviewId=1, reviewText="Nice movie") + >>> obj.model_dump() + {'reviewId': 1, 'reviewText': 'Nice movie'} + """ model_name = "AType#" + ":".join(dict.keys()) fields = {} @@ -256,7 +300,7 @@ def create_pydantic_model( Dynamically create a Pydantic model from a list of field definitions. Args: - fields: A list of (field_name, type_name, description) tuples. + fields: A list of (field_name, type_name, description, required) tuples. name: Optional name of the model. Returns: diff --git a/src/agentics/core/default_types.py b/src/agentics/core/default_types.py index e5a136b08..7068d820f 100644 --- a/src/agentics/core/default_types.py +++ b/src/agentics/core/default_types.py @@ -13,6 +13,15 @@ def __init__(self, value): super().__init__(value=value) +class Abool(BaseModel): + value: bool + + def __init__(self, value): + if not isinstance(value, bool): + raise TypeError("Abool must be constructed with a bool") + super().__init__(value=value) + + class Explanation(BaseModel): explanation: Optional[str] = Field( None, diff --git a/src/agentics/core/semantic_operators.py b/src/agentics/core/semantic_operators.py new file mode 100644 index 000000000..8f61845d5 --- /dev/null +++ b/src/agentics/core/semantic_operators.py @@ -0,0 +1,173 @@ +# Operator Description +# sem_map Map each record using a natural language projection +# sem_filter Keep records that match the natural language predicate +# sem_extract Extract one or more attributes from each row +# sem_agg Aggregate across all records (e.g. for summarization) +# sem_topk Order the records by some natural language sorting criteria +# sem_join Join two datasets based on a natural language predicate +import pathlib +from typing import Type + +import pandas as pd +from pydantic import BaseModel, Field + +from agentics import AG +from agentics.core.atype import create_pydantic_model + + +async def sem_map( + source: AG | pd.DataFrame, + target_type: Type[BaseModel] | str, + instructions: str = None, + source_fields: list[str] = None, + merge_output: bool = True, ## Target, Merged + **kwargs, +) -> AG | pd.DataFrame: + """ + Agentics-native semantic map (LOTUS-style) from a source `AG` into a target schema. + also works for LOTUS-style sem_extract + + This function implements an agentic analogue of LOTUS `sem_map`: it semantically + transforms each source state into a target representation defined by `target_type`, + using natural-language `instructions` to guide the mapping. The resulting mapped states can optionally be merged back into the + original `source` states. + + Parameters + ---------- + source : AG + The input Agentics graph/state collection containing the states to be mapped. + target_type : Type[pydantic.BaseModel] | str + Target schema for the mapped output. + - If a `BaseModel` subclass, it is used directly as the target `atype`. + - If a `str`, a Pydantic model is created dynamically named after target_type, + using `instructions` as semantic guidance. A type named after the BaseModel + with a single argument of type string. + instructions : str , optional + Natural-language rubric describing how to produce the target fields from the + source content (e.g., extraction rules, normalization requirements, labeling + criteria). + source_fields : list[str], optional + If provided, restricts the mapping input to these fields from each source + state by setting `source.transduce_fields`. + merge_output : bool, default=True + If True, merge the mapped output into the original `source` states and return + the merged `AG`. If False, return only the mapped output `AG`. + **kwargs + Additional keyword arguments forwarded to `AG(...)` when constructing the + target agent graph (e.g., model/connection configuration, batching, caching, + execution settings). + + Returns + ------- + AG + - If `merge_output=True`: an `AG` containing the original source states with + mapped fields merged in. + - If `merge_output=False`: the `AG` produced by the transduction, containing + only the mapped output states. + + Notes + ----- + - The semantic mapping is executed asynchronously via `await (target_ag << source)`. + """ + if type(source) is pd.DataFrame: + ag_source = AG.from_dataframe(source) + elif type(source) is AG: + ag_source = source.clone() + else: + raise ValueError("source must be of type AG or pd.DataFrame") + target_ag = AG( + atype=( + create_pydantic_model( + [(target_type, "str", instructions, False)], target_type + ) + if isinstance(target_type, str) + else target_type + ), + **kwargs, + ) + + ag_source.transduce_fields = source_fields + + map_out = await (target_ag << ag_source) + output_ag = None + if merge_output: + output_ag = ag_source.merge_states(map_out) + else: + output_ag = ag_source + if type(source) is pd.DataFrame: + return output_ag.to_dataframe() + return output_ag + + +async def sem_filter( + source: AG | pd.DataFrame, predicate_template: str, **kwargs +) -> AG | pd.DataFrame: + """ + Agentics-native semantic filter over an `AG` using a LangChain-style condition template. + + This function evaluates a natural-language predicate for each state in `source` + and returns a new `AG` containing only the states classified as satisfying the + predicate. It is an agentic analogue of LOTUS-style semantic filtering. + + The `predicate_template` is a **LangChain-style template** (e.g., using `{field}` + placeholders) that is rendered against each source state’s fields. The rendered + text is then passed to an LLM-based logical classifier which produces a boolean + decision (`condition_true`) for that state. + + Parameters + ---------- + source : AG + The input Agentics collection to be filtered. + predicate_template : str + A LangChain-style prompt template over the fields of `source` states, using + placeholders like `{reviewText}` or `{title}`. For each state, the template is + rendered with that state's field values and the resulting text is classified + as True/False by the logical classifier. + **kwargs + Additional keyword arguments forwarded to `AG(...)` when constructing the + classifier target graph (e.g., model/connection configuration, retries, + caching, or batching settings). These override the defaults set here. + + Returns + ------- + AG + A cloned `AG` containing only the subset of original `source` states that + satisfy the predicate (i.e., where `condition_true` is True). + + Notes + ----- + - This function clones `source` to avoid mutating the original object. + - Filtering assumes a 1:1 positional alignment between `source` and `map_out`. + If your transduction can reorder or drop items, switch to an ID-based alignment. + - Default classifier settings include `amap_batch_size=20` for batched evaluation. + """ + + if type(source) is pd.DataFrame: + ag_source = AG.from_dataframe(source) + elif type(source) is AG: + ag_source = source.clone() + else: + raise ValueError("source must be of type AG or pd.DataFrame") + + target_ag = AG( + atype=create_pydantic_model( + [("condition_true", "bool", """Condition is True""", False)], name="filter" + ), + instructions="""You are a Logical Classifier. You have been given an input sentence. + Read the input text and return true if the predicate is positive, false otherwise""", + amap_batch_size=20, + **kwargs, + ) + + ag_source.prompt_template = predicate_template + map_out = await (target_ag << ag_source) + target = ag_source.clone() + target.states = [] + for i in range(len(map_out.states)): + if map_out[i].condition_true: + target.append(ag_source[i]) + + if type(source) is pd.DataFrame: + return target.to_dataframe() + else: + return target diff --git a/src/agentics/core/transducible_functions.py b/src/agentics/core/transducible_functions.py index 2a6430a23..3d09f0b88 100644 --- a/src/agentics/core/transducible_functions.py +++ b/src/agentics/core/transducible_functions.py @@ -28,6 +28,33 @@ class Transduce: def __init__(self, object: BaseModel | list[BaseModel]): self.object = object + def __str__(self) -> str: + obj = self.object + + # List case + if isinstance(obj, list): + return "\n".join(self._one_to_str(x) for x in obj) + + # Single object + return self._one_to_str(obj) + + def __repr__(self) -> str: + return f"Transduce(object={self._one_to_str(self.object)})" + + @staticmethod + def _one_to_str(x: Any) -> str: + if isinstance(x, BaseModel): + return x.model_dump_json(indent=2) + if isinstance(x, (dict, list, tuple)): + # readable generic fallback + import json + + try: + return json.dumps(x, indent=2, ensure_ascii=False, default=str) + except TypeError: + return str(x) + return str(x) + class TransductionResult: def __init__(self, value, explanation): @@ -107,6 +134,7 @@ def transducible( timeout: int = 300, post_processing_function: Optional[Callable[[BaseModel], BaseModel]] = None, persist_output: str = None, + transduce_fields: list[str] = None, ): if tools is None: tools = [] @@ -147,6 +175,7 @@ def _transducible(fn: Callable): amap_batch_size=batch_size, transduction_timeout=timeout, save_amap_batches_to_path=persist_output, + transduce_fields=transduce_fields, ) target_ag_template.instructions = f""" @@ -516,8 +545,10 @@ async def semantic_merge(instance1: BaseModel, instance2: BaseModel) -> BaseMode async def generate_prototypical_instances( type: Type[BaseModel], n_instances: int = 10, llm: Any = AG.get_llm_provider() ) -> list[BaseModel]: + DynamicModel = create_model( - "ListOfObjectsOfGivenType", instances=(list[type], ...) # REQUIRED field + "ListOfObjectsOfGivenType", + instances=(list[type] | None, None), # REQUIRED field ) target = AG( diff --git a/uv.lock b/uv.lock index e4304f807..f7bba9711 100644 --- a/uv.lock +++ b/uv.lock @@ -55,10 +55,12 @@ name = "agentics-py" source = { editable = "." } dependencies = [ { name = "aiosqlite" }, + { name = "async-lru" }, { name = "crewai", extra = ["google-genai", "tools"] }, { name = "crewai-tools", extra = ["mcp"] }, { name = "datasets" }, { name = "ddgs" }, + { name = "duckdb" }, { name = "google" }, { name = "hnswlib" }, { name = "ipywidgets" }, @@ -72,6 +74,7 @@ dependencies = [ { name = "numerize" }, { name = "openai" }, { name = "openapi-python-client" }, + { name = "pandas" }, { name = "plotly" }, { name = "pydantic" }, { name = "pyyaml" }, @@ -108,11 +111,13 @@ rhel = [ [package.metadata] requires-dist = [ { name = "aiosqlite", specifier = ">=0.21.0" }, + { name = "async-lru", specifier = ">=2.0.5" }, { name = "crewai", extras = ["google-genai"] }, { name = "crewai", extras = ["tools"] }, { name = "crewai-tools", extras = ["mcp"] }, { name = "datasets", specifier = ">=4.0.0,<5.0.0" }, { name = "ddgs" }, + { name = "duckdb", specifier = ">=1.4.3" }, { name = "google", specifier = ">=3.0.0,<4.0.0" }, { name = "hnswlib", specifier = ">=0.8.0" }, { name = "ipywidgets", specifier = ">=8.1.7" }, @@ -126,6 +131,7 @@ requires-dist = [ { name = "numerize", specifier = ">=0.12" }, { name = "openai", specifier = ">=1.88.0,<2.0.0" }, { name = "openapi-python-client", specifier = ">=0.24.3,<0.25.0" }, + { name = "pandas", specifier = ">=2.3.3" }, { name = "plotly", specifier = ">=6.5.0" }, { name = "pydantic" }, { name = "pyyaml", specifier = ">=6.0.2,<7.0.0" }, @@ -322,6 +328,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d2/39/e7eaf1799466a4aef85b6a4fe7bd175ad2b1c6345066aa33f1f58d4b18d0/asttokens-3.0.1-py3-none-any.whl", hash = "sha256:15a3ebc0f43c2d0a50eeafea25e19046c68398e487b9f1f5b517f7c0f40f976a", size = 27047, upload-time = "2025-11-15T16:43:16.109Z" }, ] +[[package]] +name = "async-lru" +version = "2.0.5" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b2/4d/71ec4d3939dc755264f680f6c2b4906423a304c3d18e96853f0a595dfe97/async_lru-2.0.5.tar.gz", hash = "sha256:481d52ccdd27275f42c43a928b4a50c3bfb2d67af4e78b170e3e0bb39c66e5bb", size = 10380, upload-time = "2025-03-16T17:25:36.919Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/03/49/d10027df9fce941cb8184e78a02857af36360d33e1721df81c5ed2179a1a/async_lru-2.0.5-py3-none-any.whl", hash = "sha256:ab95404d8d2605310d345932697371a5f40def0487c03d6d0ad9138de52c9943", size = 6069, upload-time = "2025-03-16T17:25:35.422Z" }, +] + [[package]] name = "attrs" version = "25.4.0" @@ -1091,6 +1106,28 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/55/e2/2537ebcff11c1ee1ff17d8d0b6f4db75873e3b0fb32c2d4a2ee31ecb310a/docstring_parser-0.17.0-py3-none-any.whl", hash = "sha256:cf2569abd23dce8099b300f9b4fa8191e9582dda731fd533daf54c4551658708", size = 36896, upload-time = "2025-07-21T07:35:00.684Z" }, ] +[[package]] +name = "duckdb" +version = "1.4.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7f/da/17c3eb5458af69d54dedc8d18e4a32ceaa8ce4d4c699d45d6d8287e790c3/duckdb-1.4.3.tar.gz", hash = "sha256:fea43e03604c713e25a25211ada87d30cd2a044d8f27afab5deba26ac49e5268", size = 18478418, upload-time = "2025-12-09T10:59:22.945Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ec/bc/7c5e50e440c8629495678bc57bdfc1bb8e62f61090f2d5441e2bd0a0ed96/duckdb-1.4.3-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:366bf607088053dce845c9d24c202c04d78022436cc5d8e4c9f0492de04afbe7", size = 29019361, upload-time = "2025-12-09T10:57:59.845Z" }, + { url = "https://files.pythonhosted.org/packages/26/15/c04a4faf0dfddad2259cab72bf0bd4b3d010f2347642541bd254d516bf93/duckdb-1.4.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:8d080e8d1bf2d226423ec781f539c8f6b6ef3fd42a9a58a7160de0a00877a21f", size = 15407465, upload-time = "2025-12-09T10:58:02.465Z" }, + { url = "https://files.pythonhosted.org/packages/cb/54/a049490187c9529932fc153f7e1b92a9e145586281fe4e03ce0535a0497c/duckdb-1.4.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:9dc049ba7e906cb49ca2b6d4fbf7b6615ec3883193e8abb93f0bef2652e42dda", size = 13735781, upload-time = "2025-12-09T10:58:04.847Z" }, + { url = "https://files.pythonhosted.org/packages/14/b7/ee594dcecbc9469ec3cd1fb1f81cb5fa289ab444b80cfb5640c8f467f75f/duckdb-1.4.3-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2b30245375ea94ab528c87c61fc3ab3e36331180b16af92ee3a37b810a745d24", size = 18470729, upload-time = "2025-12-09T10:58:07.116Z" }, + { url = "https://files.pythonhosted.org/packages/df/5f/a6c1862ed8a96d8d930feb6af5e55aadd983310aab75142468c2cb32a2a3/duckdb-1.4.3-cp311-cp311-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a7c864df027da1ee95f0c32def67e15d02cd4a906c9c1cbae82c09c5112f526b", size = 20471399, upload-time = "2025-12-09T10:58:09.714Z" }, + { url = "https://files.pythonhosted.org/packages/5b/80/c05c0b6a6107b618927b7dcabe3bba6a7eecd951f25c9dbcd9c1f9577cc8/duckdb-1.4.3-cp311-cp311-win_amd64.whl", hash = "sha256:813f189039b46877b5517f1909c7b94a8fe01b4bde2640ab217537ea0fe9b59b", size = 12329359, upload-time = "2025-12-09T10:58:12.147Z" }, + { url = "https://files.pythonhosted.org/packages/b0/83/9d8fc3413f854effa680dcad1781f68f3ada8679863c0c94ba3b36bae6ff/duckdb-1.4.3-cp311-cp311-win_arm64.whl", hash = "sha256:fbc63ffdd03835f660155b37a1b6db2005bcd46e5ad398b8cac141eb305d2a3d", size = 13070898, upload-time = "2025-12-09T10:58:14.301Z" }, + { url = "https://files.pythonhosted.org/packages/5a/d7/fdc2139b94297fc5659110a38adde293d025e320673ae5e472b95d323c50/duckdb-1.4.3-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:6302452e57aef29aae3977063810ed7b2927967b97912947b9cca45c1c21955f", size = 29033112, upload-time = "2025-12-09T10:58:16.52Z" }, + { url = "https://files.pythonhosted.org/packages/eb/d9/ca93df1ce19aef8f799e3aaacf754a4dde7e9169c0b333557752d21d076a/duckdb-1.4.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:deab351ac43b6282a3270e3d40e3d57b3b50f472d9fd8c30975d88a31be41231", size = 15414646, upload-time = "2025-12-09T10:58:19.36Z" }, + { url = "https://files.pythonhosted.org/packages/16/90/9f2748e740f5fc05b739e7c5c25aab6ab4363e5da4c3c70419c7121dc806/duckdb-1.4.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:5634e40e1e2d972e4f75bced1fbdd9e9e90faa26445c1052b27de97ee546944a", size = 13740477, upload-time = "2025-12-09T10:58:21.778Z" }, + { url = "https://files.pythonhosted.org/packages/5f/ec/279723615b4fb454efd823b7efe97cf2504569e2e74d15defbbd6b027901/duckdb-1.4.3-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:274d4a31aba63115f23e7e7b401e3e3a937f3626dc9dea820a9c7d3073f450d2", size = 18483715, upload-time = "2025-12-09T10:58:24.346Z" }, + { url = "https://files.pythonhosted.org/packages/10/63/af20cd20fd7fd6565ea5a1578c16157b6a6e07923e459a6f9b0dc9ada308/duckdb-1.4.3-cp312-cp312-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4f868a7e6d9b37274a1aa34849ea92aa964e9bd59a5237d6c17e8540533a1e4f", size = 20495188, upload-time = "2025-12-09T10:58:26.806Z" }, + { url = "https://files.pythonhosted.org/packages/8c/ab/0acb4b64afb2cc6c1d458a391c64e36be40137460f176c04686c965ce0e0/duckdb-1.4.3-cp312-cp312-win_amd64.whl", hash = "sha256:ef7ef15347ce97201b1b5182a5697682679b04c3374d5a01ac10ba31cf791b95", size = 12335622, upload-time = "2025-12-09T10:58:29.707Z" }, + { url = "https://files.pythonhosted.org/packages/50/d5/2a795745f6597a5e65770141da6efdc4fd754e5ee6d652f74bcb7f9c7759/duckdb-1.4.3-cp312-cp312-win_arm64.whl", hash = "sha256:1b9b445970fd18274d5ac07a0b24c032e228f967332fb5ebab3d7db27738c0e4", size = 13075834, upload-time = "2025-12-09T10:58:32.036Z" }, +] + [[package]] name = "durationpy" version = "0.10" From 651fd9d5615a731cf6887d4fc55cbbcfeefcb602 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Fri, 9 Jan 2026 11:46:57 -0500 Subject: [PATCH 08/27] Discovery Demo --- examples/discovery/agentic_db.py | 130 ++ examples/discovery/atypes.py | 116 ++ ...on-in-south-american-freshwater-fishes.csv | 461 +++++++ .../evolution_freshwater_fish/metadata_0.json | 158 +++ .../evolution_freshwater_fish/metadata_1.json | 158 +++ .../evolution_freshwater_fish/metadata_2.json | 152 +++ .../evolution_freshwater_fish/metadata_3.json | 152 +++ .../invaded_niche_pathways.csv | 633 ++++++++++ .../invasion_success_pathways.csv | 82 ++ .../metadata_0.json | 305 +++++ .../metadata_1.json | 299 +++++ .../metadata_2.json | 299 +++++ .../metadata_3.json | 304 +++++ .../metadata_4.json | 304 +++++ .../metadata_5.json | 304 +++++ .../phylogenetic_tree.txt | 1 + .../temporal_trends_contingency_table.csv | 13 + ..._heterogeneity_in_replication_projects.csv | 87 ++ .../data/demo/meta_regression/metadata_0.json | 287 +++++ .../data/demo/meta_regression/metadata_1.json | 277 ++++ .../demo/meta_regression/metadata_10.json | 277 ++++ .../demo/meta_regression/metadata_11.json | 277 ++++ .../demo/meta_regression/metadata_12.json | 277 ++++ .../demo/meta_regression/metadata_13.json | 287 +++++ .../demo/meta_regression/metadata_14.json | 272 ++++ .../demo/meta_regression/metadata_15.json | 282 +++++ .../demo/meta_regression/metadata_16.json | 277 ++++ .../demo/meta_regression/metadata_17.json | 282 +++++ .../demo/meta_regression/metadata_18.json | 282 +++++ .../demo/meta_regression/metadata_19.json | 292 +++++ .../data/demo/meta_regression/metadata_2.json | 277 ++++ .../data/demo/meta_regression/metadata_3.json | 282 +++++ .../data/demo/meta_regression/metadata_4.json | 272 ++++ .../data/demo/meta_regression/metadata_5.json | 282 +++++ .../data/demo/meta_regression/metadata_6.json | 282 +++++ .../data/demo/meta_regression/metadata_7.json | 277 ++++ .../data/demo/meta_regression/metadata_8.json | 277 ++++ .../data/demo/meta_regression/metadata_9.json | 272 ++++ .../worldbank_education_gdp/metadata_0.json | 216 ++++ .../worldbank_education_gdp/metadata_1.json | 212 ++++ .../worldbank_education_gdp/metadata_2.json | 212 ++++ .../worldbank_education_gdp/metadata_3.json | 212 ++++ .../worldbank_education_gdp/metadata_4.json | 212 ++++ .../worldbank_education_gdp.csv | 13 + ...ducation_expenditure_percentage_of_GNI.csv | 3 + ..._and_services_annual_percentage_growth.csv | 3 + .../GNI_per_capita_constant_2015_USdollar.csv | 3 + ...pulation_ages_15+_modeled_ILO_estimate.csv | 3 + ...ol_enrollment_primary_percentage_gross.csv | 3 + ..._enrollment_secondary_percentage_gross.csv | 3 + .../metadata_0.json | 1109 +++++++++++++++++ .../metadata_1.json | 1104 ++++++++++++++++ .../metadata_2.json | 1104 ++++++++++++++++ .../metadata_3.json | 1104 ++++++++++++++++ .../metadata_4.json | 1104 ++++++++++++++++ examples/discovery/demo.py | 155 +++ examples/discovery/discovery_playground.ipynb | 64 + examples/discovery/my_sql_db.py | 149 +++ examples/discovery/transductions.py | 85 ++ src/agentics/core/semantic_operators.py | 57 +- 60 files changed, 16663 insertions(+), 15 deletions(-) create mode 100644 examples/discovery/agentic_db.py create mode 100644 examples/discovery/atypes.py create mode 100644 examples/discovery/data/demo/evolution_freshwater_fish/body-size-evolution-in-south-american-freshwater-fishes.csv create mode 100644 examples/discovery/data/demo/evolution_freshwater_fish/metadata_0.json create mode 100644 examples/discovery/data/demo/evolution_freshwater_fish/metadata_1.json create mode 100644 examples/discovery/data/demo/evolution_freshwater_fish/metadata_2.json create mode 100644 examples/discovery/data/demo/evolution_freshwater_fish/metadata_3.json create mode 100644 examples/discovery/data/demo/introduction_pathways_non-native_plants/invaded_niche_pathways.csv create mode 100644 examples/discovery/data/demo/introduction_pathways_non-native_plants/invasion_success_pathways.csv create mode 100644 examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_0.json create mode 100644 examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_1.json create mode 100644 examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_2.json create mode 100644 examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_3.json create mode 100644 examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_4.json create mode 100644 examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_5.json create mode 100644 examples/discovery/data/demo/introduction_pathways_non-native_plants/phylogenetic_tree.txt create mode 100644 examples/discovery/data/demo/introduction_pathways_non-native_plants/temporal_trends_contingency_table.csv create mode 100644 examples/discovery/data/demo/meta_regression/meta-regression_joined_data_heterogeneity_in_replication_projects.csv create mode 100644 examples/discovery/data/demo/meta_regression/metadata_0.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_1.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_10.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_11.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_12.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_13.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_14.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_15.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_16.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_17.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_18.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_19.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_2.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_3.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_4.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_5.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_6.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_7.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_8.json create mode 100644 examples/discovery/data/demo/meta_regression/metadata_9.json create mode 100644 examples/discovery/data/demo/worldbank_education_gdp/metadata_0.json create mode 100644 examples/discovery/data/demo/worldbank_education_gdp/metadata_1.json create mode 100644 examples/discovery/data/demo/worldbank_education_gdp/metadata_2.json create mode 100644 examples/discovery/data/demo/worldbank_education_gdp/metadata_3.json create mode 100644 examples/discovery/data/demo/worldbank_education_gdp/metadata_4.json create mode 100644 examples/discovery/data/demo/worldbank_education_gdp/worldbank_education_gdp.csv create mode 100644 examples/discovery/data/demo/worldbank_education_gdp_indicators/Adjusted_savings_education_expenditure_percentage_of_GNI.csv create mode 100644 examples/discovery/data/demo/worldbank_education_gdp_indicators/Exports_of_goods_and_services_annual_percentage_growth.csv create mode 100644 examples/discovery/data/demo/worldbank_education_gdp_indicators/GNI_per_capita_constant_2015_USdollar.csv create mode 100644 examples/discovery/data/demo/worldbank_education_gdp_indicators/Labor_force_participation_rate_total_percentage_of_total_population_ages_15+_modeled_ILO_estimate.csv create mode 100644 examples/discovery/data/demo/worldbank_education_gdp_indicators/School_enrollment_primary_percentage_gross.csv create mode 100644 examples/discovery/data/demo/worldbank_education_gdp_indicators/School_enrollment_secondary_percentage_gross.csv create mode 100644 examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_0.json create mode 100644 examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_1.json create mode 100644 examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_2.json create mode 100644 examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_3.json create mode 100644 examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_4.json create mode 100644 examples/discovery/demo.py create mode 100644 examples/discovery/discovery_playground.ipynb create mode 100644 examples/discovery/my_sql_db.py create mode 100644 examples/discovery/transductions.py diff --git a/examples/discovery/agentic_db.py b/examples/discovery/agentic_db.py new file mode 100644 index 000000000..54d186d3d --- /dev/null +++ b/examples/discovery/agentic_db.py @@ -0,0 +1,130 @@ +from dotenv import load_dotenv + +load_dotenv() + +import csv +import json +import os +import sqlite3 +from pathlib import Path +from typing import IO, Optional, Union + +import pandas as pd +from pydantic import BaseModel, ConfigDict, PrivateAttr + + +def read_table_smart(path: str) -> pd.DataFrame: + # read a small sample (first line or two) + with open(path, "r", encoding="utf-8") as f: + sample = f.readline() + + # count potential delimiters + commas = sample.count(",") + tabs = sample.count("\t") + semicolons = sample.count(";") + + # choose the most frequent one + if tabs > commas and tabs > semicolons: + sep = "\t" + elif semicolons > commas: + sep = ";" + else: + sep = "," + + # print(f"Detected delimiter: {repr(sep)}") + return pd.read_csv(path, sep=sep, engine="python") + + +class AgenticDB(BaseModel): + model_config = ConfigDict(arbitrary_types_allowed=True) + name: Optional[str] = None + # _conn: sqlite3.Connection = PrivateAttr(default=None) + dataset_description: Optional[str] = None + name: Optional[str] = None + df: Optional[dict] = None + path: Optional[str] = None + columns: Optional[list] = None + metadata: Optional[dict] = None + + @classmethod + def import_from_discovery_bench_metadata(cls, metadata_path: str): + output_dbs = [] + metadata = json.load(open(metadata_path, "r")) + + datadaset_csvs = [ + metadata["datasets"][i]["name"] + for i in range(len(metadata["datasets"])) + if metadata["datasets"][i]["name"].endswith("csv") + ] + datadaset_txts = [ + metadata["datasets"][i]["name"] + for i in range(len(metadata["datasets"])) + if metadata["datasets"][i]["name"].endswith("txt") + ] + datadaset_descriptions = [ + metadata["datasets"][i]["description"] + for i in range(len(metadata["datasets"])) + ] + columns = [ + metadata["datasets"][i]["columns"]["raw"] + for i in range(len(metadata["datasets"])) + ] + for dataset_csv, dataset_description, column in zip( + datadaset_csvs, datadaset_descriptions, columns + ): + data_path = Path(os.path.dirname(metadata_path)) / dataset_csv + database = AgenticDB( + dataset_description=dataset_description, + df=read_table_smart(data_path).to_dict(), + path=str(data_path), + name=dataset_csv, + columns=column, + ) + # database.import_db_from_csv(data_path) + output_dbs.append(database) + return output_dbs + + def import_db_from_csv(self, source: Union[str, IO[bytes]] = None): + """ + Import a CSV file into an in-memory or file-based SQLite database. + Supports both: + - path to a CSV file (string) + - file-like buffer (from Streamlit uploader or similar) + """ + # Handle file path or buffer + if isinstance(source, str): + self.path = source + self.df = pd.read_csv(source) + elif source is not None: + self.path = ":memory:" # store in memory if no path + self.df = pd.read_csv(source) + elif self.path: + self.df = pd.read_csv(self.path) + else: + raise ValueError( + "Either a CSV file path or a file-like object must be provided." + ) + + # Connect to SQLite (memory or file) + # self._conn = sqlite3.connect(self.path) + + # Write dataframe to DB + # self.df.to_sql("data", self._conn, index=False, if_exists="replace") + + return self.df + + def query_db(self, sql: str) -> pd.DataFrame: + """Execute an SQL query and return the result as a pandas DataFrame.""" + try: + df = pd.read_sql_query(sql, self._conn) + return df + except Exception as e: + print(f"Error executing query: {e}") + return pd.DataFrame() + + def get_description(self) -> str: + return f"""=============================== +Dataset Path: {self.path} +Dataset Description: {self.dataset_description} +Columns: {self.columns} +""" diff --git a/examples/discovery/atypes.py b/examples/discovery/atypes.py new file mode 100644 index 000000000..3ce513499 --- /dev/null +++ b/examples/discovery/atypes.py @@ -0,0 +1,116 @@ +import json +import os +from pathlib import Path +from typing import Optional, Union + +from agentic_db import AgenticDB +from pydantic import BaseModel, Field + +from agentics import AG + + +class IntermediateEvidence(BaseModel): + evidence_found: Optional[bool] = Field( + None, + description="Return True if you found any relevant evidence for the QUESTION, False otherwise", + ) + original_question: Optional[str] = None + evidence: Optional[str] = Field( + None, + description="Identify useful information needed to answer the given QUESTION", + ) + partial_answer: Optional[str] = Field( + None, + description="Provide a partial answer for the given Question bsed on the SOURCE data", + ) + + +class Answer(BaseModel): + short_answer: Optional[str] = Field( + None, + description="Provide a one sentence answer which specifically addresses the question", + ) + full_answer: Optional[str] = Field( + None, + description="Provide a detailed answer for the given question taking into consideration the evidence sources provided", + ) + selected_evidence: Optional[list[IntermediateEvidence]] + confidence: Optional[float] = None + + +class Question(BaseModel): + qid: Optional[Union[int, str]] = None + true_hypothesis: Optional[str] = None + generated_hypothesis: Optional[str] = Field( + None, + description="A specific hypothesis that supports the question, derived from the analysis of the input dataset", + ) + domain_knowledge: Optional[str] = None + question_type: Optional[str] = None + question: Optional[str] = None + dataset: Optional[str] = None + dbs: Optional[list[AgenticDB]] = None + intermediate_evidence: Optional[list[IntermediateEvidence]] = [] + full_answer: Optional[Answer] = None + + @classmethod + def import_questions_from_metadata_as_ag( + cls, metadata_path: str, import_dbs: bool = False + ) -> AG: + metadata = json.load(open(metadata_path, "r")) + dbs = AgenticDB.import_from_discovery_bench_metadata(metadata_path) + output = AG(atype=Question) + for question in metadata["queries"][0]: + question_obj = Question(**question) + question_obj.domain_knowledge = metadata.get("domain_knowledge") + if import_dbs: + question_obj.dbs = dbs + output.append(question_obj) + return output + + +class Dataset(BaseModel): + metadata_path: Optional[str] = None + datasets_descriptions: Optional[list[str]] = None + dbs: Optional[list[AgenticDB]] = None + questions: Optional[list[Question]] = [] + + @classmethod + def import_from_discovery_bench_metadata( + cls, + dataset, + metadata_path="/Users/gliozzo/Code/agentics911/agentics/sandbox/discoverybench/discoverybench/real/demo", + ) -> str: + dataset_obj = Dataset() + + if not dataset_obj.questions: + dataset_obj.questions = [] + base_path = Path(metadata_path) / dataset + print("Importing dataset", end="") + for metadata in os.listdir(base_path): + if metadata.endswith(".json"): + if not dataset_obj.dbs: + dataset_obj.dbs = AgenticDB.import_from_discovery_bench_metadata( + base_path / metadata + ) + print(".", end=".") + dataset_obj.questions += Question.import_questions_from_metadata_as_ag( + base_path / metadata, import_dbs=False + ).states + dataset_obj.get_source_descriptions() + return dataset_obj + + def get_source_descriptions(self) -> str: + self.datasets_descriptions = [] + for db in self.dbs: + + self.datasets_descriptions.append( + f""" + Dataset Description: {db.dataset_description} + Columns: {db.columns} + """ + ) + return self.datasets_descriptions + + def get_questions_as_ag(self) -> AG: + return AG(atype=Question, states=self.questions) diff --git a/examples/discovery/data/demo/evolution_freshwater_fish/body-size-evolution-in-south-american-freshwater-fishes.csv b/examples/discovery/data/demo/evolution_freshwater_fish/body-size-evolution-in-south-american-freshwater-fishes.csv new file mode 100644 index 000000000..878ad0126 --- /dev/null +++ b/examples/discovery/data/demo/evolution_freshwater_fish/body-size-evolution-in-south-american-freshwater-fishes.csv @@ -0,0 +1,461 @@ +"HYBAS_ID","long","lat","BAMM_speciation","BAMM_extinction","BAMM_NetDiv","DR","BEL_evol","MBL_evol","OGP_evol","RES_evol","RML_evol","bio1","bio12","runoff","aet","Elevation","sgr","soil_div","area","diversity" +6050000010,-76.4774219549273,7.74269320604155,0.13739216495154,0.0268068244251531,0.110585340526387,0.0892012154727722,0.228050405622689,0.0581830421011884,0.00171409048193576,0.0018735306482271,0.00403574579104354,25.5799587992937,3147.20070629782,2036,1387,330.150088287228,166,0.482401894369818,72363.7,68 +6050000740,-74.6287248003526,9.80358581864657,0.117234507531523,0.0257962743626441,0.091438233168879,0.0750202234241873,0.10472873360246,0.0364916586400268,0.00157123758885633,0.00148526432582295,0.00570267949783489,27.7163529411765,1335.01294117647,297,1082,69.4752941176471,23,0.457436241926039,17944.3,35 +6050068100,-75.2959945117735,8.44881479274636,0.119381313095128,0.0238261332096198,0.0955551798855081,0.0799968703622726,0.276340804798677,0.0529347216691029,0.00142876869071969,0.0015633757420501,0.00425768695181854,27.0975247524752,2078.82797029703,827,1312,143.032178217822,74,0.378793033593816,17105.5,44 +6050068110,-74.6084083928771,8.92286264757212,0.132477018629283,0.027776699348703,0.10470031928058,0.086327187291336,0.186213859960254,0.0532608473724975,0.00166250243875469,0.00146743925078974,0.00396205416276185,28.0206896551724,2548.65517241379,359,1445,14.7241379310345,3,0.468328194931536,610.1,48 +6050070260,-75.5915884355367,5.77009277616235,0.120127176612387,0.0229399280699747,0.0971872485424118,0.0851862856041229,0.245060668890157,0.0533734868517244,0.00178581255600299,0.00133388075169597,0.00346222531357547,20.9343360995851,2583.68914246196,1343,1371,1378.72994467497,421,0.158870009276968,61901.9,81 +6050070270,-74.1785398899462,8.88344262005263,0.124389649583059,0.0257511726630834,0.098638476919976,0.0835608570457271,0.213317038972472,0.0584318792432892,0.00138049282816502,0.00139781965861856,0.00444893847794361,28.1291139240506,2303.6835443038,368,1442,49.6962025316456,11,0.386611216423905,1690.4,50 +6050069460,-73.7670872255361,7.79429629145814,0.125289162382217,0.0244970130009082,0.100792149381309,0.0873373310332919,0.237757539957317,0.0554832652372863,0.00149820708182469,0.00156484144745499,0.00362094360274314,25.3597602739726,2197.15496575342,729,1396,561.403253424658,211,0.3015826810468,25043,59 +6050069430,-73.5370079300652,9.9555634383509,0.129043112742464,0.0247506515783749,0.104292461164089,0.0881020492088156,0.286270222541636,0.0545917979050759,0.00151519440846914,0.00148225703349778,0.00361853546969057,25.1005730659026,1649.48519579752,529,1199,509.430754536772,198,0.259359575237461,22211.6,49 +6050089660,-73.1606221533684,6.22583420861767,0.120692088535734,0.0222208496058962,0.0984712389298377,0.0854718692438426,0.236529930738534,0.0543570941830004,0.00171969996833433,0.00145136963518753,0.00387997191656929,15.6924528301887,1674.06320754717,899,1103,2242.7141509434,549,0.174086282673598,22559,66 +6050089740,-74.922594933914,4.56384320404757,0.118464748611877,0.0240726256824197,0.0943921229294571,0.0828392241725277,0.240156410880003,0.0516461916405247,0.00165213021006492,0.00138647184164375,0.00350984279802282,20.6540559440559,2048.03193473193,1078,1298,1402.66923076923,395,0.182303649656356,91668.8,90 +6050000750,-72.6714760075139,10.8408027456384,0.13488799735329,0.0323986069128087,0.102489390440481,0.0787654313099504,0.240470747325315,0.0436470537087491,0.00176444220175469,0.00167320147956497,0.00366597947940408,25.7819157392687,1253.48608903021,330,896,398.31319554849,186,0.35444530479056,53087,50 +6050001940,-72.711608183555,8.51247183586993,0.141135558894402,0.0334566025615871,0.107678956332815,0.0851932724121202,0.185688364320946,0.0530012527225258,0.00227555392253237,0.00182448911452671,0.00349561274829568,23.7303076923077,1980.02153846154,822,1349,754.18,284,0.315537979207089,27553.5,42 +6050001950,-70.6214614458899,10.1541527442494,0.147370395505444,0.0384007945476762,0.108969600957768,0.0806422484936051,0.119203543204827,0.0339262758511252,0.00200932762331676,0.00135806929940346,0.00461680642651656,24.5893188230552,1005.72390165256,258,843,533.085046352277,218,0.287036219096805,52530.4,32 +6050003210,-69.7094509871525,10.3283845189236,0.149304585406224,0.0387266629613161,0.110577922444908,0.0875888760373884,0.174660092055074,0.0393102466912786,0.00202667765280741,0.00131885119499177,0.0042090985699784,23.3854952830189,846.33608490566,128,713,663.923349056604,145,0.201520086342132,17722,32 +6050003220,-67.2623865819549,10.3480077031076,0.129901719044563,0.0318296215069917,0.0980720975375711,0.080657842198783,0.256331728592092,0.0382708958504927,0.00197560514325452,0.000959260141197006,0.00140621239412875,24.2244831338411,1272.31882480958,218,1049,470.078346028292,184,0.334642748826987,19353.5,12 +6050003680,-65.2753916602088,9.40135168848879,0.144581840249466,0.0391622970685481,0.105419543180918,0.084995475861051,0.186299553627945,0.0405844205208226,0.00224012250029614,0.00159514047783115,0.00380220666825874,26.5632575757576,996.659090909091,91,842,140.041666666667,29,0.515447776637816,22318.1,25 +6050003690,-63.5720156241077,9.98723503413084,0.165641369298878,0.0648223359090184,0.10081903338986,0.10517842513995,0.314451674270876,0.335797399322428,0.0120527026381583,0.00536467246114818,0.01231534262626,25.1903163331181,1271.07165913493,222,1030,274.72046481601,94,0.405404024894812,32719.1,7 +6050004060,-63.0514521337987,9.04149858941877,0.151581613103284,0.0375086112473693,0.114073001855915,0.0860391612652054,0.196470635502647,0.0516592293966886,0.00124190999151478,0.00142743051308846,0.00400575628756104,26.5784023668639,1134.68343195266,133,958,86.1005917159763,18,0.641474370596991,21624.8,28 +6050004070,-61.511761749264,9.23447445655203,0.139110508234375,0.0329620383302099,0.106148469904165,0.0841565616752278,0.233847252340036,0.111896388761813,0.00420874838668048,0.00279870910999182,0.00643895503011458,26.2618834080717,1804.50560538117,400,1340,11.237668161435,8,1.24866418675813,18935.5,34 +6050004470,-61.7849635224301,8.2886260802714,0.124498387503573,0.027207661175401,0.0972907263281714,0.0863554456759261,0.220801918925573,0.0436941849427393,0.00314806142203308,0.0020682404219019,0.00537695664241216,25.8062246278755,1410.68606224628,309,1163,171.886332882273,47,1.16743790301238,15679.2,153 +6050075390,-63.6592217179798,8.21338903164683,0.124764178946324,0.0331423965347317,0.0916217824115925,0.0842733329372122,0.213855895523879,0.0446784340520209,0.00229655719081482,0.00124238801335947,0.00510679793033273,27.1945266272189,1057.15236686391,247,892,112.840236686391,34,0.453006845258492,14241.1,87 +6050075410,-62.6554777511762,5.57888269305716,0.135861762817586,0.0379564887671735,0.0979052740504129,0.0972856841617145,0.159339917810867,0.0993324704172646,0.00432337702781397,0.00380902901897118,0.00618985864997045,23.8937268518519,2630.52060185185,1511,1423,636.619212962963,124,0.57938015733644,92047.1,98 +6050079390,-63.9539346653299,7.07796794470845,0.134257819304247,0.0346491374545204,0.0996086818497265,0.0928367708278935,0.194690028306833,0.0606467580017118,0.00267135228794553,0.00166291505663809,0.00479886269641167,26.4083687943262,1756.56595744681,796,1211,242.673758865248,31,0.545642634440622,15091.1,141 +6050079310,-64.5742634934515,8.06382664588778,0.132520637996159,0.0322028308258265,0.100317807170333,0.0919564401146618,0.219249452935647,0.0730930234843438,0.00272855971248086,0.00200471702451821,0.0050346285343127,27.4547206165703,1234.18689788054,366,998,125.651252408478,28,0.403315622256632,11058,121 +6050084190,-65.6997739798733,8.34971396197758,0.136583884489975,0.0338177025020178,0.102766181987957,0.0934653770245851,0.218636075335537,0.0794840161756483,0.00295699209546313,0.0021009489415848,0.00506478855534778,27.1569696969697,1268.08398268398,409,1011,104.792207792208,19,0.466574271081511,24419,124 +6050084230,-64.5981720992154,5.73017549798689,0.13724750490652,0.0338012843613289,0.103446220545191,0.0941343279618445,0.194677087600108,0.0695443888157431,0.00296178135022111,0.00235270504436655,0.00528183266472332,25.1381997313032,2842.96641289745,1571,1467,439.780564263323,138,1.27428625606917,47861.3,186 +6050084040,-66.1914901812577,7.47246169536496,0.135253226268597,0.0324295249679732,0.102823701300624,0.0913997429428218,0.129255618881167,0.0575839853815777,0.00230431273109431,0.00136485001583345,0.00449555716288595,27.5470588235294,1582.57352941176,798,1066,71.5588235294118,22,0.445465994275248,1480.2,112 +6050084130,-65.7609695119894,6.60999858534421,0.13435535096815,0.035909193645339,0.0984461573228112,0.0885195610787921,0.165464438437499,0.0477037946140928,0.0023624441345354,0.00195250796031845,0.00470526101106802,25.4937423312883,2252.47975460123,1169,1261,420.894478527607,138,0.715226195227135,17320.6,121 +6050084460,-66.3117700043054,7.64014892943218,0.12862326546243,0.0320869243924269,0.0965363410700028,0.086943182228601,0.169601543146635,0.0542662912065515,0.00271452812257565,0.00228538095049394,0.0046505759564936,27.75,1541.5,744,1047,41,1,0.370424687079419,48,148 +6050084380,-66.8223958883636,8.26920755060317,0.131544338158736,0.0355910925491971,0.0959532456095389,0.0927240438255342,0.19786238476496,0.0793633276103005,0.00450483090212975,0.00337440890341717,0.00715020100504587,27.3007788161994,1360.20872274143,464,1029,71.6105919003115,10,0.576251430577121,13613.3,91 +6050084750,-67.1085513002423,7.70743627669637,0.14661984652755,0.0322319327949625,0.114387913732587,0.0988511567433251,0.183193054575229,0.0577805973745068,0.00296321364231491,0.00185079031595849,0.00491780036039445,27.4815068493151,1609.46575342466,708,1056,43.3287671232877,4,0.409776051838928,3180.5,114 +6050084790,-68.6962492220027,6.86834635205959,0.139164221416753,0.0347450309764817,0.104419190440272,0.0958617372831603,0.202271171899231,0.0748520145756479,0.00307191125149745,0.00230560717552111,0.00549485427272718,26.0170737664875,2055.39179286761,1177,1192,326.759404005862,103,0.504241621822489,87129.7,322 +6050081020,-67.8503953192256,7.87354476550965,0.132795740767951,0.0273919305422093,0.105403810225742,0.0918123070406378,0.243670062697097,0.0486478540478777,0.00327657240171606,0.00198977363508228,0.00441190113804934,27.4384615384615,1663.38461538462,701,1078,55.4153846153846,6,0.630982392605209,1329.8,90 +6050080920,-66.9718028111806,9.36218838206793,0.144116458920013,0.0323110125619264,0.111805446358087,0.0969072171400757,0.226407689946635,0.0584792011300413,0.00257657427832178,0.00195947049418974,0.00363674439105437,26.0427322404372,1157.44918032787,228,962,258.609836065574,39,0.291517768709275,19463.5,84 +6050079970,-67.8351596234938,7.96995617226676,0.147900947818001,0.029694012092602,0.118206935725399,0.103942867736925,0.23260031885515,0.0583943382791713,0.00328072204954867,0.00195175209271073,0.0056494910658355,27.4555555555556,1662.22222222222,668,1088,54.4444444444444,4,0.489389772771435,401.4,81 +6050079860,-68.6367063883128,9.25656777638745,0.144420269764612,0.0302077089556797,0.114212560808932,0.101544542036766,0.25042007251496,0.0550975171838551,0.00312496985356422,0.00187360999316422,0.00475441010825478,26.0675894378194,1391.63628620102,423,1101,288.250851788756,54,0.338764718810625,49730.4,152 +6050079280,-69.8329428690005,8.14993771316916,0.142531417289703,0.0338516940396442,0.108679723250059,0.097170777706846,0.237955741686276,0.0591132528286278,0.00363137715607331,0.00245663309118946,0.00606157966226698,25.8644015444015,1568.07104247104,676,1138,399.743629343629,112,0.398165185818207,27441.9,157 +6050079250,-68.9030339768887,8.32028609354726,0.14377761034443,0.0307397767078908,0.113037833636539,0.103841478917712,0.190497546421736,0.0667282992077556,0.00377554992552365,0.00236345364133627,0.00438139418370159,27.6610169491525,1455.90960451977,572,1103,78.954802259887,6,0.741522544252601,7511.9,82 +6050102250,-68.7835476189347,6.09257923503931,0.134546686723775,0.0338110643916525,0.100735622332123,0.0953794026575926,0.241728526234671,0.0769085203025046,0.00531893776889385,0.00421284022170357,0.00585108605457046,27.7899749373434,2292.03258145363,1590,1293,87.4385964912281,17,0.605329397027641,8479.9,162 +6050102330,-69.1065133474363,4.85560775591376,0.138521152098311,0.0351948915827661,0.103326260515545,0.0947492263526595,0.218465888442394,0.0826369398187585,0.00393533628124946,0.00300051727772208,0.00607405066983447,27.1547481243301,2650.51468381565,1776,1422,175.347481243301,48,0.712234469741955,99834.5,426 +6050134450,-67.723614167922,4.06387190823366,0.137128513848487,0.0320784206716353,0.105050093176852,0.0914656096758782,0.218863311098879,0.0730313371008778,0.00346668312571301,0.00222711290460646,0.00700285442049289,27.5,2934,1995,1543,96,0,0.799030572878243,11.5,166 +6050134460,-67.342334194182,4.00254876479367,0.12908954378835,0.0332547831609499,0.0958347606274005,0.0843529543068524,0.162175472890628,0.0688579426788992,0.00277081964887798,0.00220560442633089,0.0056254902265042,27.4298076923077,3091.65384615385,2003,1544,114.442307692308,19,1.20462958272331,2264.6,150 +6050134890,-67.8082742093376,3.96124910909495,0.134327576346751,0.0319536009139479,0.102373975432803,0.0924120564634889,0.197808820724647,0.0675715487190102,0.00396611210095261,0.00365717140480128,0.00607789500925845,27.3176470588235,2910.64705882353,2026,1534,100.705882352941,22,1.4682837284074,363.7,167 +6050134790,-67.3565239636009,3.28625420975961,0.127455496585386,0.0345409708417091,0.0929145257436772,0.081495587096007,0.196122344355422,0.0770344438023013,0.0030852645599218,0.00222516912813386,0.00699037101145052,26.9382252559727,3134.7866894198,2255,1652,107.848122866894,17,1.57478897188935,12532.3,126 +6050083060,-70.9950117058177,7.88945352177822,0.137335130389276,0.0327114250479281,0.104623705341348,0.0958132430811639,0.321598063721809,0.0631840969719228,0.00467660374926642,0.00278356828065232,0.00557988209738155,24.9142640364188,1661.54779969651,905,1241,522.538694992413,155,0.305380791178002,14044.6,81 +6050083130,-71.4829352912219,7.52345735114238,0.149065523666207,0.0345241892370602,0.114541334429147,0.109015123555608,0.282160477401239,0.0594135258143026,0.00436730764244538,0.00192444065729256,0.00621613511711052,23.2288401253918,1895.85736677116,1128,1244,724.470219435737,216,0.341432632187836,13619,50 +6050104180,-71.2373118538995,6.30263410257663,0.148909813205211,0.0336123243959896,0.115297488809222,0.104594814416724,0.263623258453781,0.0794650643075791,0.00293830064779185,0.00206747997701369,0.00491372273634017,25.0331975560081,2309.4816700611,1261,1328,440.284114052953,128,0.42130623744262,21147.7,172 +6050104230,-71.3531531272555,5.42338134407189,0.138404078934357,0.0333645030995804,0.105039575834777,0.0958606403790819,0.23235021392209,0.0800946837610879,0.00359644299849155,0.00270747000191337,0.00575490994431332,25.8700391644909,2153.19778067885,1444,1307,335.338120104439,73,0.458876869950173,32711.7,312 +6050137270,-69.2483958694811,3.73675225665104,0.139194950638982,0.0320355882651838,0.107159362373799,0.0898515791700976,0.165182740968913,0.05152590744003,0.00307333681652769,0.00292520066642449,0.00411789515663291,26.8857610474632,2950.59410801964,1643,1514,143.504091653028,22,1.64721004528264,13041,172 +6050137380,-70.2225414889378,2.59763345679106,0.129703220858812,0.0302107722187785,0.099492448640033,0.0850488118069828,0.166172246173835,0.0588349194663671,0.00294901680232524,0.00238378264390753,0.00573795074023254,26.7029411764706,2913.04054054054,1772,1546,179.79213036566,25,1.61316871924933,53814.6,219 +6050136480,-66.5124925644818,3.50393318794213,0.127513475861665,0.0312964937060088,0.0962169821556559,0.0874196626655405,0.177305888803754,0.0626536140943585,0.00340249528840971,0.00294953466368709,0.00614627781466145,26.9294,3338.876,2104,1713,142.036,36,1.46426945240404,10747.4,204 +6050136450,-65.9585801554701,4.84246816342685,0.138567792315632,0.034937843190602,0.10362994912503,0.0936827694852334,0.200053901682597,0.0788757118847628,0.00296417447654009,0.00247523726580408,0.00618272982481499,25.2187336473051,2900.66562009419,1690,1468,489.784929356358,178,0.892638699021304,40859.4,277 +6050150590,-65.5971827746731,3.12482466132382,0.134119564133784,0.0367177465594361,0.0974018175743482,0.0877685242511089,0.222525222234936,0.0883935298404127,0.00362715370364429,0.00337671776147569,0.00582089449809771,26.1680412371134,2955.15979381443,1824,1699,246.314432989691,111,1.27145300395115,4054.6,126 +6050150580,-65.7056925052366,3.73774703193431,0.133530332489572,0.0358248769308223,0.0977054555587494,0.091107843837939,0.171379660879896,0.0636471862902549,0.00327903250317358,0.00300743684220686,0.00626411989965347,23.5702422145329,3206.17993079585,1903,1608,725.387543252595,234,1.14683926002108,6205.4,148 +6050126730,-72.48934804138,5.05872350120982,0.147839938026332,0.0328829092212766,0.114957028805055,0.109056765079797,0.278279639188918,0.0789138235125358,0.00351729945568352,0.00257173200996558,0.00465457426197227,23.9925619834711,2681.15289256198,1578,1320,723.487603305785,210,0.298007596133268,5201.7,178 +6050126850,-72.0029016515794,4.48422322391613,0.13929364485414,0.0338196586548676,0.105473986199273,0.0992150620039616,0.254119116003888,0.056390012232904,0.00295920231782206,0.00212307933088127,0.00503178391203418,26.875,2152.20833333333,1553,1304,152.583333333333,12,0.611355097432895,507,153 +6050129600,-72.5380495835559,4.44223855075017,0.135662413804364,0.0330714971578611,0.102590916646503,0.0953766201242267,0.228220625904037,0.0885640841322457,0.00410691983056638,0.00314778273103594,0.00626750361710117,26.5508241758242,2561.42032967033,1721,1347,226.472527472527,39,0.501398573628636,7737,230 +6050129810,-72.6316079689954,3.58680366021797,0.136889385161688,0.0339694583213731,0.102919926840315,0.0961941217608971,0.221283512417291,0.0900265992696706,0.00431296255571771,0.00372923204917191,0.00643052121156319,26.9120430107527,2477.38709677419,1730,1387,204.466666666667,26,0.516518987374959,10052.1,204 +6050154590,-65.00736475498,3.58734447347186,0.133924626925518,0.0351955713460195,0.0987290555794984,0.086003086847853,0.179067108841549,0.086988377382296,0.00309389593342167,0.0025875633288613,0.00695805127364478,23.4366167023555,2787.93576017131,1634,1585,719.886509635974,188,1.29191559132187,9985.6,161 +6050154700,-65.2260102455515,2.8486515725736,0.124772660033421,0.0290088473730993,0.0957638126603222,0.0849517200160301,0.169489106562957,0.0633466431125775,0.00284011056042582,0.00255164208504614,0.00573381452438601,26.78,2502.2,1401,1639,149.6,25,1.7404661374202,150.3,156 +6050156880,-64.6121458498554,2.24223753778295,0.135226702985251,0.0327869639820678,0.102439739003183,0.0895660774737047,0.184715009940774,0.0547031553485347,0.00306749556257441,0.00283266132916138,0.00650507821682379,24.9851805728518,2319.41843088418,1132,1590,440.579078455791,128,1.23644724252424,17297.4,138 +6050156750,-64.5339967309063,3.04959964876806,0.131022052831253,0.033733290932067,0.097288761899186,0.0889949157506463,0.193391714022851,0.0801878015464826,0.00224202301619373,0.00229057059043403,0.00625867720403604,23.7878378378378,2436.33558558559,1310,1583,646.585585585586,196,1.03930568446313,9409.5,157 +6050143540,-71.3322750049981,3.04984671753498,0.141938710352393,0.0362439901632012,0.105694720189192,0.100431507752342,0.263317511516305,0.0726403766524053,0.00484108791508641,0.00454310411260274,0.00684728336573996,26.9446054750403,2633.83172302738,1567,1443,195.453301127214,27,1.12620005297896,26636.9,141 +6050143380,-70.5891888973976,3.68663111655628,0.137954006653031,0.0313597614853201,0.106594245167711,0.0933526040867954,0.189596620197125,0.0542280143981182,0.00275588185895253,0.00224595268790385,0.00480012282370872,26.68,2640.26046511628,1688,1422,172.893023255814,27,1.32237414403185,9221.6,191 +6050130600,-73.4629282034576,4.12845327405238,0.140155493036677,0.0320462959123984,0.108109197124278,0.10197566793197,0.257970965177228,0.0654934831119275,0.00349419708710638,0.00237921082497195,0.00508477285699681,22.5178519593614,2874.05805515239,1887,1365,961.624092888244,277,0.303866960852905,14630.1,266 +6050130490,-73.2832846763354,5.00762964039262,0.150647463580848,0.0263946953994775,0.12425276818137,0.121931507050925,0.475812258514739,0.0599569438876027,0.00340265504664915,0.00219280164934147,0.00537319262470666,17.2341463414634,2256.72195121951,1244,1140,1994.43902439024,584,0.179538001305355,8799.7,58 +6050160900,-73.9169046880528,2.63045845007947,0.141609415567939,0.032942970414507,0.108666445153432,0.0999467254828317,0.284389068742653,0.0586315134632277,0.00325496008352996,0.00220115567428365,0.00383011083016627,24.6418213969938,2493.92926613616,1307,1496,596.061892130858,143,0.588959236115421,24345.7,133 +6050160730,-73.6169881835275,3.2815838632285,0.143079427542268,0.0341558495488848,0.108923577993383,0.103069994261392,0.280988992397559,0.0776291551772656,0.00391045337306075,0.00280669437383951,0.00507291142139431,23.802380952381,2741.47985347985,1578,1428,756.186813186813,219,0.333700140245698,11638.3,185 +6050004480,-59.8219918725973,7.67017465818332,0.123562837554595,0.0300566124914188,0.0935062250631764,0.0886484550429087,0.137884944554126,0.0543667049381258,0.00229048569219919,0.00129651753004579,0.00445043951261214,26.2498883928571,2357.15625,985,1422,61.0703125,21,1.69258872580635,38300.7,88 +6050004870,-58.5886034782679,6.44281643790279,0.141508929197858,0.0346044336180407,0.106904495579817,0.0954458800460673,0.153828145135661,0.0706579030937844,0.00268336512665832,0.00280880362192918,0.00459128282034345,26.3,1504,997,1257,5,0,1.73189397971047,1.1,125 +6050099060,-58.624775277363,5.77528486409685,0.132500121983193,0.0302593693886729,0.10224075259452,0.0941116598228586,0.159614454574178,0.0724006886517953,0.00330865925467683,0.00380326428790281,0.00619171252483009,26.2387978142077,2619.91803278689,1352,1485,73.0327868852459,34,1.60477367067514,4022.7,144 +6050099010,-60.7925944348186,6.53121309905731,0.133707781083506,0.0318868836009383,0.101820897482568,0.0927264119255152,0.18118083239676,0.0768324603356191,0.00305098323248698,0.0034181832572236,0.00593270545081355,25.2560109972507,1834.69557610597,995,1319,311.896775806048,78,1.14919745819483,85209,185 +6050113570,-59.4785661299246,5.22486098275591,0.132609897386965,0.0339299601742298,0.0986799372127355,0.0902508562332823,0.175950441160918,0.0878279243144364,0.0032156742722378,0.00381123123616748,0.00621743502699155,24.2712074303406,2738.21671826625,1894,1461,470.724458204334,115,1.32333017365481,6842,135 +6050113660,-58.9976758859033,5.05350786051925,0.126562042464097,0.0296998151128534,0.096862227351244,0.0922228863612462,0.165216224058286,0.0644648172832807,0.00251582996734196,0.00389945172341384,0.00654679775383884,26.0323076923077,2797.49230769231,1880,1500,183.5,125,1.47822198648101,2829.4,97 +6050121460,-58.5916778789472,4.44208030943297,0.13799920788124,0.030590148950748,0.107409058930492,0.0967373981550362,0.183342974979352,0.0585428925433886,0.00266758469348008,0.00327824727751919,0.00523215211836328,26.6787878787879,2209.32575757576,988,1391,119.55303030303,34,1.71208897726507,2829.2,150 +6050121450,-59.1052807382697,4.45799937652513,0.146051911996984,0.0360153851028719,0.110036526894112,0.101457606313307,0.207195018394787,0.0610055995965896,0.00292598194431839,0.00318291911408948,0.00569989360192171,26.0580441640379,1997.81703470032,1168,1299,231.917981072555,99,1.57552166162405,6762.2,89 +6050134950,-58.6162113047743,2.3697551372399,0.132898377389727,0.0302035128530584,0.102694864536668,0.0893301101090981,0.127359800684581,0.0651913977362336,0.00240186085932053,0.00226853419486928,0.00389336937448014,26.3555977229602,1835.06736242884,813,1234,256.012333965844,35,1.7707878223257,22678.7,91 +6050134940,-59.0116897218953,3.11749537122418,0.134610923437645,0.0303047792243065,0.104306144213338,0.0966867306499939,0.188538087829637,0.0642931522808981,0.00276955879526512,0.00319660870170625,0.00465100340571941,26.7742805755396,1575.0773381295,756,1107,187.586330935252,38,1.06598615490944,23684.5,176 +6050004880,-58.026301370436,5.56527244993643,0.135143433899079,0.0292082801609013,0.105935153738178,0.0951975181122904,0.144404421154538,0.0728051408448666,0.00349894836634563,0.00437272032835616,0.00508276324855662,26.6575799721836,2111.62656467316,873,1396,55.2579972183588,22,0.981477219659615,30805.5,166 +6050005180,-57.249876301357,3.19428169298551,0.138236820851446,0.0331988711867023,0.105037949664744,0.0972418428168914,0.164673848746358,0.0563519897405625,0.00289120013417551,0.00283207411904418,0.00368481008729494,26.3839416058394,2056.24883875249,550,1325,221.259455872595,29,1.69524604908807,64285.5,151 +6050005190,-55.8441880035468,4.87827820401707,0.138474828298887,0.0328120401230835,0.105662788175804,0.100202507001175,0.162391823809355,0.0599637254716705,0.00342156577490494,0.00381168575306473,0.00484721680790488,26.6339920948617,2279.97719671633,838,1434,90.7093341441168,28,1.58029580286965,70315.5,183 +6050005490,-54.4854245570973,3.4379970154255,0.135895462290878,0.0327627668082714,0.103132695482606,0.0943573629346422,0.20241871916192,0.0726264873861445,0.00297922949516657,0.00293277842879547,0.00538869510637302,25.5292714377821,2355.07414571244,882,1392,198.539651837524,40,1.39274915178538,66409.3,195 +6050005500,-52.3352424810605,3.56148921403784,0.140357781547161,0.0313797642329277,0.108978017314233,0.100093652068246,0.172411949353525,0.0594190952155426,0.00303984679763159,0.00292268045701024,0.004492053787728,25.6547624358688,2948.10930180683,1609,1392,98.6328351550301,32,1.10785442913329,96410.4,236 +6050006470,-51.7472036547113,1.35856026071142,0.136891918460217,0.0361362952111615,0.100755623249055,0.0930438784976886,0.121237343121369,0.0638917572364675,0.00240731222077292,0.00264138696036003,0.00580878302300168,25.9817768803635,2507.33064109036,1093,1340,135.857142857143,27,0.631672342767814,42497.1,94 +6050006540,-51.4957995918293,-0.0451598588429001,0.119860823067717,0.031864303433062,0.0879965196346551,0.0822991582096396,0.170761498512356,0.0902394553104654,0.00457638729822352,0.00397603833898789,0.0070684852913286,26.7852515905147,2444.78253325622,1073,1304,48.9947946790052,25,0.441886184856981,37353.9,76 +6050007000,-51.3664071500667,-1.16521489408922,0.110193860393538,0.0258257494553622,0.0843681109381757,0.090489549922219,0.346197262959694,0.0457289197909677,0.00532619492211656,0.00450699569473527,0.00757815840134165,26.906711409396,2510.86912751678,1203,1484,20.6442953020134,13,1.53862857486662,6440.6,66 +6050239720,-51.9477319164333,-1.4885035497936,0.137119506308198,0.0321616906475514,0.104957815660647,0.0955833405124787,0.20436995364316,0.0637003860649671,0.00526169810095605,0.00467654267651752,0.00663412006641952,27,2385.6,1092,1423,11.8,10,0.752173461891841,142.6,175 +6050239820,-52.04528023864,-3.69057215238406,0.134878596585339,0.0302129639391728,0.104665632646166,0.0949030127707975,0.22202714829979,0.0603034936005093,0.00334776773330281,0.00293143561639953,0.00517905679805708,25.6728571428571,1960.60253968254,839,1239,144.809841269841,36,0.439698734370439,67377.7,390 +6050238750,-52.4235560497456,-1.32280523081059,0.0958983248903265,0.0180436221425022,0.0778547027478243,0.0792414932349674,0.297382612006386,0.022862553837557,0.00147053712344315,0.00103056489819067,0.00686839239694369,26.8793103448276,2162.22413793103,895,1355,56.2586206896552,19,0.644377869762217,2387.9,26 +6050238630,-53.3955196729035,0.748097743493833,0.130675764084763,0.0309739891626171,0.0997017749221456,0.0911474088600481,0.181184962946129,0.0497423141204545,0.00329393745343794,0.00271092384821634,0.0049019773112259,25.591687748464,2269.77990603542,677,1360,252.434044091073,42,1.26268621827237,59470.7,130 +6050241280,-53.8959278729574,-1.50702308296733,0.130605941497021,0.0320719621875181,0.0985339793095027,0.092639176200244,0.228976534223476,0.0533950263447579,0.00340919662091966,0.00298448950904532,0.00505728924215215,26.2507763082231,1809.45888441633,593,1224,181.525014376078,39,0.535835847195952,37364.2,208 +6050241250,-54.2765729174693,0.408951423744895,0.134136787224705,0.0318085846980657,0.102328202526639,0.0945502552373399,0.2263055968818,0.0627108316950377,0.00482003166470521,0.00401629577006508,0.00503589256155202,25.5978987068966,2029.671875,570,1313,308.549030172414,45,1.31996504937743,39822.3,184 +6050261280,-54.3620318403745,-2.40611929200803,0.134540334243595,0.0325849521916363,0.101955382051959,0.0918901841549987,0.251221032577845,0.0611338323553112,0.00427998721976868,0.0037193104963865,0.00598672931296293,26.1450819672131,1957.76229508197,697,1205,35.1885245901639,16,0.411568670064667,2613.5,178 +6050261380,-54.2250667999077,-3.43616077962729,0.124441611292916,0.0333998695858674,0.0910417417070485,0.0832426937422469,0.192509736997203,0.0582304565122185,0.00143422063375526,0.00146070376970309,0.00351709377691801,25.3915068493151,1831.43767123288,725,1211,170.943150684932,57,0.325067379786487,31253.4,63 +6050263530,-54.7026853927215,-2.41913836889395,0.136044465874351,0.0296710556369618,0.106373410237389,0.0939867550024259,0.170610980255827,0.056199428734584,0.00402813355385439,0.00352278120101877,0.00490994898963265,26.0833333333333,2104,742,1211,14.1666666666667,16,0.306319635250875,176.4,225 +6050263540,-55.1637737356328,-0.843614024498758,0.134443152993146,0.0312968803193749,0.103146272673771,0.0897466290114684,0.196158106715875,0.0554791856300905,0.00401727664237915,0.00316256866337683,0.00467192191815348,25.8312641937926,1996.08705526117,557,1326,276.165026495079,34,0.780199439928478,28404.3,166 +6050262220,-55.6483454711751,-3.63941491139777,0.135206320089854,0.0310977268177352,0.104108593272119,0.093418827406303,0.19151572448397,0.0720248045312064,0.00394505201361185,0.00361867006706156,0.00586760904696738,26.4220050125313,2067.96441102757,861,1325,110.49373433584,40,0.408389325282196,42675.3,396 +6050262110,-55.4495267607977,-2.07876214905221,0.129454291168877,0.0295352138182208,0.0999190773506567,0.0917401834051137,0.185186048417555,0.0634673352142984,0.00373625733661321,0.00326393590442621,0.00620345867497452,26.6852112676056,1945.27464788732,877,1254,31.8697183098592,16,0.335425498606407,6049,336 +6050298170,-54.1079098059147,-6.96477663618547,0.137920745217702,0.0301905199857064,0.107730225231995,0.0923909264535322,0.146900344549147,0.0693568952585778,0.00182279540292609,0.00154599284788459,0.00402398460976536,25.2134080717489,2144.50313901345,686,1353,278.567414050822,29,1.13668745086301,142352.3,145 +6050298270,-52.5561479806894,-5.56394848564537,0.135412036999282,0.028825784709549,0.106586252289733,0.0947149991482314,0.157011498869418,0.0647788331454912,0.00217613246326352,0.00290968814944195,0.00426863866643067,24.8362030905077,1867.07858719647,590,1285,236.323620309051,29,0.541050165362125,48321.9,150 +6050250260,-56.3025750093423,-2.34545986302748,0.130835587721666,0.0299961675743869,0.100839420147279,0.0950233603148868,0.248792778673806,0.0508864978433926,0.00440993962405395,0.00291545293588067,0.00527504314516137,27.2847533632287,2262.32735426009,1048,1330,30.8789237668161,20,0.580148209726077,4717.5,86 +6050250180,-57.0342121140715,-0.0282109778681508,0.128952219808501,0.0286108054022361,0.100341414406265,0.0904600117566344,0.205930999299868,0.0636079593002151,0.00347369538279199,0.00264864523279205,0.00516239368602619,26.5452172724788,2240.0688712763,839,1394,216.62380431812,36,0.900809591808614,156958.6,303 +6050266340,-57.2636975101868,-2.57422209386607,0.140786649703384,0.0353698398771339,0.10541680982625,0.0941922066809174,0.115143768636159,0.0651483073059434,0.00404069318510859,0.00392633196112838,0.00658751116424438,27.5779569892473,2134.13440860215,995,1321,15.1182795698925,9,0.480846716018988,4009.4,116 +6050266470,-58.3953826668372,-5.12939718891801,0.13040878107987,0.0319279535257644,0.0984808275541055,0.0888034550304086,0.135392371496954,0.0590849217793875,0.00291014217166189,0.00270676862254387,0.00475050413835118,26.6775637939336,2258.88621409084,1111,1416,95.2625581768577,32,1.13385740653858,133363.2,256 +6050325720,-57.3364256812115,-6.52733596989359,0.133110564603743,0.0352857865086694,0.0978247780950736,0.0914978216107269,0.181981319346758,0.103761536003888,0.00405951783291099,0.00316695733668866,0.00616395038275868,26.2354480286738,2291.83261648746,1538,1397,194.534408602151,46,1.01746674777878,59323.4,173 +6050325820,-55.8880981585339,-6.80811953578862,0.130252323167821,0.0303277081495418,0.0999246150182792,0.0905807235509494,0.204888680272209,0.0709127575824826,0.00340827372675302,0.00301725234871425,0.00539309303359091,25.7593362509117,2343.15207877462,1167,1402,257.619620714807,41,0.753126671660829,58770.3,203 +6050262360,-59.1968376760299,-1.06889798201692,0.129886471505544,0.0327896005134069,0.0970968709921374,0.0884511624271659,0.16026409999086,0.0746907691737882,0.00373201099056098,0.00305740080621369,0.00550360587955936,26.9311320754717,2255.23616352201,1064,1424,125.373270440252,34,1.34143969100926,68223.1,242 +6050262490,-57.7467916746817,-2.70282653313052,0.125494298346361,0.0289185162353274,0.0965757821110333,0.0884412589365941,0.195169433307783,0.0578875570938225,0.00332123449587664,0.00246067924909975,0.00539683731320379,27.3878787878788,2128.54545454545,938,1350,11.6969696969697,5,1.29596725893916,730.3,231 +6050272340,-59.1551650881914,-2.63423236145042,0.134714111500064,0.0364390248184175,0.0982750866816467,0.0904904008400886,0.181501970067361,0.0694322948532585,0.00283368927490165,0.00265846957929782,0.00517485163628322,26.9052892561983,2354.00495867769,1122,1445,80.9272727272727,30,0.833633134326787,12881.4,188 +6050272470,-58.3063669172098,-3.23508510793294,0.111595023635595,0.0245522268332431,0.0870427968023522,0.0830884536218753,0.212525586236261,0.0454964733137947,0.00253931028371229,0.00208364336682051,0.00582374074750204,27.0147058823529,2201.22352941176,1011,1389,19.2823529411765,9,0.673399840173379,3618.4,108 +6050384210,-51.3677742078154,-7.80014064165888,0.140480134852206,0.0327645781471647,0.107715556705041,0.10129915705252,0.178495527805038,0.0847839686799777,0.00465764303786159,0.00431192631488773,0.00619903041097961,24.8692567567568,1963.29777992278,592,1358,344.570945945946,40,0.322654868972837,43970,185 +6050384320,-52.523844831142,-9.40003066882996,0.136058981218627,0.0306918639165194,0.105367117302108,0.0982236951283456,0.181085999126424,0.0655022580694552,0.00296117191152916,0.00302364471013544,0.0059813231035807,25.347965738758,2074.21975374732,637,1359,314.125,32,0.557226166487517,79227.1,154 +6050285930,-59.5420544154053,-2.97153735322134,0.129352920255664,0.0316004816502075,0.0977524386054564,0.0928587518065815,0.250406354782843,0.0652335580494364,0.0038683813337859,0.00354926569850476,0.00585972802231356,27.1331034482759,2248.81379310345,1138,1437,54.7275862068966,25,0.512347913979197,6217.2,360 +6050285990,-60.6381033855677,-4.47567405413693,0.132715841350674,0.0317104576940689,0.101005383656605,0.0926684377118865,0.228298325388407,0.0660590191354016,0.00409330669693499,0.00336219393392965,0.00593778654428224,26.850367220719,2362.56474681098,1134,1456,43.873598763046,18,1.27395160398805,55327,344 +6050280410,-61.356689281545,-3.25156027149759,0.129567944798752,0.0299566994145394,0.0996112453842131,0.0935910147666002,0.236490498146057,0.0615176754457196,0.00451008033816529,0.00416667984625963,0.00594912185825782,26.9882978723404,2391.99903288201,1252,1473,39.8413926499033,17,1.1562285998004,22133.1,311 +6050280400,-61.4169008792983,-2.24186343500303,0.132740564058829,0.0323756149378791,0.10036494912095,0.0931709217444071,0.203841444089858,0.064638605129772,0.00381886809665662,0.00321981162772112,0.00582074269224767,26.9703887510339,2525.81265508685,1312,1540,60.1906534325889,20,1.36133891870842,51968.8,503 +6050405310,-57.9411473527225,-11.9397837890927,0.133624666906145,0.0360018538800108,0.0976228130261343,0.095574873273154,0.238285042552302,0.0713728821822325,0.00311431347679821,0.00286172305087328,0.00472936132286409,24.4663493105712,1987.45217771941,719,1318,368.950207922959,43,0.334899169513046,191776.6,139 +6050405320,-56.7589691463152,-8.70957595550764,0.137420995775262,0.0388294606571848,0.0985915351180775,0.0956462373188192,0.252363459540345,0.115276617783338,0.00510601153620785,0.00451784092928581,0.00776622719059849,25.4380864197531,2373.23518518519,1241,1392,288,50,1.36183432274879,34417.8,116 +6050337190,-61.7827523256219,-7.01290280686087,0.136530335896645,0.0311757042574037,0.105354631639242,0.0930742638804276,0.178955615058222,0.069187737740524,0.00387384909291388,0.00321394459509111,0.00532204773323744,26.2919103580213,2468.33643071878,1595,1433,74.3899972670128,22,1.44772879636446,78025,230 +6050337320,-60.1744941866006,-9.52295075774937,0.138114210187176,0.0340051921038807,0.104109018083296,0.095774801138193,0.190514770064004,0.0875126422257948,0.00367793618573483,0.00377944367642511,0.00534243459831448,24.8168582926128,2106.58062661684,984,1319,201.256108077034,34,0.824192939431257,147160.2,250 +6050294270,-63.932953962393,-4.03340833108842,0.131809039110558,0.0308389598803626,0.100970079230195,0.0912285681157708,0.145328438387168,0.0526792575840527,0.00287914090983996,0.0028357329218757,0.00516202149789871,26.519831432821,2522.49851264254,1270,1526,62.7885473475459,19,1.63419966742725,86295,342 +6050294360,-66.7721305020767,-7.97733214761062,0.134836548101724,0.0304522313593233,0.1043843167424,0.0963447587002471,0.212988421467688,0.0591601935095157,0.00358389011709111,0.00296227200458009,0.00580092726595622,25.778657168499,2210.97335651784,815,1434,150.92966120709,22,1.01922171621547,378743.9,534 +6050247260,-63.3679229789291,-1.86084071385353,0.104156655595516,0.0222812043791238,0.0818754512163921,0.0800836401779749,0.236242716615897,0.024734811431171,0.00180377189696878,0.000684034606986336,0.00756622508382678,26.6552036199095,2684.81900452489,1650,1630,57.1508295625943,16,1.68255736517342,28476.9,38 +6050247270,-60.7254898306187,-0.0878598678761201,0.127877417711465,0.0323031362059018,0.0955742815055636,0.0917759845242773,0.265880270805642,0.0779519933945234,0.00441773526510745,0.00396695731185027,0.00691300787775786,27.0856060606061,2016.03787878788,1042,1420,81.1969696969697,20,1.56668010112218,39823.5,217 +6050237360,-63.1654684858761,-0.925891205670282,0.122992755841443,0.0306944858446909,0.0922982699967524,0.0825368665150054,0.213124048001688,0.0391933601816368,0.00338947141288229,0.00269874543892519,0.00732288004205094,26.7639187574671,2420.4348864994,1238,1573,49.621266427718,14,1.64898847215827,35819,151 +6050237120,-61.4645194234659,2.64502469038547,0.133363691673689,0.0342612738974478,0.0991024177762417,0.0934667235008155,0.201610752650877,0.0700141728616952,0.00346462443592948,0.00315998749769377,0.00637832053439657,25.9687731923985,1810.03542111773,793,1269,273.083099066682,58,0.786612228629776,190789.9,434 +6050457830,-57.0504078640437,-10.1747402176207,0.133583314224821,0.0304627303617825,0.103120583863038,0.0955027502449436,0.266491653055954,0.078661510311121,0.00562044415102795,0.00630822167731117,0.00629568722496604,25.2064729194188,2172.58520475561,773,1373,298.371202113606,34,0.512951047158535,16091.5,88 +6050457840,-55.994685537733,-9.71019968751108,0.137116336074588,0.0330503644112742,0.104065971663314,0.0998131839946624,0.251507862010213,0.0968662988687651,0.00444599681798508,0.00424364872264045,0.00557022041374757,25.2727567567568,2310.39027027027,1429,1398,309.434594594595,40,0.50758796739349,19466.2,204 +6050506430,-52.5801036192141,-12.1288511994594,0.137595639604361,0.040367378212632,0.0972282613917295,0.0965778971158237,0.302712950070067,0.15091459739027,0.00395962823240969,0.0035442857734066,0.0103228875661654,25.4036927621861,1881.36558345643,476,1294,332.314623338257,24,0.34905572813645,28584.5,33 +6050506420,-54.5140756226242,-11.5833513740982,0.124949865753551,0.0345529520492007,0.0903969137043506,0.0997777525433382,0.33127302245428,0.177850972472315,0.00541097702157603,0.00637263841965484,0.00916748427469209,24.9598806860552,2074.24235645041,626,1346,348.0365398956,34,0.417565640170545,28125.5,47 +6050224360,-66.102995473921,-0.412343689783798,0.132129718389656,0.0349788730473808,0.0971508453422749,0.0892766659492263,0.216990183028348,0.0762402534740213,0.00361575536544184,0.00312264453359743,0.00658136857449317,26.1909834775767,2889.61699449253,1309,1639,110.457120377655,35,1.65842480240221,136301.8,344 +6050224470,-63.3038907384023,0.947919860521558,0.126661690506487,0.0340414796412309,0.0926202108652557,0.0831983455361977,0.17600587027058,0.0761272959305711,0.0034831852350248,0.00273815975182813,0.00606494082013295,26.4404312668464,2121.47654986523,858,1484,154.170889487871,50,1.46112353695971,39769.3,149 +6050532340,-54.3662047167459,-13.0025675088899,0.120556307367565,0.0329282025288017,0.0876281048387636,0.0960980729534977,0.24493331058005,0.114282617997268,0.00225129854064165,0.00240246330022746,0.00744140840859904,24.3893543956044,1843.82417582418,517,1289,395.916895604396,35,0.361295717842713,30434.9,42 +6050483010,-55.5350499670495,-12.4821408233228,0.144203614137301,0.0381246839243003,0.106078930213,0.104212194407143,0.180905811352136,0.101687922470301,0.00272277065104788,0.00301216305904272,0.00563417522801227,24.7338793447863,1869.88613663604,594,1308,390.107471034758,34,0.176555142699838,52499.9,109 +6050532350,-53.1714068327482,-13.5030504438142,0.136020262932729,0.0316228279866679,0.104397434946061,0.0988475428983349,0.163737785117466,0.0860176524560011,0.00344073237365027,0.00344579830453826,0.00570147037810586,24.3632407407407,1744.94490740741,549,1263,394.840740740741,34,0.315113134202046,44898.6,178 +6050483020,-54.7982921625004,-10.2273698606316,0.135620334905142,0.0360337148539089,0.099586620051233,0.0925258989463794,0.169170161753353,0.0832866658612117,0.00366497009703253,0.00345348027941985,0.00501856136807441,25.0155097613883,2257.25704989154,717,1400,357.973969631236,37,0.360040683268978,19512,161 +6050287830,-71.6890829037079,-0.302676029582835,0.135595514755865,0.0313006001656318,0.104294914590234,0.0971881383907335,0.244987955953813,0.0701899887459899,0.0035908202708389,0.00327234886608365,0.00573802084260949,25.5482917527411,2943.54878436358,1919,1566,257.696011441284,54,1.13763559236402,270156.3,563 +6050287820,-65.6040888905054,-4.01804263482118,0.126629070960485,0.0289545687173099,0.0976745022431751,0.089415536182785,0.201587652118742,0.065479204067471,0.00426900324966208,0.00356489499659398,0.00558965471068348,26.4269390171699,2734.88869153345,1652,1579,82.4179988158674,22,1.64858506481533,36309.2,310 +6050425550,-61.9209455304936,-10.6318893642433,0.135567293336039,0.0312850864611535,0.104282206874885,0.0965843176659844,0.18628355783482,0.0635704911605759,0.00350126608991308,0.00329528257204087,0.00584237209307382,24.348083427283,1961.27480270575,995,1303,214.197294250282,35,0.433366570282465,75038.6,320 +6050425540,-64.8930297115041,-9.9178567034888,0.134343493165768,0.0316230047951511,0.102720488370617,0.0954340414755654,0.236711197671398,0.070701371984053,0.00365432597760346,0.00306712595471824,0.00544477846801884,25.8904094096358,1979.06288170097,1219,1340,160.883510517982,32,0.508985128376595,93459.2,474 +6050266740,-72.127938953786,-3.3132401712122,0.136042568958228,0.0309484735585794,0.105094095399648,0.0994435267567034,0.226916224512307,0.066590738416262,0.00348003078773079,0.00282157476502755,0.00530530001937779,25.3334928428762,2772.75532623169,1835,1619,238.215962050599,48,1.42899035376129,514725.6,786 +6050266880,-70.4541743112148,-7.35747439017895,0.131066668120995,0.0304392401503749,0.10062742797062,0.0903451495721992,0.185856208103932,0.0567788626708462,0.00401513735557985,0.00352450307411788,0.00535043435850636,25.6628978036755,2257.0500896459,1035,1491,205.104325414612,25,1.13720240966299,189721.5,318 +6050207670,-67.3095156903433,1.69868102452845,0.132557034639416,0.0354271065671909,0.0971299280722253,0.0892686938063939,0.235398720624788,0.0853582534783068,0.00405409206212008,0.00389580112533642,0.00721057367423752,26.036840330467,3225.50902040128,2051,1746,190.916708818075,35,1.70225449123991,127417.4,328 +6050207660,-70.1408402515431,0.807760249505622,0.131903271861038,0.0337371542804488,0.098166117580589,0.0901164606655436,0.194733193589712,0.0603993858896531,0.00287233910831759,0.00270137279309562,0.00559009389531829,25.9133266533066,3186.76285905144,2025,1588,182.637608550434,26,1.65566120949404,64371.9,175 +6050491730,-65.2502917977735,-11.7113295230953,0.134255470364307,0.0309273864714689,0.103328083892839,0.093734637238202,0.219399471540867,0.0652495631783037,0.00396893549626088,0.00276409352069197,0.00547017442110922,26.1744350908285,1602.40762073549,627,1256,176.653965440851,22,0.958136081748242,47420.7,312 +6050491720,-68.6867828966776,-13.2945318850998,0.1448351244251,0.0307148565824482,0.114120267842652,0.105121011515619,0.245575240740115,0.0575412155355484,0.00396647614268084,0.00330795804865764,0.00484510391017066,22.0960227692763,1915.05832778887,956,1282,1001.90286094478,207,0.312641229811746,282425.1,412 +6050532210,-62.1083290193922,-15.1352120077201,0.139212756651145,0.0319922945662207,0.107220462084924,0.100307477184213,0.216657541417889,0.0665245908572491,0.00343845151606589,0.00289555509523514,0.00570990507284908,24.4018347551712,1324.74634801917,302,1071,299.489657590277,28,0.615410747170977,354282.6,411 +6050532200,-65.0037457924553,-16.4964324910604,0.141011128524419,0.0325735529609627,0.108437575563456,0.0976122446096243,0.191408213528631,0.0528342280874127,0.00319614586997455,0.00221731574042563,0.00507393773636191,22.3762374508948,1588.13548668704,625,1114,935.694543867307,107,0.170519714795098,235711.3,280 +6050315920,-74.0119599215108,-10.6095867920292,0.143654980136541,0.0342302847048519,0.109424695431689,0.101454647938034,0.252961884050102,0.0659667955637585,0.00297033305840005,0.00258651001742479,0.00483334271129166,18.5778049362108,1571.73596041493,1235,1202,1781.75086443305,322,0.193803551898366,353711.5,427 +6050315910,-76.9906022795791,-5.13226052251101,0.143637939918528,0.0299047245691652,0.113733215349363,0.105516651525009,0.233438481054587,0.0653036040448899,0.00408683506984171,0.00317470730178646,0.00522616790483627,21.1710724942952,1881.88479316599,1424,1330,1225.90936750336,288,0.234056922385951,364869.2,406 +6050007010,-50.926607647978,-0.646902724194793,0.127031078553717,0.0296776152318939,0.0973534633218228,0.092233596486344,0.175489578753476,0.0646349036824892,0.00516285968147425,0.00501490735988728,0.00659525587916733,26.9235294117647,2760.76470588235,1265,1452,18.3529411764706,21,1.68098196497908,374,66 +6050007050,-50.6035695678439,-1.22029581812215,0.119072653156871,0.0283709338144839,0.0907017193423871,0.0891748322585561,0.162539392259008,0.0706934070172341,0.00477476554771101,0.00454985980557158,0.0061351816446865,26.8587257617729,2622.2243767313,1288,1524,19.7562326869806,12,1.55592362065115,7585.9,77 +6050007060,-50.6521495389434,-0.814018184543452,0.12534207178127,0.0279173688749401,0.0974247029063302,0.0861730958451369,0.148639129612418,0.060349152621154,0.00418015242552709,0.00427999305551601,0.00625028158003285,26.9088235294118,2861.61764705882,1335,1514,19.9411764705882,24,1.62235701179735,744.6,78 +6050007090,-50.0981773326626,-0.832644458541428,0.127761263796922,0.0326807965128152,0.0950804672841071,0.09144592865416,0.118838333970338,0.065947257097984,0.00358795267257722,0.00353912628355739,0.00552964707922725,26.9533864541833,2968.12749003984,1456,1475,19.5378486055777,12,1.66434730761507,5465.7,81 +6050007100,-50.07423404435,-0.377158248911897,0.124571877540213,0.0303723576229366,0.0941995199172769,0.0890276875211057,0.144720594265869,0.0729384674167929,0.0042767380210769,0.00367946290542868,0.00559474502267564,27.0381877022654,3133.87702265372,1539,1368,12.9320388349515,13,0.546794916181814,6618.9,85 +6050007240,-49.2859856830768,-0.651256104013875,0.119802967098159,0.028443527579077,0.0913594395190819,0.0847947618389552,0.152389218331887,0.0723272680245889,0.00209750446739329,0.00122316927767217,0.0040721988151634,27.1993265993266,2716.23905723906,1460,1315,7.90572390572391,4,0.270593301718434,6395.2,32 +6050007250,-49.4691335196553,-1.22056753320418,0.127074888462491,0.0289892022315522,0.0980856862309385,0.0953996330231574,0.134974175864528,0.0785016223256186,0.00462685811278486,0.0046464951978909,0.00546293382719657,27.0452991452991,2578.89499389499,1323,1375,12.4871794871795,11,0.452101155235385,17624.3,66 +6050007650,-50.8641109448053,-3.146849754275,0.123403255905894,0.0282244964071328,0.0951787594987614,0.0881211177191657,0.151447996042433,0.0492066335225235,0.00308551713443505,0.00326057559583332,0.005399350580921,26.4768356997972,2263.0953346856,1233,1316,88.1375253549696,28,0.422155619481411,52965.8,83 +6050007660,-49.9105276210622,-2.28862044436186,0.131365010811968,0.0317599199280283,0.0996050908839402,0.0928840951612874,0.122626111897426,0.0545879534928763,0.00234988718989515,0.00259506032399162,0.00428848514181186,26.9052734375,2541.853515625,1240,1405,24.677734375,16,0.871007147014018,11060.2,83 +6050344660,-50.0518564529339,-6.157313869345,0.143135572522194,0.0315081379010513,0.111627434621143,0.0996456859246246,0.105340251474437,0.0589014630587082,0.00294390716576236,0.00268291233964142,0.00451418207687844,25.5642051282051,1856.37230769231,470,1270,257.068205128205,36,0.240512679437026,41529.3,95 +6050345180,-48.7167083783454,-6.68030582721049,0.147666769482761,0.0297270700920324,0.117939699390729,0.101900567820205,0.113538255608944,0.0834682887402733,0.00480821419988771,0.00487067939184159,0.00488833883374727,25.9445294117647,1706.39529411765,573,1239,203.775882352941,31,0.266017865368169,36174,105 +6050345000,-47.4405384450357,-6.24225904842931,0.14274877741187,0.0308329251596888,0.111915852252181,0.101591457838876,0.193235011574694,0.0616404700226257,0.00461793983793936,0.00428238560558248,0.00437843653394516,25.6937702790396,1558.45295262816,681,1151,236.720960415315,41,0.294718687071237,32991.5,74 +6050411440,-50.0027054697928,-7.99823456748413,0.146205012883384,0.0298043552465052,0.116400657636878,0.0983006211469545,0.125710313722301,0.092136930304919,0.00488478973912237,0.00451061610428948,0.00528677559500935,25.7684542586751,1851.93533123028,767,1335,229.760252365931,29,0.323263676379494,13504.4,87 +6050411450,-49.0148592013872,-8.23535253139519,0.148467840688371,0.0306802486114541,0.117787592076917,0.100199655125185,0.115453872868857,0.0932279174148421,0.00518153286785916,0.00499247007941312,0.00541818078942572,26.0277056277056,1849.04473304473,912,1320,231.698412698413,35,0.226614449200411,14809.8,90 +6050408150,-48.0106081022252,-8.2555963965222,0.141318093336098,0.0292264149609446,0.112091678375154,0.102212203757689,0.212340686763873,0.0619744198182705,0.00506609859904992,0.00545593821100157,0.00616213722803837,26.2836257309942,1800.73489278752,801,1273,248.449317738791,37,0.286994225773629,21765.2,80 +6050408040,-46.993150151298,-7.95081297499383,0.14388091020621,0.0291869992864672,0.114693910919743,0.0987327997177108,0.180221106933598,0.0468856016105961,0.00264023128828293,0.00228604142186456,0.00403480539809459,26.049354005168,1421.83204134367,509,1133,309.497416020672,35,0.272862379141319,16564,88 +6050443080,-49.583275837715,-9.30839830382452,0.145987449490248,0.0312218632099336,0.114765586280315,0.104967289155009,0.106731380592872,0.0728700310657013,0.00481309287417897,0.00501156566280278,0.00464893052231331,26.6075949367089,2118.03164556962,1033,1366,200.946202531646,21,0.597337784327282,6609.5,94 +6050442950,-49.1387489645035,-9.20456423511221,0.142773777410092,0.0308535875895029,0.111920189820589,0.0959489313202602,0.12420800119077,0.083187164532324,0.00342945926963545,0.00353128577642466,0.00614107494143773,26.3680701754386,2159.26666666667,1039,1341,248.912280701754,35,0.357098289273112,6037.8,74 +6050449500,-50.1424052527275,-9.00263867886602,0.14043505578255,0.0296542906478508,0.1107807651347,0.093338253497713,0.085914202770393,0.0613573230970291,0.00371542968253853,0.0040379037302918,0.00599934057951598,26.3081818181818,1787.97272727273,813,1322,208.8,21,0.379006726151528,2358.1,76 +6050449170,-50.4537881305092,-8.66618730839662,0.151844342423335,0.0307634371900368,0.121080905233298,0.101168257740609,0.111218671576871,0.0561890584266267,0.00369823290018853,0.00431106336534492,0.0043753291916977,25.8698630136986,1795.19520547945,761,1328,250.017123287671,29,0.338057599328481,6214.9,77 +6050458930,-50.6491950968272,-9.43162791598804,0.141207365377033,0.0297905301556588,0.111416835221374,0.0974602615492124,0.133564890986384,0.0774986590044359,0.00356290675084452,0.00396520037453516,0.005626620917801,26.3226252158895,1764.14853195164,578,1311,237.018998272884,32,0.337447265749073,12152,121 +6050459050,-49.4410236209765,-9.92627533829494,0.137756283535466,0.0273794039113904,0.110376879624076,0.0955315349930665,0.127591634321588,0.0570184162683824,0.00388551168459686,0.00403116433127687,0.00608334134343289,26.7958466453674,2071.29073482428,922,1347,231.38338658147,26,0.430071873864554,6696.7,74 +6050451380,-48.3603318856253,-11.5309571069934,0.144317674820932,0.0327459066840372,0.111571768136895,0.0997662506511351,0.230498198941826,0.0645640210515757,0.00466848139466533,0.00427693819192453,0.00495209075275021,25.9978385251113,1622.00890019072,560,1188,341.233630006357,41,0.286386349826958,66112.9,89 +6050451390,-47.225618634243,-10.133884495359,0.148721716066294,0.0316342559921367,0.117087460074157,0.10187028462861,0.178454841293706,0.0545493473075252,0.00410113214389656,0.0034086732296326,0.00393274589508803,25.6279445727483,1524.48498845266,657,1175,401.68129330254,53,0.261391104021742,45771.7,87 +6050476790,-51.1770943265075,-10.8877534334683,0.134365089801638,0.0354410529775426,0.0989240368240955,0.0972195688905638,0.167471879634691,0.0900366504289718,0.00322135671879207,0.00295970208791409,0.00486515636042165,26.9769230769231,1746.71512385919,649,1276,220.700130378096,18,0.387935603106294,32437.7,67 +6050527670,-52.5747638486024,-14.3817561672712,0.138412987450091,0.0329584372507745,0.105454550199316,0.097297618602207,0.163299009725563,0.0912422632329243,0.00387436899036323,0.0043396407605926,0.00543736782929348,24.4987516869096,1647.6858974359,537,1241,405.740215924426,35,0.219173795296957,61613.7,111 +6050527680,-50.7194795554594,-12.3503320705038,0.130644306571235,0.0304692114831615,0.100175095088074,0.0914194621105498,0.136257269808006,0.0468420571614032,0.00133206255355836,0.00188511471299204,0.00346955265451011,27.1550724637681,1656.92028985507,518,1235,204.275362318841,4,0.635014004261859,2936.4,82 +6050550960,-50.4389413451211,-13.1300731535087,0.141209400552053,0.0329255727010372,0.108283827851015,0.0964301736083327,0.0814896845891083,0.0613913284988629,0.00319349996262363,0.00362112399730544,0.00553600030799981,26.6482269503546,1675.58156028369,450,1201,245.404255319149,17,0.405603898461341,2889.8,103 +6050551090,-51.1957930687264,-13.8662338376567,0.142867404690609,0.0324257531275834,0.110441651563026,0.100965852304819,0.137658607118979,0.0751764845031601,0.00325343337690356,0.00295675214602182,0.00473364461567449,26.2798230088496,1572.16991150442,421,1198,235.603539823009,4,0.641631486302568,11804.2,106 +6050546240,-48.2047687993412,-13.9680874133317,0.144602324491738,0.0293415181972606,0.115260806294478,0.100552907554822,0.183181873168155,0.065513795484035,0.00515490547788533,0.00471138228523845,0.00460425492019464,24.2639502762431,1666.45718232044,533,1148,669.591160220994,95,0.144760457232267,30127.8,101 +6050546360,-46.9475463247621,-13.4619494324761,0.144504469714194,0.0302731270375009,0.114231342676693,0.105553163659894,0.123557210859925,0.0830897247113778,0.0059029001361999,0.00586658972103325,0.00591434009506185,24.9175983436853,1445.65527950311,435,1118,579.187025534852,75,0.206607773308951,60333.9,79 +6050565980,-49.9025274184508,-14.0103068732568,0.147332284771933,0.0336662263207796,0.113666058451153,0.102643602984752,0.105481575673331,0.0696782755770217,0.00355505244638421,0.00363962885475912,0.00454957414265516,26.0155656108597,1600.007239819,468,1148,337.364705882353,34,0.228076075372509,23196.2,94 +6050565970,-50.8461047151342,-15.4218183005027,0.143756232161525,0.0320058253720301,0.111750406789494,0.101412232550546,0.149867980860571,0.078014562540936,0.00339689281575565,0.00351445568159604,0.00482909571074625,25.4823668081094,1590.11692597831,394,1158,378.986327204149,34,0.208257797336108,43754,119 +6050594780,-49.4826942076805,-15.485399961907,0.156216008115426,0.0296131153346542,0.126602892780772,0.109972587665036,0.126768428217414,0.0734864519557096,0.00444295342967831,0.00431694244435377,0.00519445421218566,23.9856516976999,1652.10624315444,527,1141,699.167579408543,63,0.115002443020967,18969,44 +6050594790,-48.3615531110799,-15.2044695624227,0.152686168269643,0.032100211385774,0.120585956883869,0.108888863798728,0.116404055324285,0.0717308172257285,0.00370741430339478,0.0035042583040667,0.00572956531681132,23.403557312253,1459.42160737813,475,1103,782.375494071146,82,0.148962911913443,15699.3,67 +6050622980,-52.0684618825083,-15.9263503946946,0.131869038245446,0.0297175850426122,0.102151453202834,0.092291669333759,0.136456956315213,0.0780740888524354,0.00160509709335964,0.0017849020978378,0.00435830299456444,25.4705882352941,1538.96078431373,503,1169,360.78431372549,45,0.216829063706012,1072,83 +6050622990,-51.6291915841492,-16.6069433033749,0.144258993988756,0.0328734296191723,0.111385564369584,0.1045090087294,0.188553040058535,0.0569862371153066,0.00128477218457527,0.00198928459825205,0.00315476828808862,24.3018456375839,1600.71644295302,520,1172,574.692953020134,68,0.131182561392224,12177.4,40 +6050624770,-53.2222480418847,-16.0462294356284,0.13551891770677,0.0336760429583522,0.101842874748418,0.0985899629766804,0.227859350976348,0.0893592268313332,0.00402333872547138,0.00331523650194056,0.00510024876843856,23.085911179173,1662.40581929556,581,1247,519.218989280245,65,0.156592325341189,13484.6,74 +6050624870,-52.7713614884036,-16.8217474678011,0.156760097549763,0.0329953982473761,0.123764699302387,0.116773863486101,0.161974298148948,0.0473014889199895,0.00210703530725186,0.00217005712872556,0.00330943513139249,23.3994713656388,1659.04581497797,553,1233,608.639647577093,66,0.145896201243961,23383.4,31 +6050007960,-48.2725666006195,-2.90289723724553,0.127199290020154,0.0265751982883294,0.100624091731824,0.0883686059603354,0.148619940753877,0.0681568435508639,0.00319538061299406,0.0027518919398004,0.00470327122488363,26.6650945840891,2229.99093029282,905,1284,88.1272350349832,29,0.32474102021162,82901.6,108 +6050008320,-47.0041849197554,-1.28446950254005,0.105410803476779,0.0250833659252934,0.0803274375514856,0.085865470320353,0.120399636436975,0.0567120982333001,0.00120829467098418,0.000699500694076746,0.00324190089811585,26.2877828054299,2391.74547511312,1450,1284,37.7364253393665,22,0.243027545430635,18906.2,17 +6050008620,-46.9637328662554,-3.30024479203745,0.140736884920845,0.0370140588887276,0.103722826032117,0.108603895708261,0.101375877700979,0.0456261137240485,0.00149138384504939,0.000880632968080138,0.00260577094629789,26.3672615384615,1773.59076923077,572,1157,138.144615384615,32,0.301315095584314,34830.8,19 +6050008630,-45.4119707257864,-2.2996489214044,0.140147705630856,0.0300072824958801,0.110140423134976,0.105849315311082,0.0987365533234996,0.0450288107385499,0.00116978691109737,0.0012042790459911,0.00242897787719052,26.9220640569395,2114.3084223013,1098,1192,33.5308422301305,17,0.25839158661692,36049.4,20 +6050009220,-46.2620676040268,-4.31487810858401,0.130834422798712,0.0348236149907665,0.0960108078079459,0.0949147283118059,0.0541223082214165,0.0260087312887982,0.00132643921098943,0.00093700667148031,0.00304446446721213,26.3238798102267,1512.77016341592,398,1078,176.890353189246,40,0.177552562827214,40533.4,22 +6050009230,-44.8150438481383,-3.30845043881006,0.128801770617672,0.0298195046107804,0.0989822660068919,0.0906318908657723,0.0620373005056247,0.0348975236482834,0.000996626188643126,0.00117456694024234,0.00352886894387501,27.4,1623,841,1121,7,27,0.14469783531187,11.4,27 +6050009250,-45.4344501222947,-5.24283402604655,0.137317150395841,0.0321932606245091,0.105123889771331,0.095543955175691,0.159368943517392,0.0369067930163887,0.00135487056416661,0.00105925761258803,0.0031250854690953,26.3849808429119,1352.79961685824,398,1022,180.356704980843,33,0.263586022121534,55843.1,21 +6050009260,-44.4123850396756,-2.98446869175686,0.144387317461768,0.0332740076001264,0.111113309861641,0.100252736097978,0.0970624545160525,0.0372105718627992,0.00115798951596149,0.00100139870627214,0.00316609271568749,27.1916666666667,1672.74305555556,944,1096,14.5277777777778,11,0.267453852972493,3088.6,24 +6050009430,-44.1972737940884,-5.3966036023689,0.12071876943532,0.025651374488068,0.0950673949472523,0.103628588902276,0.134384786276545,0.0554963861691451,0.00125728695637858,0.00055837965943101,0.0028328015279265,26.6073701167942,1398.74506645187,298,1042,207.919452275473,34,0.430315036337169,52991.6,12 +6050009440,-43.313722677665,-3.24802872033177,0.153307535943736,0.0391361569447451,0.114171378998991,0.0993529883066466,0.14871692564731,0.0238464134910116,0.000786024287436712,0.000890885157586386,0.0025285546546836,27.0502435629784,1659.85386221294,775,1056,60.6235212247738,21,0.528926771900486,30706.1,17 +6050009760,-42.1442927669249,-4.05847159841645,0.145746612739772,0.0365414374414697,0.109205175298302,0.0979634374823335,0.122276307813651,0.0236155590563793,0.000831725562830161,0.000828949068593523,0.00234944408902572,26.6360746720484,1422.17255297679,349,994,145.592835519677,35,0.450741383370402,42461.4,23 +6050334790,-43.072791687557,-5.94572822984083,0.130572221546066,0.0312652810385475,0.0993069405075188,0.0960238014343155,0.115536687943853,0.0426961704950244,0.000878063042223985,0.000975719684228425,0.00344754561816331,26.6769709543568,1332.64315352697,226,1007,204.622406639004,42,0.492593671658748,10352.8,24 +6050372330,-43.2074535470248,-7.49088518171814,0.218733813992665,0.056127548642341,0.162606265350323,0.11185529272551,0.0237754080198039,0.0292583871933813,0.000736630519232573,0.00117489550896069,0.0033539129572542,26.6161616161616,984.904761904762,135,824,321.997113997114,41,0.663448066287018,14762.2,7 +6050372340,-41.9969368936739,-7.92007769168591,0.148105924737914,0.032104862165336,0.116001062572578,0.154622476353413,0.132748051559585,0.0613991243691435,0.00114375879483039,0.00437495723840634,0.00627805325021188,26.1634311512415,742.650959367946,47,625,371.730812641084,50,0.452703233516036,75429,3 +6050401900,-45.4724321565911,-8.75446904649116,0.14284788841366,0.0296046425736442,0.113243245840015,0.108263979100772,0.139538855272918,0.043666804471665,0.000941890539571234,0.000924415808597025,0.00270655171275123,25.4973834832379,1155.61447260834,271,967,447.858544562551,78,0.311264338782531,51975.8,25 +6050401890,-46.0096964354234,-7.69545963982841,0.120467320063217,0.0304449899742872,0.0900223300889296,0.0907292796704303,0.0158785193751663,0.0667500041908866,0.000587606003383736,0.000567537817650865,0.00193852201517154,25.684775374376,1210.56572379368,323,1014,379.500831946755,43,0.26202818867136,25645.9,3 +6050009770,-40.9047655509002,-3.25202610486824,0.134924954166016,0.0438267212962227,0.0910982328697935,0.105955551322639,0.120870653186249,0.0479830044196178,0.00177338240874288,0.00188651923585624,0.00386679641577655,26.9834738617201,1105.92917369309,377,857,121.733558178752,40,0.280298010796853,12633,15 +6050010100,-40.3032639084594,-4.14437195722962,0.134430715419232,0.0429030647093262,0.0915276507099054,0.108395743618698,0.0674379945675036,0.0358940620824543,0.00155181244794439,0.00196571951528464,0.00296415143224944,26.0424375917768,876.199706314244,175,720,242.164464023495,51,0.214834501074647,14578.5,14 +6050010110,-39.1416088993013,-4.00588780893235,0.101448414723303,0.024129220865896,0.0773191938574067,0.0772250035465267,0.181664285546738,0.0200949125105991,0.00129350904942017,0.000760107355669098,0.00182307081631193,26.0859591298616,959.773236651285,189,764,140.673038892551,37,0.17026387467687,32471,10 +6050010510,-39.3118527812217,-6.05238606131664,0.135727630844849,0.0492243086056973,0.0865033222391522,0.114060711416531,0.0631132410478605,0.0429696848682678,0.00174696430217988,0.00122164327908559,0.0020144634882536,25.7258515283843,785.191557496361,74,662,352.48442503639,45,0.322235249383111,73490.9,9 +6050010520,-37.6191744406518,-5.58133857367396,0.135098224198078,0.0454304488579828,0.0896677753400946,0.101059453562224,0.168839212257213,0.0360697264033154,0.00158331540883805,0.00121949518702938,0.00292128826245061,26.861282932417,796.310423825888,97,672,154.258877434135,39,0.547457941337319,18573.5,14 +6050010670,-37.4279508588395,-6.65315200235825,0.152617797242349,0.0543740425458032,0.0982437546965455,0.11476821206449,0.0694466846730298,0.0515749859120642,0.00153738430330899,0.00217635093096078,0.0041804981620398,25.440932642487,747.958549222798,102,630,323.321761658031,53,0.305967002462814,41203.5,13 +6050010680,-35.7472718674047,-6.09478045715012,0.127359647140392,0.0363534521756433,0.0910061949647488,0.0960005455361708,0.173460817834005,0.0406807863096022,0.00167288459393544,0.00169061332021855,0.00276798085555796,24.8322491349481,799.414532871972,210,654,188.817301038062,47,0.273726399661729,30866,21 +6050011240,-36.3116313530698,-7.48321411124634,0.131521417902617,0.0445635356953787,0.0869578822072387,0.108585010835157,0.223521600066206,0.0619399105636561,0.00205607647891289,0.00203580518567947,0.00357165864967073,23.185376344086,599.755913978495,109,503,463.92688172043,49,0.266185256682846,19753.8,14 +6050011250,-35.8451895959667,-8.63187153201551,0.128153756645738,0.0410807546143645,0.0870730020313737,0.100065488140333,0.186393205624427,0.0442651608918311,0.00185824809949505,0.00184145602304643,0.00318601824001122,23.2472966390648,1218.99220652703,493,854,325.185582075012,60,0.0777405660357426,43552.8,18 +6050011780,-37.5027711797687,-9.29012246619581,0.150729035880681,0.0307350385880211,0.11999399729266,0.113420113461988,0.154569130557634,0.0631338678117447,0.00122404211392616,0.00167224286111297,0.00273417125117993,23.8460766423358,644.934762773723,175,540,363.315693430657,56,0.365260979147027,46326.6,45 +6050450420,-38.1268029781271,-8.05885194862221,0.15211447951966,0.0395198226421058,0.112594656877554,0.130703559226508,0.160670931272792,0.0991879036596022,0.00141419026314739,0.00254489349819927,0.0031972944042354,23.2215365239295,651.612090680101,159,551,524.775818639798,42,0.387946607854699,16914.1,18 +6050450410,-39.1843969845948,-8.81257972102606,0.145499421681563,0.0374731797683612,0.108026241913202,0.110312704570696,0.141290784972862,0.0754356449309655,0.00183166480804739,0.00226297956252017,0.00320923648466146,24.0411192214112,515.95498783455,131,435,422.170316301703,29,0.487778632509981,17368.5,33 +6050441200,-39.9975730806107,-7.84417711120328,0.174010554502853,0.0400655218965647,0.133945032606288,0.129740126068238,0.0980911141895107,0.0742417948794856,0.00102900107113088,0.0019806977714233,0.00234518907690886,23.8987804878049,669.378048780488,138,566,557.653963414634,53,0.364304022637722,13822.6,14 +6050441330,-40.2466077146602,-9.10477276074465,0.136738318477234,0.0330214893215492,0.103716829155685,0.10045518679932,0.0795100620854235,0.0477236989357443,0.0011370968292939,0.00155897320401074,0.00316235969946739,24.2122532894737,505.258223684211,104,425,449.495888157895,29,0.496184548674726,25776.6,37 +6050466270,-40.9198617942937,-10.5628445179331,0.13218261004728,0.0318427743886085,0.100339835658671,0.106705197908393,0.133341037767218,0.0648998879753057,0.00130709162563628,0.00176180042385368,0.00342196411854705,22.4642547033285,500.850940665702,22,421,697.289435600579,73,0.621532662492865,14495.8,34 +6050466150,-41.5189885874391,-9.51400376681023,0.132527097084035,0.0319393516387238,0.100587745445312,0.113151205798004,0.0989366533045362,0.0649396652350594,0.00109858647557281,0.00174416405430089,0.0032016321147429,25.4983365949119,541.56457925636,27,456,482.863992172211,33,0.435996059025938,21585.6,31 +6050481680,-41.7439678242225,-11.1265437157858,0.131252071339102,0.0291093962089267,0.102142675130176,0.112826760225693,0.0885521851174424,0.0682372839679579,0.0010945911554193,0.00189596619525148,0.00325964934773134,22.9201911589008,610.671445639188,44,514,733.364396654719,82,0.425783842671379,17659.9,27 +6050481670,-42.7908263186425,-10.5938246610679,0.146061175896559,0.0315921343843353,0.114469041512224,0.116215167685546,0.0873906267864763,0.0469692087790805,0.00167904864452811,0.00230768597405125,0.0030955137651773,24.8969359331476,699.168245125348,47,590,547.252367688022,52,0.450227774680293,37975.3,43 +6050510750,-45.0169085230828,-11.6888019083951,0.154868861184073,0.0344517915207854,0.120417069663287,0.110476412738685,0.146543110879095,0.0485785079439514,0.0020238693661383,0.00226972866552401,0.00284638169972268,24.0194798157681,1157.71010566242,275,943,658.06177187754,42,0.367146229026384,77450.9,46 +6050562020,-44.1873040611248,-14.2939849227058,0.143974512611182,0.0337051180340879,0.110269394577094,0.100407233428631,0.123588288434242,0.0464466508446882,0.00173634161367012,0.00184506022113746,0.00283213230403119,23.6760808534531,928.380123526109,245,784,620.078607523863,42,0.301763988236694,37079.6,60 +6050562010,-45.0560547735502,-13.5971099634889,0.147664145609411,0.034345559394502,0.113318586214909,0.107829223886319,0.130548506075483,0.0567016997275387,0.00208956392716826,0.00242006629372197,0.00383235626175346,23.5863718958207,1070.16535433071,411,900,737.57238037553,46,0.373305665446424,34406.2,38 +6050595200,-45.3776284547879,-15.6793493301181,0.146280940298875,0.0328782654252806,0.113402674873594,0.106873151510792,0.133683193970889,0.0458153543220753,0.00163471990393942,0.00174724898313858,0.00285870906933956,23.1252983293556,1124.42840095465,308,944,665.53937947494,57,0.250932961790279,52028,66 +6050595210,-43.4090461778471,-15.5799110514449,0.146403818465989,0.0319009374483364,0.114502881017653,0.115462656745582,0.0856602912290276,0.0465209376876235,0.00135529912561344,0.00213487092717718,0.00299593606157645,23.2016139878951,885.024209818426,199,748,635.151311365165,59,0.173158683079344,30808.9,32 +6050639920,-44.5338555960439,-16.6868781349445,0.141334404637389,0.0323489330321714,0.108985471605218,0.106494246841709,0.108452463485055,0.0505652207710096,0.00154853690698793,0.00175889854614747,0.00325301254680622,22.1797142857143,1186.60285714286,317,1001,720.531428571429,55,0.245881819458312,7221.4,58 +6050639910,-46.3656472524532,-17.1639481308846,0.136663203784537,0.0339364603725529,0.102726743411984,0.104488311779557,0.135617481991866,0.0510867479146484,0.00211941750494687,0.00216327996625828,0.00370215402677133,22.9436760691538,1297.95814376706,407,1075,690.406733393995,53,0.211340716397748,45168.1,53 +6050649360,-44.1676758908206,-17.3024990979972,0.146516328612145,0.0350798657825145,0.11143646282963,0.111994443434099,0.142562431921426,0.0564944277868593,0.00199215775582223,0.00240050478015791,0.00332586603713022,21.8592941176471,1161.52235294118,337,977,778.524705882353,111,0.211739291849924,8690.8,46 +6050649500,-44.9438579810695,-17.1722431630019,0.154038726191702,0.0345590080698031,0.119479718121899,0.114452877180934,0.144483941576917,0.0568658901081794,0.00195905868171393,0.0022975608542698,0.00254741852565879,22.952,1090.4,370,924,618.92,42,0.220205507234235,508.2,40 +6050652910,-44.1348927158654,-18.755470911952,0.152576292975346,0.0325241787083363,0.12005211426701,0.113107561604688,0.140892208612325,0.0500816503613173,0.00185888215495042,0.0021670384209769,0.00334279013401658,21.1454215116279,1295.14026162791,400,1048,830.96511627907,102,0.149143364679428,27927.6,59 +6050653020,-45.065327775198,-17.7695769500048,0.153591070393263,0.0323226320986953,0.121268438294567,0.113549328453454,0.13184234615328,0.056769675112674,0.00199888599721529,0.00248322735521749,0.00354959070479395,22.5947368421053,1231.1954887218,372,1033,701.706766917293,81,0.161811945403447,5412.5,44 +6050670380,-45.8896347683598,-18.5795770530717,0.142390441716073,0.0361189101402445,0.106271531575829,0.111911352980117,0.129107423432754,0.054718697746768,0.00191578337986382,0.00215032919058811,0.00207326234080386,22.0299295774648,1422.26408450704,452,1127,847.338028169014,77,0.11564213781136,5856.1,32 +6050670510,-45.4978756458625,-18.7266942428562,0.145975870929485,0.0285881347936383,0.117387736135847,0.109234085444309,0.122612234105645,0.0537301370912384,0.00116879118166656,0.00159451888840827,0.00281895978751983,22.2588377723971,1335.30992736077,418,1088,756.179176755448,63,0.111149870020068,8325.8,48 +6050684130,-44.3895070508123,-19.7806618543446,0.143726557017905,0.0337621439397398,0.109964413078165,0.106277838983305,0.153217908343135,0.0522104359384523,0.00193102905535527,0.00234294958078828,0.00326385579393518,21.0573116691285,1343.38552437223,421,1085,835.218611521418,54,0.153422860420132,13683,50 +6050684180,-45.3974843574177,-19.9084411865931,0.149765341073094,0.0326758347354445,0.11708950633765,0.110923939127713,0.136377426516507,0.0533222772206347,0.00176244746443188,0.00221987196440781,0.003105678988292,21.2686813186813,1389.62293956044,419,1123,786.427197802198,52,0.12652736914956,29394.9,56 +6050011790,-37.2232611996048,-10.5544307372234,0.125666569636208,0.0374838885643644,0.0881826810718435,0.102232146487754,0.185865277740623,0.0464476738804802,0.00140838119750136,0.0023430557386214,0.00322151952057666,24.5227106227106,1094.25641025641,404,833,137.40293040293,36,0.233617533853874,5872.5,10 +6050011890,-38.5180298859596,-10.1869151423274,0.140273460885495,0.0391384676407207,0.101134993244775,0.116078994198062,0.0658558969457684,0.0431428204243234,0.00110102000146886,0.00251690152767549,0.00360955622614239,23.3875979112272,649.251958224543,86,539,388.498694516971,59,0.464141987096865,16213.7,8 +6050011900,-37.8176944193581,-11.1107769243799,0.120275851174391,0.0323209846440049,0.0879548665303864,0.096996057497342,0.185080475817339,0.0426690934481571,0.00129818896452354,0.00204475001381331,0.00276830775914346,24.0306666666667,1083.80666666667,308,861,189.111111111111,36,0.281618446719994,9604.3,12 +6050012020,-39.390688272208,-10.9166047438352,0.119475819211307,0.0310266824715046,0.0884491367398022,0.0913769102138816,0.153754342695095,0.0403448030300823,0.00131186641588255,0.00192960574346258,0.00256440037351919,23.5224228305183,680.144437973209,49,572,385.855562026791,50,0.356790960166518,36065.2,13 +6050012030,-38.4203639363778,-12.1526128776319,0.110073114714698,0.0273123092892808,0.0827608054254175,0.0913490005309611,0.227354457320189,0.0361828306873686,0.00363636549306707,0.00145158456585034,0.00149716332337903,23.7491358024691,1314.07160493827,411,976,156.155555555556,34,0.175127828847936,16991.4,8 +6050012280,-40.4895742063643,-12.3339052165272,0.114184688159292,0.0318726469570449,0.0823120412022468,0.0888497393243603,0.352660613012208,0.0451488909727321,0.00147390743596013,0.00224859404497958,0.00298541948280756,22.249443378119,730.856046065259,69,618,526.440690978887,68,0.311496604384667,54758.3,11 +6050012290,-39.0950773767129,-12.9442294443862,0.150168687063528,0.0452602324242419,0.104908454639286,0.113576360828742,0.0844742548819285,0.0774923803737924,0.00124931126396399,0.00516646086560081,0.00735244529286546,23.6788571428571,1365.35428571429,460,1016,138.88,36,0.108153908545632,3669.1,3 +6050012370,-39.8320894653546,-13.1844843904686,0.123693528510179,0.0350846960808544,0.0886088324293248,0.110442001356087,0.207671861772263,0.0651669825598327,0.00397736142581819,0.00232701331603357,0.00468193952175753,21.0792899408284,885.307692307692,198,727,557.165680473373,81,0.132451501756636,7042.2,14 +6050012380,-39.3311134727306,-13.6717875530985,0.117936651960652,0.0301599583681178,0.087776693592534,0.101198263419966,0.304127574751224,0.0617909317605711,0.00201977281784991,0.0022917745002849,0.00495253430377386,23.2388101983003,1751.10198300283,589,1179,211.974504249292,59,0.13381552048426,7427,9 +6050012580,-41.0702189098329,-14.2048738301285,0.118067791736749,0.0308401006264095,0.08722769111034,0.0957041728097582,0.224358503976188,0.0500992543216138,0.00170427097663722,0.00189304613216112,0.00432315885212878,22.1557512176845,794.032596478082,106,663,591.984638441364,83,0.156294986240615,55394,13 +6050012590,-40.7215460701324,-15.3247136247856,0.117499955869711,0.0277295213223909,0.0897704345473197,0.10102832317398,0.203294567788066,0.0499593099734027,0.00232838484854629,0.00245754441676621,0.00462926995957751,22.098439859084,976.208857574233,154,812,567.930045294414,61,0.156260332138073,41325.6,17 +6050012790,-41.9763191501401,-16.7750568148406,0.137250621106033,0.0397383912997874,0.0975122298062461,0.106255922697462,0.057197586600351,0.0529964190030741,0.00203901756695835,0.00234581411453993,0.00386145201426945,21.8714244354372,986.835842501448,198,830,686.308627678054,104,0.130689478245137,70980.9,14 +6050012800,-40.3556927805847,-17.6730216336501,0.11726450304772,0.0252002313634423,0.0920642716842781,0.0807377483698721,0.248400509048281,0.0433788230453452,0.00223428198335839,0.0011361407436135,0.00285276853047516,23.6387973137973,1126.62057387057,299,951,255.491147741148,47,0.103835626555243,66863.6,20 +6050013420,-42.2390871728674,-19.4447208457195,0.115923808422806,0.0228890263014535,0.0930347821213527,0.0812939794031891,0.147944710966299,0.035550067902914,0.00205819830938597,0.00116139269145802,0.00223308813005992,21.6832804491091,1271.25799365389,406,1049,562.527947278496,94,0.0828182388029392,82935.9,19 +6050013430,-41.0189316280934,-20.5455345049501,0.146196768628509,0.0368284038277543,0.109368364800754,0.121172091272206,0.20117169005457,0.103880052348804,0.0030004802028933,0.00259598213270124,0.00424128843725127,21.9037072243346,1194.79752851711,392,1008,429.761406844106,123,0.114594589305832,21133.6,54 +6050013750,-43.4980138153644,-22.0504075017079,0.149068770445525,0.0375639827115534,0.111504787733971,0.115176292678894,0.157688945448618,0.054450315023967,0.00168983729482088,0.00180022665091567,0.00243386839817988,20.1605849582173,1415.03586350975,482,1104,605.196378830084,107,0.10245886108236,57233.3,79 +6050013760,-42.9185682707582,-22.6255515416965,0.143802501430091,0.0366956290540123,0.107106872376079,0.109800655733085,0.156678001487109,0.0654828923804315,0.00190733649899396,0.00182744079134636,0.00348871301256157,21.8287224669604,1392.2872246696,558,1082,244.355066079295,139,0.274267079088264,22525.3,70 +6050014330,-49.2645882888067,-27.0348420098153,0.150327638757242,0.040263243901578,0.110064394855664,0.112963143490005,0.148478057746419,0.0734771899892487,0.00289740808545077,0.00188435483317523,0.00368571907923788,18.919846091535,1635.30984204131,835,1175,363.426083434589,149,0.286294265675748,94497.1,100 +6050015560,-52.4742848545968,-29.4235383576415,0.142343140286823,0.039636601380416,0.102706538906407,0.11277151066982,0.153689394853073,0.0878297057874528,0.00370019278533473,0.00227812752869454,0.0048379916236023,18.3462058212058,1576.20036382536,851,1155,374.005457380457,96,0.124311699989836,71807.3,63 +6050015570,-52.6161859536363,-30.9430995408656,0.148943659740779,0.0432257027827089,0.10571795695807,0.117629302089388,0.177196227114124,0.0969977561813957,0.00398920690817893,0.00250532400871485,0.00544979071455413,18.2957607872824,1409.53368660106,803,1032,182.878122634368,57,0.258872888093955,24361.6,57 +6050015940,-53.7793955682525,-32.8221296189839,0.143287899108726,0.0423525528903474,0.100935346218378,0.115704769339912,0.163124116427498,0.106079775553514,0.00459654238507275,0.00266763874071786,0.00607634612527891,17.2459615384615,1243.30128205128,648,968,95.6201923076923,31,0.58094433625241,56212.3,52 +6050015950,-55.9026570264615,-34.1038153739401,0.140446385614775,0.042562006652361,0.097884378962414,0.1113805838817,0.180879916060659,0.116234971282809,0.00488703752558783,0.00263563166703369,0.00644449079430551,16.8849886621315,1150.07029478458,587,918,68.6267573696145,27,0.501704956654441,39407.1,46 +6050016720,-55.9503955154437,-32.5636993629294,0.145625109288399,0.0456243086383253,0.100000800650074,0.117092872208725,0.169911524332828,0.108427941246741,0.00514216221194429,0.00292505378882816,0.00558330174059408,17.6034552845528,1223.6155995935,497,1003,130.821900406504,21,0.625231570566727,71073.9,42 +6050016760,-57.5010033442798,-31.4857427007843,0.157408232021951,0.0356395010030026,0.121768731018949,0.126303392157009,0.184552865427296,0.0883996870671399,0.00327242864613215,0.00341933406931574,0.00543309268778687,18.4756990941315,1252.3497439937,551,1053,93.5773926742812,23,0.630588710925024,46433.4,173 +6050852400,-57.6496374228954,-30.1815251659359,0.158837012565614,0.0363934024387402,0.122443610126874,0.124609207291291,0.147776948497161,0.0742193516057571,0.00241325516642508,0.00241216148086756,0.00520789319572867,19.7,1343,570,1132,41,0,0.902154594219419,5,85 +6050852410,-56.6022768532593,-30.465208390781,0.158001081829304,0.0445435435724548,0.113457538256849,0.134318565665643,0.171926578930809,0.104082903984679,0.00376016576141586,0.00304437556317469,0.00662895363547646,19.0922292993631,1337.67898089172,665,1109,151.070063694268,31,0.522239943236942,14641.1,66 +6050852210,-57.0232439599456,-29.5740305708614,0.147725458038705,0.0366988720155447,0.11102658602316,0.109216881261525,0.135647882780901,0.0946186907742683,0.00414198125680856,0.00351841405142983,0.00528837789814114,20.104748603352,1325.46089385475,646,1113,70.7486033519553,13,0.439158986065203,6649.4,109 +6050852180,-57.6079040482805,-29.3408174069422,0.157239354634507,0.0366254694697238,0.120613885164783,0.127513226845311,0.145937043432242,0.0855984418922735,0.00316380399993707,0.0025977204558184,0.00566908505143598,20.1734234234234,1278.25525525526,448,1080,72.1186186186186,7,0.890417773461672,12473.7,86 +6050842120,-53.8904501376324,-27.7759240688181,0.159362851408651,0.0369010933590806,0.12246175804957,0.127450763311114,0.190622306011298,0.0832067980677881,0.00329014867764252,0.00330320777014226,0.00524946655577633,19.2821072533517,1785.83155723616,893,1304,414.844963905122,74,0.0683847654727268,110464,198 +6050842130,-55.1237623435508,-29.9833143575017,0.153008433543664,0.0361284819977924,0.116879951545872,0.120079376936025,0.17602037325666,0.0901685394296275,0.00370961223554602,0.00377453213570731,0.00556537666768888,18.8583136327817,1565.05200945626,731,1174,175.176122931442,34,0.218031941000724,47203.5,132 +6050818720,-50.4377944422338,-27.5151485010246,0.181203125246006,0.0377661049098558,0.14343702033615,0.158349366098974,0.183823370620684,0.0813389979153346,0.0024332590232885,0.00243331869566303,0.00468019438173425,15.7049744897959,1628.35586734694,730,1173,960.288265306122,88,0.368191052148807,14884,39 +6050818840,-50.4722336867316,-28.2214845236714,0.172761069420882,0.0422068884717393,0.130554180949142,0.1500325042808,0.192884003173823,0.111801464137099,0.00376429148517612,0.00328402780333161,0.00610392690648659,15.5290322580645,1673.56100981767,818,1139,1035.29172510519,129,0.196789986343641,13379.1,53 +6050016770,-58.712718813452,-32.9454693755701,0.135532159080341,0.0347868645777773,0.100745294502563,0.10432573279573,0.114385186572208,0.048546147398726,0.00161259569848765,0.00218379057214933,0.00342193658099993,17.5843971631206,1024.81276595745,375,869,25.8879432624113,8,0.469518753412997,12638.9,44 +6050016970,-58.8730889329443,-33.9107518533568,0.14101691296508,0.031425267038177,0.109591645926903,0.109840462838153,0.199823218341052,0.0806047282413274,0.00324080145799701,0.00297074063778445,0.00512721903958806,16.6975308641975,958.283950617284,362,812,3.17283950617284,3,1.6228153372074,1407.9,170 +6050893010,-60.2538533532446,-33.9798364321551,0.140392042734588,0.0321856546358905,0.108206388098697,0.110404999695741,0.220426235406768,0.0888723832645361,0.00393248896474034,0.00349083301231569,0.00662293163689711,16.5313272877164,968.882110469909,311,821,53.2192910140148,11,0.242642918269804,21657.1,134 +6050892920,-59.4457335855894,-33.469740966794,0.139767676823588,0.0309296088093462,0.108838068014242,0.108675985550807,0.218522645537643,0.0745140122278129,0.0031414183297717,0.00271549717230712,0.00546674970840116,17.254,1019.98,291,864,3.24,3,1.43082435137145,1800.1,206 +6050887830,-59.6955505803339,-33.2283763147258,0.136511618519524,0.0301475821689075,0.106364036350617,0.107106140749037,0.194889384720097,0.0589088987151386,0.00327887845888544,0.00260594613349628,0.00488754515286243,17.5347826086957,1001.04347826087,253,848,3.95652173913043,3,1.21017980955779,431.7,133 +6050887840,-59.1088702280906,-31.87372398914,0.138792248448127,0.0292978579322014,0.109494390515925,0.117425907213739,0.231236010343573,0.0754492815618234,0.00286056514417394,0.00232385410270166,0.00473355836647,18.2541736227045,1082.63606010017,338,917,55.9908180300501,9,0.443865647738891,21794.3,100 +6050887140,-60.8920803412961,-33.2229878774592,0.144842218196787,0.0310778024777936,0.113764415718993,0.114316494034095,0.217205851536843,0.0731091337466743,0.00286989328857547,0.00287342906760886,0.00590091905474373,16.8129958960328,928.27633378933,173,787,54.8399452804378,9,0.294044034275157,13024.5,211 +6050887050,-60.0005570377956,-32.5295108685354,0.144348593593094,0.0307385546584549,0.113610038934639,0.115903038100533,0.217744319363039,0.0870460926563004,0.00378696356189619,0.00329850202006998,0.00492952431378013,17.9490272373541,984.385214007782,236,834,44.0778210116732,13,0.278878666715309,9334.7,129 +6050879900,-60.605527352613,-32.0758829914916,0.146060314496246,0.0305143916761031,0.115545922820143,0.115298315543858,0.211138323873492,0.0723972804540715,0.00300151155545084,0.00274130546395749,0.00554435152866865,18.0868965517241,992.6,165,841,38.351724137931,14,0.281741014990975,2650.4,226 +6050879890,-62.8039978229617,-32.6998768081501,0.137448782489364,0.0255838298955782,0.111864952593786,0.10906987923097,0.172693918405011,0.051545200166777,0.00166318844255375,0.00260080844545485,0.00399818356931242,16.9128314393939,830.018465909091,40,703,216.739109848485,26,0.191453337292759,76390.7,91 +6050870380,-59.25782202716,-30.6263540581271,0.144852296852683,0.031833642866281,0.113018653986402,0.113440701052523,0.195854843337523,0.0757020443707212,0.00304341935579084,0.00273973635554102,0.00591303172816147,19.2423416789396,1061.87776141384,249,899,56.9455081001473,10,0.593008279429408,25197.8,208 +6050870370,-60.5752338780519,-31.6549253923682,0.143437807291828,0.0316203891902552,0.111817418101573,0.110581116364305,0.220183353488407,0.076747840314362,0.00286487328958713,0.00276920124299089,0.00549183008631059,18.5307692307692,953.307692307692,162,809,13.6923076923077,4,1.32329797487422,217.1,204 +6050869680,-60.1786401579626,-30.4303930675151,0.136469156211654,0.0293448931111012,0.107124263100553,0.108780880802968,0.233817083794416,0.0941634107580975,0.00345582427203506,0.00373832857272808,0.00667092946501475,19.4695180722892,1005.69518072289,190,852,32.7734939759036,5,0.642575827477031,15259.7,99 +6050869670,-61.1196344377,-30.6007069354112,0.145650835776074,0.0341837908190559,0.111467044957018,0.112296751509333,0.229768120652802,0.0843926946029481,0.00387956351345224,0.00344231900520542,0.00560720627146663,19.0233171408744,946.10478834143,119,801,66.265787647467,7,0.369085862989531,26612.6,97 +6050849780,-59.8654702036325,-28.839822481551,0.143246014467194,0.0316991814531222,0.111546833014072,0.109344343961911,0.220494277472675,0.0759543710773757,0.00288535701980577,0.00274581803124368,0.00538984118255057,20.3634584013051,1138.36704730832,138,964,51.2985318107667,4,0.98443972523874,11412.4,205 +6050849790,-57.9402291365873,-28.6754318360072,0.139339442346885,0.0336056564382006,0.105733785908685,0.103630247900827,0.144695777825738,0.0654368970719144,0.00270090520884941,0.00173845551086147,0.00592459481198795,20.8470300489168,1282.26554856744,378,1077,63.643605870021,5,1.11459684958264,26872.7,118 +6050846530,-61.6814076165317,-27.7092734255406,0.145905846914522,0.0343183890116516,0.11158745790287,0.109593061190793,0.159736996364902,0.100619263147797,0.00379017521864666,0.00337266653702714,0.00558556462459049,21.2858495670996,840.531114718615,15,710,116.050865800866,6,0.547349094319572,70446.3,116 +6050846520,-62.0019373539936,-29.288495014925,0.14197364603826,0.0281024853982463,0.113871160640014,0.116999449075219,0.264925278895517,0.102511177449922,0.00391053946217203,0.004273096551037,0.0068111819611169,20.027446300716,842.291169451074,32,712,82.3508353221957,3,0.568694243510964,15716.6,95 +6050837900,-60.4789272582472,-27.452188359267,0.135154675897493,0.0285812696591834,0.106573406238309,0.102491664944131,0.210799449602851,0.0933563409557903,0.00412429670348426,0.00343507843527748,0.00559597942912681,21.3420879120879,1003.2956043956,67,847,90.9714285714286,4,0.753864891808458,17440.7,113 +6050837910,-59.370867769379,-27.5185718415201,0.138757298480063,0.031641084779513,0.10711621370055,0.104102939895212,0.211212255942234,0.0804409371326155,0.00312324031055434,0.00281625242102922,0.00545761116543826,21.3517312661499,1111.82532299742,179,942,76.7359173126615,4,0.990724875503552,36858.2,176 +6050816640,-58.8176542974551,-27.4565948159672,0.146572339036722,0.0323939433966694,0.114178395640052,0.109598037996778,0.206037346941139,0.0679575086996289,0.0026957936978989,0.00184714423407527,0.00535580632729012,21.8,1269,239,1086,63,0,1.19207916907893,40.6,119 +6050816630,-60.7725572570486,-26.1915922952366,0.142185732621287,0.0341391750957064,0.10804655752558,0.103296256681041,0.181508483252663,0.0753871605975394,0.00239565331862281,0.00186623864620015,0.00629826698150086,21.8817769718948,921.681776971895,54,779,126.325475974615,5,1.08198717082968,21120.8,122 +6050816240,-58.6997861736695,-27.3963548906031,0.146090103936414,0.0347942423865399,0.111295861549874,0.113215259027614,0.191208713276003,0.10243399066971,0.00411643551236159,0.00361150896675533,0.00689426714281464,21.9230769230769,1261.30769230769,246,1067,54.6923076923077,8,1.08234182516846,208.2,99 +6050816160,-61.5883907254643,-25.5518691348529,0.140449981683425,0.033987022005924,0.106462959677501,0.102711948774111,0.142426239707362,0.074567625809131,0.003399397990798,0.00257762069731881,0.00649142267796021,22.1221922731357,787.867026055705,38,668,173.011680143756,5,1.09832326412867,21657,131 +6050814820,-58.8073914073323,-26.8224099529635,0.136936405257794,0.032309231917942,0.104627173339852,0.0963488535500543,0.143063032675957,0.0859581930629813,0.0034728140639903,0.00296986948148938,0.00560885111364695,21.7876865671642,1255.85074626866,224,1063,64.5522388059701,5,1.04193690519907,5156.7,135 +6050814830,-55.620021537952,-26.6821776789253,0.154495155209314,0.0323572626416391,0.122137892567674,0.121478081763894,0.187792031126841,0.070412633627276,0.00300020705265793,0.00266374348927846,0.00548757025158723,20.8522089896274,1625.97003457549,687,1307,216.294275835574,47,0.18008746003128,50030.4,268 +6050830430,-63.1187202904657,-27.0122125577911,0.140947604909957,0.031845763420463,0.109101841489494,0.109271928264264,0.132528214206527,0.0900487590418616,0.00472009529300462,0.00310694389054496,0.00679483944380521,21.4887096774194,617.639784946237,6,521,180.958781362007,5,0.995652314936034,21223.6,91 +6050830420,-63.462808549922,-27.9975074534937,0.167278525574892,0.0401774213123784,0.127101104262513,0.121327045110804,0.180186122603702,0.0672017809358454,0.0020464672896835,0.00152984868504274,0.00378783281529235,20.986875,593.56875,19,501,138.16875,4,0.758013285538453,3072.6,21 +6050808660,-63.676081817376,-23.8569360751785,0.15152710921203,0.032299268233136,0.119227840978894,0.114654547853533,0.204297717318203,0.0975151717377485,0.00364725604015566,0.00339679074411784,0.00433099462505892,19.4061049168342,702.936940230305,130,594,994.813745201974,156,0.207526051262819,107168,111 +6050808670,-59.6005459872034,-24.9682862742631,0.142962432887473,0.0322575776663104,0.110704855221162,0.10718060412916,0.152712436560482,0.0836597122550728,0.00325292753445873,0.00289877349967579,0.00555312351659584,22.3651374680307,1096.92551150895,160,922,163.36668797954,13,0.567079425594669,121712.6,181 +6050819650,-65.3535927894615,-25.73272072058,0.154604770910144,0.033623052655998,0.120981718254146,0.124200083394692,0.189873134060089,0.0716490066252587,0.00244265667265825,0.00151923982422087,0.00408300305831493,14.5456655290102,382.078156996587,102,326,1974.3795221843,280,0.173026799926299,56623.1,61 +6050819640,-64.3403060092225,-26.8855804448618,0.156349998672582,0.0488361350988486,0.107513863573733,0.119143829358039,0.151354293001736,0.156148777943335,0.00230930621303518,0.00221170015130422,0.00425039559868174,20.4675458715596,669.775229357798,16,566,340.215596330275,45,0.454769689363383,16672.5,13 +6050790150,-64.0343925874445,-21.501071263657,0.138220510302575,0.0294416058750186,0.108778904427557,0.106335065646211,0.185061278600119,0.0845999458673428,0.00347858151397808,0.00306616531582998,0.00451350010900655,15.7263826086957,533.935130434783,69,453,2319.66904347826,233,0.0866518409508088,114707.3,158 +6050790160,-58.4510693493821,-22.7419422914492,0.147919151636698,0.0321511943648311,0.115767957271867,0.108022529069009,0.160064185264407,0.0709039467314325,0.00307288692219899,0.00264753670489878,0.00502826233883876,23.8282362641481,1058.93373810059,124,896,157.797166629188,17,0.507095458498477,264041.2,219 +6050793610,-54.7469834046653,-24.8022441293157,0.149087452416889,0.0313133274746539,0.117774124942235,0.116435278303273,0.170930857111783,0.0891725910672029,0.00372769316948839,0.00265024461314422,0.00508097291802071,21.012652232747,1630.46414073072,744,1352,315.29702300406,41,0.0839961296730638,28898.8,161 +6050793620,-51.6587736271703,-25.8771454150084,0.1436371070113,0.0319052787347819,0.111731828276518,0.111395102622383,0.141658421441396,0.0866393218381011,0.00411081050284129,0.00276589201710602,0.00554355566916911,17.2401144492132,1695.31502145923,839,1305,795.592846924177,80,0.102779401234165,67535.5,158 +6050774200,-53.0300075322096,-24.4431758794614,0.150944876346793,0.030722953575145,0.120221922771648,0.118338578121839,0.202261740757306,0.084752280889163,0.00317442709507792,0.00214768916689299,0.00473692651474224,19.8083521444695,1593.33559066968,715,1307,516.166290443943,73,0.0536794899334994,26049,120 +6050774150,-54.753624798329,-23.3075429005376,0.151999663972648,0.0291027875964193,0.122896876376229,0.116530400301074,0.173013195637622,0.0877171083036982,0.00303750672910872,0.00208056976796081,0.00456585288562026,22.0620517097581,1549.95079232694,332,1312,371.095079232694,44,0.229925157822355,23471.2,103 +6050762760,-54.027008128535,-22.9581558671703,0.140074451065309,0.0305630960743422,0.109511354990967,0.108627907322738,0.186423504367379,0.0761967731412751,0.00280614797337409,0.00171008044564664,0.00437598213808105,22.5160377358491,1477.6320754717,263,1254,322.61320754717,30,0.311509343230388,2084.6,75 +6050762870,-52.0229672029275,-24.0520996172279,0.165161777182158,0.0315032609450891,0.133658516237068,0.133010760645202,0.237799489958966,0.104400219011046,0.0031069436114635,0.00226820827198644,0.0038195262336516,19.72714209686,1475.29909526344,610,1234,565.497072911123,85,0.0681222552114344,36699,77 +6050759380,-53.3135255738563,-22.6750019574912,0.140206196584717,0.0286334984109534,0.111572698173763,0.107092973708616,0.182073733827439,0.0850851701510341,0.0042838792285028,0.00281434231388739,0.0060169448786899,22.6454545454545,1310.97727272727,234,1109,300.787878787879,25,0.422925839299164,2650,115 +6050759370,-54.5984346375228,-22.0117691658648,0.150930785575446,0.0321107643829162,0.11882002119253,0.112191054349217,0.199209900028741,0.0976314259585032,0.0041638114326111,0.00286060097716332,0.00456562344488319,23.1005246589717,1450.57607555089,260,1228,393.707240293809,37,0.186147071252487,37869.9,139 +6050752460,-52.5270790696189,-22.1162577241341,0.146014372092143,0.0308306525893647,0.115183719502778,0.11672498963098,0.184719570491413,0.105677111578184,0.00508192882305125,0.00358731389180751,0.00548243004378481,22.7155844155844,1219.59925788497,242,1032,339.820037105751,36,0.363445171816041,10822.3,77 +6050752470,-51.7212095909974,-22.7362363755198,0.15048116482101,0.0312549507741869,0.119226214046823,0.115588395328486,0.175864458675766,0.0943175553675982,0.00422782602355761,0.00283810789847494,0.0050449289433887,21.4466393442623,1274.90491803279,283,1079,428.865573770492,52,0.124264842110553,24160,120 +6050710030,-57.0409052366256,-19.2813927920853,0.145992012341707,0.0328945724250217,0.113097439916685,0.106810703208949,0.180486807454951,0.0734378022458963,0.00323077566795004,0.00263065834202766,0.00531182029238232,24.8946724055476,1222.84218077475,168,1029,242.93400286944,34,0.431468920062161,211620.1,222 +6050710170,-61.3573867742337,-20.4898619611564,0.149592433271643,0.0347891253993451,0.114803307872298,0.109566831434661,0.179637057762413,0.0765620421762707,0.00341367575755449,0.00297343269251118,0.00637714310745161,24.4052455644124,589.787606068398,26,497,296.892517356647,14,0.991919258051206,78297.2,136 +6050738860,-53.7282698939668,-20.9038466260003,0.14830289224552,0.0319982372848221,0.116304654960698,0.115145072915646,0.172303571604651,0.10275258563684,0.00431694448289154,0.00318660471703891,0.00482700777563284,23.4970693779904,1399.71710526316,286,1184,444.418660287081,46,0.373188969249373,33608.6,99 +6050738870,-51.4009093620181,-21.7700863831851,0.151923833891815,0.035680011008661,0.116243822883154,0.111403990934918,0.192920422279568,0.0879273427015335,0.00409408047539955,0.00205947790493238,0.00507758193952782,22.2459119496855,1197.18490566038,221,1012,395.109433962264,37,0.206037823883495,15781.2,61 +6050728580,-50.7665607377282,-21.5345240738355,0.14722952169667,0.0313531282457928,0.115876393450878,0.111378298584538,0.188385134372153,0.0851924912997146,0.00427534139025379,0.00292372422887436,0.00522152833601305,22.0945205479452,1180.09452054795,220,999,403.902739726027,34,0.15980541477081,14642.8,90 +6050728570,-53.044092756508,-20.1084112198527,0.151100019113769,0.0309140008618341,0.120186018251935,0.110591497981953,0.165188589197647,0.0992173336009202,0.0048482280562709,0.00316985046493283,0.00556983987249647,23.8622082585278,1413.95780969479,284,1189,453.231597845601,48,0.261526690262583,22470.9,76 +6050755160,-50.704049738393,-22.877945960962,0.169942887417701,0.044596836040207,0.125346051377493,0.115497589052524,0.146724760918818,0.0747382743352301,0.00204674762537792,0.00185999780376265,0.00196542330980136,21.7078651685393,1268.44943820225,258,1073,400.910112359551,33,0.0748977924608134,1731.3,11 +6050755240,-50.5716463839114,-24.356098378181,0.152082449764823,0.0291070195393859,0.122975430225437,0.11704859448599,0.193999839604117,0.0826528406080754,0.00313137815341374,0.00198326157186605,0.00494106307455039,18.4530243519246,1453.21681068342,556,1223,793.437549096622,69,0.0888696532221356,24892.7,113 +6050721300,-52.5060590637739,-19.5231055128059,0.15766294165041,0.0370462739573054,0.120616667693105,0.121842396717004,0.165890610201244,0.114534231403028,0.00493249620689933,0.00350498418193672,0.00612943879151557,23.8102893890675,1483.20980707396,433,1227,509.842443729904,55,0.198124295731302,25219.2,93 +6050721310,-51.6018753678446,-20.6544605278531,0.162332893414866,0.0440210014497867,0.11831189196508,0.12354287378856,0.262799643739482,0.15341126650141,0.00455881310260199,0.00323050785561001,0.00509413730300516,23.8857142857143,1267.57142857143,201,1072,343.571428571429,30,0.26466285626587,644.7,27 +6050718170,-50.9926725209555,-20.3051841296554,0.151400745902482,0.0326410086865796,0.118759737215903,0.112432934041579,0.18417578373901,0.115319486296135,0.00429096870040396,0.00226805898056775,0.00530563044040127,23.2053921568627,1306.87091503268,302,1102,413.235294117647,35,0.133926071494425,12306.3,88 +6050718180,-48.4402390799622,-22.2575119800149,0.161171086092993,0.0345758673955957,0.126595218697397,0.120046364785984,0.199674040995674,0.0957075561104014,0.00331192681858059,0.00257016771993199,0.0039974842712698,20.4406043956044,1300.81895604396,458,1054,592.221978021978,47,0.134928178377511,72166.9,140 +6050757580,-50.1808552923099,-23.659067814595,0.150367223224122,0.0268055372008876,0.123561686023235,0.118856233678843,0.191531289495721,0.0822868144786418,0.00218741058344474,0.00223797836393505,0.00388841832545806,19.8125256673511,1336.4681724846,302,1131,620.618069815195,62,0.0746689014873683,9645.5,90 +6050757590,-50.1720631939299,-22.7432696512693,0.163973153068393,0.0301292015465199,0.133843951521873,0.133320692019765,0.212611783011024,0.093610791497797,0.00261316629072763,0.00251388665461465,0.00383378662859144,21.1436170212766,1282.60638297872,255,1086,484.170212765957,39,0.0914712603238088,3664.6,75 +6050709810,-51.4944790240585,-18.583461337844,0.15712887838381,0.0313336408046486,0.125795237579162,0.122514020801797,0.173970840192647,0.0888539120514562,0.00307879820252453,0.00194773852951663,0.00478402181595145,23.5543068640646,1538.00942126514,433,1226,614.090511440108,52,0.134074640165329,60319.7,96 +6050709820,-50.3352496078292,-19.9587979275632,0.156765820197739,0.0370806348641076,0.119685185333632,0.116354884060189,0.180377292229615,0.103437097599044,0.00286809660351613,0.00216490333902721,0.00476510385919645,23.2533980582524,1232.43446601942,314,1042,433.769417475728,31,0.12447422988747,8325.4,53 +6050757070,-49.2695656024597,-22.7623138650893,0.159837992403221,0.0363017784402989,0.123536213962922,0.116047674898628,0.220573826273774,0.11078386997108,0.00302791293887868,0.00208389965362698,0.00430659369955089,20.3964444444444,1288.49777777778,285,1090,624.057777777778,45,0.204592914596742,9033.7,49 +6050757160,-49.8673407364998,-23.1098149516087,0.16999714294835,0.0309284603527145,0.139068682595636,0.134923513227436,0.228233076001452,0.109189227782026,0.00263435363475573,0.00278103089182955,0.00448725019183678,20.8533333333333,1369.28888888889,262,1160,518.133333333333,61,0.0725209922408038,886.4,52 +6050760520,-48.6429734319625,-23.6921318175322,0.171420826777183,0.0366737275827347,0.134747099194448,0.135452703159774,0.214438427336138,0.0655384343710186,0.00167128828980866,0.00205168097081712,0.00266088280544644,19.1913846153846,1253.42461538462,321,1056,701.019487179487,45,0.198725367742047,19215.4,62 +6050760600,-49.5296542515184,-23.932066843498,0.159976642875262,0.0358667776704914,0.12410986520477,0.130552241277893,0.160074107964788,0.090055443338105,0.00285619626726108,0.00188323627758522,0.00237733098807524,19.1347826086957,1303.01830663616,300,1102,729.503432494279,67,0.111738315406908,8511.4,41 +6050705380,-49.1381615869281,-19.9381006653724,0.164059160619085,0.0340782452270684,0.129980915392017,0.120592570270331,0.183585420330397,0.101088271439112,0.00305536013364004,0.00222885479822389,0.00417529392834507,23.6164744645799,1348.23887973641,401,1123,522.561779242175,39,0.114036369494844,12213.9,66 +6050705490,-49.2160415882413,-20.6774551949651,0.162981609949008,0.0342287806901845,0.128752829258823,0.120459430977966,0.188627203680393,0.0825281452048655,0.00288783833167251,0.00218126678491754,0.00415077556569573,22.8398773006135,1267.24948875256,384,1061,513.208588957055,30,0.0804249696472012,9913.9,85 +6050667100,-55.3950454974208,-16.4841304714961,0.146574256816097,0.0318765156381069,0.11469774117799,0.107449208956971,0.169859065349909,0.0763007855523572,0.00335789346415082,0.00274372302654167,0.00551242467807376,24.7694931222913,1451.91313359714,319,1175,315.080082909365,46,0.230304450372408,109239.9,208 +6050667090,-58.1405681040799,-16.289672478206,0.14650266776868,0.0336590837944373,0.112843583974243,0.103732571848455,0.168253050807533,0.0669917785652797,0.00290603165847265,0.00216933099582157,0.0054323780150137,25.3462632869992,1431.79820114473,261,1177,229.590024529845,34,0.394359629032442,125982,183 +6050681620,-50.1054665590304,-18.5576790314963,0.15283819942406,0.0288144649295103,0.124023734494549,0.116450842685534,0.153661751903349,0.0760428670244084,0.0023166473968944,0.00156043494467439,0.00329492552234936,24.875,1442.25,310,1197,429.625,12,0.190738170390741,152.9,56 +6050681720,-49.0455725413411,-19.1698732086367,0.155253223540824,0.031978430899652,0.123274792641172,0.119001615823206,0.134436174170617,0.0543157233298922,0.00208545991683519,0.00176801637829763,0.003432934229543,23.0810126582278,1403.31364275668,372,1153,660.82841068917,41,0.124443886522562,14430.4,51 +6050680410,-49.2763198974054,-17.6760372747344,0.162822250726771,0.0289177598859684,0.133904490840803,0.124665886963718,0.246391418197298,0.129131491392337,0.00389850534224332,0.00256003509760901,0.00446354571498967,23.4670113753878,1351.31437435367,363,1112,694.907962771458,63,0.100631505973221,19715.5,70 +6050680380,-50.2964918956566,-17.3829697299623,0.169046470469099,0.0336450246166293,0.13540144585247,0.131890706655511,0.209919386651765,0.115671890903533,0.00352917944544223,0.00278400315192049,0.00496383243848489,23.8473498233216,1487.76383981154,331,1178,649.378680800942,44,0.103562880477518,34881.1,52 +6050709700,-48.1989671027042,-19.757411533434,0.170767768457771,0.0337789030621854,0.136988865395585,0.131292863279886,0.239030685000656,0.119764384240964,0.00351606663909727,0.00266351195236048,0.00406209996023848,22.5921875,1490.828125,481,1186,694.578125,56,0.109749827343975,2565.8,55 +6050709930,-47.4008349260177,-21.5610849686118,0.162117397253495,0.033874252908083,0.128243144345412,0.120069283127357,0.181545605990037,0.0801480544970001,0.00249971029571891,0.00215068471877253,0.00356499380058575,20.5733445566779,1433.87766554433,502,1123,736.957351290685,67,0.0882905818629085,35672.9,112 +6050708750,-47.1935494768512,-20.368886692539,0.153347311031697,0.0346289498464726,0.118718361185225,0.109225329149129,0.147244396672035,0.0667450313816185,0.00190285359553468,0.00174499493480102,0.00329606988931531,20.8140776699029,1499.1395631068,527,1126,821.42354368932,102,0.0850197705461584,16487.1,63 +6050708820,-47.5580282380966,-20.6692527565344,0.186261115496759,0.0406265659575187,0.14563454953924,0.134449661794345,0.160288283918239,0.0940152363662664,0.00246513366440237,0.00203083596155203,0.00260340007047103,21.1114457831325,1498.65060240964,539,1102,762.454819277108,82,0.0863968898721913,6670.3,36 +6050674150,-48.3772376103694,-16.820092804611,0.145201450239537,0.029722691972924,0.115478758266613,0.109945025315204,0.163622180787928,0.0754315076273896,0.00277852832472249,0.00192797157655743,0.00387033832600868,22.4814389359129,1404.2345828295,395,1135,873.899637243047,65,0.108302301622612,34071,101 +6050674210,-48.7912685898416,-18.4426038790797,0.165521440229232,0.0294927293244702,0.136028710904762,0.144643280551368,0.0880426063626452,0.0623409735551427,0.00104760494396578,0.00238884476029709,0.00517697569894476,23.2333333333333,1286.27272727273,326,1074,665.606060606061,64,0.0811986063011994,680.9,17 +6050675150,-47.2915910608088,-19.3402072147827,0.158406879241452,0.0302212676028783,0.128185611638574,0.118555862401391,0.185696653588228,0.101387087927277,0.00318085641459633,0.0019792203270981,0.00368871924416867,20.6666972477064,1526.52660550459,419,1142,959.128440366972,78,0.0973990981148443,22096.3,65 +6050675090,-47.4315517500832,-17.9901021337368,0.159819319279111,0.0323511794825723,0.127468139796539,0.120867757435843,0.145766900470048,0.0726268757409091,0.00218709731607169,0.00188231244515951,0.0034993020343045,22.2562363238512,1419.5306345733,380,1139,857.542122538293,60,0.107927520866501,37235.9,75 +6050719760,-44.774825349071,-21.214116765622,0.158401870939097,0.0370234124322803,0.121378458506817,0.126590530949821,0.132517046238286,0.0765599678947546,0.00248450393834316,0.0019442143111162,0.00279278333261026,19.3078549848943,1529.71072507553,525,1140,997.416918429003,61,0.12532055700586,26432,52 +6050719850,-45.6781086337172,-21.8539864628663,0.155860916144979,0.0356597079769816,0.120201208167997,0.107514983277275,0.216144457908505,0.0825153074874307,0.00301445110815013,0.00143518888527737,0.00268067124852865,18.9522943037975,1539.0332278481,514,1159,1000.21835443038,86,0.0909350548768758,25200.1,41 +6050016980,-58.4709455977817,-34.8490110047485,0.139870925178069,0.031570041264255,0.108300883913814,0.105840163605435,0.21899959931735,0.0770129225813578,0.00295337229053319,0.0028122564501532,0.00585262729074628,16.2430656934307,988.699817518248,372,839,20.3156934306569,8,0.484131697292388,19412.8,203 +6050017380,-59.9934628231741,-36.1358821512222,0.139025999572323,0.0296747307744065,0.109351268797917,0.111202076707605,0.192807057303566,0.0550531103378321,0.00133740420089406,0.00186680009175868,0.00385156238345818,15.0883303048657,927.14455885322,208,786,93.2457096709065,8,0.358039418544828,86109,36 +6050017540,-57.7291615657806,-37.563001646552,0.122572139402915,0.0394508068803294,0.0831213325225852,0.0901821659746607,0.0924735936720189,0.0595781984055207,0.00213384290453058,0.00174518024303482,0.0097915842501493,14.0778743961353,822.193236714976,322,698,43.5004830917874,12,0.625363224926258,17652.6,6 +6050017980,-59.8725258744563,-38.4187969571899,0.142451818016236,0.0541893634560417,0.0882624545601942,0.11576372111946,0.106541979408692,0.0913597302734211,0.00291011065534374,0.00230327666030694,0.015777427785825,14.0261802575107,766.722460658083,267,652,91.1945636623748,16,0.492197629063495,11733.6,3 +6050018290,-62.8815089476421,-38.5852714987735,0.142451818016236,0.0541893634560417,0.0882624545601942,0.11576372111946,0.106541979408692,0.0913597302734211,0.00291011065534374,0.00230327666030694,0.015777427785825,14.8485091743119,528.426987767584,57,449,117.92125382263,36,0.648168264996823,43879.3,3 +6050018850,-64.1739574935507,-39.4467221113666,0.117012046808746,0.0415170047002983,0.0754950421084482,0.083994802139893,0.0773474769787554,0.0726064820157771,0.0032913967177703,0.00161370968720775,0.00914362000487952,14.9773550724638,323.865942028986,7,276,106.669685990338,16,1.28761940287781,27381.9,7 +6050933820,-68.6128265159227,-37.347324594422,0.111802730597666,0.0389810575523757,0.0728216730452906,0.0861472091171811,0.0809263008956295,0.0765980730995295,0.00294727843579077,0.00188033552428852,0.00748580808953348,11.5180040733198,299.997148676171,131,252,1123.87657841141,197,0.247371227030132,41850.9,10 +6050933800,-66.3135290446676,-37.1833063274466,0.124391773719776,0.0511209879794837,0.0732707857402921,0.0890634890808473,0.0819852301169578,0.0937382076972666,0.00341181672167882,0.00207145973564243,0.0124292388824824,15.0573163678427,381.916714864083,10,324,363.537015615963,28,0.955421344733274,58969.4,5 +6050913910,-67.700051558634,-33.8442954628686,0.10575066120377,0.0358270461551845,0.0699236150485852,0.0721973819118404,0.216163139622197,0.0612524113756677,0.00333345718264645,0.00146551602748515,0.00747037533728308,14.9267410104688,344.392125625853,31,292,851.2369139736,94,0.256512367397023,78538.3,9 +6050913900,-68.5379681737319,-35.1202001442438,0.0975385122686125,0.0318913938202486,0.0656471184483639,0.058898594043658,0.0532568376889361,0.0578494325061737,0.0036858110124897,0.00107002403572866,0.00417865386285799,11.6517127071823,355.224861878453,92,300,1355.66685082873,172,0.211707108441297,31857.6,4 +6050877660,-69.2448620008383,-31.8895417481345,0.106752969589203,0.0381264411676353,0.0686265284215672,0.0757771397786052,0.233793471271965,0.066693842593141,0.00339123167363386,0.00154384813415756,0.00826671633307588,8.12528127068167,199.791197882197,62,179,2350.79086697551,389,0.0729396881011761,55089.2,8 +6050862860,-69.1028971234418,-29.7199590878522,0.147489799027118,0.0360680576587087,0.111421741368409,0.0955114541825527,0.277554016649107,0.0438462246754671,0.00200321833981392,0.00142213644951921,0.00229747986949108,6.65566180171112,127.867639657776,12,123,3026.80976346251,331,0.138713831001297,37026,8 +6050862810,-68.2972794691915,-29.2451115467055,0.0977608297830426,0.017266764176313,0.0804940656067296,0.0645988365520554,0.0626290575094486,0.0194765185011277,0.0030386104842381,0.000421453890110626,0.00098159624394001,12.1724420190996,151.351296043656,6,133,2195.01841746248,317,0.148292368516616,27497.9,1 +6050019090,-65.5656955965758,-39.7215319603848,0.0886009530947613,0.0293116735985428,0.0592892794962185,0.0569444182084971,0.112131116639046,0.0475602873425654,0.00241982648718435,0.0014350747882342,0.0108199007589649,14.6719626168224,268.281619937695,8,230,181.760124610592,25,1.33591045738712,26576.9,14 +6050934620,-68.7852415699674,-39.3721550422911,0.0898983225823105,0.0309508002849451,0.0589475222973653,0.0562422834212211,0.0601144881084736,0.0617271480161819,0.0027875541989524,0.00113305616554514,0.0050207294786183,12.5492413793103,140.76,9,124,582.579310344828,84,1.22970071637971,11998.4,5 +6050934630,-70.1023170046753,-37.9948579550779,0.0807624268550697,0.0281828726190273,0.0525795542360424,0.0502284892381608,0.117169955551339,0.0570492328136463,0.00262324327057111,0.0013845658760755,0.0103041417015976,10.2845342706503,423.467926186292,354,329,1235.5795254833,230,0.298736682360626,38327.7,6 +6050937230,-69.9244702377867,-39.3176360318664,0.0921803644459821,0.0332681130185576,0.0589122514274245,0.0626450907354577,0.101279155920887,0.0582790028526118,0.00283254862839572,0.00158628625123286,0.0103467248772407,11.06875,217.57421875,187,191,868.75,107,0.787859859702398,4280.6,10 +6050937260,-69.7606482045439,-40.0173752893778,0.0862171446655485,0.0309927988153506,0.0552243458501979,0.0516553053236932,0.0977408737831181,0.0630830102100621,0.00307409701677832,0.00147265288835825,0.0111264836204975,10.3706775700935,222.23714953271,140,195,850.896028037383,119,1.15164444913023,14111.8,9 +6050942390,-70.3869352003607,-40.53767223631,0.0881353826419864,0.0265255024364968,0.0616098802054897,0.0599602671018396,0.0931018792612346,0.039215440268864,0.00202556024002199,0.00120498677904061,0.00700437172576777,9.55116279069767,385.395348837209,314,335,825.395348837209,148,1.06341487810395,688.8,11 +6050942410,-70.223276376288,-40.9892719270764,0.0926089192213838,0.0330739943810896,0.0595349248402942,0.0555922786987975,0.0596525714102997,0.0667609293861988,0.00323314806529254,0.00132647019581211,0.00792714372099792,8.1258865248227,329.719858156028,297,289,1070.63475177305,179,0.948679848968267,4531,8 +6050942470,-71.1947702336496,-40.9162382876941,0.0706354557730161,0.0243018628310643,0.0463335929419517,0.0398799634280283,0.0990236601404,0.0441015482436936,0.00279930775970371,0.00127900464734397,0.00801928077356173,7.01334405144695,845.633440514469,922,538,1143.98874598071,269,0.48352073641507,10010,7 +6050942450,-71.0223739415389,-39.7183065055215,0.0661097309449794,0.0254751797070104,0.0406345512379689,0.0343650810699132,0.105511256682536,0.0485669123080596,0.00270321206576136,0.00142364316264918,0.00920264287706281,7.84309109518936,699.273285568065,713,499,1235.63766632549,255,0.560932997564699,16192,6 +6050020000,-65.8626292784825,-43.5732662661329,0.0730695613685898,0.0273931007352878,0.045676460633302,0.0374459560690471,0.0503199976340057,0.045426949109911,0.00285191548893719,0.00109107529241807,0.00335143414655257,12.6456852791878,180.517766497462,19,162,186.131979695431,40,1.46097097159951,6082.2,6 +6050954860,-69.6550427399185,-43.0313959200252,0.0676439135208302,0.0255325258859968,0.0421113876348334,0.0349764649624489,0.0996521045310235,0.0437346819323206,0.00270221397030697,0.00131239787597897,0.00812133894931743,9.14712863705972,280.097626339969,244,245,749.037136294028,144,0.746538199327952,41050.1,7 +6050954870,-67.2575462718719,-44.878606061249,0.0635737928580847,0.0241338644364269,0.0394399284216577,0.0331243466325001,0.137501503060354,0.0424753037909016,0.00258964830260168,0.00147861244507252,0.0116963219327145,11.1607648725212,180.78895184136,32,166,378.78328611898,76,1.29928816573599,10779.1,8 +6050960240,-69.4170428144405,-46.1216637569116,0.0676429023472007,0.0255288341001664,0.0421140682470342,0.0349764649624489,0.100623896938528,0.0437459073514907,0.00270188308032682,0.00131265231189082,0.00811854395654422,9.74471830985916,170.325704225352,13,160,547.894366197183,47,1.33628004346489,8444.2,7 +6050960220,-69.1080816651312,-45.5335376667001,0.076510776089744,0.0332751339270544,0.0432356421626895,0.0370934937782185,0.0498448810000481,0.0578752082674436,0.00311892456505226,0.00127008280168585,0.00443661306699543,10.2926517571885,153.445686900958,14,144,422.658146964856,62,1.19551107527234,9421.4,4 +6050960610,-70.8931635515015,-45.7552034969424,0.0635737928580847,0.0241338644364269,0.0394399284216577,0.0331243466325001,0.137501503060354,0.0424753037909016,0.00258964830260168,0.00147861244507252,0.0116963219327145,7.20857632933105,295.42538593482,288,264,738.408233276158,77,1.27235146009265,8731.2,8 +6050960580,-70.2316650795077,-45.3316321302218,0.0635737928580847,0.0241338644364269,0.0394399284216577,0.0331243466325001,0.137501503060354,0.0424753037909016,0.00258964830260168,0.00147861244507252,0.0116963219327145,8.62131578947368,217.021052631579,160,199,627.657894736842,59,1.32063208864196,5701.3,8 +6050958450,-70.9456697884271,-44.7948025306087,0.0635737928580847,0.0241338644364269,0.0394399284216577,0.0331243466325001,0.137501503060354,0.0424753037909016,0.00258964830260168,0.00147861244507252,0.0116963219327145,6.74478417266187,369.032374100719,821,306,874.176258992806,115,0.878538638644458,8478.5,8 +6050958420,-70.6289933845139,-44.1256146099274,0.0635737928580847,0.0241338644364269,0.0394399284216577,0.0331243466325001,0.137501503060354,0.0424753037909016,0.00258964830260168,0.00147861244507252,0.0116963219327145,7.37147239263804,230.003067484663,666,212,887.40490797546,92,0.999483130404388,5020,8 +6050020960,-66.2986204050643,-48.1827834403375,0.0556645970535177,0.0249922318093333,0.0306723652441843,0.0269585280317138,0.161013871416483,0.0509176723690013,0.00249744446147595,0.00179345641052148,0.0151625047973906,10.0635135135135,225.445945945946,42,209,38.0540540540541,50,1.65791543587996,1078.2,6 +6050021020,-67.7223721914236,-48.7504392619663,0.0556645970535177,0.0249922318093333,0.0306723652441843,0.0269585280317138,0.161013871416483,0.0509176723690013,0.00249744446147595,0.00179345641052148,0.0151625047973906,9.60780141843972,231.985815602837,23,214,162.971631205674,38,1.48925974551733,1996.3,6 +6050021030,-68.1245084507807,-49.1388556297486,0.0556645970535177,0.0249922318093333,0.0306723652441843,0.0269585280317138,0.161013871416483,0.0509176723690013,0.00249744446147595,0.00179345641052148,0.0151625047973906,9.05527426160338,222.662447257384,9,207,201.810126582278,53,1.41840029257409,3265.4,6 +6050021060,-68.2463866707712,-49.4535166958452,0.0556645970535177,0.0249922318093333,0.0306723652441843,0.0269585280317138,0.161013871416483,0.0509176723690013,0.00249744446147595,0.00179345641052148,0.0151625047973906,9.28211920529801,216.860927152318,8,202,75.0662251655629,83,1.24710838041984,2145.7,6 +6050021070,-68.0890951971185,-49.7765230374353,0.0556645970535177,0.0249922318093333,0.0306723652441843,0.0269585280317138,0.161013871416483,0.0509176723690013,0.00249744446147595,0.00179345641052148,0.0151625047973906,8.80521739130435,210.730434782609,8,198,68.6130434782609,33,1.58931201534757,3151.6,6 +6050021160,-69.0970300889055,-49.4550581417909,0.0556645970535177,0.0249922318093333,0.0306723652441843,0.0269585280317138,0.161013871416483,0.0509176723690013,0.00249744446147595,0.00179345641052148,0.0151625047973906,8.93380281690141,182.640845070423,38,173,156.112676056338,54,1.2980647854732,1956.1,6 +6050021170,-68.5517840438076,-49.8652595236103,0.0556645970535177,0.0249922318093333,0.0306723652441843,0.0269585280317138,0.161013871416483,0.0509176723690013,0.00249744446147595,0.00179345641052148,0.0151625047973906,8.95,196,6,186,10.5,24,1.62599702196179,32,6 +6050021180,-70.9051330319683,-49.0671842884782,0.0556645970535177,0.0249922318093333,0.0306723652441843,0.0269585280317138,0.161013871416483,0.0509176723690013,0.00249744446147595,0.00179345641052148,0.0151625047973906,6.92632275132275,197.085537918871,77,189,574.993827160494,107,0.757205354078989,31903.4,6 +6050021190,-68.6488680175636,-49.9270740242471,0.0556645970535177,0.0249922318093333,0.0306723652441843,0.0269585280317138,0.161013871416483,0.0509176723690013,0.00249744446147595,0.00179345641052148,0.0151625047973906,8.7,192.333333333333,43,182,47,100,1.47588455069484,44.1,6 +6050021210,-71.7479899119659,-49.9827846140839,0.0659536519169919,0.0303102699440778,0.0356433819729141,0.0303580328861412,0.041997797455026,0.0595663246365653,0.00284474332041636,0.00136957266426428,0.00437826455048569,5.73024928092042,507.585330776606,187,328,623.31975071908,173,0.365366923945424,28902.3,4 +6050024060,-72.3020995613942,-46.9262703262327,0.0661922771056731,0.0230602645759626,0.0431320125297105,0.0374149077898822,0.136101295861991,0.0427864895715791,0.00267489489705645,0.00144917073859402,0.0116094666476048,5.11105552776388,739.565782891446,494,436,887.206103051526,363,0.325889594729347,29292.3,8 +6050025410,-71.8769747244305,-43.9356193601566,0.0454507870198038,0.0148066031907877,0.0306441838290161,0.0260831387445969,0.159275913694134,0.0251981459682701,0.00187532312816858,0.00141017243139785,0.0130706385529443,6.32286049237984,1241.46776084408,2193,599,904.071512309496,501,0.353506272938556,13162.2,6 +6050025420,-72.2117973259409,-42.2593920471495,0.0858679164613738,0.0347083345537053,0.0511595819076684,0.0517275066089171,0.14644424573563,0.0570877472683545,0.00389859421216897,0.00164760463424257,0.008732457813952,8.10133630289532,1762.78619153675,1399,677,639.583518930958,441,0.416550820362364,49783.1,17 +6050026140,-72.675717181263,-39.4558703473771,0.0844880018576858,0.0312840228266245,0.0532039790310613,0.0508869445476675,0.169573422051307,0.049999703014349,0.00444983507525674,0.00147337641601855,0.007308962292852,10.2200165425972,1716.76730079956,1225,735,436.08547008547,235,0.995315839921746,60207.6,21 +6050026750,-71.9576333188409,-37.7380965455128,0.0870006765780341,0.0321467430163559,0.0548539335616781,0.053300892229467,0.157064954209697,0.0505562218918191,0.00458963249821059,0.00139827583206074,0.00582841295837997,10.0692092372288,1422.04198740378,1157,686,801.296011196641,242,0.447526156384756,24251.9,20 +6050026760,-72.1331579695813,-36.5788754256729,0.0831335199615419,0.0317734530724103,0.0513600668891316,0.0460099691290187,0.161672798838022,0.0542668615349331,0.00489644041072763,0.00136081177078653,0.00623006177673201,11.6542920847269,1129.0635451505,874,635,496.188405797101,192,0.549398683702705,15445.1,18 +6050026990,-71.4395593142744,-35.8636947305183,0.0890946488440812,0.0339266287862493,0.0551680200578318,0.0483894114399296,0.156293072937619,0.057860680271122,0.00521823477983135,0.00132289455183145,0.00463598741313792,10.8477722772277,948.558580858086,739,605,883.480198019802,274,0.219741372701649,21104.8,16 +6050027000,-71.4482024500269,-34.9605282214069,0.0971051817976583,0.0401161323821443,0.0569890494155141,0.052279802174506,0.173450160306431,0.0658265311633226,0.00560999790054678,0.001473144613061,0.00521176112285632,11.8867579908676,832.089802130898,500,568,718.354642313546,283,0.254646431163333,11562.6,13 +6050027220,-70.9110785667614,-34.4148104258678,0.0889940866010271,0.0359820630943154,0.0530120235067118,0.0493837329150821,0.123890626663335,0.0608329960103938,0.0045719204287798,0.00130767725554039,0.00472613720196911,11.1939820742638,715.856594110115,379,523,1155.71318822023,366,0.164437618470858,13785.3,11 +6050027230,-71.5378647566015,-33.8868103850556,0.0945205666123632,0.0401671432770504,0.0543534233353128,0.0501598948852813,0.193211400892676,0.0751303238481868,0.00623745821784373,0.00155323617974857,0.00576328573713375,16.5693548387097,477.645161290323,165,407,181,57,1.06340648053969,1067.7,11 +6050027290,-70.587435875422,-33.5751760210071,0.0934152133852005,0.0373081693512104,0.0561070440339901,0.0512144807596985,0.174397663098094,0.0660424868388372,0.00580174575469613,0.0013909812746338,0.00507472377361672,9.6875,494.566666666667,305,412,1670.30714285714,446,0.106116859441047,15100,13 +6050027300,-70.9893124065472,-32.2538712037861,0.0978216712115608,0.0429412425945328,0.054880428617028,0.0539938356989206,0.203048077038299,0.0797669543575735,0.00667071586307015,0.00162138477709105,0.00612769501145759,11.1958390177353,307.210777626194,130,266,1319.66234652115,417,0.176639066589254,26557.6,10 +6050027770,-70.8698271952336,-30.7683586055983,0.0930497485150708,0.0389769682719114,0.0540727802431594,0.0494015826916135,0.183371831537218,0.0700774364585821,0.00594014278524578,0.00147320130223549,0.00542015127085343,9.30800627943485,150.425431711146,65,139,1675.00627943485,459,0.139346198819495,11697.9,12 +6050027780,-70.8252442962804,-29.6723597230357,0.0897279787031712,0.0344237730283555,0.0553042056748157,0.0513491933035518,0.119125377927106,0.0572321706263526,0.00453600884205651,0.00123239922952982,0.00440975858570303,9.55234234234234,87.3333333333333,26,87,1659.34864864865,466,0.114709178125285,20821.6,12 +6050028110,-70.3215140646944,-28.9716177962746,0.107845955006731,0.0441674247545803,0.0636785302521506,0.0610500855240038,0.194052743076247,0.0739033510406148,0.00644962359472981,0.00157605709492613,0.00578294426758003,5.73747609942639,79.4569789674952,16,85,2736.58508604207,649,0.0727666368700986,9789.5,11 +6050028120,-70.6098963265221,-28.0649656084912,0.0930497485150708,0.0389769682719114,0.0540727802431594,0.0494015826916135,0.183371831537218,0.0700774364585821,0.00594014278524578,0.00147320130223549,0.00542015127085343,13.2520979020979,37.8653846153846,10,39,955.146853146853,240,0.144693271802732,10693.6,12 +6050029280,-70.7458531459989,-17.1533212928998,0.137516485216716,0.0383586004262624,0.0991578847904538,0.112206334481835,0.0772098173112507,0.0756302413728244,0.00132253764073254,0.00144513784824849,0.00325381075582166,11.6999438832772,188.038159371493,36,161,2506.8367003367,456,0.12942041459537,36532.9,9 +6050029580,-71.594066245533,-16.2147612573948,0.156395717043609,0.0538299325406205,0.102565784502988,0.0933546525903739,0.178503411064818,0.132152286368549,0.00258618363745962,0.00375462871771397,0.00372247406052929,9.21480916030534,261.688549618321,80,226,3290.53129770992,321,0.151585020499319,13581.6,10 +6050029590,-72.5548519244139,-16.6218856930165,0.120233114394569,0.0324858176790631,0.0877472967155056,0.0970830497655297,0.140374795696808,0.0703451929180896,0.0013353823156189,0.00141018230052634,0.00308965566135385,19.3545454545455,0.954545454545455,4,5,469.5,516,0.215332498650742,448.9,10 +6050029610,-72.0212403694377,-15.652391458382,0.105894911304201,0.0269953246336962,0.0788995866705051,0.0792651068799697,0.153340460823015,0.0687612343771933,0.0014248232855417,0.00141715612532567,0.00300679731602415,7.66835138387485,438.075812274368,215,366,3600.08182912154,527,0.107190007770204,17093.8,10 +6050029620,-72.8542229813998,-16.2293724105846,0.125314201166729,0.0300517300905081,0.0952624710762211,0.0981897271234648,0.124192616015697,0.101470986189965,0.000991520349925977,0.00244586810913436,0.00526725891976722,16.2606837606838,48.6068376068376,16,43,1651.96581196581,388,0.408375918904453,2479,4 +6050029720,-73.0839747138381,-15.2422702992889,0.16319396867462,0.0592377027109379,0.103956265963682,0.107664816741488,0.18153806052453,0.134271101746088,0.00256745481419328,0.0037752888308619,0.00394871258791147,7.23840206185567,605.288659793814,205,475,3745.70618556701,721,0.0935855307745594,16063.1,10 +6050029730,-77.2330524382589,-10.6121338569275,0.220067939531778,0.0879468046696663,0.132121134862112,0.131491774974045,0.19955799357117,0.248240104051805,0.0042622856583368,0.00679250501097365,0.00622907061690466,15.0430385756677,294.116201780415,183,241,1830.46991097923,502,0.0722751867242752,177191.9,5 +6050032100,-80.0038964278121,-4.4978006069035,0.106864011875606,0.0255192126254838,0.0813447992501219,0.0772977923068385,0.158511906484791,0.0588860836259417,0.000963416606210605,0.00158301959485851,0.00332640235388959,20.8059393939394,684.396363636364,280,554,1161.1696969697,331,0.0733054958329502,17761.9,8 +6050032110,-80.4942016462831,-4.06804545029524,0.12110561519375,0.0339303111729625,0.0871753040207875,0.0853985871220527,0.119222032227456,0.0520187283869717,0.000952798005007383,0.00131361896401195,0.00391106965740559,22.4664783427495,561.84934086629,192,431,584.131826741996,229,0.135523546202635,11236.5,13 +6050032290,-79.5615179297974,-3.05550293545965,0.144443663388101,0.0364673144331075,0.107976348954993,0.0962350289783203,0.0736692155451364,0.0567558130737932,0.00108184280714846,0.00130516528973041,0.00361621500817321,19.8419270833333,894.092447916667,475,709,1252.96223958333,449,0.173350528241306,16478.7,14 +6050032610,-79.5453587330328,-1.39851168514494,0.127301899903774,0.027737482118637,0.099564417785137,0.0786643259604514,0.0742320098642003,0.0388061422410708,0.00196299022211479,0.00107417700316952,0.0025093514622387,21.9990795529257,1565.6449704142,976,939,708.96449704142,240,0.127160737464326,32684.5,31 +6050032620,-80.2961289430846,-1.14125222570092,0.162743185079745,0.0454293494321621,0.117313835647583,0.115135544550709,0.109064923692435,0.0895009471666224,0.001334469334508,0.00196407082011822,0.00620676224102927,24.4433537832311,775.792433537832,479,599,146.191206543967,71,0.148827325886737,21050.5,7 +6050033220,-78.9915219300931,0.0134396147828197,0.11892287163937,0.0301096429099756,0.0888132287293942,0.0743957970653886,0.0805213867705081,0.0491854615304097,0.00165936777489316,0.00109889163000889,0.0038323892939641,18.5403788634098,1931.29112662014,1287,1099,1494.97607178465,340,0.186459375814009,21626,23 +6050033230,-78.5584734280413,0.99821312146679,0.122484166317339,0.0268256881535923,0.0956584781637469,0.0726189339667479,0.135499908415756,0.0428008046523829,0.00131741137043022,0.00123430236373965,0.00282685919149341,21.0990548204159,2350.25992438563,1651,1239,1016.06994328922,314,0.328287352386946,22671.2,31 +6050033560,-77.5426682927249,1.66327347136259,0.148207839353798,0.0335295036827029,0.114678335671095,0.0881475270247808,0.241003147447728,0.0627676583738182,0.00245959688885233,0.00174503422273081,0.00337297895214554,19.4056405353728,2635.01816443595,1981,1277,1461.43116634799,426,0.157391318246397,22420.9,27 +6050033570,-77.3731182226486,2.87402409676697,0.118635081453126,0.0246144293110656,0.0940206521420604,0.0753173593277276,0.190733003734749,0.0434204645924245,0.00156757544003472,0.000941926001249791,0.0025049986096721,23.9659025787966,4855.07831900669,3477,1499,581.407831900669,306,0.483173120811786,22325,37 +6050034060,-76.5946667923958,4.63714123331166,0.14037041833582,0.0265392392479065,0.113831179087914,0.0895012822369315,0.27090308357422,0.0643734872719981,0.00195895197101478,0.00231534551579782,0.00359186870303018,23.472060857538,4768.70262793914,3834,1471,685.856154910097,341,0.306759481642309,15397.6,45 +6050034070,-77.8483469685518,7.51310249730352,0.136045707173069,0.0260822491280857,0.109963458044983,0.0850879960028216,0.245805359206255,0.0525040715831721,0.00146375012513029,0.00166013154817966,0.00360137977768916,25.4442143727162,3534.5700365408,2747,1308,207.289890377588,153,0.438196307977268,35190.6,44 +6050035990,-68.8282268518957,-54.0860075079467,0.034988778467156,0.0143145709936827,0.0206742074734734,0.0183799991831052,0.178605284931071,0.0263424714616986,0.00164266565695467,0.00160791613965529,0.0154884470147451,4.71958290946083,883.041369955917,825,421,240.028653780943,193,0.423972721096339,74850.8,5 +6050040460,-66.3499767293231,-29.5131463118973,0.119444116123671,0.0270061142614456,0.092438001862225,0.0926563774284055,0.122959382539813,0.0544409732841452,0.00291790796701207,0.00110724782186982,0.00451801455907032,16.340571497468,352.892572944297,21,300,1315.25006028454,171,0.246454113405124,155117.6,13 +6050040470,-68.7726797673518,-17.236874832798,0.58521450376044,0.302494553973572,0.282719949786868,0.214363261940545,0.401289307600611,0.620709461901403,0.0132219904483592,0.023228657361547,0.00958660797790356,6.63711123564023,480.848136732978,145,408,4074.19038946484,101,0.219185710670381,146228.6,22 +6050040480,-63.9510814642831,-29.4271651639736,0.157343211266402,0.0382970205015349,0.119046190764867,0.126353936236257,0.221032438431968,0.0682171025823388,0.00262755423743408,0.00218224516199963,0.00439990063774311,18.6,713.791660675773,64,604,400.267577282531,74,0.339177945961966,129875.1,51 +6050040490,-67.5126036752522,-20.8548069180289,0.565664823296234,0.2911254618914,0.274539361404835,0.206935051121684,0.389475972840008,0.600326557385644,0.012822202176791,0.0222988615678064,0.00925055344647188,7.0257217847769,104.391404199475,2,94,3986.38254593176,101,0.158629261504401,61465.2,12 +6050040500,-67.6132359963091,-42.8523631236104,0.0907802412107016,0.0330392198988753,0.0577410213118263,0.0570620596157919,0.0988834728831354,0.0429919981025153,0.00214725924335886,0.00144914087318048,0.00751531976804257,11.0778804682686,177.160813308688,18,158,576.770178681454,90,1.02877181367081,25505.9,10 +6050040510,-62.7132022838083,-35.7534341580381,0.142451818016236,0.0541893634560417,0.0882624545601942,0.11576372111946,0.106541979408692,0.0913597302734211,0.00291011065534374,0.00230327666030694,0.015777427785825,15.8509689213894,757.15831809872,29,642,108.254844606947,5,0.288215688572897,47611.4,3 +6050040520,-66.0479382723615,-23.1823752762022,0.147222377717676,0.0360622500122921,0.111160127705384,0.105132940523289,0.134490815077544,0.0740190322952972,0.00264049074275934,0.00209356593742473,0.0051452929047176,7.9094495412844,155.42752293578,80,135,3841.01926605505,135,0.147030417372173,21476.2,124 +6050040530,-68.3737038552907,-36.5489232084889,0.115294577659545,0.0322231347319135,0.0830714429276318,0.0784699946370659,0.0744785678754874,0.0607240814424535,0.00342022770048465,0.000964538407055309,0.0048477262423478,12.4599901816397,256.070692194404,25,219,1069.04270986745,94,0.559317868449925,35292.5,5 +6050040540,-68.4588410797861,-23.7397309334529,0.579841309737003,0.299424150192688,0.280417159544315,0.217441113844164,0.398000652907831,0.61389770989224,0.0131405290149926,0.0229679648281066,0.0095405995166755,9.96640735502122,28.4229137199434,2,29,3196.50495049505,286,0.346805316364874,27715.5,18 +6050040550,-67.6448156173837,-39.8631467913336,0.0907802412107016,0.0330392198988753,0.0577410213118263,0.0570620596157919,0.0988834728831354,0.0429919981025153,0.00214725924335886,0.00144914087318048,0.00751531976804257,12.8764307228916,198.876506024096,5,172,509.704819277108,62,1.29308850600786,21950,10 +6050040590,-69.1567894836296,-41.3257529883608,0.0907802412107016,0.0330392198988753,0.0577410213118263,0.0570620596157919,0.0988834728831354,0.0429919981025153,0.00214725924335886,0.00144914087318048,0.00751531976804257,8.30989010989011,202.2696534235,84,181,1123.59425190194,90,0.939800309909803,19034.3,10 +6050040600,-64.8895353410468,-33.8199065648967,0.131271912042254,0.0449999941059258,0.086271917936328,0.0977126205840252,0.0986866061625333,0.0729500381140362,0.00290039830519459,0.00193717228875666,0.0121079826821039,16.3099730458221,671.854447439353,11,569,472.644204851752,39,0.223681753726498,33013.1,4 +6050040610,-66.9350100421272,-31.5331764378294,0.122358179761249,0.0329969621469809,0.0893612176142684,0.10240742891561,0.209850913239241,0.0800196023781447,0.00224492211256954,0.00236701086504316,0.00425123013548911,17.9939702427565,348.743931088489,9,295,726.228660924041,95,0.369804095830746,23242.7,18 +6050040670,-69.1021822733092,-37.9918691122619,0.0827309305563989,0.0301024410661957,0.0526284894902032,0.0565180731885253,0.123942011897633,0.0530388908196043,0.00259849898517886,0.00158923070817271,0.012470201905079,13.4737793851718,147.339963833635,43,127,679.962025316456,153,0.886523740831659,9365.7,12 +6050040680,-67.2492321933467,-40.833163190743,0.125779686402275,0.0489402734632137,0.0768394129390613,0.0847623246933269,0.0763140418945806,0.0636778110329957,0.00217716925676053,0.00175041845944867,0.00662354721802807,11.6147581139008,205.778934476424,17,180,682.298224127373,74,1.20340227311439,26545.5,5 +6050040710,-67.609477873725,-24.8407460970081,0.571919479828468,0.294726362794918,0.277193117033551,0.212031248474294,0.393878630200396,0.607864077632024,0.0130405000248106,0.0226000482868116,0.00939152618199451,5.13773416592328,65.7024977698483,33,67,4264.05352363961,257,0.258771135507289,87433.4,14 +6050040750,-62.385558418716,-34.3296277096002,0.154753963340367,0.0326800960582413,0.122073867282126,0.124043899411931,0.213729936417018,0.0839481285283611,0.00263456807223894,0.00296004567921517,0.0063330715440955,16.2507331378299,807.140762463343,38,684,114.149560117302,6,0.241571343885575,18125.8,100 +6050040760,-64.8057015712983,-35.9919393246484,0.135297210121513,0.0270999455639712,0.108197264557542,0.104272547794593,0.208445683071461,0.0635341963929396,0.00315879795459273,0.00313445244065361,0.00618397335202089,15.8079710144928,599.389130434783,4,508,250.035507246377,19,0.453532065232496,24029,79 +6050040780,-69.1781132181979,-42.2656599864851,0.096968042852527,0.035113680244278,0.061854362608249,0.0611623419816733,0.0659099981020249,0.044037655795375,0.00218539861945144,0.0013167861504956,0.00426804776827989,7.85040404040404,198.866666666667,118,180,1132.79494949495,94,0.818434355900029,15709.5,9 +6050040790,-65.6030422980616,-40.1917487788449,0.0872542752886886,0.0333259017552032,0.0539283735334854,0.0515263580443422,0.0979003291612824,0.0396618833555561,0.00227770752828928,0.00152663523879283,0.00746951266151163,15.0834728033473,259.338912133891,2,223,128.679916317992,55,1.35271656774048,7800.9,9 +6050040810,-69.0434027535916,-44.3679767194564,0.0635737928580847,0.0241338644364269,0.0394399284216577,0.0331243466325001,0.137501503060354,0.0424753037909016,0.00258964830260168,0.00147861244507252,0.0116963219327145,9.50945340501792,158.881720430108,42,148,662.610215053763,112,0.956877633836612,34229.6,8 +6050040890,-68.944963158564,-48.0684369800868,0.0586899304402335,0.0251148787593004,0.0335750516809331,0.0286279909297065,0.148197566710483,0.0457608445465836,0.00252551084808219,0.00162963509578136,0.0132269970311109,9.0396865520728,202.481041456016,37,189,452.845803842265,70,1.1278596999158,56959.8,7 +6050040900,-63.1033242847071,-37.4124296849081,0.144750268252116,0.0380630415429347,0.106687226709182,0.112577641822245,0.180116433368624,0.0577093228862042,0.00167721263015926,0.00186502598682251,0.0062274796116073,14.7219814241486,655.030340557276,52,556,188.520743034056,24,0.390105827918265,27544.1,15 +6050040910,-61.1654922056713,-35.1213062032707,0.143049463944694,0.0300714337502336,0.11297803019446,0.113044110640556,0.20974174626317,0.0768875022212908,0.00301193604995864,0.00287440461830284,0.00614492607750985,15.78,971.945882352941,106,823,77.2258823529412,5,0.301875479905864,7489.5,183 +6050040920,-67.9183050247002,-41.7459677462763,0.096968042852527,0.035113680244278,0.061854362608249,0.0611623419816733,0.0659099981020249,0.044037655795375,0.00218539861945144,0.0013167861504956,0.00426804776827989,8.5675,183.146428571429,13,165,1150.34285714286,70,0.94924938753233,4501.3,9 +6050041010,-67.6938040155507,10.1897844207702,0.152552036239645,0.0303935722001586,0.122158464039486,0.108543790633668,0.249627489421233,0.0566472825046516,0.00377336830845589,0.00234037249428871,0.00425272332731712,23.2356643356643,1091.55944055944,273,919,659.867132867133,97,0.186416842717448,3024.2,81 +6050041070,-65.4546328944167,-34.8225397506355,0.15379157845333,0.0328447966163777,0.120946781836952,0.122178615617176,0.187999285244865,0.105709749491842,0.00457977459592267,0.00411459100633812,0.00551540234786772,16.3456166419019,530.925705794948,3,449,366.015601783061,19,0.544596835997782,23585.6,90 +6050041130,-65.8682721035797,-40.968859708354,0.0907802412107016,0.0330392198988753,0.0577410213118263,0.0570620596157919,0.0988834728831354,0.0429919981025153,0.00214725924335886,0.00144914087318048,0.00751531976804257,13.4832618025751,227.068669527897,15,197,355.515021459227,52,1.37116751110907,3799.6,10 +6050041170,-68.9905924506127,-27.1604625760091,0.571385642808122,0.294595888227944,0.276789754580178,0.206150988827739,0.392863099279127,0.602656564127153,0.0127853833489756,0.0225854188815747,0.00931137807821136,2.53521126760563,73.868544600939,2,86,4485.67605633803,234,0.898744667762448,4094.7,14 +6050041200,-64.3912334710431,-36.8752003479128,0.13672254739202,0.0284591379828721,0.108263409409148,0.103942636745078,0.223125881888751,0.0759754023168017,0.00330684232314598,0.00288532090090692,0.00596723059443042,15.1789772727273,567.566761363636,4,482,216.005681818182,35,0.498015259282173,12058.3,109 +6050041310,-67.0606308957112,-41.831530564078,0.0900046202582192,0.03479171497916,0.0552129052790592,0.0562246399562071,0.102911741257989,0.0456048291693362,0.0020482202165945,0.0015633283157438,0.00824128904849841,10.2264840182648,187.808219178082,13,168,866.739726027397,114,1.14625819033889,3491.6,9 +6050042900,-67.0683117006138,-43.5245575262343,0.0907802412107016,0.0330392198988753,0.0577410213118263,0.0570620596157919,0.0988834728831354,0.0429919981025153,0.00214725924335886,0.00144914087318048,0.00751531976804257,12.3009523809524,176.885714285714,9,158,294.204761904762,36,1.49813015116491,3307,10 diff --git a/examples/discovery/data/demo/evolution_freshwater_fish/metadata_0.json b/examples/discovery/data/demo/evolution_freshwater_fish/metadata_0.json new file mode 100644 index 000000000..23d3c28d4 --- /dev/null +++ b/examples/discovery/data/demo/evolution_freshwater_fish/metadata_0.json @@ -0,0 +1,158 @@ +{ + "id": 0, + "domain": "biology", + "workflow_tags": "regression", + "domain_knowledge": "", + "datasets": [ + { + "name": "body-size-evolution-in-south-american-freshwater-fishes.csv", + "description": "This dataset contains the drivers of speciation rates in South American freshwater fishes, employing an integrative approach that considers multiple biotic and abiotic factors.", + "max_depth": 0, + "columns": { + "raw": [ + { + "name": "HYBAS_ID", + "description": "Identifier for hydrological basins as defined by the HydroBASINS framework", + "depth": 0 + }, + { + "name": "long", + "description": "The longitude for specific geographic locations used to map the presence/absence of species.", + "depth": 0 + }, + { + "name": "lat", + "description": "The latitude for specific geographic locations used to map the presence/absence of species.", + "depth": 0 + }, + { + "name": "BAMM_speciation", + "description": "This variable represents the average speciation rates of species within each sub-basin, as estimated by BAMM", + "depth": 0 + }, + { + "name": "BAMM_extinction", + "description": "This variable represents the mean extinction rates for each tip of the phylogenetic tree, as estimated by the BAMM analysis", + "depth": 0 + }, + { + "name": "BAMM_NetDiv", + "description": "This variable stands for the net diversification rates, which are calculated by subtracting the mean extinction rates from the mean \nspeciation rates for each tip in the phylogenetic tree.", + "depth": 0 + }, + { + "name": "DR", + "description": "Diversification Rate, a transformed measure of evolutionary distinctness to understand phylogenetic diversity. The lower the \"ed\" value, the higher the \"DR\" metric, indicating a species with less unique evolutionary history compared to others in the phylogeny.", + "depth": 0 + }, + { + "name": "BEL_evol", + "description": "Rates of Body Elongation evolution", + "depth": 0 + }, + { + "name": "MBL_evol", + "description": "Rates of Maximum Body Length evolution", + "depth": 0 + }, + { + "name": "OGP_evol", + "description": "Rates of Oral Gape Position Evolution", + "depth": 0 + }, + { + "name": "RES_evol", + "description": "Rates of Relative Eye Size Evolution", + "depth": 0 + }, + { + "name": "RML_evol", + "description": "Rate of Relative Maxillary Length evolution", + "depth": 0 + }, + { + "name": "bio1", + "description": "This variable represents the annual mean temperature.", + "depth": 0 + }, + { + "name": "bio12", + "description": "This variable represents the annual mean precipitation.", + "depth": 0 + }, + { + "name": "runoff", + "description": "This variable is used to represent the quantity of water from precipitation that flows over the land's surface and does not get absorbed into the ground. It is being extracted from a geographic database (HydroAtlas).", + "depth": 0 + }, + { + "name": "aet", + "description": "Actual evapotranspiration, which is the sum of evaporation and plant transpiration from the Earth's land and ocean surface to the atmosphere.", + "depth": 0 + }, + { + "name": "Elevation", + "description": "Mean elevation data indicating the height above sea level.", + "depth": 0 + }, + { + "name": "sgr", + "description": "Stream gradient is a measure of the steepness or incline of a stream or river. It can affect water flow and sediment transport, which in turn can influence the habitat conditions for freshwater species. Higher stream gradients usually correspond to faster-moving \nwater and can create different ecological conditions compared to lower gradients.", + "depth": 0 + }, + { + "name": "soil_div", + "description": "It measures the diversity of soil types or conditions within each sub-basin studied. Soil diversity is computed using a dataset that includes eight variables related to substrate types and soil conditions.", + "depth": 0 + }, + { + "name": "area", + "description": "The geographic area of a sub-basin, possibly in square meters, used as one of the predictors in the analysis.", + "depth": 0 + }, + { + "name": "diversity", + "description": "Species diversity, which in ecological studies typically refers to the number of species and their relative abundances in a given area.", + "depth": 0 + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + { + "depth": 0, + "target": "", + "expr": "", + "expr_ind": "", + "text": "The rate of maximum body length evolution emerged as the most influential factor explaining spatial variation in speciation rates. The relationship is positive with linear coefficient 0.82." + } + ], + "intermediate": [] + }, + "workflow": "1. OLS Regression with 10 independent varialbes\n['RML_evol','MBL_evol','OGP_evol','BEL_evol','diversity','runoff','Elevation','sgr','soil_div','area']\n2. And 'BAMM_speciation' as the dependent variable", + "queries": [ + [ + { + "qid": 0, + "true_hypothesis": "The rate of maximum body length evolution emerged as the most influential factor explaining spatial variation in speciation rates. The relationship is positive with linear coefficient 0.82.", + "question_type": "variable", + "question": "Is the maximum body length evolution the most impactful factor in explaining the speciation rates?" + }, + { + "qid": 1, + "true_hypothesis": "The rate of maximum body length evolution emerged as the most influential factor explaining spatial variation in speciation rates. The relationship is positive with linear coefficient 0.82.", + "question_type": "variable", + "question": "What explained the spatial variation in speciation rates, where the rate of maximum body length evolution was identified as the most influential factor with a positive relationship coefficient of 0.82?" + }, + { + "qid": 2, + "true_hypothesis": "The rate of maximum body length evolution emerged as the most influential factor explaining spatial variation in speciation rates. The relationship is positive with linear coefficient 0.82.", + "question_type": "relationhsip", + "question": "What is the linear coefficient that describes the positive relationship between the rate of maximum body length evolution and spatial variation in speciation rates, where the former emerges as the most influential factor?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/evolution_freshwater_fish/metadata_1.json b/examples/discovery/data/demo/evolution_freshwater_fish/metadata_1.json new file mode 100644 index 000000000..205c82329 --- /dev/null +++ b/examples/discovery/data/demo/evolution_freshwater_fish/metadata_1.json @@ -0,0 +1,158 @@ +{ + "id": 0, + "domain": "biology", + "workflow_tags": "regression", + "domain_knowledge": "", + "datasets": [ + { + "name": "body-size-evolution-in-south-american-freshwater-fishes.csv", + "description": "This dataset contains the drivers of speciation rates in South American freshwater fishes, employing an integrative approach that considers multiple biotic and abiotic factors.", + "max_depth": 0, + "columns": { + "raw": [ + { + "name": "HYBAS_ID", + "description": "Identifier for hydrological basins as defined by the HydroBASINS framework", + "depth": 0 + }, + { + "name": "long", + "description": "The longitude for specific geographic locations used to map the presence/absence of species.", + "depth": 0 + }, + { + "name": "lat", + "description": "The latitude for specific geographic locations used to map the presence/absence of species.", + "depth": 0 + }, + { + "name": "BAMM_speciation", + "description": "This variable represents the average speciation rates of species within each sub-basin, as estimated by BAMM", + "depth": 0 + }, + { + "name": "BAMM_extinction", + "description": "This variable represents the mean extinction rates for each tip of the phylogenetic tree, as estimated by the BAMM analysis", + "depth": 0 + }, + { + "name": "BAMM_NetDiv", + "description": "This variable stands for the net diversification rates, which are calculated by subtracting the mean extinction rates from the mean \nspeciation rates for each tip in the phylogenetic tree.", + "depth": 0 + }, + { + "name": "DR", + "description": "Diversification Rate, a transformed measure of evolutionary distinctness to understand phylogenetic diversity. The lower the \"ed\" value, the higher the \"DR\" metric, indicating a species with less unique evolutionary history compared to others in the phylogeny.", + "depth": 0 + }, + { + "name": "BEL_evol", + "description": "Rates of Body Elongation evolution", + "depth": 0 + }, + { + "name": "MBL_evol", + "description": "Rates of Maximum Body Length evolution", + "depth": 0 + }, + { + "name": "OGP_evol", + "description": "Rates of Oral Gape Position Evolution", + "depth": 0 + }, + { + "name": "RES_evol", + "description": "Rates of Relative Eye Size Evolution", + "depth": 0 + }, + { + "name": "RML_evol", + "description": "Rate of Relative Maxillary Length evolution", + "depth": 0 + }, + { + "name": "bio1", + "description": "This variable represents the annual mean temperature.", + "depth": 0 + }, + { + "name": "bio12", + "description": "This variable represents the annual mean precipitation.", + "depth": 0 + }, + { + "name": "runoff", + "description": "This variable is used to represent the quantity of water from precipitation that flows over the land's surface and does not get absorbed into the ground. It is being extracted from a geographic database (HydroAtlas).", + "depth": 0 + }, + { + "name": "aet", + "description": "Actual evapotranspiration, which is the sum of evaporation and plant transpiration from the Earth's land and ocean surface to the atmosphere.", + "depth": 0 + }, + { + "name": "Elevation", + "description": "Mean elevation data indicating the height above sea level.", + "depth": 0 + }, + { + "name": "sgr", + "description": "Stream gradient is a measure of the steepness or incline of a stream or river. It can affect water flow and sediment transport, which in turn can influence the habitat conditions for freshwater species. Higher stream gradients usually correspond to faster-moving \nwater and can create different ecological conditions compared to lower gradients.", + "depth": 0 + }, + { + "name": "soil_div", + "description": "It measures the diversity of soil types or conditions within each sub-basin studied. Soil diversity is computed using a dataset that includes eight variables related to substrate types and soil conditions.", + "depth": 0 + }, + { + "name": "area", + "description": "The geographic area of a sub-basin, possibly in square meters, used as one of the predictors in the analysis.", + "depth": 0 + }, + { + "name": "diversity", + "description": "Species diversity, which in ecological studies typically refers to the number of species and their relative abundances in a given area.", + "depth": 0 + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + { + "depth": 0, + "target": "", + "expr": "", + "expr_ind": "", + "text": "Evolutionary rates of oral gape position and relative maxillary length both exhibited a negative relationship with speciation rates. Their respective coefficient of relation is -4.6 and -4.9." + } + ], + "intermediate": [] + }, + "workflow": "1. OLS Regression with 10 independent varialbes\n['RML_evol','MBL_evol','OGP_evol','BEL_evol','diversity','runoff','Elevation','sgr','soil_div','area']\n2. And 'BAMM_speciation' as the dependent variable", + "queries": [ + [ + { + "qid": 0, + "true_hypothesis": "Evolutionary rates of oral gape position and relative maxillary length both exhibit a negative relationship with speciation rates. Their respective coefficient of relation is -4.6 and -4.9.", + "question_type": "variables", + "question": "Which evolutionary traits exhibit a negative relationship with speciation rates, with respective coefficients of -4.6 and -4.9?" + }, + { + "qid": 1, + "true_hypothesis": "Evolutionary rates of oral gape position and relative maxillary length both exhibit a negative relationship with speciation rates. Their respective coefficient of relation is -4.6 and -4.9.", + "question_type": "variables", + "question": "Which factors exhibit a negative relationship with speciation rates?" + }, + { + "qid": 2, + "true_hypothesis": "Evolutionary rates of oral gape position and relative maxillary length both exhibited a negative relationship with speciation rates. Their respective coefficient of relation is -4.6 and -4.9.", + "question_type": "relationship", + "question": "What relationship did the evolutionary rates of oral gape position and relative maxillary length have with speciation rates, and what were their coefficients?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/evolution_freshwater_fish/metadata_2.json b/examples/discovery/data/demo/evolution_freshwater_fish/metadata_2.json new file mode 100644 index 000000000..4384c3404 --- /dev/null +++ b/examples/discovery/data/demo/evolution_freshwater_fish/metadata_2.json @@ -0,0 +1,152 @@ +{ + "id": 0, + "domain": "biology", + "workflow_tags": "regression", + "domain_knowledge": "", + "datasets": [ + { + "name": "body-size-evolution-in-south-american-freshwater-fishes.csv", + "description": "This dataset contains the drivers of speciation rates in South American freshwater fishes, employing an integrative approach that considers multiple biotic and abiotic factors.", + "max_depth": 0, + "columns": { + "raw": [ + { + "name": "HYBAS_ID", + "description": "Identifier for hydrological basins as defined by the HydroBASINS framework", + "depth": 0 + }, + { + "name": "long", + "description": "The longitude for specific geographic locations used to map the presence/absence of species.", + "depth": 0 + }, + { + "name": "lat", + "description": "The latitude for specific geographic locations used to map the presence/absence of species.", + "depth": 0 + }, + { + "name": "BAMM_speciation", + "description": "This variable represents the average speciation rates of species within each sub-basin, as estimated by BAMM", + "depth": 0 + }, + { + "name": "BAMM_extinction", + "description": "This variable represents the mean extinction rates for each tip of the phylogenetic tree, as estimated by the BAMM analysis", + "depth": 0 + }, + { + "name": "BAMM_NetDiv", + "description": "This variable stands for the net diversification rates, which are calculated by subtracting the mean extinction rates from the mean \nspeciation rates for each tip in the phylogenetic tree.", + "depth": 0 + }, + { + "name": "DR", + "description": "Diversification Rate, a transformed measure of evolutionary distinctness to understand phylogenetic diversity. The lower the \"ed\" value, the higher the \"DR\" metric, indicating a species with less unique evolutionary history compared to others in the phylogeny.", + "depth": 0 + }, + { + "name": "BEL_evol", + "description": "Rates of Body Elongation evolution", + "depth": 0 + }, + { + "name": "MBL_evol", + "description": "Rates of Maximum Body Length evolution", + "depth": 0 + }, + { + "name": "OGP_evol", + "description": "Rates of Oral Gape Position Evolution", + "depth": 0 + }, + { + "name": "RES_evol", + "description": "Rates of Relative Eye Size Evolution", + "depth": 0 + }, + { + "name": "RML_evol", + "description": "Rate of Relative Maxillary Length evolution", + "depth": 0 + }, + { + "name": "bio1", + "description": "This variable represents the annual mean temperature.", + "depth": 0 + }, + { + "name": "bio12", + "description": "This variable represents the annual mean precipitation.", + "depth": 0 + }, + { + "name": "runoff", + "description": "This variable is used to represent the quantity of water from precipitation that flows over the land's surface and does not get absorbed into the ground. It is being extracted from a geographic database (HydroAtlas).", + "depth": 0 + }, + { + "name": "aet", + "description": "Actual evapotranspiration, which is the sum of evaporation and plant transpiration from the Earth's land and ocean surface to the atmosphere.", + "depth": 0 + }, + { + "name": "Elevation", + "description": "Mean elevation data indicating the height above sea level.", + "depth": 0 + }, + { + "name": "sgr", + "description": "Stream gradient is a measure of the steepness or incline of a stream or river. It can affect water flow and sediment transport, which in turn can influence the habitat conditions for freshwater species. Higher stream gradients usually correspond to faster-moving \nwater and can create different ecological conditions compared to lower gradients.", + "depth": 0 + }, + { + "name": "soil_div", + "description": "It measures the diversity of soil types or conditions within each sub-basin studied. Soil diversity is computed using a dataset that includes eight variables related to substrate types and soil conditions.", + "depth": 0 + }, + { + "name": "area", + "description": "The geographic area of a sub-basin, possibly in square meters, used as one of the predictors in the analysis.", + "depth": 0 + }, + { + "name": "diversity", + "description": "Species diversity, which in ecological studies typically refers to the number of species and their relative abundances in a given area.", + "depth": 0 + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + { + "depth": 0, + "target": "", + "expr": "", + "expr_ind": "", + "text": "The rate of body elongation evolution has no significant association with speciation rates." + } + ], + "intermediate": [] + }, + "workflow": "1. OLS Regression with 10 independent varialbes\n['RML_evol','MBL_evol','OGP_evol','BEL_evol','diversity','runoff','Elevation','sgr','soil_div','area']\n2. And 'BAMM_speciation' as the dependent variable", + "queries": [ + [ + { + "qid": 0, + "true_hypothesis": "The rate of body elongation evolution has no significant association with speciation rates.", + "question_type": "variables", + "question": "Which two aspects of evolutionary biology have been found to have no significant association?" + }, + { + "qid": 1, + "true_hypothesis": "The rate of body elongation evolution has no significant association with speciation rates.", + "question_type": "relationship", + "question": "What is the nature of the relationship between the rate of body elongation evolution and speciation rates?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/evolution_freshwater_fish/metadata_3.json b/examples/discovery/data/demo/evolution_freshwater_fish/metadata_3.json new file mode 100644 index 000000000..1ee93c4a0 --- /dev/null +++ b/examples/discovery/data/demo/evolution_freshwater_fish/metadata_3.json @@ -0,0 +1,152 @@ +{ + "id": 0, + "domain": "biology", + "workflow_tags": "regression", + "domain_knowledge": "", + "datasets": [ + { + "name": "body-size-evolution-in-south-american-freshwater-fishes.csv", + "description": "This dataset contains the drivers of speciation rates in South American freshwater fishes, employing an integrative approach that considers multiple biotic and abiotic factors.", + "max_depth": 0, + "columns": { + "raw": [ + { + "name": "HYBAS_ID", + "description": "Identifier for hydrological basins as defined by the HydroBASINS framework", + "depth": 0 + }, + { + "name": "long", + "description": "The longitude for specific geographic locations used to map the presence/absence of species.", + "depth": 0 + }, + { + "name": "lat", + "description": "The latitude for specific geographic locations used to map the presence/absence of species.", + "depth": 0 + }, + { + "name": "BAMM_speciation", + "description": "This variable represents the average speciation rates of species within each sub-basin, as estimated by BAMM", + "depth": 0 + }, + { + "name": "BAMM_extinction", + "description": "This variable represents the mean extinction rates for each tip of the phylogenetic tree, as estimated by the BAMM analysis", + "depth": 0 + }, + { + "name": "BAMM_NetDiv", + "description": "This variable stands for the net diversification rates, which are calculated by subtracting the mean extinction rates from the mean \nspeciation rates for each tip in the phylogenetic tree.", + "depth": 0 + }, + { + "name": "DR", + "description": "Diversification Rate, a transformed measure of evolutionary distinctness to understand phylogenetic diversity. The lower the \"ed\" value, the higher the \"DR\" metric, indicating a species with less unique evolutionary history compared to others in the phylogeny.", + "depth": 0 + }, + { + "name": "BEL_evol", + "description": "Rates of Body Elongation evolution", + "depth": 0 + }, + { + "name": "MBL_evol", + "description": "Rates of Maximum Body Length evolution", + "depth": 0 + }, + { + "name": "OGP_evol", + "description": "Rates of Oral Gape Position Evolution", + "depth": 0 + }, + { + "name": "RES_evol", + "description": "Rates of Relative Eye Size Evolution", + "depth": 0 + }, + { + "name": "RML_evol", + "description": "Rate of Relative Maxillary Length evolution", + "depth": 0 + }, + { + "name": "bio1", + "description": "This variable represents the annual mean temperature.", + "depth": 0 + }, + { + "name": "bio12", + "description": "This variable represents the annual mean precipitation.", + "depth": 0 + }, + { + "name": "runoff", + "description": "This variable is used to represent the quantity of water from precipitation that flows over the land's surface and does not get absorbed into the ground. It is being extracted from a geographic database (HydroAtlas).", + "depth": 0 + }, + { + "name": "aet", + "description": "Actual evapotranspiration, which is the sum of evaporation and plant transpiration from the Earth's land and ocean surface to the atmosphere.", + "depth": 0 + }, + { + "name": "Elevation", + "description": "Mean elevation data indicating the height above sea level.", + "depth": 0 + }, + { + "name": "sgr", + "description": "Stream gradient is a measure of the steepness or incline of a stream or river. It can affect water flow and sediment transport, which in turn can influence the habitat conditions for freshwater species. Higher stream gradients usually correspond to faster-moving \nwater and can create different ecological conditions compared to lower gradients.", + "depth": 0 + }, + { + "name": "soil_div", + "description": "It measures the diversity of soil types or conditions within each sub-basin studied. Soil diversity is computed using a dataset that includes eight variables related to substrate types and soil conditions.", + "depth": 0 + }, + { + "name": "area", + "description": "The geographic area of a sub-basin, possibly in square meters, used as one of the predictors in the analysis.", + "depth": 0 + }, + { + "name": "diversity", + "description": "Species diversity, which in ecological studies typically refers to the number of species and their relative abundances in a given area.", + "depth": 0 + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + { + "depth": 0, + "target": "", + "expr": "", + "expr_ind": "", + "text": "There is a weak but significant, positive relationship between speciation rates and species diversity. The weak relationship is determined by a very small coefficient 0.00003018." + } + ], + "intermediate": [] + }, + "workflow": "1. OLS Regression with 10 independent varialbes\n['RML_evol','MBL_evol','OGP_evol','BEL_evol','diversity','runoff','Elevation','sgr','soil_div','area']\n2. And 'BAMM_speciation' as the dependent variable", + "queries": [ + [ + { + "qid": 0, + "true_hypothesis": "There is a weak but significant, positive relationship between speciation rates and species diversity. The weak relationship is determined by a very small coefficient 0.00003018.", + "question_type": "variables", + "question": "What entities show a weak but significant, positive relationship with a coefficient of 0.00003018?" + }, + { + "qid": 1, + "true_hypothesis": "There is a weak but significant, positive relationship between speciation rates and species diversity. The weak relationship is determined by a very small coefficient 0.00003018.", + "question_type": "relationship", + "question": "What type of relationship exists between speciation rates and species diversity?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/introduction_pathways_non-native_plants/invaded_niche_pathways.csv b/examples/discovery/data/demo/introduction_pathways_non-native_plants/invaded_niche_pathways.csv new file mode 100644 index 000000000..b994d1478 --- /dev/null +++ b/examples/discovery/data/demo/introduction_pathways_non-native_plants/invaded_niche_pathways.csv @@ -0,0 +1,633 @@ +"n.gard" "n.unint" "n.agfo" "n.total" "habitat" "elevation" "cropland.1956.50m" "urban.1956.50m" "cropland.1993.50m" "urban.1993.50m" "cropland.2009.50m" "urban.2009.50m" "cropland.1956.500m" "urban.1956.500m" "cropland.1993.500m" "urban.1993.500m" "cropland.2009.500m" "urban.2009.500m" "cropland.1956.1000m" "urban.1956.1000m" "cropland.1993.1000m" "urban.1993.1000m" "cropland.2009.1000m" "urban.2009.1000m" "progressive.1956.2009.50m" "regressive.1956.2009.50m" "no.changes.1956.2009.50m" "progressive.1993.2009.50m" "regressive.1993.2009.50m" "no.changes.1993.2009.50m" "distance.stream" "distance.road" "longitude" "latitude" "annual.temperature" "annual.rainfall" "annual.radiation" +0 1 0 1 "Croplands" 97 0.68 0 0.81 0 0.67 0.1 0.84 0.01 0.39 0.57 0.25 0.61 0.81 0.11 0.29 0.65 0.17 0.65 0 0.19 0.81 0.04 0.29 0.67 189.844 684.402 427105 4594035 15.333 617 14865.5 +0 0 0 0 "Croplands" 675 0.82 0 0.69 0 0.47 0 0.29 0 0.26 0 0.21 0.01 0.36 0 0.32 0 0.26 0.01 0.35 0.05 0.6 0.23 0.06 0.71 168.539 1678.179 427485 4626135 11.849 794 15017.583 +0 0 0 0 "Croplands" 225 1 0 1 0 1 0 0.86 0.01 0.83 0.02 0.71 0.13 0.67 0.1 0.53 0.14 0.44 0.23 0 0 1 0 0 1 173.959 589.292 429695 4609675 14.067 708 14893.667 +0 0 0 0 "Croplands" 212 1 0 1 0 1 0 0.95 0.02 0.96 0.01 0.77 0.18 0.91 0.02 0.87 0.03 0.7 0.17 0 0 1 0 0 1 543.346 102.109 429975 4607799 14.887 664 14862.167 +0 0 0 0 "Croplands" 83 0.94 0.06 1 0 0.85 0.11 0.92 0.02 0.83 0.06 0.59 0.26 0.92 0.01 0.69 0.13 0.51 0.33 0.04 0.05 0.91 0.04 0.11 0.85 59.362 295.332 430135 4597595 15.464 632 14671.25 +0 2 0 2 "Croplands" 81 0.87 0 0.83 0 0.61 0.2 0.93 0.04 0.64 0.21 0.41 0.49 0.91 0.04 0.67 0.2 0.42 0.46 0.19 0.21 0.6 0.06 0.21 0.73 110.656 88.735 430605 4598355 15.579 610 14692.833 +0 0 0 0 "Croplands" 194 1 0 0.76 0.24 0.73 0.27 0.93 0.04 0.58 0.34 0.48 0.46 0.77 0.12 0.45 0.32 0.32 0.47 0 0.27 0.73 0.01 0.03 0.96 308.511 18.804 431235 4608695 14.506 700 14860.167 +0 0 0 0 "Croplands" 72 1 0 0.99 0.01 0.67 0 0.89 0.05 0.88 0.08 0.4 0.24 0.83 0.12 0.68 0.28 0.32 0.43 0.33 0 0.67 0.34 0 0.66 220.205 783.61 431255 4594855 15.579 610 14692.833 +0 0 0 0 "Croplands" 210 0.69 0 0.55 0 0.3 0 0.68 0.01 0.58 0.11 0.31 0.35 0.5 0.01 0.39 0.22 0.2 0.4 0.41 0.08 0.51 0.3 0.06 0.64 2.609 934.062 434645 4608575 15.105 658 14805.75 +0 0 0 0 "Croplands" 380 0.02 0 0 0 0 0 0.18 0.01 0.13 0 0.06 0.02 0.17 0 0.05 0.06 0.03 0.12 0.04 0 0.96 0 0 1 188.272 1060.639 434578 4616936 14.017 732 14843.917 +0 1 0 1 "Croplands" 160 0.63 0 0.71 0 0.48 0 0.71 0.01 0.62 0.02 0.47 0.05 0.67 0.02 0.48 0.22 0.31 0.31 0.17 0.02 0.81 0.26 0.03 0.71 137.701 544.592 435915 4607475 15.105 658 14805.75 +0 0 0 0 "Croplands" 134 1 0 1 0 1 0 0.9 0.01 0.88 0.01 0.77 0.1 0.76 0.02 0.67 0.1 0.52 0.23 0 0 1 0 0 1 42.887 340.503 437115 4608205 15.105 658 14805.75 +0 1 0 1 "Croplands" 217 0.93 0 0.74 0 0.76 0 0.57 0.02 0.51 0.15 0.42 0.27 0.48 0.02 0.42 0.13 0.31 0.27 0.17 0 0.83 0.03 0.05 0.93 159.096 155.127 452572 4610830 14.165 761 14841 +0 2 0 2 "Croplands" 105 1 0 0.46 0.54 0 1 0.46 0.13 0.27 0.48 0.16 0.58 0.38 0.05 0.17 0.4 0.07 0.48 0 1 0 0 0.46 0.54 20.614 112.167 462225 4618832 14.553 790 14624.083 +0 0 0 0 "Croplands" 268 0 0 0 0 0 0 0.25 0 0.08 0 0.01 0 0.25 0 0.09 0.04 0.03 0.09 0.09 0 0.91 0 0.31 0.69 51.444 2675.923 465789 4610693 13.95 803 14690.833 +0 4 0 4 "Croplands" 88 0.1 0 0 0 0 0 0.21 0 0.21 0.1 0.1 0.08 0.12 0 0.17 0.07 0.09 0.08 0.1 0.01 0.89 0 0.01 0.99 205.358 15.767 465955 4619485 14.543 781 14599.25 +0 0 0 0 "Croplands" 747 0 0 0.99 0 0.77 0 0.06 0 0.29 0 0.23 0 0.18 0 0.25 0 0.2 0.01 0.01 0.99 0 0.22 0.01 0.77 36.075 1916.99 375405 4583845 12.742 676 15273.75 +0 1 0 1 "Croplands" 907 0.6 0 0.81 0 0.46 0 0.33 0 0.36 0 0.2 0.01 0.28 0 0.26 0 0.15 0.01 0.38 0.16 0.45 0.35 0.03 0.62 207.226 1063.364 429335 4661075 11.158 763 14941.167 +0 0 0 0 "Croplands" 319 1 0 1 0 1 0 0.96 0.02 0.93 0.03 0.9 0.04 0.91 0.02 0.9 0.03 0.83 0.06 0 0 1 0 0 1 12.33 106.747 382495 4582795 14.17 632 15049.25 +0 0 0 0 "Croplands" 499 1 0 0.84 0 0.71 0 1 0 0.67 0.01 0.64 0.01 0.82 0.01 0.6 0.01 0.55 0.02 0.29 0 0.71 0.18 0.05 0.76 31.907 301.852 382805 4592255 13.177 658 15220.417 +1 1 0 2 "Croplands" 483 1 0 0.9 0.1 0.9 0.1 0.86 0.01 0.63 0.01 0.59 0.02 0.7 0.01 0.5 0.02 0.45 0.03 0 0.1 0.9 0 0 1 196.728 11.032 382815 4591915 13.177 658 15220.417 +0 0 0 0 "Croplands" 464 1 0 1 0 1 0 0.81 0.02 0.57 0.01 0.48 0.02 0.63 0.01 0.36 0.02 0.3 0.03 0 0 1 0 0 1 1.434 426.302 383115 4594605 13.177 658 15220.417 +0 1 0 1 "Croplands" 289 0.88 0.12 1 0 0.79 0.21 0.9 0.01 0.85 0.02 0.85 0.04 0.83 0.01 0.8 0.01 0.78 0.04 0 0.09 0.91 0 0.21 0.79 385.387 134.817 386245 4583294 14.464 626 15027.25 +1 1 0 2 "Croplands" 35 1 0 0.52 0 0.33 0.41 0.93 0.01 0.58 0.07 0.28 0.32 0.86 0.02 0.38 0.23 0.18 0.48 0.26 0.41 0.33 0.11 0.51 0.39 381.567 0.066 386782 4562825 15.781 580 15130.667 +0 1 0 1 "Croplands" 484 0.47 0 0.9 0 0.83 0 0.44 0.02 0.24 0.15 0.18 0.29 0.34 0.02 0.15 0.16 0.12 0.34 0.02 0.36 0.62 0.07 0 0.93 19.801 133.929 388535 4594585 13.488 654 15153.917 +1 2 0 3 "Croplands" 45 0.76 0.05 0.73 0 0.55 0.06 0.76 0.02 0.32 0.18 0.2 0.29 0.75 0.01 0.4 0.15 0.26 0.27 0.36 0.04 0.6 0.29 0.18 0.53 16.476 105.589 388845 4565315 15.538 600 15094.75 +0 1 0 1 "Croplands" 354 0.66 0 0.68 0 0.85 0 0.64 0 0.66 0 0.63 0 0.57 0 0.58 0 0.56 0.01 0.07 0.26 0.67 0.09 0.32 0.59 185.433 830.599 390285 4593965 14.114 630 15085 +0 1 0 1 "Croplands" 209 1 0 1 0 1 0 0.85 0 0.79 0 0.71 0 0.88 0.01 0.84 0.01 0.78 0.03 0 0 1 0 0 1 10.724 378.379 393545 4585645 14.865 623 14992.083 +0 0 0 0 "Croplands" 42 1 0 0.93 0.07 0.79 0.21 0.91 0.07 0.78 0.18 0.52 0.38 0.93 0.05 0.71 0.15 0.47 0.36 0 0.21 0.79 0.02 0.15 0.83 9.904 194.295 393705 4566485 15.69 604 14994.417 +0 0 0 0 "Croplands" 194 1 0 1 0 1 0 0.88 0.04 0.82 0.02 0.75 0.04 0.91 0.03 0.88 0.02 0.82 0.04 0 0 1 0 0 1 183.782 92.102 394155 4583635 14.865 623 14992.083 +1 4 0 5 "Croplands" 235 1 0 1 0 1 0 0.99 0 1 0 0.99 0.01 0.97 0.01 0.94 0.02 0.93 0.04 0 0 1 0 0 1 146.004 702.815 395645 4581745 14.906 623 15008 +0 0 0 0 "Croplands" 270 1 0 1 0 1 0 0.97 0.02 0.96 0.03 0.93 0.06 0.95 0.03 0.94 0.05 0.91 0.08 0 0 1 0 0 1 462.726 132.121 395775 4580995 14.723 643 15025.5 +1 2 0 3 "Croplands" 213 1 0 1 0 1 0 0.93 0.05 0.88 0.09 0.81 0.15 0.85 0.03 0.79 0.06 0.7 0.13 0 0 1 0 0 1 348.716 58.074 395033 4577506 14.874 635 14951 +0 1 0 1 "Croplands" 175 1 0 0.8 0 0.59 0 0.87 0.03 0.88 0.02 0.7 0.1 0.87 0.02 0.83 0.02 0.66 0.13 0.41 0 0.59 0.33 0.02 0.65 296.159 89.565 395955 4584185 14.906 623 15008 +1 2 0 3 "Croplands" 271 1 0 0.92 0.08 0.58 0.08 0.67 0.04 0.74 0.01 0.5 0.13 0.48 0.02 0.45 0 0.33 0.05 0.34 0.08 0.58 0.34 0 0.66 16.353 541.925 396980 4576710 14.723 643 15025.5 +1 0 0 1 "Croplands" 271 0.81 0.13 0.26 0.27 0.71 0.25 0.84 0.11 0.79 0.13 0.68 0.22 0.79 0.05 0.73 0.06 0.68 0.12 0.03 0.13 0.84 0.02 0.43 0.55 88.106 84.624 398068 4579833 14.723 643 15025.5 +1 1 0 2 "Croplands" 361 1 0 0.51 0 0.32 0 0.94 0.03 0.86 0.01 0.64 0.07 0.78 0.02 0.65 0.03 0.51 0.05 0.68 0 0.32 0.27 0.43 0.31 236.771 100.408 400482 4584090 14.385 646 15149.667 +0 3 0 3 "Croplands" 194 0.95 0 1 0 0.92 0 0.68 0 0.5 0 0.45 0.01 0.73 0.03 0.53 0.1 0.48 0.11 0.08 0.01 0.91 0.08 0 0.92 179.375 976.549 402028 4592341 14.783 621 14994.167 +0 2 0 2 "Croplands" 375 1 0 0.64 0 0.65 0 0.68 0.04 0.57 0.02 0.49 0.05 0.4 0.02 0.29 0.02 0.24 0.03 0.35 0 0.65 0.07 0.2 0.73 54.638 218.606 402555 4583265 14.385 646 15149.667 +2 1 0 3 "Croplands" 188 1 0 0.82 0.15 0.5 0.14 0.75 0.2 0.43 0.48 0.29 0.55 0.7 0.13 0.34 0.32 0.2 0.42 0.35 0.14 0.51 0.33 0 0.67 80.279 573.182 405215 4599665 14.894 613 14986.5 +0 4 0 4 "Croplands" 65 0.67 0 0.71 0 0.66 0 0.61 0.03 0.55 0.18 0.36 0.29 0.54 0.03 0.35 0.22 0.25 0.31 0.06 0.13 0.7 0.11 0 0.83 147.965 18.046 410074 4595413 15.097 612 15037.833 +0 0 0 0 "Croplands" 380 0.94 0 0.3 0.65 0.24 0.76 0.74 0.09 0.58 0.24 0.34 0.51 0.64 0.09 0.42 0.26 0.25 0.4 0 0.76 0.24 0 0.12 0.88 36.558 23.6 409635 4575885 14.303 684 15105 +0 1 0 1 "Croplands" 124 1 0 0.64 0 0.4 0 0.84 0.1 0.5 0.34 0.18 0.49 0.79 0.04 0.28 0.37 0.12 0.47 0.6 0 0.4 0.55 0.31 0.14 173.117 176.992 415380 4592325 15.344 617 14895.333 +0 0 0 0 "Croplands" 383 1 0 1 0 0 0.07 0.75 0.06 0.51 0.3 0.17 0.46 0.77 0.04 0.47 0.34 0.23 0.54 0.93 0.07 0 0.93 0.07 0 2.563 412.122 418655 4604625 13.764 697 15030.333 +0 2 0 2 "Croplands" 200 0.17 0 0.54 0 0.25 0.07 0.73 0.01 0.58 0.04 0.34 0.05 0.68 0.02 0.4 0.2 0.23 0.33 0.15 0.35 0.48 0.17 0.07 0.71 95.071 596.643 418832 4596260 14.951 642 14934.083 +1 2 1 4 "Croplands" 870 0.67 0.33 1 0 0.04 0.25 0.48 0.08 0.39 0.11 0.16 0.16 0.38 0.03 0.32 0.04 0.1 0.06 0.71 0.09 0.21 0.71 0.25 0.04 79.682 143.357 426115 4663525 11.022 771 14940 +1 2 0 3 "Croplands" 542 0.07 0 0.48 0 0.31 0 0.5 0.01 0.57 0 0.4 0.05 0.44 0.02 0.46 0.04 0.34 0.1 0.53 0.21 0.26 0.25 0.12 0.63 164.262 542.507 435895 4655205 12.292 727 14824.583 +0 0 0 0 "Croplands" 617 0.86 0 0.85 0 0.84 0 0.76 0.01 0.81 0 0.71 0.04 0.71 0.01 0.72 0.01 0.63 0.06 0.02 0.05 0.93 0.01 0.06 0.93 204.505 1247.307 403495 4658165 11.714 753 14926.083 +0 1 0 1 "Croplands" 766 0.6 0 0.53 0 0.5 0 0.69 0.02 0.73 0.01 0.68 0.02 0.46 0.02 0.5 0.01 0.44 0.02 0.09 0 0.91 0.03 0 0.97 69.585 151.505 393545 4653645 11.459 760 15138.75 +0 2 0 2 "Croplands" 532 1 0 1 0 1 0 0.5 0 0.44 0.01 0.47 0 0.51 0.01 0.46 0.01 0.45 0.02 0 0 1 0 0 1 183.42 2367.993 389965 4648175 12.379 732 15086.5 +0 3 0 3 "Croplands" 610 0.64 0 0.63 0 1 0 0.25 0 0.22 0.01 0.35 0 0.33 0 0.35 0.01 0.4 0 0 0.36 0.64 0 0.37 0.63 74.746 3308.412 397195 4649895 12.235 727 15049.25 +0 3 0 3 "Croplands" 70 0 0 0 0 0.35 0.05 0.38 0.01 0.04 0.11 0.04 0.15 0.4 0.05 0.17 0.12 0.07 0.14 0 0.57 0.43 0 0.56 0.44 214.772 88.391 470163 4621008 14.491 772 14629.417 +1 4 0 5 "Croplands" 397 0.83 0 0.84 0 0.75 0 0.71 0.01 0.67 0.02 0.66 0.04 0.56 0.01 0.53 0.02 0.49 0.03 0.09 0 0.91 0.1 0.11 0.79 45.166 285.459 378915 4604605 13.394 626 15240.583 +0 1 0 1 "Croplands" 850 0.04 0 0.04 0 0.07 0 0.34 0.02 0.35 0 0.27 0.02 0.25 0.01 0.25 0 0.2 0.01 0 0.04 0.96 0 0.03 0.97 97.005 299.434 431715 4655995 11.338 764 14974.333 +1 3 2 6 "Croplands" 717 0.01 0 0 0 0 0 0.56 0 0.41 0 0.4 0.01 0.57 0.01 0.43 0.01 0.42 0.02 0.01 0 0.99 0.01 0 0.99 279.086 530.397 369688 4612626 12.121 684 15312.5 +0 0 0 0 "Croplands" 581 0.37 0 0.51 0 0.53 0 0.23 0 0.29 0 0.26 0 0.24 0 0.23 0 0.21 0 0.13 0.16 0.71 0 0.03 0.97 263.724 2231.408 374105 4600405 12.672 651 15352.5 +0 0 0 0 "Croplands" 524 0.47 0 0.55 0 0.8 0 0.19 0.01 0.17 0 0.16 0 0.18 0 0.12 0.01 0.11 0.01 0.02 0.35 0.62 0.08 0.29 0.63 87.635 427.504 374125 4605035 12.722 647 15329.667 +0 0 0 0 "Croplands" 692 0.96 0 0.94 0 0.62 0 0.74 0.2 0.67 0.27 0.51 0.32 0.83 0.13 0.67 0.24 0.56 0.32 0.35 0 0.64 0.32 0 0.68 200.477 238.965 376165 4621745 12.254 710 15082 +0 0 0 0 "Croplands" 850 0.72 0 0.84 0 0.35 0 0.52 0 0.4 0 0.3 0.01 0.42 0 0.26 0 0.19 0.01 0.58 0.01 0.41 0.49 0.01 0.5 146.76 1228.838 376255 4589105 12.183 695 15348.667 +0 0 0 0 "Croplands" 619 1 0 1 0 1 0 0.95 0 0.89 0 0.81 0.01 0.76 0.01 0.68 0.04 0.59 0.06 0 0 1 0 0 1 177.32 413.69 376525 4592495 12.606 668 15322.75 +0 0 0 0 "Croplands" 652 0.88 0 0.93 0 0.92 0.01 0.9 0.02 0.88 0.01 0.74 0.06 0.89 0.02 0.9 0.03 0.78 0.08 0.07 0.05 0.88 0.07 0.01 0.92 97.801 2.531 379305 4620925 12.314 704 15180 +0 0 0 0 "Croplands" 475 1 0 0.7 0 0.69 0 0.63 0.02 0.55 0.01 0.5 0.03 0.34 0.01 0.29 0.01 0.26 0.02 0.31 0 0.69 0.04 0.03 0.93 29.893 16.398 379955 4598135 12.976 653 15276 +0 1 0 1 "Croplands" 459 0.57 0.43 0.94 0.06 0.58 0.42 0.71 0.04 0.76 0.01 0.65 0.06 0.54 0.01 0.56 0.01 0.51 0.03 0.1 0.09 0.8 0.01 0.37 0.63 58.488 42.816 379486 4609554 12.816 673 15241.667 +0 0 0 0 "Croplands" 674 0.3 0 0.55 0 0.24 0 0.25 0 0.22 0 0.18 0.01 0.24 0 0.2 0.01 0.17 0.01 0.14 0.19 0.66 0.49 0.01 0.5 231.034 702.041 381635 4611415 12.816 673 15241.667 +0 0 0 0 "Croplands" 376 1 0 1 0 1 0 0.74 0 0.74 0 0.68 0.02 0.76 0.01 0.72 0.01 0.62 0.09 0 0 1 0 0 1 160.493 556.974 384785 4602275 13.797 619 15112.417 +0 0 0 0 "Croplands" 671 0.88 0 0.97 0 0.97 0 0.12 0 0.19 0 0.18 0 0.25 0.01 0.32 0 0.31 0.01 0 0.09 0.91 0 0 1 54.472 730.062 387645 4623205 12.266 709 15251.667 +0 0 0 0 "Croplands" 290 0.76 0 0.22 0 0.18 0 0.58 0.03 0.28 0.09 0.22 0.11 0.58 0.03 0.37 0.07 0.19 0.18 0.64 0 0.36 0.82 0 0.18 48.234 108.813 388845 4602065 14.032 617 15041.75 +0 1 0 1 "Croplands" 251 0.56 0 0.54 0 0.43 0 0.15 0 0.09 0 0.08 0 0.34 0 0.21 0.01 0.18 0.02 0.16 0.02 0.82 0.11 0 0.89 202.141 1410.186 393155 4593015 14.549 613 15066.417 +0 1 0 1 "Croplands" 616 0.81 0 0.78 0 0.81 0 0.33 0.01 0.3 0 0.24 0.02 0.25 0.01 0.14 0 0.12 0.01 0 0 1 0 0.03 0.97 216.185 320.869 393305 4629775 12.967 674 15096.583 +0 2 0 2 "Croplands" 370 0.9 0 0.4 0 0.53 0 0.44 0.01 0.29 0 0.25 0.05 0.36 0.01 0.25 0.01 0.2 0.04 0.47 0.1 0.43 0.03 0.15 0.82 104.68 257.131 395015 4612945 13.358 650 15144 +1 0 0 1 "Croplands" 305 1 0 0.41 0.18 0.21 0.32 0.97 0.01 0.7 0.18 0.47 0.43 0.94 0.04 0.67 0.25 0.53 0.39 0.26 0.32 0.21 0.09 0.5 0.39 536.837 99.334 404455 4619915 14.381 603 14899.083 +0 0 0 0 "Croplands" 494 0.92 0 0 0 0 0 0.32 0 0.22 0 0.3 0 0.24 0.01 0.32 0 0.34 0.01 1 0 0 0.9 0 0.1 73.414 629.788 412644 4637241 13.091 655 15044.917 +0 1 0 1 "Croplands" 849 0.45 0.19 0.9 0.1 0.16 0.09 0.86 0.04 0.85 0.02 0.48 0.08 0.64 0.05 0.61 0.05 0.26 0.07 0.18 0.05 0.45 0.37 0 0.25 163.367 112.05 414845 4677335 9.216 881 15080.5 +0 0 0 0 "Croplands" 593 0.84 0 0.62 0 0.62 0 0.58 0.01 0.55 0.02 0.43 0.14 0.44 0.03 0.4 0.05 0.3 0.12 0.33 0.11 0.55 0.03 0.02 0.95 56.939 93.134 420391 4637346 12.946 663 15034.75 +0 1 0 1 "Croplands" 574 0.92 0 0.61 0 0.67 0.17 0.44 0.01 0.4 0.02 0.3 0.13 0.42 0.02 0.37 0.03 0.28 0.1 0.16 0.18 0.65 0.01 0.4 0.59 172.768 42.578 420475 4637525 12.946 663 15034.75 +0 0 0 0 "Croplands" 676 0.95 0 0.92 0 0.91 0 0.26 0.03 0.25 0.02 0.22 0.03 0.28 0.01 0.24 0.01 0.19 0.01 0.04 0 0.96 0.01 0 0.99 305.655 72.729 421035 4629045 12.148 744 15089.333 +0 0 0 0 "Croplands" 680 0.85 0.15 0.87 0.13 0.7 0.12 0.91 0.04 0.8 0.14 0.51 0.31 0.87 0.08 0.7 0.22 0.53 0.33 0.22 0.02 0.76 0.2 0.04 0.76 1.643 95.121 425995 4629265 11.831 776 15080.917 +0 0 0 0 "Croplands" 675 0.76 0 0.83 0 0.67 0 0.36 0.01 0.39 0.01 0.36 0.02 0.41 0.01 0.42 0 0.39 0.01 0.28 0.05 0.67 0.18 0.16 0.67 264.976 1741.098 426315 4651415 11.798 726 14944.333 +0 0 0 0 "Croplands" 736 1 0 1 0 1 0 0.62 0 0.66 0 0.53 0.03 0.59 0.01 0.58 0.01 0.49 0.04 0 0 1 0 0 1 257.045 707.567 427125 4630255 11.831 776 15080.917 +0 0 0 0 "Croplands" 739 0.36 0 0.26 0 0.19 0 0.44 0.01 0.38 0 0.3 0 0.38 0 0.37 0 0.32 0.01 0.17 0 0.83 0.07 0 0.92 72.473 704.984 429340 4649027 11.559 749 15031.917 +1 1 0 2 "Croplands" 840 0.96 0 1 0 0.82 0 0.63 0.02 0.63 0.02 0.5 0.05 0.63 0.01 0.66 0.01 0.52 0.04 0.14 0 0.86 0.18 0 0.82 208.86 218.997 430290 4649411 11.559 749 15031.917 +0 1 1 2 "Croplands" 644 1 0 0.96 0 0.87 0 0.51 0.01 0.48 0 0.47 0.03 0.25 0.01 0.22 0.01 0.24 0.02 0.13 0 0.87 0.13 0.04 0.84 103.181 679.829 432735 4646825 11.682 753 15012 +0 0 0 0 "Croplands" 502 1 0 1 0 1 0 0.84 0.02 0.86 0.02 0.69 0.11 0.78 0.01 0.84 0.01 0.69 0.1 0 0 1 0 0 1 318.792 426.725 437505 4647935 12.62 722 14837.417 +0 0 1 1 "Croplands" 477 0.84 0.16 0.81 0.19 0.85 0.15 0.57 0.09 0.39 0.27 0.38 0.41 0.68 0.07 0.56 0.19 0.46 0.34 0.02 0.01 0.97 0.04 0 0.96 279.384 304.309 437925 4651455 12.452 724 14847.25 +1 0 0 1 "Croplands" 502 0.95 0.05 1 0 0.61 0.09 0.86 0.01 0.91 0 0.79 0.09 0.85 0.02 0.92 0 0.8 0.07 0.29 0.06 0.65 0.29 0.09 0.61 176.698 1176.709 438735 4639005 12.733 732 14898.417 +1 2 0 3 "Croplands" 501 0.92 0 1 0 1 0 0.86 0.03 0.93 0.03 0.81 0.09 0.81 0.05 0.78 0.11 0.61 0.23 0 0.08 0.92 0 0 1 384.622 577.141 440575 4641225 12.961 723 14852.083 +0 0 0 0 "Croplands" 913 0.79 0 0.92 0 0.7 0 0.17 0 0.29 0 0.15 0.01 0.26 0.01 0.24 0.02 0.14 0.04 0.27 0.02 0.7 0.28 0 0.72 398.976 693.724 452515 4649585 12.24 765 14862.75 +0 1 0 1 "Croplands" 439 1 0 1 0 1 0 0.51 0.01 0.52 0.01 0.51 0.01 0.46 0.01 0.45 0.02 0.44 0.03 0 0 1 0 0 1 236.299 383.855 379855 4602295 13.394 626 15240.583 +0 0 0 0 "Croplands" 816 0.56 0 0.45 0 0.44 0 0.18 0 0.1 0 0.08 0 0.27 0 0.19 0 0.15 0 0.17 0.35 0.48 0.01 0.55 0.44 84.502 2371.242 385265 4626975 11.97 725 15234.5 +0 2 0 2 "Croplands" 1 1 0 1 0 1 0 0.97 0.03 0.97 0.01 0.61 0.22 0.97 0.03 0.91 0.07 0.38 0.3 0 0 1 0 0 1 425.042 820.51 425765 4574045 15.911 632 14821.917 +1 2 0 3 "Croplands" 5 1 0 1 0 0.9 0.1 0.98 0.02 0.95 0.04 0.88 0.1 0.97 0.01 0.92 0.05 0.67 0.3 0 0.1 0.9 0 0.1 0.9 202.332 469.985 419995 4574535 15.877 643 14736.5 +3 0 0 3 "Wetlands" 420 0.22 0 0 0 0 0 0.61 0 0.55 0 0.53 0.02 0.65 0.01 0.61 0.01 0.56 0.02 0.54 0 0.46 0.28 0.01 0.7 59.734 858.348 378120 4602575 13.253 626 15251.5 +2 0 0 2 "Wetlands" 393 0.62 0.11 0.3 0.13 0.32 0.15 0.72 0.01 0.64 0.02 0.63 0.03 0.65 0.01 0.62 0.01 0.59 0.03 0.42 0.21 0.36 0.01 0.43 0.56 85.833 47.028 378485 4604235 13.253 626 15251.5 +0 0 0 0 "Wetlands" 374 0.49 0 0.11 0 0.12 0 0.7 0.02 0.63 0.02 0.59 0.02 0.64 0.01 0.6 0.02 0.57 0.03 0.37 0.06 0.57 0.22 0.13 0.64 138.932 158.627 378855 4604145 13.394 626 15240.583 +0 0 0 0 "Wetlands" 358 0.26 0 0 0 0 0 0.56 0 0.53 0 0.52 0 0.6 0.01 0.58 0 0.54 0.01 0.54 0.07 0.39 0.01 0.23 0.76 76.937 812.142 381155 4603815 13.394 626 15240.583 +0 0 0 0 "Wetlands" 419 0.65 0 0.59 0 0.52 0 0.83 0 0.29 0 0.2 0 0.57 0.01 0.26 0.02 0.19 0.03 0.41 0.01 0.58 0.1 0.08 0.82 0.551 557.88 385260 4592568 13.177 658 15220.417 +0 0 0 0 "Wetlands" 252 0.63 0 0.65 0 0.61 0 0.91 0.02 0.91 0 0.85 0.06 0.84 0.02 0.83 0.01 0.74 0.08 0.37 0.01 0.62 0.04 0.01 0.95 130.231 920.469 385835 4581105 14.793 618 15009.5 +0 0 0 0 "Wetlands" 419 0.77 0 0.7 0 0.64 0 0.79 0 0.28 0 0.19 0 0.56 0.01 0.25 0.02 0.19 0.03 0.34 0 0.66 0.06 0.01 0.92 28.091 591.258 385266 4592535 13.177 658 15220.417 +1 1 1 3 "Wetlands" 4 1 0 1 0 0.47 0 0.77 0.05 0.52 0.13 0.38 0.24 0.79 0.03 0.51 0.21 0.3 0.31 0.53 0 0.47 0.53 0 0.47 300.472 66.086 391015 4562985 15.538 600 15094.75 +0 0 0 0 "Wetlands" 120 0.5 0 0.51 0 0.52 0 0.76 0.01 0.78 0 0.74 0.04 0.87 0.01 0.84 0 0.79 0.02 0.13 0.09 0.78 0.07 0.41 0.52 73.078 883.653 385045 4570145 15.139 610 15125.083 +0 0 0 0 "Wetlands" 149 0.83 0 0.99 0 0.68 0 0.61 0.17 0.28 0.39 0.2 0.5 0.64 0.18 0.32 0.4 0.16 0.53 0.2 0.17 0.62 0.3 0.01 0.68 139.173 446.339 406181 4599623 14.894 613 14986.5 +1 1 0 2 "Wetlands" 146 0.41 0 0.53 0 0.42 0 0.37 0.04 0.24 0.04 0.24 0.04 0.38 0.03 0.14 0.02 0.14 0.03 0.03 0.03 0.94 0.12 0 0.88 26.037 42.678 406385 4602693 14.696 628 15030.917 +2 6 0 8 "Wetlands" 98 0.06 0 0.33 0 0 0 0.57 0.03 0.51 0.16 0.07 0.55 0.63 0.06 0.44 0.1 0.2 0.28 0.06 0.19 0.63 0.46 0 0.42 54.948 131.64 406435 4601275 14.696 628 15030.917 +2 0 0 2 "Wetlands" 103 0 0 0 0.06 0 0.06 0.39 0.04 0.23 0.04 0.22 0.04 0.41 0.03 0.14 0.02 0.16 0.04 0.27 0.07 0.66 0 0.02 0.98 34.85 109.539 406517 4602647 14.696 628 15030.917 +3 0 0 3 "Wetlands" 109 0.77 0 0.6 0 0.24 0.02 0.69 0.03 0.44 0.11 0.27 0.27 0.59 0.02 0.41 0.09 0.26 0.17 0.53 0.03 0.35 0.48 0.05 0.37 17.774 134.612 407025 4589845 14.422 646 15134.417 +1 3 0 4 "Wetlands" 81 0.52 0 0.52 0 0.56 0 0.61 0.01 0.51 0.09 0.38 0.21 0.7 0.03 0.38 0.31 0.28 0.43 0.05 0.09 0.86 0.03 0.07 0.89 16.413 398.658 409462 4595195 15.271 609 14864.75 +2 0 0 2 "Wetlands" 70 0.65 0 0 0.16 0 0.01 0.51 0.12 0.02 0.72 0 0.83 0.5 0.04 0.1 0.42 0.04 0.49 0.92 0.01 0.07 0.42 0 0.58 117.699 179.798 416365 4581655 15.154 636 14905.167 +1 3 0 4 "Wetlands" 4 0 0.5 0 0 0 1 0.8 0.04 0.71 0.06 0.59 0.38 0.93 0.01 0.83 0.07 0.64 0.29 0 0.5 0.5 0 1 0 449.653 270.619 417086 4571138 15.686 626 14961.167 +0 0 0 0 "Wetlands" 479 0 0 0 0 0 0 0.35 0.02 0.15 0 0.16 0.02 0.43 0.02 0.19 0.09 0.09 0.13 0 0 1 0 0 1 377.958 181.21 430260 4614232 13.375 736 14929.833 +1 0 0 1 "Wetlands" 345 0.42 0 0.2 0 0 0 0.71 0 0.14 0.03 0.15 0.04 0.61 0.01 0.19 0.05 0.16 0.07 0.42 0.01 0.56 0.2 0.02 0.78 123.603 453.365 432315 4614695 14.071 716 14862.167 +1 0 0 1 "Wetlands" 310 0.4 0 0.05 0 0.09 0 0.71 0 0.29 0.12 0.23 0.17 0.57 0.01 0.15 0.18 0.11 0.27 0.3 0.2 0.49 0 0.4 0.6 68.305 1199.729 432555 4613985 14.071 716 14862.167 +1 1 0 2 "Wetlands" 310 0.2 0 0.09 0 0.05 0 0.69 0 0.26 0.14 0.21 0.19 0.55 0.01 0.15 0.18 0.1 0.26 0.15 0.23 0.63 0.06 0.29 0.65 75.179 1206.971 432595 4613995 14.071 716 14862.167 +0 0 0 0 "Wetlands" 35 0.93 0.07 0 0 0 0.38 0.74 0.17 0.27 0.42 0.06 0.57 0.71 0.2 0.28 0.39 0.1 0.53 0.62 0.31 0.06 0.03 0.42 0.55 220.923 69.437 471385 4607585 15.074 730 14685.583 +0 1 0 1 "Riparian" 132 0 0.07 0 0.26 0 0.42 0.09 0.05 0 0.08 0.01 0.09 0.07 0.02 0 0.03 0.01 0.05 0.02 0.38 0.46 0 0.2 0.68 82.275 167.903 404390 4605143 14.248 622 15029.25 +0 0 0 0 "Wetlands" 303 0.35 0 0.25 0 0.25 0 0.58 0.01 0.39 0.04 0.38 0.03 0.6 0.01 0.34 0.08 0.3 0.1 0.1 0 0.9 0 0 0.99 61.686 699.884 392443 4592517 14.549 613 15066.417 +0 0 0 0 "Wetlands" 270 0.81 0 0.18 0 0.41 0 0.48 0.01 0.36 0.01 0.33 0.02 0.54 0.01 0.34 0.06 0.3 0.07 0.51 0.01 0.49 0.37 0.28 0.35 124.156 681.335 392455 4592725 14.549 613 15066.417 +0 0 0 0 "Wetlands" 157 0.52 0 0.42 0 0.4 0.01 0.46 0.16 0.23 0.29 0.11 0.38 0.46 0.08 0.19 0.18 0.1 0.23 0.17 0.01 0.79 0.18 0.05 0.72 180.956 24.857 405495 4609255 14.407 602 14967.667 +0 0 0 0 "Wetlands" 161 0.14 0.14 0.26 0 0.07 0.15 0.49 0.12 0.26 0.27 0.11 0.33 0.48 0.08 0.22 0.18 0.12 0.24 0.11 0.16 0.66 0.16 0.15 0.6 137.816 18.919 405285 4609265 14.407 602 14967.667 +1 3 0 4 "Wetlands" 165 0 0.16 0 0.1 0 0.1 0.39 0.32 0.12 0.63 0.08 0.69 0.51 0.17 0.22 0.45 0.13 0.54 0.09 0.19 0.55 0.31 0.01 0.62 2.052 4.27 405055 4613365 14.407 602 14967.667 +0 0 0 0 "Wetlands" 208 0.45 0 0.45 0 0.02 0 0.39 0.02 0.17 0.03 0.15 0.02 0.29 0.01 0.13 0.03 0.09 0.03 0.43 0.04 0.53 0.44 0.02 0.54 18.663 6.813 403465 4611295 14.407 602 14967.667 +1 0 0 1 "Wetlands" 405 0.68 0 0.73 0 0.73 0 0.76 0.01 0.71 0 0.67 0.02 0.69 0.03 0.65 0.04 0.58 0.08 0.1 0.07 0.83 0.04 0.04 0.91 34.985 763.118 377866 4602506 13.253 626 15251.5 +0 1 0 1 "Wetlands" 1 0 0.05 0 0.03 0 0.72 0 0.15 0 0.2 0 0.58 0.08 0.16 0 0.3 0 0.5 0 0.68 0.22 0 0.8 0.03 345.321 688.069 422195 4571085 15.752 633 14934.583 +0 1 0 1 "Wetlands" 3 1 0 1 0 0 0 0.99 0.01 0.98 0 0.23 0.08 0.9 0.02 0.92 0.02 0.31 0.22 1 0 0 1 0 0 276.99 1570.331 426415 4573395 15.911 632 14821.917 +0 0 0 0 "Wetlands" 0 0 0 0.96 0 0 0 0.37 0.01 0.68 0 0 0.03 0.69 0.02 0.79 0.01 0.12 0.23 0 0 0 0.96 0 0.04 317.105 2440.289 427045 4572685 15.955 638 14662.167 +1 0 0 1 "Wetlands" 4 0.42 0 0 0 0.26 0.13 0.78 0.01 0.13 0.58 0.06 0.69 0.87 0.02 0.28 0.56 0.07 0.73 0.33 0.62 0.01 0.6 0.39 0.01 728.343 526.727 426605 4575165 15.911 632 14821.917 +0 0 0 0 "Wetlands" 4 0.45 0 0 0 0.26 0.2 0.78 0.01 0.11 0.61 0.05 0.7 0.87 0.02 0.26 0.56 0.05 0.76 0.33 0.57 0.08 0.49 0.46 0.05 807.428 606.3 426695 4575145 15.911 632 14821.917 +3 1 0 4 "Wetlands" 4 0.14 0 0.12 0 0 0.57 0.73 0.01 0.29 0.28 0 0.86 0.8 0.02 0.29 0.45 0 0.94 0.01 0.68 0.25 0.26 0.64 0.1 1529.076 2309.498 428415 4574825 15.955 638 14662.167 +1 0 0 1 "Wetlands" 5 1 0 1 0 0.11 0.22 0.74 0.03 0.45 0.25 0.16 0.29 0.55 0.02 0.38 0.2 0.24 0.22 0.57 0.22 0.11 0.57 0.22 0.11 419.614 261.218 420809 4570160 15.752 633 14934.583 +1 1 0 2 "Wetlands" 3 0.56 0.42 0.62 0.38 0 0.97 0.7 0.09 0.6 0.27 0.09 0.82 0.77 0.07 0.55 0.23 0.22 0.71 0.03 0.54 0.42 0.03 0.59 0.38 776.804 20.042 421535 4573295 15.877 643 14736.5 +0 0 0 0 "Wetlands" 3 0 0 0 0.32 0 0.41 0 0 0 0.06 0 0.05 0.23 0.01 0.25 0.14 0.14 0.12 0.02 0.41 0.42 0 0.09 0.81 1885.721 3171.633 425835 4571685 15.768 631 14901.333 +1 0 0 1 "Coastal" 1 0 0 0 0.32 0 0.41 0.54 0.21 0.08 0.81 0.01 0.79 0.67 0.2 0.39 0.55 0.25 0.65 0.02 0.41 0.42 0 0.09 0.81 116.191 194.814 450339 4595289 15.259 655 14755 +0 1 0 1 "Coastal" 8 0.07 0.12 0 0.45 0.14 0.2 0.69 0.1 0.64 0.24 0.41 0.4 0.8 0.08 0.69 0.26 0.47 0.43 0.01 0.3 0.26 0.34 0 0.22 90.99 142.122 450725 4595715 15.259 655 14755 +1 1 1 3 "Coastal" 10.074 0 0.25 0 0.57 0 0.73 0.75 0.11 0.72 0.14 0.46 0.31 0.82 0.09 0.78 0.16 0.54 0.32 0.03 0.26 0.41 0.04 0.13 0.7 444.457 23.113 455475 4599535 15.05 707 14754.5 +1 1 0 2 "Coastal" 18 0.07 0.26 0.09 0.62 0.06 0.78 0.75 0.11 0.71 0.16 0.46 0.31 0.83 0.07 0.8 0.13 0.55 0.29 0 0.28 0.41 0.01 0.04 0.79 373.699 24.746 455535 4599575 15.05 707 14754.5 +2 4 0 6 "Coastal" 9 0 0 0 0.48 0 0.52 0.06 0.12 0 0.48 0 0.58 0.07 0.06 0 0.35 0 0.45 0.05 0.52 0.43 0.05 0 0.91 305.434 26.204 410805 4568715 14.879 657 15046.167 +2 0 0 2 "Coastal" 44 0 0.12 0 0.24 0 0.37 0.08 0.05 0 0.24 0 0.23 0.05 0.03 0 0.35 0 0.39 0.02 0.37 0.48 0.22 0.11 0.41 172.908 14.037 409425 4568165 14.879 657 15046.167 +1 3 0 4 "Coastal" 3 0 0 0 0 0 0 0.07 0.42 0 0.69 0 0.7 0.22 0.36 0 0.83 0 0.85 0 0 0.39 0 0 1 141.374 464.461 414605 4568795 15.506 628 14965.25 +1 2 0 3 "Coastal" 10 0.62 0 0 0 0 0 0.65 0.03 0.21 0.33 0.09 0.41 0.62 0.04 0.35 0.28 0.26 0.4 0.62 0.02 0.36 0.08 0.29 0.64 328.329 363.447 416765 4569565 15.686 626 14961.167 +2 1 0 3 "Coastal" 10 1 0 0 0 0 0 0.57 0.04 0.16 0.33 0.06 0.44 0.59 0.04 0.32 0.29 0.25 0.4 1 0 0 0.01 0.17 0.82 265.474 304.867 416835 4569505 15.686 626 14961.167 +2 0 0 2 "Coastal" 0.103 0 0 0 0 0 0 0.56 0.03 0.62 0.05 0.61 0.06 0.76 0.04 0.73 0.14 0.49 0.31 0 0 0 0.01 0.17 0.82 492.763 805.964 475800 4608513 15.102 711 14698.583 +0 2 0 2 "Coastal" 0.05 0 0 0 0 0 0 0.58 0 0.37 0.4 0.04 0.6 0.79 0 0.67 0.21 0.45 0.32 0 0 0 0.01 0.17 0.82 518.548 1712.556 480617 4610595 15.313 690 14624.167 +0 0 0 0 "Coastal" 20 0 0 0 0 0 0 0.65 0.03 0 0.04 0 0.03 0.69 0.02 0.03 0.06 0.01 0.09 0.53 0 0.43 0.63 0 0.33 116.378 1872.755 396545 4564125 15.65 600 15103.333 +0 0 0 0 "Coastal" 557 0.03 0 0.03 0 0.03 0 0.24 0 0.21 0 0.19 0 0.32 0 0.34 0 0.32 0 0.07 0.08 0.85 0.1 0.07 0.83 182.481 1329.701 374495 4626805 12.633 671 15142.667 +0 1 0 1 "Coastal" 551 0 0 0.24 0 0.2 0 0.39 0.01 0.41 0.01 0.41 0.01 0.44 0.02 0.41 0.01 0.41 0.02 0.02 0.57 0.42 0.1 0.17 0.72 143.553 338.202 372165 4625545 12.633 671 15142.667 +0 0 0 0 "Coastal" 559 0.39 0 0.36 0 0.4 0 0.22 0 0.13 0.05 0.13 0.07 0.34 0 0.27 0.01 0.26 0.02 0.08 0.4 0.52 0.06 0.23 0.72 96.359 1011.662 369045 4626055 12.686 659 15156 +0 0 0 0 "Coastal" 557 0 0 0 0 0 0 0.07 0 0.09 0 0.08 0 0.27 0 0.25 0 0.25 0.01 0.94 0 0.06 0.9 0.03 0.06 160.309 806.555 368645 4626865 12.686 659 15156 +0 0 0 0 "Coastal" 364 0 0 0 0 0 0 0.09 0 0.09 0 0.08 0 0.21 0.01 0.12 0.08 0.09 0.09 0.12 0 0.88 0 0 1 0.009 553.32 398285 4632325 13.318 633 15111 +0 0 0 0 "Coastal" 497 0 0 0 0 0 0 0 0 0 0 0 0 0.04 0 0.05 0 0.04 0 0 0.78 0.22 0 0.72 0.28 49.037 1210.073 399445 4632635 13.318 633 15111 +0 0 0 0 "Coastal" 611 0 0 0 0 0 0 0.18 0 0.15 0 0.14 0 0.21 0 0.23 0 0.22 0 0 0.67 0.33 0.14 0.11 0.75 213.286 1024.131 374665 4627205 12.633 671 15142.667 +0 0 0 0 "Coastal" 576 0.29 0 0.03 0 0 0 0.23 0 0.17 0 0.16 0 0.31 0 0.31 0 0.3 0.01 0.29 0.69 0.02 0.51 0.02 0.47 316.677 1387.908 374315 4627105 12.633 671 15142.667 +0 0 0 0 "Coastal" 565 0.63 0 0.46 0 0.47 0 0.73 0 0.74 0 0.74 0.01 0.54 0 0.56 0.01 0.55 0.02 0.31 0 0.69 0.01 0.11 0.88 276.459 985.856 372845 4625555 12.633 671 15142.667 +0 0 0 0 "Coastal" 519 0 0 0 0 0 0 0.24 0.09 0.15 0.07 0.1 0.11 0.29 0.04 0.17 0.03 0.15 0.04 0 0.45 0.55 0.47 0.04 0.49 159.047 284.675 370425 4626055 12.686 659 15156 +0 0 0 0 "Coastal" 7 0 0 0 0.13 0 0 0.12 0.01 0 0.23 0 0.03 0.21 0.01 0.12 0.14 0.05 0.03 0.16 0.49 0.33 0.26 0.33 0.34 88.931 1144.505 421695 4570005 15.752 633 14934.583 +1 0 0 1 "Coastal" 4 0 0 0 0 0 0 0.09 0 0.06 0.01 0.01 0.02 0.36 0.01 0.41 0.07 0.26 0.08 0 0 0.82 0 0 0.8 1615.178 2904.975 425955 4571965 15.911 632 14821.917 +0 0 0 0 "Coastal" 0 0 0 0 0 0 0 0.21 0.01 0.2 0.01 0.12 0.01 0.31 0.01 0.39 0.02 0.21 0.04 0 0.01 0.55 0 0 0.93 1390.07 3003 426285 4571925 15.911 632 14821.917 +0 0 0 0 "Coastal" 1 1 0 0.35 0 0 0 0.39 0.02 0.4 0.01 0.28 0.03 0.45 0.01 0.54 0.01 0.29 0.04 1 0 0 0.35 0 0.09 1246.387 2685.223 426215 4572235 15.911 632 14821.917 +4 1 0 5 "Coastal" 7 0.45 0 0 0 0 0 0.5 0.01 0.25 0.26 0.07 0.22 0.5 0.02 0.31 0.23 0.16 0.26 0.45 0.3 0.26 0 0.32 0.68 553.38 496.659 420915 4569855 15.752 633 14934.583 +4 1 0 5 "Coastal" 7 0.92 0 0 0.25 0 0 0.57 0.01 0.32 0.25 0.11 0.22 0.48 0.02 0.31 0.22 0.17 0.24 0.92 0 0.08 0.25 0 0.75 467.65 458.31 420930 4569959 15.752 633 14934.583 +1 1 0 2 "Coastal" 3 0 0 0 0.38 0 0.47 0.25 0.05 0.07 0.44 0.04 0.69 0.55 0.03 0.32 0.3 0.23 0.41 0 0.48 0.52 0 0.08 0.86 88.907 219.32 420125 4569455 15.752 633 14934.583 +2 1 1 4 "Urban" 85 0 0 0 0.47 0 1 0.26 0 0.06 0.26 0.05 0.43 0.35 0.01 0.13 0.19 0.11 0.26 0 1 0 0 0.53 0.47 71.236 550.079 387487 4565714 15.396 597 15185.5 +0 1 0 1 "Urban" 486 0.91 0.09 0.05 0.9 0.04 0.95 0.78 0.04 0.31 0.18 0.22 0.35 0.45 0.02 0.18 0.12 0.13 0.24 0 0.87 0.13 0 0.06 0.94 82.321 18.906 387795 4594175 13.488 654 15153.917 +0 0 0 0 "Urban" 300 1 0 0.47 0.32 0 1 0.76 0.04 0.59 0.07 0.15 0.48 0.63 0.02 0.45 0.06 0.2 0.25 0 1 0 0 0.47 0.32 178.491 399.427 389065 4602645 13.881 626 15160.25 +1 1 0 2 "Urban" 65 0.75 0 1 0 0.67 0 0.93 0 0.66 0.02 0.55 0.11 0.94 0.01 0.53 0.24 0.32 0.42 0.02 0.25 0.64 0.16 0 0.67 328.624 330.725 389798 4564684 15.538 600 15094.75 +0 2 0 2 "Urban" 13 0.91 0 0 0.55 0 0.92 0.77 0.19 0.12 0.59 0.01 0.85 0.66 0.26 0.18 0.63 0.02 0.8 0.08 0.92 0 0.08 0.37 0.55 54.077 399.758 394325 4564595 15.69 604 14994.417 +0 3 0 3 "Urban" 270 0.45 0.44 0.2 0.72 0.06 0.92 0.76 0.1 0.68 0.15 0.5 0.27 0.75 0.04 0.67 0.05 0.55 0.1 0.08 0.55 0.37 0 0.2 0.8 177.321 89.828 399475 4582345 14.385 646 15149.667 +1 3 0 4 "Urban" 36 0.95 0.05 1 0 0.6 0.05 0.87 0.07 0.47 0.32 0.12 0.66 0.68 0.17 0.26 0.46 0.08 0.63 0.36 0 0.64 0.35 0.05 0.6 153.879 549.952 399707 4566651 15.543 603 15116.75 +0 4 0 4 "Urban" 43 1 0 0 1 0 1 0.54 0.24 0.07 0.77 0.03 0.77 0.48 0.22 0.09 0.62 0.01 0.75 0 1 0 0 0 1 315.868 8.766 431845 4593565 15.523 608 14842.083 +0 3 0 3 "Urban" 40 0 0.88 0 1 0 1 0.17 0.35 0 0.66 0 0.62 0.22 0.27 0.03 0.51 0 0.48 0 0.12 0.88 0 0 1 262.295 223.478 432187 4592551 15.523 608 14842.083 +0 1 0 1 "Urban" 40 0 1 0 1 0 1 0.16 0.39 0.01 0.68 0 0.63 0.23 0.26 0.03 0.51 0 0.49 0 0 1 0 0 1 233.206 180.906 432233 4592699 15.523 608 14842.083 +1 2 0 3 "Urban" 365 0.51 0 0.1 0 0.11 0 0.56 0 0.34 0 0.25 0.06 0.36 0 0.2 0 0.14 0.02 0.4 0.01 0.59 0.02 0.03 0.95 17.618 2325.103 444763 4598052 14.84 677 14855.583 +0 0 0 0 "Urban" 11 0.31 0.69 0 1 0 1 0.5 0.39 0.07 0.9 0.03 0.9 0.68 0.25 0.32 0.64 0.15 0.78 0 0.31 0.69 0 0 1 18.726 144.969 446135 4593355 15.543 606 14735.667 +0 2 0 2 "Urban" 126 1 0 0.6 0 0 0.92 0.93 0.03 0.46 0.39 0.14 0.68 0.72 0.09 0.31 0.53 0.13 0.72 0.08 0.92 0 0.01 0.92 0.07 83.925 99.018 447609 4596719 14.84 677 14855.583 +1 1 0 2 "Urban" 367 0.46 0 0 0.63 0 0.64 0.51 0.03 0.16 0.38 0.03 0.71 0.41 0.02 0.12 0.3 0.04 0.53 0 1 0 0.08 0.37 0.54 1.559 204.284 400673 4602175 14.004 653 15044.833 +1 1 0 2 "Urban" 369 0.66 0 0.49 0.49 0.44 0.34 0.33 0.01 0.13 0.3 0.06 0.44 0.29 0.02 0.16 0.14 0.07 0.27 0.15 0.34 0.5 0.22 0.01 0.78 43.703 349.442 408295 4577125 13.939 687 15134.833 +0 2 0 2 "Urban" 73 0 0 0 0.11 0.07 0.01 0.63 0.06 0.51 0.25 0.23 0.45 0.67 0.08 0.45 0.23 0.21 0.4 0.24 0.14 0.36 0.52 0.09 0.23 38.667 315.199 408473 4598916 15.271 609 14864.75 +0 2 0 2 "Urban" 32 1 0 0 0.39 0 1 0.63 0.07 0.02 0.41 0 0.84 0.44 0.17 0.06 0.52 0 0.72 0 1 0 0 0.34 0.39 306.241 1162.459 414121 4570981 15.506 628 14965.25 +3 1 0 4 "Urban" 155 0.46 0 0.26 0.13 0 0.8 0.48 0.01 0.07 0.2 0.04 0.28 0.5 0.01 0.06 0.36 0.02 0.42 0.1 0.8 0.1 0 0.67 0.33 80.503 1024.276 415765 4582855 15.154 636 14905.167 +0 3 0 3 "Urban" 8 1 0 0 0.41 0 1 0.98 0 0.54 0.12 0.35 0.57 0.93 0.02 0.55 0.15 0.39 0.51 0 1 0 0 0.59 0.41 517.955 28.304 416015 4571035 15.506 628 14965.25 +0 0 0 0 "Urban" 329 0 1 0 1 0 1 0.14 0.79 0 0.99 0 1 0.31 0.58 0.03 0.88 0 0.99 0 0 1 0 0 1 105.906 241.051 417845 4603015 14.688 673 14730.25 +0 0 0 0 "Urban" 286 0 1 0 1 0 1 0.37 0.44 0.05 0.95 0 1 0.44 0.42 0.05 0.83 0 0.98 0 0 1 0 0 1 553.395 643.801 418845 4601265 14.688 673 14730.25 +0 1 0 1 "Urban" 25 0.79 0.21 0 1 0 1 0.56 0.37 0.4 0.59 0.21 0.76 0.71 0.23 0.43 0.51 0.21 0.75 0 0.79 0.21 0 0 1 191.813 684.115 420103 4581374 15.263 645 14934.75 +0 0 0 0 "Urban" 14 1 0 0 0.25 0 1 0.82 0.06 0.58 0.22 0.34 0.44 0.78 0.14 0.54 0.3 0.32 0.46 0 1 0 0 0.75 0.25 238.782 6.783 421335 4578495 15.855 634 14772.833 +0 1 0 1 "Urban" 12 1 0 0.99 0.01 0.87 0.01 0.76 0.14 0.54 0.32 0.33 0.5 0.74 0.18 0.48 0.35 0.28 0.53 0.12 0.01 0.87 0.12 0 0.88 31.694 75.878 421541 4578521 15.855 634 14772.833 +0 1 0 1 "Urban" 177 1 0 0 1 0 1 0.71 0.02 0.19 0.48 0.01 0.88 0.62 0.07 0.27 0.34 0.13 0.64 0 1 0 0 0 1 87.905 655.974 423055 4598065 15.042 649 14906.417 +0 3 0 3 "Urban" 9 0.12 0.88 0 0.5 0 1 0.33 0.65 0 0.81 0 1 0.52 0.46 0 0.87 0 1 0 0.12 0.88 0 0.5 0.5 504.76 0.891 427220 4579208 15.951 634 14684.417 +0 1 0 1 "Urban" 9 1 0 0 1 0 1 0.95 0.04 0 0.91 0 1 0.96 0.03 0 0.95 0 1 0 1 0 0 0 1 588.386 1331.953 427721 4576071 15.955 638 14662.167 +0 1 0 1 "Urban" 85 0 0.95 0 1 0 1 0.03 0.8 0 1 0 1 0.04 0.87 0 1 0 1 0 0.05 0.95 0 0 1 71.244 1763.919 427699 4583108 15.728 644 14640.333 +1 0 0 1 "Urban" 203 0.75 0.06 0 1 0 1 0.36 0.25 0 0.71 0.01 0.65 0.23 0.2 0 0.56 0 0.53 0 0.94 0.06 0 0 1 148.749 444.94 427970 4586744 15.133 654 14940.333 +1 1 0 2 "Urban" 22 0.74 0.26 0 1 0 1 0.41 0.5 0 1 0.01 0.98 0.38 0.47 0 0.95 0 0.96 0 0.74 0.26 0 0 1 118.737 792.427 428240 4579156 15.951 634 14684.417 +4 0 0 4 "Urban" 53 0 0.35 0 1 0 1 0.19 0.38 0.02 0.7 0.03 0.63 0.29 0.23 0.03 0.57 0.03 0.52 0 0.65 0.35 0 0 1 423.04 260.54 431711 4591336 15.523 608 14842.083 +2 1 0 3 "Urban" 50 0 0 0 0.27 0 0.05 0.14 0.27 0 0.59 0 0.53 0.26 0.21 0.04 0.5 0.03 0.48 0.27 0.11 0.62 0.23 0.1 0.67 57.365 218.08 432078 4591234 15.523 608 14842.083 +0 1 0 1 "Urban" 14 0 1 0 1 0 1 0 0.71 0 0.78 0 0.88 0 0.87 0 0.9 0 0.93 0 0 0.99 0 0 1 855.025 1698.583 432188 4580660 15.942 605 14949.417 +0 7 0 7 "Urban" 179 1 0 1 0 0.57 0 0.37 0 0.31 0.19 0.19 0.33 0.55 0.01 0.4 0.2 0.27 0.33 0.43 0 0.57 0.43 0 0.57 267.222 816.95 433005 4604885 15.054 661 14805.917 +1 3 0 4 "Urban" 10 0 1 0 1 0 1 0 1 0 1 0 1 0.01 0.96 0 0.98 0 0.98 0 0 1 0 0 1 80.519 2632.017 433129 4583489 15.928 622 14660 +3 3 0 6 "Urban" 21 0.04 0.96 0 1 0 1 0.45 0.36 0 0.85 0 0.92 0.48 0.38 0 0.92 0 0.96 0 0.04 0.96 0 0 1 136.232 241.443 433793 4587983 15.747 621 14674.417 +0 1 0 1 "Urban" 7 0.01 0.13 0 0.03 0 0.32 0.37 0.41 0 0.83 0 0.85 0.45 0.41 0 0.9 0 0.92 0.21 0.27 0.17 0.25 0.29 0.25 173.944 941.784 435608 4586111 15.812 606 14673.583 +1 2 0 3 "Urban" 237 0 0 0 0 0 0 0.09 0.02 0 0.14 0 0.19 0.14 0.04 0.06 0.13 0.04 0.2 0.03 0 0.97 0 0 1 23.327 436.223 436615 4594571 15.154 643 14953.833 +4 5 1 10 "Urban" 148 0.47 0.49 0 1 0 1 0.62 0.09 0.42 0.28 0.23 0.49 0.76 0.05 0.62 0.17 0.42 0.32 0 0.51 0.49 0 0 1 218.728 176.462 436675 4607235 15.105 658 14805.75 +0 0 0 0 "Urban" 15 0 1 0 1 0 1 0.02 0.97 0 1 0 1 0.21 0.75 0 0.95 0 0.97 0 0 1 0 0 1 12.97 452.266 436775 4588825 15.812 606 14673.583 +1 0 0 1 "Urban" 166 0.58 0.19 0.75 0.02 0.57 0.16 0.5 0.01 0.1 0.34 0.04 0.36 0.58 0.02 0.18 0.29 0.11 0.32 0.11 0.03 0.87 0.07 0.17 0.76 140.58 292.168 440068 4598528 14.763 668 15012.167 +0 2 0 2 "Urban" 157 1 0 0.77 0.23 0 1 0.7 0.16 0.35 0.57 0.11 0.84 0.69 0.18 0.34 0.58 0.06 0.86 0 1 0 0 0.77 0.23 254.897 196.608 440235 4608045 15.135 664 14705.25 +1 1 0 2 "Urban" 230 0 0 0 0 0 0 0.22 0 0.2 0.03 0.11 0.06 0.16 0 0.13 0.02 0.07 0.05 0 0 1 0 0 1 40.056 1108.024 454111 4604473 14.145 757 14815.917 +1 3 0 4 "Urban" 243 0.97 0 0.96 0 0.01 0.99 0.52 0.14 0.35 0.32 0.16 0.53 0.54 0.05 0.42 0.15 0.28 0.29 0 0.99 0.01 0 0.99 0.01 113.473 279.488 452982 4617545 14.321 774 14748 +1 4 0 5 "Urban" 189 0.95 0 0 0.9 0 0.81 0.59 0.14 0.32 0.43 0.13 0.58 0.54 0.12 0.34 0.29 0.15 0.46 0.19 0.81 0 0.18 0 0.82 90.053 137.404 450970 4610133 14.479 735 14835.5 +3 4 0 7 "Urban" 354 0.71 0.29 1 0 0.75 0.25 0.57 0.05 0.55 0.05 0.42 0.14 0.42 0.03 0.37 0.15 0.25 0.24 0.11 0.07 0.82 0 0.25 0.75 200.194 7.371 446165 4615555 13.983 762 14774.833 +1 1 0 2 "Urban" 45 0.51 0.1 0 0.99 0 1 0.76 0.04 0.04 0.4 0.02 0.4 0.75 0.06 0.17 0.35 0.06 0.41 0 0.9 0.1 0 0.01 0.99 117.769 42.428 470665 4606795 15.074 730 14685.583 +1 1 0 2 "Urban" 196 1 0 0.33 0 0.31 0 0.63 0.02 0.34 0.04 0.12 0.07 0.58 0.03 0.31 0.09 0.12 0.11 0.69 0 0.31 0.06 0.37 0.56 70.554 252.547 462316 4608015 14.818 751 14736.167 +1 6 0 7 "Urban" 104 0 0 0 0.51 0 0.08 0.19 0.01 0.12 0.1 0.06 0.11 0.31 0.01 0.18 0.03 0.1 0.08 0 0.8 0.2 0.43 0.29 0.28 77.453 237.48 472915 4610495 14.889 738 14714.833 +0 1 0 1 "Urban" 67 0 0 1 0 0.09 0 0.9 0.04 0.9 0.1 0.5 0.13 0.91 0.05 0.79 0.14 0.44 0.26 0 0 0 0.91 0 0.09 5.586 753.868 431335 4595265 15.579 610 14692.833 +0 0 0 0 "Urban" 82 0 1 0 1 0 1 0 1 0 1 0 1 0.01 0.96 0 1 0 1 0 0 1 0 0 1 197.87 2082.711 429835 4584525 15.728 644 14640.333 +1 0 0 1 "Urban" 522 0.49 0.51 0 1 0 1 0.5 0.3 0.02 0.77 0.01 0.91 0.58 0.14 0.26 0.43 0.17 0.58 0 0.49 0.51 0 0 1 175.385 28.68 438765 4655775 12.39 735 14800.833 +0 0 0 0 "Urban" 463 0.27 0.53 0 0.77 0 1 0.43 0.31 0.42 0.42 0.32 0.47 0.59 0.11 0.62 0.17 0.44 0.3 0 0.47 0.53 0 0.23 0.77 114.856 21.921 407615 4654445 12.749 681 14951.5 +0 2 0 2 "Urban" 494 0 0.53 0 0.72 0 0.67 0.35 0.27 0.03 0.47 0.04 0.45 0.46 0.16 0.24 0.32 0.19 0.34 0 0.14 0.85 0.05 0.28 0.67 349.593 3.222 390895 4641335 12.931 676 15129.5 +0 0 0 0 "Urban" 413 0.49 0.21 0 0.4 0.14 0.54 0.37 0.05 0.25 0.13 0.22 0.15 0.36 0.1 0.2 0.2 0.19 0.18 0.19 0.39 0.42 0.03 0.32 0.65 144.243 40.527 391685 4641685 12.766 694 15133.5 +1 2 0 3 "Urban" 256 0 1 0 1 0 1 0.07 0.83 0.01 0.99 0 1 0.32 0.63 0.1 0.87 0.02 0.96 0 0 1 0 0 1 1193.13 506.187 402769 4620706 14.333 597 14889.833 +0 1 0 1 "Urban" 838 0.67 0 0 0.3 0 0.58 0.67 0.02 0.3 0.08 0.12 0.35 0.59 0.01 0.27 0.04 0.15 0.19 0.39 0.61 0 0.05 0.32 0.63 205.302 85.988 424785 4630755 11.831 776 15080.917 +1 2 0 3 "Urban" 318 0 1 0 1 0 1 0.59 0.31 0.23 0.64 0.19 0.69 0.83 0.09 0.62 0.22 0.53 0.29 0 0 1 0 0 1 140.228 119.857 413065 4628085 13.757 615 15072.167 +1 3 0 4 "Urban" 242 0.71 0.29 0 1 0.4 0.5 0.7 0.12 0.46 0.4 0.38 0.39 0.7 0.06 0.44 0.28 0.33 0.35 0.25 0.38 0.37 0.5 0 0.5 246.863 25.776 408575 4622935 14.219 593 15029.583 +1 1 0 2 "Urban" 491 0.68 0 0.42 0.41 0.11 0.62 0.31 0.08 0.34 0.24 0.17 0.22 0.49 0.03 0.49 0.07 0.34 0.09 0.07 0.82 0.11 0.15 0.34 0.51 5.714 229.083 420205 4636365 12.762 686 15077.917 +1 3 0 4 "Urban" 596 0.31 0.48 0.35 0.65 0 0.75 0.41 0.21 0.43 0.39 0.11 0.55 0.4 0.08 0.49 0.13 0.15 0.24 0.23 0.4 0.37 0.25 0.13 0.62 284.754 152.778 435615 4661575 11.787 747 14855.5 +1 0 0 1 "Urban" 442 0.24 0.09 0.25 0.72 0.09 0.15 0.66 0.03 0.59 0.15 0.45 0.15 0.68 0.08 0.62 0.16 0.52 0.19 0.22 0.29 0.45 0.73 0 0.25 20.855 69.926 440475 4649195 12.902 719 14821.917 +1 2 0 3 "Urban" 512 0.9 0.1 0.16 0.84 0.17 0.83 0.83 0.07 0.48 0.44 0.23 0.63 0.83 0.03 0.73 0.17 0.59 0.26 0 0.73 0.27 0.01 0 0.99 102.3 117.363 440505 4639545 12.733 732 14898.417 +0 0 0 0 "Urban" 156 0.93 0 1 0 0 0.91 0.63 0.02 0.53 0.07 0.14 0.43 0.62 0.02 0.54 0.09 0.22 0.34 0.03 0.91 0.06 0.09 0.91 0 124.433 283.694 445571 4607433 14.682 693 14880.583 +1 3 0 4 "Croplands" 144 0.98 0 1 0 0 0 0.57 0.06 0.47 0.17 0.08 0.33 0.65 0.03 0.51 0.15 0.14 0.44 0.98 0 0.02 1 0 0 198.994 7.567 445985 4606505 14.682 693 14880.583 +0 0 0 0 "Urban" 442 0.54 0 0.58 0 0 0 0.12 0.02 0.07 0.01 0 0.02 0.07 0.02 0.03 0.02 0 0.03 0.58 0 0.41 0.74 0 0.26 98.701 336.431 423345 4585435 15.263 645 14934.75 +0 4 0 4 "Riparian" 68 0.65 0 0.31 0 0.36 0.32 0.75 0.03 0.37 0.09 0.34 0.45 0.5 0.07 0.18 0.26 0.14 0.44 0.26 0.35 0.39 0.29 0.37 0.35 81.373 1053.457 428775 4592735 15.333 617 14865.5 +0 0 0 0 "Broad-leaved forests" 187 0.15 0 0.15 0 0 0 0.31 0 0.43 0 0.32 0 0.39 0.01 0.41 0.02 0.26 0.04 0.15 0.15 0.7 0.15 0.14 0.71 129.168 1016.979 414045 4579855 14.898 648 15021 +0 0 0 0 "Broad-leaved forests" 672 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.01 0.25 0 0.75 0 0 1 35.362 493.34 426795 4611825 13.38 726 15025.667 +0 0 0 0 "Broad-leaved forests" 583 1 0 0.82 0.18 0.74 0.26 0.82 0.02 0.52 0.04 0.35 0.08 0.6 0.03 0.4 0.06 0.16 0.15 0 0.26 0.74 0 0.09 0.91 149.192 527.6 429712 4616665 13.375 736 14929.833 +0 0 0 0 "Broad-leaved forests" 708 0 0 0 0 0 0 0.15 0 0.2 0 0.16 0 0.16 0 0.14 0 0.11 0 0.25 0.11 0.64 0.47 0 0.53 38.851 1438.315 430805 4621245 12.544 779 15006.083 +1 0 0 1 "Broad-leaved forests" 223 0.28 0 0.29 0 0.01 0.29 0.69 0.03 0.69 0.02 0.31 0.47 0.62 0.02 0.6 0.04 0.43 0.27 0 0.29 0.71 0.01 0.29 0.71 512.14 394.019 439385 4611595 14.81 697 14745.75 +0 0 0 0 "Broad-leaved forests" 296 0 0 0 0 0 0 0.05 0 0 0 0 0 0.23 0.01 0.04 0.07 0.02 0.07 0.35 0 0.65 0 0 1 87.784 713.006 440294 4596889 14.763 668 15012.167 +0 0 0 0 "Broad-leaved forests" 518 0 0 0 0 0 0 0.09 0 0.05 0 0.04 0.02 0.15 0 0.09 0.01 0.06 0.02 0 0 1 0 0 1 92.106 630.608 448465 4623685 11.794 830 14794.667 +0 0 0 0 "Broad-leaved forests" 255 0 0 0 0 0 0 0.07 0.01 0.06 0 0.03 0.08 0.2 0.01 0.13 0.06 0.04 0.17 0 0 1 0 0 1 133.583 496.847 449140 4608582 14.479 735 14835.5 +0 0 0 0 "Broad-leaved forests" 739 0 0 0 0 0 0 0.12 0 0.02 0 0.01 0.01 0.12 0 0.05 0 0.01 0.01 0 0 1 0 0 1 28.965 825.31 449925 4620795 12.823 812 14738.25 +0 0 0 0 "Broad-leaved forests" 419 0.33 0 0 0 0 0 0.16 0.01 0.02 0 0.01 0.01 0.17 0 0.07 0 0.04 0.01 0.33 0.02 0.66 0 0.02 0.98 107.936 2013.917 453545 4609145 14.165 761 14841 +0 0 0 0 "Broad-leaved forests" 278 0 0 0 0 0 0 0.13 0.01 0.14 0 0.09 0.01 0.14 0 0.08 0 0.03 0.01 0 0 1 0 0 1 145.955 1814.588 455654 4604987 13.84 782 14808.25 +0 0 0 0 "Broad-leaved forests" 454 0 0 0 0 0 0 0 0 0 0 0 0 0.02 0 0.02 0 0.01 0 0 0 1 0 0 1 11.228 2159.809 456125 4609905 13.767 785 14815.167 +0 0 0 0 "Broad-leaved forests" 357 0 0 0 0 0 0 0 0 0.01 0.02 0.01 0.01 0.04 0 0.02 0.03 0.02 0.02 0 0 1 0 0 1 67.446 2957.039 456995 4607255 13.84 782 14808.25 +0 0 0 0 "Broad-leaved forests" 389 0.05 0 0 0 0 0 0.13 0.01 0 0.03 0 0.03 0.08 0 0.01 0.02 0.01 0.04 0.05 0.01 0.94 0.05 0.04 0.91 146.407 1280.1 457935 4605435 13.84 782 14808.25 +0 0 0 0 "Broad-leaved forests" 236 0.99 0 0.44 0 0 0 0.18 0.02 0.15 0.02 0.01 0.08 0.13 0.01 0.12 0.06 0.04 0.11 0.99 0 0.01 0.44 0.2 0.35 50.479 33.062 457908 4611312 13.767 785 14815.167 +0 0 0 0 "Broad-leaved forests" 546 0 0 0 0 0 0 0.15 0 0.06 0 0.03 0.01 0.11 0 0.06 0 0.03 0.01 0 0 1 0 0 1 101.302 1046.517 459235 4623705 13.193 815 14672.417 +0 0 0 0 "Broad-leaved forests" 323 0 0 0 0 0 0 0.08 0 0.01 0.01 0 0.07 0.06 0 0.04 0.03 0.01 0.1 0 0 1 0 0 1 64.341 1899.207 460675 4614645 14.201 795 14741 +0 0 0 0 "Broad-leaved forests" 262 0 0 0 0 0 0 0 0 0 0 0 0 0.03 0 0.04 0 0.03 0 0 0 1 0 0 1 54.959 2480.298 464255 4616055 13.887 803 14750.083 +0 0 0 0 "Broad-leaved forests" 280 0 0 0 0 0 0 0 0 0 0 0 0 0.04 0 0.05 0 0.03 0 0.66 0 0.34 0 0 1 0.572 2338.405 464345 4616245 13.887 803 14750.083 +0 0 0 0 "Broad-leaved forests" 44 0.17 0 0 0 0 0.16 0.46 0.04 0.04 0.39 0.03 0.49 0.67 0.08 0.13 0.29 0.06 0.41 0.02 0.16 0.82 0.04 0.16 0.8 81.721 356.776 469455 4606379 15.074 730 14685.583 +0 0 0 0 "Broad-leaved forests" 193 0 0 0 0 0 0 0 0 0 0.1 0 0.08 0 0 0 0.09 0 0.13 0 0.04 0.96 0 0.04 0.96 97.33 2278.779 471515 4617005 14.099 791 14736.333 +0 0 0 0 "Broad-leaved forests" 69 0 0 0 0 0 0 0.07 0 0.06 0.05 0.04 0.07 0.05 0.01 0.04 0.06 0.03 0.09 0 0.53 0.47 0 0.53 0.47 143.609 784.712 479325 4619885 14.476 752 14727.25 +0 0 0 0 "Broad-leaved forests" 868 0 0 0 0 0 0 0.14 0 0.05 0 0.05 0 0.17 0 0.08 0 0.05 0 0 0 1 0.03 0 0.97 119.523 805.929 428916 4639346 11.397 773 15114.833 +0 0 0 0 "Broad-leaved forests" 881 0.08 0 0.2 0 0 0 0.25 0 0.28 0 0.1 0.01 0.21 0 0.22 0 0.11 0.03 0.23 0.07 0.7 0.2 0 0.8 74.094 1176.191 450266 4652811 11.015 813 14954.417 +0 0 0 0 "Broad-leaved forests" 947 0 0 0 0 0 0 0.24 0 0.32 0 0.25 0 0.19 0 0.23 0.01 0.17 0.01 0.01 0 0.99 0 0 1 31.788 1768.372 450825 4657355 10.472 853 14896.833 +0 0 0 0 "Broad-leaved forests" 667 0.6 0 0.7 0 0.46 0 0.09 0 0.08 0 0.07 0.02 0.18 0 0.16 0 0.12 0.01 0.25 0 0.73 0.25 0 0.73 75.242 42.131 449478 4650413 11.015 813 14954.417 +0 0 0 0 "Broad-leaved forests" 536 0 0 0 0 0 0 0.26 0.01 0.22 0 0.2 0.01 0.24 0.02 0.19 0.02 0.15 0.04 0.4 0 0.6 0 0 1 50.664 564.429 449305 4645075 12.277 761 14840.583 +0 0 0 0 "Broad-leaved forests" 562 0 0 0 0 0 0 0.11 0.02 0.1 0 0.08 0.03 0.09 0.01 0.08 0.03 0.05 0.05 0.47 0 0.53 0 0 1 29.628 472.609 449115 4639425 11.823 789 14893.5 +0 0 0 0 "Broad-leaved forests" 470 1 0 0.7 0 0 0.7 0.17 0.01 0.04 0 0 0.06 0.13 0.01 0.05 0.01 0.03 0.04 0.3 0.7 0 0 0.7 0.3 69.511 425.062 450121 4645743 12.265 755 14910 +0 0 0 0 "Broad-leaved forests" 803 0.25 0 0.09 0 0.22 0 0.18 0 0.15 0 0.12 0 0.19 0 0.17 0 0.09 0.03 0.48 0 0.51 0 0.14 0.86 72.2 586.032 447764 4659895 10.071 875 14910.167 +0 0 0 0 "Broad-leaved forests" 1144 0 0 0 0 0 0 0.11 0 0.06 0 0 0 0.11 0 0.11 0 0 0 0.3 0 0.7 0 0 1 130.839 1049.848 447335 4663325 10.071 875 14910.167 +0 0 0 0 "Broad-leaved forests" 574 0 0 0 0 0 0 0.32 0.01 0.24 0.04 0.25 0.03 0.23 0.01 0.2 0.03 0.18 0.06 0.06 0 0.94 0 0 1 351.013 87.919 445055 4645405 11.885 771 14962.5 +0 0 0 0 "Broad-leaved forests" 545 0.28 0 0.21 0 0.07 0 0.24 0 0.28 0 0.21 0.01 0.25 0 0.28 0 0.23 0.01 0.24 0.04 0.72 0.16 0.12 0.72 5.757 1567.883 444855 4648485 12.915 728 14798.583 +1 0 0 1 "Broad-leaved forests" 474 0 0 0 0 0 0 0.3 0.01 0.21 0 0.18 0 0.33 0.02 0.21 0.05 0.16 0.07 0.28 0 0.68 0 0 0.74 36.785 1237.762 444005 4647885 12.915 728 14798.583 +0 0 0 0 "Broad-leaved forests" 498 0 0 0 0 0 0 0.25 0.04 0.16 0.08 0.11 0.08 0.34 0.07 0.23 0.13 0.15 0.16 0.07 0.09 0.84 0 0.09 0.91 230.243 981.076 443535 4647665 12.915 728 14798.583 +0 0 0 0 "Broad-leaved forests" 756 0 0 0 0 0 0 0 0 0 0 0 0 0.01 0 0.01 0 0.01 0 0 0.03 0.96 0 0 1 194.171 1722.828 443465 4627955 11.306 819 15053.083 +0 0 0 0 "Broad-leaved forests" 1025 0 0 0 0 0 0 0.05 0 0 0 0.03 0 0.01 0 0 0 0.01 0 0 0 1 0 0 1 6.751 1149.034 444715 4627555 9.638 914 15104.583 +0 0 0 0 "Broad-leaved forests" 563 0.2 0 0.21 0 0.2 0 0.14 0 0.23 0.01 0.15 0.02 0.2 0.01 0.28 0.01 0.18 0.04 0.42 0.04 0.54 0.19 0.01 0.8 255.056 482.546 446305 4652405 12.212 751 14841.417 +0 0 0 0 "Broad-leaved forests" 588 0.07 0 0.14 0 0.09 0 0.4 0.01 0.34 0 0.32 0.01 0.43 0.01 0.42 0.01 0.4 0.01 0.49 0.04 0.47 0.04 0.21 0.74 48.717 24.1 376645 4625435 12.358 699 15147.583 +0 0 0 0 "Broad-leaved forests" 890 0.41 0 0.36 0 0.1 0 0.46 0.07 0.31 0.05 0.15 0.04 0.43 0.06 0.3 0.07 0.14 0.08 0.31 0 0.69 0.27 0 0.73 124.026 299.786 406422 4677717 10.073 812 15098.833 +0 0 0 0 "Broad-leaved forests" 1283 0 0 0 0 0 0 0.16 0.01 0.13 0 0 0 0.05 0 0.05 0 0 0 0.26 0 0.74 0 0 1 22.965 3675.668 394630 4679381 5.607 1188 15015.667 +0 0 0 0 "Broad-leaved forests" 367 0.53 0 0.52 0 0.13 0 0.42 0.01 0.16 0 0.1 0.01 0.35 0.01 0.11 0.02 0.08 0.03 0.44 0.03 0.53 0.4 0.03 0.57 97.719 418.96 398235 4610529 13.449 646 15138.5 +2 0 0 2 "Broad-leaved forests" 856 0 0.11 0.24 0.11 0 0.09 0.63 0.06 0.56 0.09 0.28 0.13 0.62 0.02 0.5 0.04 0.28 0.07 0.48 0.19 0.33 0.27 0.23 0.5 4.056 105.65 426615 4636215 11.157 794 15155.583 +0 0 0 0 "Broad-leaved forests" 549 0.14 0 0.13 0 0.14 0 0.74 0.01 0.8 0 0.71 0.04 0.68 0.01 0.79 0 0.7 0.04 0.05 0.02 0.93 0.01 0.02 0.96 528.301 997.833 436035 4647485 12.62 722 14837.417 +0 0 0 0 "Broad-leaved forests" 773 0.03 0 0.01 0 0.01 0 0.08 0 0.08 0 0.08 0 0.2 0.01 0.21 0 0.2 0.01 0.53 0.02 0.45 0.25 0.06 0.69 140.909 940.918 429915 4646425 11.559 749 15031.917 +0 0 0 0 "Broad-leaved forests" 825 0 0 0 0 0 0 0.18 0 0.17 0 0.09 0 0.12 0 0.13 0 0.05 0 0 0 1 0 0 1 46.243 2525.421 430070 4634200 10.936 808 15148.833 +0 0 0 0 "Broad-leaved forests" 1815 0 0 0 0 0 0 0 0 0 0 0 0 0.01 0 0.01 0 0.02 0 0.11 0.41 0.48 0.03 0.35 0.62 9.523 1639.193 412235 4682395 8.367 985 14955.667 +0 0 0 0 "Broad-leaved forests" 1332 0 0 0 0 0 0 0.08 0 0 0 0 0 0.05 0 0.01 0 0 0 0.36 0 0.64 0 0 1 119.25 937.076 416465 4681865 8.686 953 14941.083 +0 0 0 0 "Broad-leaved forests" 1312 0 0 0 0 0 0 0.05 0 0.02 0 0 0 0.06 0 0.02 0 0 0 0 0 1 0 0 1 114.4 1229.228 415355 4681065 8.686 953 14941.083 +0 0 0 0 "Broad-leaved forests" 861 0 0 0 0 0 0 0.16 0 0.14 0 0.05 0 0.12 0 0.07 0 0.04 0 0.3 0 0.7 0 0 1 99.796 4719.8 456555 4648245 12.088 782 14851.75 +0 0 0 0 "Broad-leaved forests" 903 0 0 0 0 0 0 0.12 0 0.1 0 0.07 0 0.17 0 0.16 0 0.05 0 0.6 0.03 0.37 0.39 0.2 0.41 65.935 1249.253 455135 4651775 10.32 864 14962.917 +0 0 0 0 "Broad-leaved forests" 716 0 0 0 0 0 0 0.05 0 0.05 0 0 0 0.04 0 0.04 0 0.02 0 0 0 1 0 0 1 225.992 3320.257 455065 4648955 12.24 765 14862.75 +0 0 0 0 "Broad-leaved forests" 61 0.17 0 0 0 0 0 0.21 0.02 0.09 0.04 0.06 0.04 0.24 0.02 0.17 0.02 0.1 0.02 0.17 0 0.83 0 0.06 0.94 98.411 47.405 474795 4620615 14.808 740 14651.5 +0 0 0 0 "Broad-leaved forests" 1076 0 0 0 0 0 0 0.22 0 0.28 0 0.21 0.01 0.35 0.01 0.43 0 0.31 0.02 0 0.09 0.91 0 0.09 0.91 185.975 789.007 452165 4655665 10.232 869 14929.667 +0 0 0 0 "Broad-leaved forests" 991 0 0 0 0 0 0.23 0.15 0 0.1 0.11 0.06 0.13 0.2 0 0.17 0.04 0.07 0.04 0.45 0.23 0.32 0 0.23 0.77 14.2 1678.969 431715 4634425 11.334 798 15085.917 +0 0 0 0 "Broad-leaved forests" 703 0 0 0 0 0 0 0 0 0 0 0 0 0.1 0 0.15 0 0.01 0.01 0.62 0 0.38 0.03 0 0.97 39.938 896.2 438075 4662475 11.787 747 14855.5 +0 0 0 0 "Broad-leaved forests" 803 0 0 0 0 0 0 0.02 0 0.01 0 0.01 0 0.07 0 0.06 0 0.03 0 0.15 0.05 0.79 0.09 0 0.91 22.561 1301.033 439965 4662415 10.991 801 14914.667 +0 0 0 0 "Broad-leaved forests" 706 0.44 0 0 0 0 0.01 0.18 0 0.15 0 0.13 0 0.11 0 0.07 0 0.06 0 0.49 0.01 0.5 0 0.01 0.99 12.83 2394.163 439015 4659855 10.991 801 14914.667 +0 0 0 0 "Broad-leaved forests" 848 0 0 0 0 0 0 0.1 0 0.11 0 0.06 0 0.07 0 0.07 0 0.03 0 0.02 0 0.98 0.01 0 0.99 79.089 1290.084 437685 4665225 11.044 776 14950.167 +0 0 0 0 "Broad-leaved forests" 734 0.02 0 0.41 0 0.32 0 0.56 0.02 0.6 0.02 0.55 0.04 0.53 0.02 0.55 0.03 0.51 0.05 0.47 0.31 0.23 0.12 0.03 0.85 272.23 105.863 425765 4654525 11.798 726 14944.333 +0 0 0 0 "Broad-leaved forests" 729 0.14 0 0.06 0 0.07 0 0.21 0.01 0.21 0.01 0.18 0.01 0.3 0 0.29 0 0.23 0.01 0.35 0 0.65 0 0.01 0.98 134.694 1459.563 425665 4656635 11.406 744 14959.5 +0 0 0 0 "Broad-leaved forests" 617 0 0.06 0 0 0 0.03 0.31 0.02 0.28 0 0.26 0.04 0.38 0.02 0.36 0.01 0.31 0.02 0.25 0.17 0.59 0.02 0.18 0.8 81.149 651.79 421895 4653695 12.139 704 14931 +0 0 0 0 "Broad-leaved forests" 695 0 0 0 0 0 0 0.38 0 0.39 0.03 0.35 0.03 0.54 0.01 0.58 0.01 0.51 0.03 0.51 0.11 0.39 0.18 0.45 0.38 206.392 860.589 402305 4658065 11.714 753 14926.083 +0 0 0 0 "Broad-leaved forests" 1664 0 0 0 0 0 0 0 0 0 0 0 0 0.03 0 0 0 0 0 0.08 0 0.92 0 0 1 178.908 3104.556 401115 4666985 7.702 1006 15068.583 +0 0 0 0 "Broad-leaved forests" 1322 0 0 0 0 0 0 0 0 0 0.04 0 0.03 0.04 0 0 0.02 0 0.02 0 0 1 0 0 1 15.811 202.416 407335 4682205 9.488 872 15027.417 +0 0 0 0 "Broad-leaved forests" 629 0 0 0.07 0 0.01 0 0.19 0 0.17 0 0.16 0 0.18 0 0.22 0 0.2 0 0.02 0.31 0.66 0.62 0 0.38 290.433 871.988 374835 4627165 12.633 671 15142.667 +0 0 0 0 "Broad-leaved forests" 567 0.33 0.14 0 0 0 0 0.21 0 0.17 0 0.16 0 0.29 0 0.3 0 0.29 0 0.45 0.48 0.05 0 0.6 0.4 358.23 1283.609 374455 4627005 12.633 671 15142.667 +0 0 0 0 "Broad-leaved forests" 650 0 0 0.24 0 0 0.07 0.59 0.05 0.6 0.02 0.52 0.08 0.53 0.02 0.56 0.01 0.48 0.03 0.55 0.34 0.11 0.36 0.12 0.49 100.357 44.468 422675 4653495 12.139 704 14931 +0 1 0 1 "Riparian" 152 0.75 0 0.75 0 0.74 0 0.88 0.02 0.86 0.01 0.76 0.07 0.87 0.03 0.82 0.1 0.77 0.13 0.02 0.19 0.78 0.03 0.18 0.79 126.361 416.739 385205 4573295 15.086 616 14976.667 +1 1 1 3 "Riparian" 83 0 0.1 0 0.06 0 0.13 0.45 0.15 0.42 0.16 0.25 0.26 0.6 0.07 0.38 0.16 0.1 0.36 0.47 0.2 0.18 0.45 0.1 0.29 64.697 326.003 406208 4600751 14.545 630 14979 +0 0 0 0 "Riparian" 329 0.62 0 0.16 0 0.16 0 0.69 0.02 0.11 0.08 0.08 0.14 0.65 0.01 0.17 0.05 0.13 0.09 0.47 0 0.53 0.02 0.02 0.96 65.12 212.513 431895 4615465 14.071 716 14862.167 +0 0 0 0 "Riparian" 536 0.31 0 0.49 0 0.32 0 0.54 0.01 0.51 0 0.45 0.01 0.43 0.01 0.38 0.01 0.34 0.01 0.05 0.06 0.89 0.24 0.09 0.67 147.91 749.904 373255 4602325 12.672 651 15352.5 +0 0 0 0 "Riparian" 411 0.13 0 0.12 0 0.11 0 0.5 0 0.48 0 0.4 0.01 0.54 0.03 0.53 0.03 0.46 0.06 0.37 0.19 0.44 0.15 0.14 0.72 30.268 1137.851 377467 4601547 13.253 626 15251.5 +0 0 0 0 "Riparian" 182 0.65 0 0.63 0 0.62 0 0.96 0 0.96 0 0.95 0 0.91 0.03 0.92 0.04 0.87 0.06 0.09 0.04 0.87 0.05 0.19 0.76 139.687 412.912 385485 4576055 15.086 616 14976.667 +1 0 0 1 "Riparian" 112 0.75 0 0.68 0 0.65 0 0.84 0.01 0.73 0 0.66 0.01 0.71 0.02 0.63 0.02 0.57 0.03 0.23 0.08 0.69 0.15 0.15 0.7 130.834 283.741 385895 4569865 15.139 610 15125.083 +2 1 0 3 "Riparian" 140 0.55 0 0.63 0 0.33 0 0.83 0.06 0.85 0.04 0.69 0.12 0.87 0.04 0.83 0.07 0.7 0.15 0.22 0.08 0.7 0.3 0.17 0.53 127.806 318.154 386385 4573215 15.086 616 14976.667 +0 0 0 0 "Riparian" 250 0.61 0 0.62 0 0.54 0 0.79 0.03 0.72 0.06 0.63 0.11 0.89 0.03 0.81 0.05 0.72 0.13 0.46 0.02 0.52 0.08 0 0.92 151.55 153.934 386875 4581005 14.793 618 15009.5 +0 0 0 0 "Riparian" 173 0.71 0 0.39 0 0.42 0 0.83 0.09 0.76 0.1 0.56 0.32 0.86 0.08 0.71 0.16 0.44 0.45 0.29 0 0.71 0.01 0.04 0.94 77.243 434.703 388735 4575935 14.809 631 15059.167 +0 0 0 0 "Riparian" 187 0.78 0 0.41 0 0.26 0 0.91 0.02 0.82 0.05 0.61 0.25 0.92 0.02 0.87 0.02 0.72 0.14 0.52 0 0.48 0.16 0 0.83 159.427 378.063 396745 4584805 14.906 623 15008 +2 1 0 3 "Riparian" 90 0.57 0.08 0.12 0.1 0.32 0.11 0.68 0.02 0.49 0.06 0.31 0.11 0.71 0.12 0.45 0.29 0.17 0.49 0.24 0.39 0.37 0.06 0.37 0.57 64.922 134.851 406983 4599246 15.271 609 14864.75 +1 2 0 3 "Riparian" 75 0 0 0 0.6 0 0.32 0.61 0.08 0.42 0.31 0.16 0.48 0.66 0.1 0.52 0.23 0.23 0.45 0.31 0.34 0.35 0.52 0.07 0.41 42.472 49.768 408105 4598825 15.271 609 14864.75 +0 2 0 2 "Riparian" 76 0 0 0.26 0 0.04 0 0.73 0.02 0.66 0.08 0.3 0.36 0.68 0.03 0.48 0.16 0.17 0.43 0.59 0.08 0.33 0.71 0.05 0.24 91.382 272.732 408905 4598325 15.271 609 14864.75 +0 5 0 5 "Riparian" 69 0.28 0 0.72 0 0 0.08 0.62 0.03 0.55 0.05 0.28 0.27 0.65 0.03 0.42 0.14 0.16 0.39 0.36 0.2 0.44 0.48 0.15 0.12 91.781 222.329 409095 4598345 15.271 609 14864.75 +0 5 1 6 "Riparian" 63 0 0 0.19 0 0 0 0.61 0.02 0.38 0.05 0.17 0.31 0.61 0.02 0.38 0.17 0.19 0.38 0 0.89 0.01 0.43 0 0.56 105.078 50.791 409385 4597975 15.271 609 14864.75 +0 5 0 5 "Riparian" 66 1 0 1 0 0.51 0 0.63 0.04 0.48 0.09 0.19 0.33 0.68 0.02 0.43 0.18 0.21 0.42 0.49 0 0.51 0.49 0 0.51 55.834 235.29 409247 4597792 15.271 609 14864.75 +0 2 0 2 "Riparian" 158 0.51 0.25 0.42 0.44 0.37 0.35 0.85 0.03 0.74 0.05 0.64 0.16 0.74 0.03 0.67 0.1 0.51 0.22 0.29 0.25 0.46 0.2 0.14 0.66 39.786 45.384 428664 4601398 15.198 640 14837.667 +0 0 0 0 "Riparian" 670 1 0 1 0 1 0 0.34 0.01 0.27 0 0.22 0.03 0.23 0.01 0.21 0.01 0.15 0.04 0 0 1 0 0 1 35.677 1489.727 430130 4622896 11.849 794 15017.583 +0 0 0 0 "Riparian" 703 0.87 0 0.41 0 0.29 0 0.28 0 0.2 0 0.17 0 0.15 0 0.11 0 0.09 0 0.72 0 0.28 0.13 0 0.87 35.101 1848.884 430765 4624665 11.849 794 15017.583 +0 0 0 0 "Riparian" 527 0 0 0 0 0 0 0 0 0 0.03 0 0.05 0.01 0 0.01 0.01 0 0.03 0.54 0 0.46 0 0 1 47.715 2237.002 439965 4624715 12.549 766 14970.5 +0 0 0 0 "Riparian" 559 0 0 0 0 0 0 0 0 0 0.01 0 0.03 0.02 0 0.01 0.01 0 0.02 0 0 1 0.11 0 0.89 39.809 2505.212 440345 4624245 12.549 766 14970.5 +2 5 2 9 "Riparian" 109 0.11 0.1 0.51 0.01 0.07 0.47 0.55 0.05 0.44 0.18 0.29 0.27 0.5 0.03 0.4 0.16 0.29 0.26 0.02 0.39 0.6 0.13 0.53 0.34 132.967 123.664 442245 4601495 14.87 654 14917.083 +0 0 0 0 "Riparian" 654 0 0 0 0 0 0 0.06 0 0.04 0 0 0 0.02 0 0.01 0 0 0 0.06 0 0.94 0 0 1 4.937 4349.526 444445 4620825 11.98 828 14918.333 +1 0 1 2 "Riparian" 446 0 0 0 0.17 0 0 0.05 0 0.03 0.02 0.02 0.01 0.08 0 0.06 0.01 0.04 0.01 0 0.31 0.69 0.17 0.04 0.79 142.271 1586.232 445795 4617925 11.781 852 14810.917 +0 0 0 0 "Riparian" 452 0 0 0 0 0 0 0.12 0.01 0.06 0.01 0.05 0.03 0.19 0.01 0.12 0.01 0.09 0.04 0.15 0 0.85 0 0 1 60.556 185.543 448815 4623965 11.794 830 14794.667 +1 0 0 1 "Riparian" 413 0 0 0 0 0 0 0.31 0.02 0.17 0 0.11 0.05 0.24 0.01 0.12 0.01 0.06 0.04 0.03 0 0.97 0 0 1 22.134 450.864 450705 4622325 12.823 812 14738.25 +0 0 0 0 "Riparian" 889 0 0 0 0 0 0 0.08 0 0.07 0 0.04 0 0.12 0 0.06 0 0.04 0.01 0.49 0.12 0.4 0 0.12 0.88 78.42 1757.183 451263 4624210 11.794 830 14794.667 +0 0 0 0 "Riparian" 609 0 0 0 0 0 0 0.31 0.01 0.16 0.01 0.09 0.02 0.25 0 0.12 0.02 0.06 0.03 0.08 0.03 0.89 0 0.04 0.96 49.604 718.965 451365 4623005 11.794 830 14794.667 +0 0 0 0 "Riparian" 1154 0 0 0 0 0 0 0 0 0 0 0 0 0.02 0 0.01 0 0 0.02 0.17 0 0.83 0 0 1 59.688 1743.491 452165 4626325 9.843 951 14908.583 +0 0 0 0 "Riparian" 1085 0 0 0 0 0 0 0 0 0 0 0 0 0.02 0 0.01 0 0 0.01 0.27 0 0.73 0 0 1 9.696 1699.766 452195 4626365 9.843 951 14908.583 +2 0 1 3 "Riparian" 351 0.8 0.08 0.83 0 0.7 0.21 0.43 0.06 0.28 0.09 0.19 0.23 0.31 0.02 0.18 0.05 0.12 0.11 0 0.13 0.87 0 0.21 0.79 43.627 8.08 455657 4619811 13.657 815 14642 +1 0 0 1 "Riparian" 414 0.02 0 0 0 0 0 0.23 0.01 0.14 0 0.05 0.02 0.08 0 0.05 0 0.02 0.01 0.02 0 0.98 0 0 1 80.635 2887.402 463115 4613505 13.887 803 14750.083 +3 4 1 8 "Riparian" 184 0 0 0 0 0 0 0.04 0 0.04 0 0 0 0.03 0 0.02 0 0 0 0 0 1 0 0 1 24.97 1596.71 465059 4617487 13.887 803 14750.083 +1 1 0 2 "Riparian" 292 0 0 0 0.66 0 0.49 0.13 0 0.02 0.32 0.01 0.33 0.07 0 0 0.18 0 0.2 0 0.49 0.51 0.17 0 0.83 18.621 1543.373 468603 4610939 13.95 803 14690.833 +1 0 1 2 "Riparian" 289 0.21 0 0.31 0.14 0.24 0.15 0.08 0 0.02 0.23 0.01 0.25 0.08 0 0.01 0.17 0 0.18 0.1 0.28 0.62 0.13 0.08 0.79 14.637 1740.643 468685 4611165 13.95 803 14690.833 +3 3 1 7 "Riparian" 76 0.24 0 0.62 0 0.46 0 0.06 0 0.06 0.05 0.06 0.07 0.09 0 0.09 0.08 0.05 0.12 0 0.22 0.78 0.22 0.06 0.72 166.786 2453.179 472265 4616085 14.099 791 14736.333 +2 0 1 3 "Riparian" 427 0.28 0 0 0.18 0.48 0 0.22 0.01 0.05 0.15 0.11 0.07 0.28 0.05 0.16 0.13 0.15 0.1 0.21 0.36 0.25 0.21 0.34 0.2 43.875 237.894 392215 4641665 12.766 694 15133.5 +0 0 0 0 "Riparian" 313 0 0 0 0 0 0 0.36 0.02 0.3 0.02 0.21 0.05 0.29 0.01 0.17 0.03 0.11 0.05 0.6 0.11 0.29 0.03 0.13 0.84 22.637 140.254 398155 4632765 13.318 633 15111 +3 2 0 5 "Riparian" 586 0.38 0.2 0.71 0.12 0.15 0.08 0.18 0.04 0.2 0.05 0.08 0.09 0.29 0.04 0.27 0.06 0.13 0.13 0.62 0.02 0.34 0.6 0 0.39 189.093 302.079 434505 4663025 11.578 743 14896.917 +1 0 1 2 "Riparian" 605 0.4 0 0.01 0 0.01 0 0.3 0.01 0.31 0 0.24 0.01 0.34 0.01 0.36 0 0.32 0.02 0.4 0.4 0.21 0.16 0.44 0.4 24.436 652.593 421745 4653145 12.139 704 14931 +2 0 1 3 "Riparian" 560 0.39 0 0.29 0 0.34 0 0.78 0 0.76 0 0.73 0.03 0.63 0.01 0.65 0.02 0.61 0.04 0.53 0.07 0.4 0.13 0.16 0.7 59.116 2030.513 404575 4656915 12.375 709 14908.583 +0 0 0 0 "Riparian" 738 0 0 0 0 0 0 0.02 0.01 0.02 0.01 0.02 0.01 0.15 0.01 0.1 0.01 0.08 0.01 0.29 0.19 0.52 0.07 0.06 0.87 91.837 56.103 414285 4664875 10.459 808 14959.167 +1 0 1 2 "Riparian" 585 0.35 0 0.39 0 0.42 0 0.56 0.02 0.52 0.06 0.52 0.08 0.67 0.01 0.7 0.02 0.63 0.07 0.14 0.11 0.69 0.01 0.04 0.88 157.947 1630.018 404165 4657495 12.375 709 14908.583 +2 0 0 2 "Riparian" 472 0 0 0 0 0 0 0.08 0 0 0.02 0 0.04 0.08 0 0.02 0 0.02 0.01 0 0 1 0 0 1 84.046 2014.985 451295 4643135 12.277 761 14840.583 +0 0 0 0 "Riparian" 570 0.72 0 0.05 0 0.06 0 0.59 0.03 0.58 0.06 0.55 0.07 0.57 0.03 0.5 0.14 0.41 0.22 0.94 0 0.06 0.01 0.09 0.9 161.669 76.294 444125 4643185 12.643 734 14896.583 +1 0 0 1 "Riparian" 383 0.19 0 0.17 0 0.17 0 0.26 0 0.14 0 0.1 0.01 0.23 0 0.13 0 0.09 0.01 0.12 0.17 0.71 0 0 1 41.301 792.898 416335 4625225 13.309 655 15101.667 +0 0 0 0 "Riparian" 461 0 0 0 0 0 0 0.09 0 0.03 0 0 0 0.06 0 0.02 0 0 0 0.4 0 0.6 0.23 0 0.77 19.668 2362.206 392435 4603525 13.224 678 15187.417 +0 0 0 0 "Riparian" 530 0.55 0 0.53 0 0.44 0 0.67 0.01 0.77 0.01 0.8 0.03 0.66 0.02 0.72 0.02 0.68 0.06 0.27 0 0.73 0.09 0 0.91 134.339 255.558 375305 4594245 12.606 668 15322.75 +0 0 0 0 "Riparian" 305 0.13 0 0.11 0 0.11 0 0.13 0.02 0.16 0.05 0.12 0.1 0.2 0.01 0.2 0.06 0.18 0.09 0.19 0 0.81 0 0 1 49.075 0.6 396325 4623625 13.842 609 15068 +0 0 0 0 "Riparian" 352 0.29 0 0.37 0 0.44 0 0.44 0.01 0.32 0 0.28 0.04 0.43 0.01 0.3 0 0.27 0.05 0.26 0.22 0.52 0.09 0.12 0.8 91.587 441.425 393505 4620735 13.386 637 15155.5 +1 0 0 1 "Riparian" 378 0 0 0 0 0 0 0.13 0 0.1 0 0.14 0 0.15 0.01 0.13 0 0.17 0.01 0 0.15 0.69 0 0.26 0.74 28.267 678.233 414265 4624745 13.309 655 15101.667 +0 2 0 2 "Riparian" 320 0.54 0.06 0.58 0 0.53 0.07 0.93 0 0.89 0 0.79 0.08 0.77 0.11 0.7 0.15 0.58 0.28 0.23 0.02 0.75 0.21 0.1 0.7 46.892 364.517 385145 4603155 13.797 619 15112.417 +0 0 0 0 "Riparian" 470 0.3 0.07 0.12 0 0 0 0.26 0.02 0.11 0.02 0.08 0.16 0.31 0.01 0.13 0.01 0.1 0.08 0.36 0.27 0.36 0.23 0.27 0.49 153.637 46.619 374665 4609145 12.722 647 15329.667 +0 0 0 0 "Riparian" 516 0.33 0.12 0.3 0 0.27 0.08 0.4 0.02 0.44 0 0.34 0.01 0.46 0.01 0.43 0 0.34 0.01 0.42 0.04 0.54 0.08 0.12 0.8 146.275 114.405 375275 4612315 12.956 649 15257.75 +0 0 0 0 "Riparian" 847 0 0 0 0 0 0 0.09 0.01 0.06 0.01 0.05 0.01 0.08 0.01 0.06 0 0.04 0.01 0.19 0 0.81 0 0 1 56.521 286.347 446945 4627035 10.829 868 14940.25 +0 0 0 0 "Riparian" 163 0.01 0 0.03 0.01 0 0.01 0.61 0.06 0.55 0.11 0.19 0.4 0.59 0.1 0.39 0.27 0.17 0.43 0.08 0.07 0.67 0.11 0.06 0.71 113.389 176.51 404685 4614355 14.47 599 14939.25 +0 1 0 1 "Riparian" 433 0 0 0 0 0 0 0.17 0.02 0.14 0.02 0.1 0.03 0.21 0.01 0.19 0.01 0.15 0.01 0 0 1 0 0 1 60.443 91.281 389885 4633129 12.983 665 15128.333 +0 0 0 0 "Riparian" 310 0.19 0.09 0.23 0 0.14 0.09 0.25 0.01 0.27 0.01 0.19 0.03 0.22 0.02 0.22 0.02 0.11 0.07 0.33 0.01 0.55 0.04 0.09 0.83 7.463 406.314 394735 4632375 13.399 630 15095.333 +0 0 0 0 "Coniferous forests" 527 0 0 0 0 0 0 0.07 0 0.01 0 0.01 0 0.18 0 0.09 0 0.09 0 0.57 0 0.43 0 0 1 31.668 1342.381 411055 4643595 12.703 676 15036 +0 0 0 0 "Coniferous forests" 631 0.1 0 0 0 0 0 0.04 0 0.02 0 0.02 0 0.08 0 0.05 0 0.03 0.01 0.1 0 0.9 0 0 1 113.873 827.946 423835 4622805 12.361 735 15099.75 +1 0 0 1 "Coniferous forests" 270 0.65 0 0.13 0 0.11 0 0.86 0.02 0.63 0.05 0.4 0.18 0.89 0.02 0.7 0.05 0.56 0.15 0.87 0.04 0.09 0.02 0.12 0.86 101.89 248.486 390725 4589475 14.451 624 15037.75 +0 0 0 0 "Coniferous forests" 134 0 0 0 0 0 0.62 0.33 0 0.07 0.17 0.02 0.38 0.37 0 0.07 0.09 0.02 0.2 0 0.62 0.38 0 0.62 0.38 74.023 646.936 401769 4567433 15.156 622 15143.667 +0 0 0 0 "Coniferous forests" 258 0 0 0 0 0 0 0.25 0.02 0.13 0.02 0.06 0.03 0.22 0.02 0.1 0.01 0.05 0.03 0.14 0.04 0.82 0 0.09 0.91 161.591 47.687 410915 4601345 14.446 636 15097.5 +0 0 0 0 "Coniferous forests" 133 0.26 0 0 0.03 0 0.46 0.57 0.01 0.14 0.23 0.13 0.32 0.63 0.02 0.31 0.14 0.2 0.25 0.01 0.9 0.1 0 0.89 0.11 202.811 335.403 410867 4595539 15.097 612 15037.833 +0 0 0 0 "Coniferous forests" 1328 0 0 0 0 0 0 0.25 0 0.24 0 0 0 0.1 0 0.13 0 0 0 0 0 1 0 0 1 73.041 2677.268 416825 4673005 8.915 922 15041.167 +0 0 0 0 "Coniferous forests" 160 0.44 0.13 0.04 0.3 0.08 0.24 0.41 0.01 0.09 0.18 0.04 0.24 0.36 0.01 0.09 0.18 0.03 0.3 0.28 0.12 0.59 0.14 0.07 0.79 238.215 137.279 412515 4585605 14.637 653 15018.667 +0 0 0 0 "Coniferous forests" 145 0 0 0 0 0 0 0.45 0.02 0.15 0.08 0.04 0.15 0.5 0.04 0.25 0.11 0.09 0.26 0 0.01 0.99 0 0.01 0.99 260.515 242.816 413995 4584595 15.154 636 14905.167 +0 0 0 0 "Coniferous forests" 540 0 0 0 0 0 0 0.01 0 0 0 0 0 0.06 0.01 0.04 0 0.03 0.01 0 0 1 0 0 1 221.649 515.434 414225 4605475 13.381 703 15099.167 +0 0 0 0 "Coniferous forests" 148 0 0 0 0 0 0 0.37 0 0.2 0.16 0.09 0.21 0.42 0.02 0.16 0.15 0.1 0.19 0 0 1 0 0 1 57.062 701.823 417855 4589885 15.184 622 14972.917 +0 0 0 0 "Coniferous forests" 265 0 0 0 0 0 0 0.04 0.02 0.03 0 0.01 0.03 0.06 0.02 0.05 0.02 0.02 0.03 0 0 1 0 0 1 55.356 259.275 421235 4587265 14.808 638 15091 +0 0 0 0 "Coniferous forests" 769 0 0 0 0 0 0 0.1 0 0.06 0 0.04 0 0.12 0 0.08 0 0.05 0.01 0.07 0.01 0.92 0.01 0 0.99 16.152 992.518 422145 4621675 12.179 765 15098 +0 0 0 0 "Coniferous forests" 443 0 0 0 0 0 0 0.03 0 0.02 0.01 0.01 0.02 0.03 0 0.02 0.01 0.02 0.01 0.08 0 0.92 0 0.58 0.42 111.231 963.221 426815 4610455 13.38 726 15025.667 +0 0 0 0 "Coniferous forests" 556 0 0 0 0 0 0 0 0 0 0 0 0.01 0.01 0.01 0 0.03 0 0.03 0 0 1 0 0 1 67.877 538.163 427145 4610915 13.38 726 15025.667 +0 0 0 0 "Coniferous forests" 863 0.32 0 0.32 0 0.29 0 0.23 0 0.1 0 0.08 0 0.21 0 0.12 0 0.09 0.01 0.71 0.01 0.27 0.04 0 0.96 325.124 2718.225 432055 4626995 11.613 809 15047.75 +0 0 0 0 "Coniferous forests" 189 0 0.12 0 0.16 0 0.17 0.09 0.03 0.13 0.24 0.04 0.25 0.3 0.01 0.05 0.22 0.02 0.17 0.02 0.19 0.78 0.21 0.15 0.64 124.829 99.492 434835 4592905 15.154 643 14953.833 +0 0 0 0 "Coniferous forests" 531 0 0 0 0 0 0 0.17 0.01 0.05 0 0.05 0.01 0.14 0.03 0.07 0.04 0.05 0.05 0 0 1 0 0 1 89.655 1009.14 440565 4619635 13.224 760 14851.833 +0 0 0 0 "Coniferous forests" 218 0 0 0 0 0 0 0.48 0 0.36 0 0.2 0.04 0.62 0.01 0.55 0.02 0.35 0.12 0 0 1 0 0 1 54.042 749.21 443665 4607355 14.951 677 14821.75 +0 0 0 0 "Coniferous forests" 643 0 0 0 0 0 0 0.01 0 0.01 0 0 0 0.01 0 0.01 0 0.01 0 0 0 1 0 0 1 161.642 884.61 444505 4617225 13.878 756 14817.25 +0 0 0 0 "Coniferous forests" 55 0.28 0 0.3 0 0.15 0.05 0.67 0 0.55 0 0.41 0.07 0.61 0 0.58 0.02 0.33 0.09 0.14 0.15 0.71 0.12 0.08 0.81 27.228 1694.922 481175 4616835 15.114 706 14642.75 +0 0 0 0 "Coniferous forests" 82 0.16 0.19 0.34 0 0 0.15 0.09 0.02 0.02 0 0 0.02 0.18 0.01 0.11 0 0.08 0.01 0.22 0 0.78 0.25 0.15 0.61 1.598 54.25 388066 4567141 15.396 597 15185.5 +0 0 0 0 "Coniferous forests" 819 0 0 0 0 0 0 0.15 0.01 0.14 0 0.08 0.01 0.24 0.01 0.21 0 0.16 0.01 0.02 0 0.98 0 0 1 274.552 511.566 427815 4662445 11.158 763 14941.167 +0 0 0 0 "Coniferous forests" 694 0 0 0 0 0 0 0.16 0 0.16 0 0.18 0 0.26 0 0.29 0 0.28 0 0 0 1 0 0 1 16.623 1390.082 397585 4654305 11.785 746 15079.167 +0 0 0 0 "Coniferous forests" 770 0 0 0 0 0 0 0.21 0 0.11 0 0.09 0 0.24 0.01 0.17 0 0.11 0 0.24 0 0.76 0 0 1 20.236 2317.133 420745 4659425 11.618 726 14989 +0 0 0 0 "Coniferous forests" 771 0 0 0 0 0 0 0.08 0 0.07 0 0.06 0 0.17 0 0.13 0.01 0.11 0 0.16 0 0.84 0.03 0 0.97 62.021 570.037 419005 4655005 11.894 709 15019.917 +0 0 0 0 "Coniferous forests" 716 0 0 0 0 0 0 0 0 0 0 0 0 0.11 0 0.08 0.01 0.08 0 0.05 0 0.95 0.03 0 0.97 220.809 3222.797 417455 4658225 11.618 726 14989 +0 0 0 0 "Coniferous forests" 768 0 0 0 0 0 0 0.08 0.01 0.02 0 0 0 0.07 0 0.06 0 0.04 0 0 0 1 0.06 0 0.94 91.622 2818.349 417005 4658555 11.499 742 14991.917 +0 0 0 0 "Coniferous forests" 912 0 0 0 0 0 0 0.34 0.01 0.36 0 0.36 0.02 0.25 0 0.27 0 0.25 0.01 0.44 0 0.56 0 0 1 273.515 1519.864 394635 4657315 10.085 835 15141.25 +0 0 0 0 "Coniferous forests" 1702 0 0 0 0 0 0 0 0 0 0 0 0 0.04 0 0 0 0 0 0.64 0 0.36 0 0 1 7.286 3027.344 400935 4666995 7.702 1006 15068.583 +0 0 0 0 "Coniferous forests" 1058 0 0 0 0 0 0 0.08 0 0.02 0 0.01 0 0.14 0 0.09 0 0.05 0 0 0 1 0 0 1 208.821 1049.562 394575 4663935 9.287 873 15155.25 +0 0 0 0 "Coniferous forests" 1130 0 0 0 0 0 0 0 0 0 0.01 0 0 0.12 0.01 0.11 0.01 0.12 0.01 0.34 0 0.66 0 0 1 477.254 509.532 401885 4662345 9.807 877 14878.917 +0 0 0 0 "Coniferous forests" 1079 0 0.07 0 0.1 0 0.05 0.25 0.02 0.28 0.02 0.25 0.02 0.23 0.01 0.24 0.01 0.2 0.01 0.71 0 0.29 0.05 0 0.94 15.818 58.505 400205 4661305 9.807 877 14878.917 +0 0 0 0 "Coniferous forests" 1082 0 0 0 0 0 0 0.02 0 0 0.01 0 0.02 0.13 0 0.05 0.01 0.01 0.03 0 0 1 0 0 1 150.772 128.251 416855 4667155 10.459 808 14959.167 +0 0 0 0 "Coniferous forests" 946 0.53 0 0 0 0 0 0.19 0 0.03 0.02 0.01 0 0.16 0 0.05 0.01 0.03 0 0.86 0 0.14 0.38 0 0.62 73.299 2067.024 411625 4667745 10.787 785 14946.25 +0 0 0 0 "Coniferous forests" 543 0 0 0 0 0 0 0.15 0.01 0.02 0 0.01 0 0.07 0.01 0.03 0 0.02 0 0.8 0 0.19 0.04 0.01 0.95 17.292 2343.801 407985 4661445 11.683 739 14936.75 +0 0 0 0 "Coniferous forests" 475 0.01 0 0 0 0 0 0.25 0 0.17 0.02 0.11 0.06 0.33 0.02 0.23 0.04 0.13 0.13 1 0 0 0.12 0 0.88 146.592 319.923 408415 4646805 12.829 675 14971.833 +0 0 0 0 "Coniferous forests" 1227 0 0 0 0 0 0 0.05 0 0 0 0 0 0.06 0 0 0 0 0 0.63 0 0.37 0 0 1 346.968 1409.358 419655 4679415 8.361 981 14980.5 +0 0 0 0 "Coniferous forests" 861 0.56 0 0.6 0 0.13 0.18 0.69 0.02 0.67 0.04 0.17 0.06 0.53 0.02 0.51 0.04 0.27 0.06 0.41 0.3 0.28 0.34 0.24 0.41 94.207 287.066 426025 4635365 11.157 794 15155.583 +0 0 0 0 "Coniferous forests" 831 0 0 0 0 0 0 0.38 0 0.22 0 0.06 0 0.24 0 0.17 0 0.07 0 0.13 0 0.87 0 0 1 102.203 940.261 425395 4636505 11.717 748 15103.75 +0 0 0 0 "Coniferous forests" 525 0.46 0 0.39 0 0.4 0 0.13 0 0.12 0 0.12 0.04 0.2 0 0.19 0 0.2 0.02 0.45 0 0.55 0.45 0.03 0.52 20.966 44.971 423875 4640655 12.464 697 15053.583 +0 0 0 0 "Coniferous forests" 396 0 0 0 0 0 0 0.19 0 0.17 0 0.25 0 0.14 0.01 0.12 0 0.16 0 0 0.14 0.86 0.21 0.07 0.72 112.48 719.436 414093 4624614 13.309 655 15101.667 +0 0 0 0 "Coniferous forests" 581 0 0 0 0 0 0 0.21 0.01 0.13 0 0.04 0.01 0.29 0.01 0.23 0.01 0.16 0.01 0.16 0 0.84 0 0 1 310.324 550.877 420435 4629495 12.881 684 15082.417 +0 0 0 0 "Coniferous forests" 625 0.95 0 0.85 0 0.93 0 0.55 0.01 0.51 0.01 0.47 0.07 0.38 0.01 0.34 0.01 0.31 0.05 0.04 0.01 0.95 0.04 0.08 0.88 197.84 18.118 421625 4637914 12.464 697 15053.583 +0 0 0 0 "Coniferous forests" 384 0 0 0 0 0 0 0.16 0 0.13 0 0.14 0 0.12 0 0.1 0 0.14 0 0.02 0 0.98 0.25 0 0.75 18.993 1021.925 414345 4624405 13.309 655 15101.667 +0 0 0 0 "Coniferous forests" 261 0.06 0.17 0.06 0.17 0 0.1 0.55 0.04 0.58 0.1 0.39 0.15 0.54 0.02 0.42 0.16 0.33 0.21 0.13 0.05 0.82 0.13 0.07 0.79 211.425 4.166 409275 4624265 14.219 593 15029.583 +0 0 0 0 "Coniferous forests" 763 0.24 0.03 0.04 0 0 0.04 0.11 0.03 0.04 0.01 0.03 0.02 0.06 0.01 0.03 0.01 0.01 0.01 0.59 0.01 0.41 0.01 0.1 0.89 26.629 320.562 407077 4672114 9.98 824 15060.083 +0 0 0 0 "Coniferous forests" 374 0.31 0 0.1 0 0.05 0 0.48 0.02 0.49 0 0.39 0.04 0.34 0.01 0.35 0 0.3 0.02 0.35 0 0.65 0.04 0 0.95 94.326 1581.63 404655 4638455 13.316 638 15044.5 +0 0 0 0 "Coniferous forests" 442 0 0 0 0 0 0 0.27 0 0.21 0 0.24 0.03 0.28 0 0.24 0 0.27 0.01 0 0 1 0 0 1 96.616 2106.424 402515 4640525 12.912 670 15077.583 +0 0 0 0 "Coniferous forests" 1460 0 0 0 0 0 0 0.04 0 0 0.02 0 0 0.13 0 0.04 0.02 0.01 0 0.93 0 0.07 0 0 1 241.356 3045.18 401855 4678805 8.749 926 15044.167 +0 0 0 0 "Coniferous forests" 423 0 0 0 0 0 0 0.11 0 0.1 0 0.12 0 0.11 0 0.07 0 0.1 0 0.07 0.14 0.78 0.14 0.08 0.78 20.932 841.277 398565 4635225 13.318 633 15111 +0 0 0 0 "Coniferous forests" 1247 0 0 0 0.92 0 0 0.22 0.03 0.18 0.39 0.1 0 0.18 0.02 0.15 0.16 0.09 0.01 0.05 0.73 0.22 1 0 0 422.806 248.189 403081 4670332 7.994 951 15166.083 +0 0 0 0 "Coniferous forests" 473 0 0 0 0 0 0 0.1 0 0.02 0 0.01 0 0.17 0 0.11 0 0.09 0 0 0 1 0 0 1 47.594 2134.014 393725 4615375 13.202 657 15160.333 +0 0 0 0 "Coniferous forests" 475 0 0 0 0 0 0 0.28 0 0.04 0 0.03 0 0.37 0.01 0.12 0.01 0.09 0.01 0 0.05 0.95 0 0.05 0.95 218.034 350.357 392615 4623915 12.743 680 15207.583 +0 0 0 0 "Coniferous forests" 511 0 0 0 0 0 0 0.23 0 0.15 0 0.07 0.05 0.35 0.01 0.22 0.01 0.16 0.05 0.22 0 0.78 0 0 1 124.982 653.817 392375 4612865 12.946 682 15234.75 +0 0 0 0 "Coniferous forests" 511 0.18 0 0 0 0 0 0.33 0 0.31 0 0.26 0 0.34 0.01 0.3 0.02 0.27 0.02 0.25 0 0.75 0 0 1 377.809 618.197 389525 4608645 13.543 640 15206.667 +0 0 0 0 "Coniferous forests" 691 0.92 0 0 0 0 0 0.17 0 0 0 0 0 0.08 0 0.02 0 0.02 0 0.92 0 0.08 0 0 1 128.906 951.228 389275 4630175 11.962 732 15219.833 +0 0 0 0 "Coniferous forests" 682 0.01 0 0 0 0 0 0.08 0 0.03 0 0.07 0 0.21 0 0.14 0 0.15 0 0.14 0 0.86 0.34 0 0.66 66.108 3034.636 384165 4618205 12.316 710 15225.5 +0 0 0 0 "Coniferous forests" 602 0.68 0 0 0 0 0 0.57 0.01 0.52 0 0.49 0.02 0.52 0.01 0.54 0.01 0.51 0.04 1 0 0 0.1 0.02 0.88 329.423 509.957 386045 4623155 12.266 709 15251.667 +0 0 0 0 "Coniferous forests" 850 0 0 0 0 0 0 0.37 0.01 0.26 0.01 0.13 0.02 0.34 0.01 0.26 0.01 0.2 0.01 0 0.4 0.6 0 0.48 0.52 78.545 3375.696 386635 4628075 11.962 732 15219.833 +0 0 0 0 "Coniferous forests" 591 0 0 0 0 0 0 0.25 0.01 0.26 0.01 0.24 0.01 0.22 0.01 0.22 0.01 0.23 0.01 0.11 0.4 0.49 0.23 0.06 0.71 335.463 537.154 380345 4600305 13.394 626 15240.583 +0 0 0 0 "Coniferous forests" 622 0.19 0 0.1 0 0.1 0 0.67 0 0.66 0 0.64 0 0.71 0 0.73 0 0.68 0.01 0.37 0 0.62 0.07 0.01 0.92 230.095 1561.932 375965 4617005 12.565 676 15236.417 +0 0 0 0 "Coniferous forests" 670 0 0 0 0 0 0 0.49 0.01 0.29 0 0.29 0.01 0.57 0.01 0.41 0.01 0.39 0.02 0.58 0 0.42 0 0 1 18.073 541.559 371185 4608155 12.323 666 15338 +0 0 0 0 "Coniferous forests" 691 0.5 0 0.41 0 0.38 0.09 0.37 0.03 0.34 0.05 0.34 0.05 0.52 0.01 0.51 0.02 0.5 0.02 0.43 0.18 0.4 0.14 0.16 0.7 14.199 293.872 370115 4606645 12.323 666 15338 +0 0 0 0 "Coniferous forests" 634 0 0 0.15 0 0.3 0 0.37 0.02 0.29 0.02 0.19 0.18 0.4 0.01 0.31 0.01 0.25 0.09 0.29 0.45 0.27 0.16 0.29 0.56 37.238 314.227 369955 4609425 12.323 666 15338 +0 0 0 0 "Coniferous forests" 559 0 0 0 0 0 0 0.07 0 0.08 0 0.07 0 0.26 0 0.23 0.01 0.22 0.01 0 0 1 0.27 0.04 0.69 86.915 748.202 368845 4626685 12.686 659 15156 +0 0 0 0 "Coniferous forests" 544 1 0 0.1 0 0.1 0 0.59 0.01 0.34 0.02 0.3 0.03 0.51 0.01 0.35 0.02 0.33 0.02 0.9 0 0.1 0.01 0 0.99 0.37 719.711 381899 4590473 13.239 672 15167.583 +0 0 0 0 "Coniferous forests" 619 0 0 0 0 0 0 0.15 0 0.14 0 0.09 0.02 0.1 0.01 0.11 0 0.09 0.05 0.13 0.15 0.72 0 0.28 0.72 126.258 236.754 446305 4637435 11.969 774 14931.417 +0 0 0 0 "Coniferous forests" 686 0 0 0.01 0.07 0 0 0.17 0 0.17 0.18 0.03 0.23 0.14 0 0.09 0.16 0.03 0.19 0.04 0.02 0.94 0.08 0.02 0.9 242.97 1935.553 438435 4627995 12.01 780 15057.667 +0 0 0 0 "Coniferous forests" 711 0 0 0 0 0 0 0.16 0 0.18 0 0.17 0 0.35 0 0.37 0 0.33 0.01 0.05 0 0.95 0 0 1 115.42 890.827 428815 4649575 11.559 749 15031.917 +0 0 0 0 "Coniferous forests" 686 0 0 0 0 0 0 0.16 0.01 0.16 0 0.14 0.01 0.14 0.01 0.16 0 0.11 0.03 0.16 0 0.84 0 0 1 49.201 67.433 441984 4632232 11.776 786 15034.833 +0 0 0 0 "Coniferous forests" 723 0.12 0 0.25 0 0.09 0 0.29 0.02 0.36 0 0.3 0.04 0.25 0.01 0.34 0 0.24 0.03 0.22 0.11 0.67 0.15 0.13 0.7 154.863 248.946 434325 4624005 12.625 764 14954.667 +1 0 0 1 "Coniferous forests" 549 0 0 0 0 0 0 0.01 0 0.01 0 0.01 0 0.07 0 0.06 0 0.05 0 0 0.18 0.82 0.07 0.05 0.88 31.093 2235.327 421545 4631575 12.148 744 15089.333 +2 0 0 2 "Coniferous forests" 6 0 0 0 0.48 0 0.5 0 0 0 0.13 0 0.11 0.16 0.01 0.16 0.16 0.05 0.17 0 0.53 0.47 0.07 0.1 0.82 2099.195 3293.242 425665 4571555 15.768 631 14901.333 +2 0 0 2 "Coniferous forests" 8 0 0 0 0 0 0 0 0 0 0.24 0 0.18 0.19 0.01 0.19 0.15 0.04 0.2 0.32 0.04 0.64 0 0.09 0.91 2207.994 3283.44 425515 4571565 15.768 631 14901.333 +0 0 0 0 "Scrublands" 720 0 0 0 0 0 0 0 0 0 0 0 0.03 0.26 0.01 0.17 0.01 0.14 0.03 0 0 1 0 0 1 123.564 822.537 380604 4588561 12.759 681 15251.417 +0 0 0 0 "Scrublands" 393 0.65 0 0.3 0 0.24 0 0.6 0 0.22 0.02 0.17 0.04 0.46 0.01 0.21 0.03 0.16 0.06 0.52 0.02 0.46 0.11 0.07 0.82 141.14 648.904 387305 4592335 13.488 654 15153.917 +0 0 0 0 "Scrublands" 270 0.03 0 0 0.17 0 0.17 0.31 0.01 0.1 0.01 0.04 0.05 0.27 0.01 0.14 0.03 0.08 0.07 0.06 0.3 0.64 0.03 0.09 0.88 95.161 15.745 392435 4573785 14.938 627 15069.25 +0 0 0 0 "Scrublands" 157 0.63 0 0 0 0 0 0.42 0.01 0.26 0 0.2 0.01 0.47 0.02 0.37 0 0.3 0.02 0.61 0.08 0.24 0.07 0.17 0.76 85.92 293.938 386875 4570099 15.139 610 15125.083 +0 0 0 0 "Scrublands" 133 1 0 0 0 0 0 0.57 0 0.17 0 0.17 0.02 0.42 0.01 0.22 0 0.16 0.03 1 0 0 0.08 0 0.92 168.255 1011.058 389846 4567772 14.774 630 15169.667 +0 0 0 0 "Scrublands" 261 1 0 0 0 0 0 0.3 0 0.07 0 0.01 0.04 0.24 0.01 0.11 0.02 0.07 0.06 1 0 0 0 0.46 0.54 84.667 160.169 392235 4573407 14.938 627 15069.25 +0 0 0 0 "Scrublands" 192 0 0 0 0 0 0 0.24 0 0.05 0.08 0.05 0.15 0.43 0.01 0.21 0.09 0.14 0.19 0 0 1 0 0 1 62.76 792.097 399215 4570445 15.156 622 15143.667 +0 0 0 0 "Scrublands" 198 0 0 0 0 0 0 0.15 0 0.03 0.01 0.03 0 0.17 0 0.02 0.02 0.02 0.03 0.18 0 0.82 0.92 0 0.08 204.928 1562.071 400035 4572385 14.787 634 15145.833 +0 0 0 0 "Scrublands" 216 0 0 0 0 0 0 0 0 0 0 0 0 0.04 0 0 0 0 0 0.45 0.06 0.48 0.68 0 0.32 248.646 2649.573 401165 4570105 15.156 622 15143.667 +0 0 0 0 "Scrublands" 322 0 0 0 0 0 0 0.04 0 0 0 0 0 0.03 0 0 0 0 0 0.89 0 0.11 1 0 0 61.235 1778.394 403181 4575913 14.375 662 15151.667 +0 0 0 0 "Scrublands" 305 0.09 0 0 0 0 0 0.02 0 0 0.03 0.01 0 0.02 0 0 0.01 0.01 0 0.21 0 0.79 0.21 0 0.79 61.219 3537.376 405955 4570955 14.647 660 15143.417 +0 0 0 0 "Scrublands" 303 0.18 0 0 0 0 0 0.06 0.01 0 0 0 0 0.08 0 0.04 0.01 0.03 0.01 0.18 0.67 0.15 0.13 0.54 0.33 189.205 738.868 406105 4605575 14.248 622 15029.25 +0 0 0 0 "Scrublands" 387 0 0 0 0 0 0 0 0 0 0.01 0 0 0.03 0 0.02 0.01 0.02 0 0 0 1 0 0 1 88.379 2608.224 406706 4570341 14.647 660 15143.417 +0 0 0 0 "Scrublands" 505 0 0 0 0 0 0 0 0 0 0.01 0 0.01 0 0 0.01 0 0 0.01 0 0 1 0 0 1 218.822 3058.832 408545 4571375 14.647 660 15143.417 +0 0 0 0 "Scrublands" 501 0 0 0 0 0 0 0 0 0.02 0 0 0 0 0 0.01 0 0 0.02 0 0 1 0 0 1 122.278 2683.101 409035 4571125 14.647 660 15143.417 +0 0 0 0 "Scrublands" 359 0 0 0 0 0 0 0 0.01 0 0.03 0 0.04 0 0.02 0.01 0.09 0 0.14 0 0.45 0.55 0 0.64 0.36 39.628 221.684 410658 4604078 14.446 636 15097.5 +0 0 0 0 "Scrublands" 180 1 0 0 0 0 0 0.42 0.1 0 0.19 0.02 0.21 0.34 0.09 0.1 0.17 0.08 0.22 1 0 0 0.44 0.01 0.55 219.892 113.981 411275 4591735 15.318 614 14915.833 +0 0 0 0 "Scrublands" 463 0 0 0 0 0 0 0.14 0 0.09 0 0.08 0.01 0.14 0.01 0.08 0.02 0.05 0.02 1 0 0 1 0 0 227.571 364.697 411805 4574951 14.303 684 15105 +0 0 0 0 "Scrublands" 373 0 0 0 0 0 0 0.05 0 0 0.1 0.02 0.18 0.13 0 0.1 0.06 0.06 0.13 0 0.97 0.03 0 1 0 136.766 2416.81 412205 4579665 14.309 677 15094.833 +0 0 0 0 "Scrublands" 118 0.55 0 0.39 0 0.36 0 0.64 0 0.06 0.16 0.08 0.08 0.56 0.02 0.12 0.16 0.06 0.19 0.26 0.11 0.63 0.04 0.13 0.83 194.588 457.883 414415 4593015 15.344 617 14895.333 +0 0 0 0 "Scrublands" 244 0 0 0 0 0 0 0.1 0 0 0 0 0 0.13 0 0.01 0.04 0.01 0.06 0 0 1 0 0 1 241.077 961.085 414625 4587505 15.211 628 14878.167 +0 0 0 0 "Scrublands" 602 0.25 0 0 0 0 0 0.15 0 0.11 0 0.06 0.01 0.08 0 0.05 0 0.03 0 0.28 0.45 0.27 0 0.56 0.44 46.977 1664.73 420445 4620915 12.179 765 15098 +0 0 0 0 "Scrublands" 411 0 0 0 0 0 0 0.12 0 0.11 0 0.19 0 0.12 0 0.1 0 0.13 0 0 0.07 0.93 0 0.07 0.93 8.617 854.885 399895 4633745 12.988 666 15123.083 +0 0 0 0 "Scrublands" 516 0 0 0 0 0 0 0.28 0.01 0.11 0.01 0.06 0.02 0.29 0.01 0.1 0.01 0.06 0.01 0 0.88 0.12 0 1 0 65.995 433.286 421365 4616455 13.024 720 15051 +0 0 0 0 "Scrublands" 685 0.51 0 0.46 0 0.22 0 0.15 0 0.09 0.01 0.08 0.02 0.26 0 0.16 0.02 0.13 0.03 0.3 0.38 0.28 0.46 0.24 0.3 72.25 671.077 421375 4619605 12.179 765 15098 +0 0 0 0 "Scrublands" 664 0 0 0 0 0 0 0.2 0 0.04 0 0.03 0.01 0.18 0 0.1 0.01 0.07 0.02 0 0 1 0 1 0 36.385 908.046 421097 4620511 12.179 765 15098 +0 0 0 0 "Scrublands" 590 0 0 0 0.14 0 0.11 0.05 0 0 0.03 0.02 0.04 0.27 0 0.12 0.02 0.1 0.04 0 0.53 0.47 0.32 0.01 0.68 77.455 88.787 432175 4617125 14.071 716 14862.167 +0 0 0 0 "Scrublands" 91 0 0 0 0 0 0 0.04 0.02 0.03 0.06 0 0.06 0.22 0.1 0.09 0.23 0.04 0.26 0 0.32 0.68 0 0 1 39.077 325.621 433035 4593355 15.523 608 14842.083 +0 0 0 0 "Scrublands" 1119 0.57 0 0 0 0 0 0.11 0 0.09 0 0 0 0.06 0 0.05 0 0 0 0.57 0 0.43 0.06 0.19 0.76 96.724 2385.394 444005 4626385 10.709 862 15098.5 +0 0 0 0 "Scrublands" 220 0 0 0 0 0 0 0.1 0.01 0.03 0 0.03 0 0.08 0 0.04 0 0.04 0.01 0 0.8 0.2 0.2 0.01 0.79 40.814 963.864 399795 4572945 14.787 634 15145.833 +0 0 0 0 "Scrublands" 751 0 0 0 0 0 0 0.05 0 0.06 0 0.08 0 0.12 0 0.11 0.01 0.12 0 0 0 1 0 0 1 177.561 2980.56 417375 4657405 11.618 726 14989 +0 0 0 0 "Scrublands" 665 0 0 0 0 0.19 0 0.45 0.02 0.48 0 0.41 0.03 0.35 0.01 0.37 0 0.34 0.02 0.05 0.38 0.57 0.03 0.6 0.36 201.421 137.394 414185 4654435 12.321 692 14966.917 +0 0 0 0 "Scrublands" 678 0 0 0 0 0.01 0 0.05 0 0.05 0 0.11 0 0.2 0 0.21 0 0.23 0 0 0.39 0.61 0 0.39 0.61 26.996 1268.744 412595 4656855 11.895 722 14957.167 +0 0 0 0 "Scrublands" 591 0 0 0 0 0 0 0.06 0 0.06 0 0.07 0 0.12 0 0.11 0 0.16 0.02 0.06 0.52 0.42 0 0.72 0.28 44.183 1819.055 412135 4653615 12.405 691 14984.583 +0 0 0 0 "Scrublands" 759 0 0 0 0 0 0 0.18 0 0.17 0 0.14 0 0.18 0 0.16 0 0.13 0 0 0.08 0.92 0.12 0.08 0.8 45.153 1212.559 415465 4657855 11.499 742 14991.917 +0 0 0 0 "Scrublands" 574 0 0 0 0 0 0 0.06 0 0.07 0 0.06 0 0.18 0 0.22 0 0.21 0.01 0 0.87 0.13 0 0.87 0.13 64.091 1199.431 405595 4655045 12.527 693 14999.083 +0 0 0 0 "Scrublands" 820 0 0 0 0 0 0 0.02 0 0 0 0 0 0.06 0 0.05 0 0.04 0.01 0 0.63 0.37 0 0.4 0.6 266.774 801.985 391025 4652695 11.365 766 15167.583 +0 0 0 0 "Scrublands" 1432 0 0 0 0 0 0 0.06 0 0.03 0 0.01 0 0.07 0 0.03 0 0 0 0.28 0.46 0.27 0.11 0.03 0.86 188.541 3253.264 394375 4667165 8.045 961 15171.583 +0 0 0 0 "Scrublands" 626 0.4 0 0 0 0 0 0.28 0 0.21 0 0.17 0.01 0.16 0 0.11 0 0.1 0 0.4 0 0.6 0 0 1 68.181 953.802 398445 4646575 12.235 727 15049.25 +0 0 0 0 "Scrublands" 660 0 0 0 0 0 0 0.09 0 0.06 0 0.15 0 0.17 0 0.13 0 0.21 0 0.1 0.35 0.55 0 0.59 0.41 110.338 2741.712 400145 4650425 12.512 702 15027.5 +0 0 0 0 "Scrublands" 646 0 0 0 0 0 0 0.15 0 0.1 0 0.18 0 0.18 0 0.18 0 0.27 0 0 0.25 0.75 0 0.25 0.75 134.254 2023.84 399455 4651405 11.785 746 15079.167 +0 0 0 0 "Scrublands" 774 0 0 0.21 0 0 0 0.03 0 0.09 0.01 0.08 0 0.07 0 0.11 0.01 0.12 0 0 0.99 0.01 0.21 0.78 0.01 165.715 1561.531 395085 4650095 12.092 743 15078.917 +0 0 0 0 "Scrublands" 620 0.57 0 0.49 0 0.48 0 0.44 0 0.35 0.02 0.39 0.01 0.28 0 0.23 0.01 0.24 0 0.11 0.16 0.73 0.02 0.15 0.83 71.702 2391.81 399345 4649275 12.235 727 15049.25 +0 0 0 0 "Scrublands" 517 0 0 0 0 0 0 0 0 0.01 0 0.01 0 0.1 0 0.09 0 0.08 0 0.42 0.05 0.53 0.07 0.12 0.81 135.17 1364.141 402505 4646765 12.512 702 15027.5 +0 0 0 0 "Scrublands" 508 0 0 0 0 0 0 0.1 0 0.07 0 0.07 0 0.17 0 0.12 0.01 0.12 0 0 0.01 0.99 0.31 0.01 0.68 96.671 2079.23 403795 4646385 12.674 690 15010.333 +0 0 0 0 "Scrublands" 555 0 0 0 0 0 0 0.1 0 0.06 0 0.05 0.03 0.2 0 0.1 0 0.12 0.02 0 0.57 0.43 0.26 0.01 0.73 45.897 1662.064 404945 4647605 12.674 690 15010.333 +0 0 0 0 "Scrublands" 582 0.9 0 0.79 0 0.69 0 0.33 0.01 0.23 0 0.23 0.02 0.32 0.01 0.28 0.01 0.24 0.02 0.31 0.01 0.68 0.1 0 0.89 229.763 566.943 406425 4649505 12.674 690 15010.333 +0 0 0 0 "Scrublands" 664 0 0 0 0 0 0 0.09 0 0.04 0 0.06 0 0.11 0 0.06 0 0.08 0 0.03 0.06 0.91 0.18 0.22 0.6 95.212 1012.484 409095 4648785 12.829 675 14971.833 +0 0 0 0 "Scrublands" 384 0.35 0 0.06 0 0.12 0 0.61 0.01 0.6 0.02 0.61 0.06 0.58 0.01 0.57 0.04 0.54 0.05 0.92 0.01 0.03 0.14 0.12 0.68 376.038 169.953 379845 4603555 13.394 626 15240.583 +0 0 0 0 "Scrublands" 373 1 0 0.94 0.05 0.93 0.05 0.69 0.01 0.72 0.02 0.7 0.04 0.59 0.01 0.58 0.03 0.55 0.05 0.02 0.05 0.93 0.02 0.01 0.97 94.105 126.601 380145 4603545 13.394 626 15240.583 +0 0 0 0 "Scrublands" 367 0.62 0 0.72 0 1 0 0.67 0.01 0.67 0.02 0.63 0.06 0.57 0.01 0.52 0.03 0.5 0.06 0 0.38 0.62 0 0.26 0.72 219.474 215.805 380260 4603859 13.394 626 15240.583 +0 0 0 0 "Scrublands" 505 0.57 0 0.11 0 0.11 0 0.15 0 0.07 0 0.08 0 0.32 0.01 0.24 0 0.23 0.01 0.5 0.2 0.3 0.46 0.1 0.44 112.153 548.232 379455 4597375 12.976 653 15276 +0 0 0 0 "Scrublands" 488 0.32 0 0.55 0 0.34 0 0.37 0 0.47 0 0.43 0 0.27 0 0.31 0 0.27 0.01 0.06 0.49 0.45 0.24 0.01 0.75 116.825 1418.913 375765 4606595 13.224 624 15273.75 +0 0 0 0 "Scrublands" 548 0 0 0 0 0 0 0.21 0.01 0.17 0.01 0.17 0.01 0.31 0.01 0.26 0.01 0.25 0.01 0 0 0.99 0.1 0.13 0.77 80.045 281.31 371625 4625195 12.686 659 15156 +0 0 0 0 "Scrublands" 510 0 0 0 0 0 0 0.27 0.01 0.26 0.01 0.26 0.01 0.29 0.01 0.22 0.01 0.22 0.01 0 0.1 0.9 0.43 0.13 0.44 11.696 261.291 369205 4627015 12.686 659 15156 +0 0 0 0 "Scrublands" 575 0 0 0 0 0 0 0.05 0 0.06 0 0.06 0 0.23 0.01 0.17 0.02 0.16 0.02 0 0.26 0.74 0.59 0.23 0.18 3.238 549.319 369175 4626605 12.686 659 15156 +0 0 0 0 "Scrublands" 560 0 0 0 0 0 0 0.11 0 0.12 0 0.11 0 0.25 0.01 0.19 0.01 0.19 0.01 0 0.94 0.06 0.23 0.01 0.75 136.009 438.378 369165 4626785 12.686 659 15156 +0 0 0 0 "Scrublands" 888 0.22 0 0.41 0 0 0 0.29 0 0.31 0 0.13 0.01 0.21 0 0.23 0 0.11 0.04 0.43 0.04 0.53 0.41 0.01 0.59 87.342 1195.115 450271 4652869 11.015 813 14954.417 +1 0 0 1 "Scrublands" 744 0.28 0 0.08 0 0.08 0 0.1 0.02 0.08 0.02 0.06 0.01 0.08 0.01 0.06 0.01 0.04 0.01 0.2 0.19 0.61 0.59 0 0.41 143.267 34.805 444155 4634175 11.776 786 15034.833 +0 0 0 0 "Scrublands" 754 0 0 0 0 0 0 0.03 0 0.03 0.02 0.03 0 0.02 0 0.06 0.03 0.03 0.03 0.22 0.05 0.73 0.72 0.08 0.2 45.339 1155.305 442285 4633825 11.776 786 15034.833 +0 0 0 0 "Scrublands" 647 0 0 0 0 0 0 0.07 0 0.04 0 0.02 0 0.07 0 0.05 0 0.04 0 0 0.05 0.95 0 0.05 0.95 200.214 2292.747 420167 4621653 12.686 720 15105.75 +0 0 0 0 "Scrublands" 437 0 0 0 0 0 0 0.39 0.02 0.11 0.01 0.1 0.02 0.47 0.02 0.28 0.01 0.21 0.03 0.5 0.1 0.41 1 0 0 232.724 1247.396 392805 4620245 13.386 637 15155.5 +0 0 0 0 "Scrublands" 548 0.21 0 0.28 0 0.26 0 0.28 0 0.22 0 0.2 0 0.39 0 0.3 0 0.29 0.01 0.03 0.08 0.89 0.74 0 0.26 256.133 1337.678 389630 4622386 13.192 644 15183.917 +0 0 0 0 "Scrublands" 508 0 0 0 0 0 0 0.13 0 0.19 0 0.18 0 0.2 0 0.17 0 0.16 0 0.83 0 0.17 0.83 0 0.17 284.884 2938.723 394095 4617445 13.202 657 15160.333 +0 0 0 0 "Scrublands" 347 0.25 0 0.01 0 0.34 0 0.35 0 0.21 0 0.13 0.02 0.21 0 0.14 0 0.07 0.02 0.08 0.2 0.72 0.59 0.33 0.09 64.055 1916.345 396465 4618805 13.867 609 15094.417 +0 0 0 0 "Scrublands" 377 0 0 0 0 0 0 0.14 0 0.1 0 0.1 0 0.24 0 0.18 0 0.16 0.01 0.62 0 0.38 1 0 0 26.567 1672.24 395845 4619085 13.386 637 15155.5 +1 2 0 3 "Scrublands" 295 0.84 0 0.81 0 0.49 0.08 0.69 0.01 0.57 0.03 0.35 0.17 0.57 0.02 0.4 0.1 0.26 0.2 0.28 0.09 0.63 0.24 0.08 0.68 12.784 120.146 397617 4615985 13.807 612 15090.667 +0 0 0 0 "Scrublands" 358 0.81 0 0.76 0 0.57 0 0.69 0.02 0.52 0 0.37 0.07 0.56 0.03 0.4 0.02 0.29 0.08 0.13 0.05 0.63 0 0.08 0.72 157.167 131.141 396457 4615351 13.807 612 15090.667 +0 0 0 0 "Scrublands" 702 0 0 0 0 0 0 0.25 0 0.15 0 0.15 0 0.35 0 0.26 0.01 0.27 0.01 0.06 0.27 0.67 0.72 0 0.28 361.774 1209.611 382555 4617275 12.316 710 15225.5 +0 0 0 0 "Scrublands" 649 0.1 0 0 0 0 0 0.3 0 0.12 0 0.12 0 0.29 0 0.1 0 0.08 0 0.1 0.78 0.12 0.15 0 0.85 13.995 3492.418 386795 4617415 12.392 707 15270.167 +0 0 0 0 "Scrublands" 739 0.08 0 0 0 0 0 0.09 0 0 0 0.01 0 0.08 0 0.02 0 0.03 0 0.23 0.13 0.63 0.06 0.06 0.87 252.606 1273.466 372985 4590035 12.014 692 15390 +0 0 0 0 "Scrublands" 422 0 0 0 0 0 0 0.13 0 0.07 0.09 0.07 0.04 0.23 0 0.12 0.09 0.12 0.07 0 0 1 1 0 0 207.415 1233.905 395905 4612125 13.358 650 15144 +0 0 0 0 "Scrublands" 414 0.61 0 0 0 0 0 0.25 0.01 0.09 0.02 0.06 0.14 0.28 0.01 0.17 0.01 0.13 0.06 0.61 0 0.39 1 0 0 137.51 5.262 393895 4612905 13.358 650 15144 +0 0 0 0 "Scrublands" 439 0.47 0 0.06 0 0 0.01 0.42 0.01 0.2 0 0.17 0.02 0.21 0 0.06 0 0.05 0.01 0.46 0.21 0.34 0.79 0.01 0.19 84.115 1015.442 398915 4609655 13.449 646 15138.5 +0 0 0 0 "Scrublands" 529 0 0 0 0 0 0 0.03 0.01 0.03 0.02 0.01 0.02 0.07 0.01 0.04 0.01 0.01 0.01 0.65 0 0.35 1 0 0 102.746 37.262 396455 4609955 13.449 646 15138.5 +0 0 0 0 "Scrublands" 610 0.01 0 0 0 0 0 0.27 0 0.09 0 0.07 0.01 0.26 0.01 0.14 0.01 0.1 0.03 0.16 0 0.84 1 0 0 1.335 1439.183 395255 4608355 12.998 677 15240.25 +0 0 0 0 "Scrublands" 483 0.66 0 0.64 0 0.46 0 0.43 0 0.34 0.02 0.29 0.02 0.31 0 0.26 0.01 0.23 0.01 0.32 0 0.68 0.26 0.02 0.72 45.262 43.132 379765 4610645 12.816 673 15241.667 +0 0 0 0 "Scrublands" 696 0.46 0 0.46 0 0.6 0 0.25 0 0.4 0 0.39 0 0.22 0 0.28 0 0.24 0.01 0.02 0.25 0.73 0.01 0.14 0.86 122.789 1345.43 382235 4612145 12.759 693 15193.917 +0 0 0 0 "Scrublands" 477 0 0 0 0 0 0 0.26 0.02 0.17 0.02 0.08 0.11 0.38 0.01 0.22 0.02 0.17 0.08 0 0.68 0.32 0.04 0 0.96 35.212 407.832 375515 4609475 12.956 649 15257.75 +0 0 0 0 "Scrublands" 131 0 0.1 0 0.21 0 0.25 0.05 0.04 0 0.07 0.01 0.08 0.06 0.02 0 0.04 0.01 0.05 0.01 0.34 0.62 0 0.22 0.76 99.203 170.845 404507 4604954 14.248 622 15029.25 +0 0 0 0 "Scrublands" 372 0.16 0 0 0 0 0 0.23 0.01 0.17 0.01 0.05 0.08 0.26 0.01 0.13 0.02 0.07 0.05 0.28 0.01 0.71 0.87 0.13 0 142.395 329.949 409755 4618335 14.129 606 15041 +0 0 0 0 "Scrublands" 317 0.29 0 0.03 0 0.02 0 0.45 0 0.45 0 0.38 0.01 0.45 0 0.38 0.02 0.3 0.06 0.63 0 0.37 0.83 0 0.17 221.549 1400.169 410645 4623045 13.861 611 15081.75 +0 0 0 0 "Scrublands" 1416 0 0 0 0 0 0 0.04 0 0 0 0.01 0 0.03 0 0.01 0 0.01 0 0 0.98 0.02 0.43 0.01 0.56 8.861 775.973 407515 4684005 4.657 1289 15117.583 +0 0 0 0 "Scrublands" 620 0.2 0 0 0 0 0 0.44 0.01 0.35 0 0.31 0.02 0.22 0 0.16 0 0.14 0.01 0.2 0 0.8 0 0 1 164.146 341.098 393055 4630065 12.967 674 15096.583 +0 0 0 0 "Scrublands" 472 0 0 0 0 0 0 0.07 0 0.05 0 0.11 0.01 0.12 0.01 0.08 0.06 0.09 0.08 0.04 0.46 0.5 0 0.46 0.54 28.544 510.015 405825 4634565 13.393 632 15079.167 +0 0 0 0 "Scrublands" 745 0 0 0 0 0 0 0.15 0 0.04 0 0.07 0.01 0.19 0 0.09 0 0.08 0.01 0 0 1 0 0 1 56.968 342.369 391555 4630505 12.207 722 15184.667 +0 0 0 0 "Scrublands" 664 0 0 0 0 0 0 0.02 0 0.01 0 0.01 0 0.07 0 0.05 0 0.04 0 0 0 1 0 0 1 175.367 1641.417 394315 4631125 12.967 674 15096.583 +0 0 0 0 "Scrublands" 464 0 0 0 0 0 0 0.06 0.01 0.01 0.01 0 0 0.11 0.01 0.05 0.01 0.03 0.01 0 1 0 0 1 0 418.822 308.954 392675 4632795 12.983 665 15128.333 +0 0 0 0 "Scrublands" 477 0 0 0.2 0 0.17 0 0.1 0 0.14 0 0.15 0 0.12 0.01 0.13 0 0.16 0.01 0 0.2 0.8 0.83 0.01 0.16 126.985 682.104 398125 4635715 13.318 633 15111 +0 0 0 0 "Scrublands" 503 0 0 0 0 0 0 0.28 0 0.25 0 0.2 0 0.19 0 0.18 0 0.16 0 0 1 0 0 1 0 122.096 1501.676 399905 4638185 12.912 670 15077.583 +0 0 0 0 "Scrublands" 719 0 0 0.2 0 0 0 0.26 0 0.3 0 0.19 0.01 0.25 0 0.27 0 0.18 0.04 0.31 0.1 0.59 0.44 0.17 0.39 39.527 1238.312 434645 4657675 12.292 727 14824.583 +0 0 0 0 "Scrublands" 524 0 0 0 0 0 0 0.12 0 0.08 0.03 0.1 0 0.16 0 0.14 0.01 0.12 0 0 0 1 0.17 0 0.83 139.17 1875.09 402395 4635015 12.988 666 15123.083 +0 0 0 0 "Scrublands" 476 0 0 0 0 0 0 0 0 0.02 0 0.08 0 0.05 0 0.04 0 0.06 0 0.09 0 0.91 0 0 1 64.425 1234.108 404045 4635545 13.393 632 15079.167 +0 0 0 0 "Scrublands" 465 0 0 0 0 0 0 0.32 0.01 0.15 0.01 0.09 0.04 0.46 0.01 0.24 0.01 0.2 0.02 0.62 0.01 0.37 0.98 0 0.02 70.619 187.917 379115 4583545 13.527 649 15147.5 +0 0 0 0 "Coniferous forests" 407 0.02 0 0 0 0 0 0.31 0.01 0.01 0.31 0.03 0.39 0.22 0 0.03 0.16 0.02 0.19 0.02 0 0.98 1 0 0 41.38 549.963 407425 4585435 13.981 678 15125.75 +1 0 1 2 "Coniferous forests" 310 0 0 0 0 0 0 0.17 0.04 0.04 0.01 0 0.04 0.18 0.04 0.05 0.08 0.02 0.1 0.79 0.06 0.16 0.3 0 0.7 228.171 35.271 424425 4584445 15.293 653 14903.333 +0 0 0 0 "Scrublands" 815 0 0 0 0 0 0 0.09 0 0.11 0 0.07 0 0.15 0 0.15 0.01 0.14 0 0.34 0.02 0.64 0 0.02 0.98 33.937 2652.161 417085 4656795 11.499 742 14991.917 +0 0 0 0 "Meadows" 311 0 0 0 0.3 0 0 0.01 0 0.03 0.03 0 0.02 0.02 0 0.01 0.03 0 0.02 0.01 0.12 0.87 0.42 0 0.58 14 1604.71 411395 4573375 14.303 684 15105 +2 3 0 5 "Meadows" 158 0.71 0 0.85 0 0.73 0 0.21 0.01 0.19 0 0.14 0.01 0.13 0 0.1 0.04 0.06 0.05 0 0.12 0.87 0.17 0.02 0.81 2.987 1632.427 412896 4572668 15.141 642 14972.333 +0 1 0 1 "Meadows" 181 0.92 0 0.94 0 0.61 0.21 0.83 0 0.63 0.1 0.35 0.28 0.64 0.01 0.49 0.21 0.28 0.39 0.12 0.23 0.65 0.13 0.21 0.65 157.756 1004.151 435150 4608324 15.105 658 14805.75 +0 0 0 0 "Meadows" 1126 0 0 0 0 0 0 0.13 0 0 0 0 0 0.08 0 0.01 0 0 0 0.14 0.02 0.84 0 0 1 134.145 1119.596 455819 4625966 10.975 899 14813.167 +0 0 0 0 "Meadows" 636 1 0 1 0 1 0 0.76 0.01 0.77 0 0.71 0.02 0.8 0.01 0.8 0.01 0.71 0.07 0 0 1 0 0 1 270.805 675.826 403895 4658765 12.375 709 14908.583 +0 0 0 0 "Meadows" 696 0 0 0 0 0.07 0 0.08 0 0.07 0.01 0.11 0.01 0.22 0 0.19 0.02 0.2 0.01 0 0.07 0.93 0 0.07 0.93 96.806 1864.222 395845 4650815 11.459 760 15138.75 +0 0 0 0 "Meadows" 821 0 0 0 0 0 0 0.09 0 0.04 0 0.02 0 0.07 0 0.02 0 0.01 0 0 0.39 0.61 0 0.39 0.61 136.98 3275.276 410305 4659545 12.369 702 14937.5 +0 2 0 2 "Meadows" 591 0.69 0 0.77 0 0.75 0 0.36 0 0.41 0 0.28 0.16 0.26 0 0.27 0 0.25 0.04 0.12 0.22 0.67 0.04 0.23 0.73 177.623 905.99 410695 4655005 12.405 691 14984.583 +0 0 0 0 "Meadows" 1280 0 0 0 0 0 0 0.02 0 0 0 0 0 0.07 0 0 0 0 0 0.31 0.04 0.65 0.11 0.16 0.73 228.367 1607.275 419295 4679505 8.361 981 14980.5 +0 0 0 0 "Meadows" 1373 0 0 0 0 0 0 0.05 0 0.15 0.01 0 0.01 0.1 0 0.2 0.01 0.02 0.01 0.66 0.22 0.12 0.26 0.09 0.64 161.146 384.274 419565 4680775 8.361 981 14980.5 +0 0 0 0 "Meadows" 380 0 0 0 0 0 0 0.24 0 0.21 0 0.12 0.04 0.29 0.05 0.2 0.1 0.14 0.13 0.75 0.25 0 0.98 0.02 0 11.628 772.28 413875 4635405 13.464 635 15021 +0 0 0 0 "Meadows" 1122 0 0 1 0 0 0 0.29 0 0.83 0 0 0 0.29 0.01 0.52 0 0.02 0 0.09 0.02 0.9 1 0 0 78.191 857.146 414575 4678805 8.686 953 14941.083 +0 0 0 0 "Meadows" 1954 0 0 0 0 0 0 0 0 0 0.02 0 0.01 0 0 0 0.01 0 0 0.39 0 0.61 0.26 0.15 0.58 62.744 278.428 417885 4683955 6.424 1131 15132.25 +0 0 0 0 "Meadows" 517 0.77 0 0.36 0 0.22 0.24 0.76 0.01 0.67 0.04 0.64 0.05 0.6 0.01 0.56 0.05 0.55 0.07 0.43 0.24 0.33 0.08 0.58 0.34 194.726 389.959 443705 4645295 12.643 734 14896.583 +0 0 0 0 "Meadows" 1015 0 0 0 0 0.44 0 0.19 0.01 0.26 0 0.23 0.02 0.31 0.01 0.36 0 0.3 0.02 0.19 0.62 0.19 0 0.8 0.2 14.636 352.816 457155 4655829 10.562 850 14917.917 +0 0 0 0 "Meadows" 834 0.26 0 0.13 0 0.02 0 0.51 0 0.47 0 0.28 0.05 0.36 0.01 0.36 0.02 0.22 0.05 0.89 0.04 0.07 0.11 0.16 0.73 201.875 63.455 451815 4650335 10.32 864 14962.917 +0 0 0 0 "Meadows" 610 0 0 0 0 0 0 0.08 0 0.03 0 0.01 0 0.11 0 0.12 0 0.08 0 0.05 0.22 0.73 0 0.38 0.62 215.165 107.372 449563 4649244 12.265 755 14910 +0 0 0 0 "Meadows" 558 0 0 0 0.09 0 0.21 0.15 0 0.27 0.01 0.16 0.04 0.22 0 0.29 0.01 0.19 0.02 0.38 0.24 0.38 0.12 0.23 0.65 57.118 2205.312 443897 4654446 12.665 731 14823.75 +0 0 0 0 "Meadows" 610 0.12 0 0.09 0 0.05 0 0.4 0 0.4 0 0.33 0.02 0.34 0 0.39 0 0.32 0.03 0.84 0.02 0.14 0.77 0 0.23 54.071 725.444 442325 4638765 12.187 758 14961.833 +0 0 0 0 "Meadows" 972 0.7 0 0.79 0 0.73 0 0.09 0 0.11 0 0.04 0 0.21 0 0.23 0 0.15 0 0.25 0.06 0.7 0.17 0.11 0.72 138.218 1599.124 427645 4633345 10.936 808 15148.833 +0 0 0 0 "Meadows" 896 0.65 0 0.21 0 0.2 0 0.51 0.01 0.5 0 0.37 0 0.36 0 0.3 0 0.23 0 0.52 0.17 0.31 0.12 0.56 0.32 205.653 1332.761 408590 4678054 10.074 812 15113.417 +0 0 0 0 "Meadows" 1114 1 0 0 0 0 0 0.2 0 0.06 0 0.05 0 0.28 0 0.16 0 0.14 0 1 0 0 0.41 0 0.59 166.663 1972.557 408415 4679185 8.318 971 15111.083 +0 0 0 0 "Meadows" 1172 0.89 0.01 0.17 0.83 0.03 0.9 0.51 0.04 0.52 0.08 0.29 0.13 0.47 0.02 0.45 0.03 0.26 0.05 0.06 0.93 0.01 0.09 0.1 0.81 37.992 239.872 402305 4672875 7.994 951 15166.083 +0 0 0 0 "Meadows" 1462 0.41 0 0.66 0 0 0 0.39 0 0.39 0.16 0.02 0.01 0.18 0 0.18 0.11 0.03 0.01 0.77 0 0.23 0.66 0 0.34 445.831 2.196 401445 4670215 7.994 951 15166.083 +0 0 0 0 "Meadows" 1215 0.06 0 0 0 0 0 0.33 0 0 0.01 0 0 0.21 0 0.05 0.01 0 0 0.63 0.04 0.33 0.16 0.21 0.64 26.398 2577.091 402262 4678030 9.202 868 15112.583 +0 0 0 0 "Meadows" 1438 0 0 0 0 0 0 0.11 0 0.09 0 0.04 0 0.19 0 0.17 0 0.08 0 0.38 0.01 0.61 0.33 0 0.67 54.07 761.494 400605 4674955 9.202 868 15112.583 +0 0 0 0 "Meadows" 1189 0.45 0 0.78 0 0.16 0 0.46 0.02 0.43 0.03 0.3 0.03 0.34 0.01 0.29 0.02 0.19 0.02 0.82 0.11 0.07 0.62 0.06 0.32 32.501 554.092 399515 4676615 8.277 928 15123.833 +0 0 0 0 "Meadows" 995 0.05 0 0.03 0 0 0 0.1 0 0.11 0 0.04 0 0.09 0 0.06 0 0.02 0 0.24 0.15 0.61 0.17 0 0.83 20.654 1392.473 396977 4677336 8.277 928 15123.833 +0 0 0 0 "Meadows" 1647 0 0 0 0 0 0 0.02 0 0 0 0 0 0.02 0 0 0 0 0 0.11 0 0.89 0 0 1 140.642 4010.469 396183 4679919 5.607 1188 15015.667 +0 0 0 0 "Meadows" 1919 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.04 0.93 0.03 0.11 0.77 0.12 131.277 3157.965 402535 4683885 6.389 1099 15263.583 +0 0 0 0 "Meadows" 1380 0.54 0 0 0 0 0 0.12 0 0.09 0 0 0 0.12 0 0.06 0 0.01 0 0.58 0.07 0.35 0.01 0 0.99 81.745 1198.654 410015 4681685 8.318 971 15111.083 +0 0 0 0 "Meadows" 882 0.08 0 0.07 0 0 0 0.35 0 0.29 0 0.23 0 0.42 0.01 0.33 0.01 0.26 0.01 0.08 0.56 0.37 0.07 0.93 0 107.839 3089.475 385895 4627825 11.962 732 15219.833 +0 0 0 0 "Meadows" 696 1 0 0 0 0 0 0.31 0 0.13 0 0.11 0 0.26 0 0.09 0 0.07 0 1 0 0 0.1 0 0.9 38.833 3448.698 386600 4617528 12.392 707 15270.167 +1 0 0 1 "Meadows" 656 0 0 0 0 0 0 0.3 0 0.17 0 0.16 0.01 0.27 0 0.12 0 0.08 0 0 0.25 0.75 0.75 0 0.25 202.845 2642.934 386995 4618248 12.392 707 15270.167 +0 0 0 0 "Meadows" 689 0.31 0 0 0 0 0 0.24 0 0.07 0 0.05 0.02 0.18 0 0.08 0 0.06 0.01 0.31 0 0.69 0 0 1 165.098 249.155 391105 4630335 12.207 722 15184.667 +1 3 0 4 "Meadows" 524 0.1 0 0.19 0 0.13 0 0.47 0.01 0.36 0 0.24 0.03 0.35 0.01 0.26 0.02 0.19 0.03 0.16 0.12 0.73 0.08 0.1 0.83 75.969 753.558 392380 4605172 13.543 640 15206.667 +0 0 0 0 "Meadows" 718 0 0 0 0 0 0 0.13 0 0.07 0 0.06 0.01 0.22 0 0.11 0 0.1 0.01 0 0 1 0 0 1 39.777 297.389 392358 4630544 12.207 722 15184.667 +0 0 0 0 "Meadows" 1746 0 0 0 0 0 0 0 0 0 0 0 0 0.03 0 0 0 0 0 0.02 0.07 0.91 0.01 0.04 0.96 71.711 1115.146 411615 4682565 8.367 985 14955.667 +0 0 0 0 "Meadows" 1872 0 0 0 0 0 0 0.03 0 0 0.03 0 0.01 0.04 0 0 0.02 0 0.01 0 0 1 0.03 0.68 0.3 195.605 209.457 418855 4683765 6.424 1131 15132.25 +0 0 0 0 "Scrublands" 268 0 0 0 0 0 0 0.05 0 0.03 0 0.02 0 0.07 0 0.02 0.01 0.01 0.01 0 0 1 0 0 1 113.675 3546.016 404135 4573105 14.375 662 15151.667 +0 0 0 0 "Scrublands" 290 0 0 0 0 0 0 0 0 0.01 0 0.01 0 0.04 0 0.01 0.02 0.01 0.02 0 1 0 0 0 1 29.934 3718.752 405015 4572825 14.375 662 15151.667 +0 0 0 0 "Scrublands" 338 0 0.22 0 0 0 0 0.04 0.01 0 0.02 0.03 0 0.01 0 0 0.01 0.01 0 0.22 0.39 0.39 0 0 1 61.681 3227.478 407209 4573195 14.047 688 15167.333 +0 0 0 0 "Scrublands" 464 0 0 0 0 0 0 0 0 0 0.06 0 0.1 0 0 0 0.12 0 0.13 0 0.09 0.91 0 0 1 142.777 2499.597 410366 4573252 14.303 684 15105 +0 0 0 0 "Scrublands" 280 0 0 0 0 0 0 0.02 0.02 0 0.02 0 0.03 0.09 0.06 0 0.13 0.01 0.11 0 0.33 0.67 0 0.05 0.95 122.895 226.168 429015 4588915 15.133 654 14940.333 +0 0 0 0 "Scrublands" 508 0.41 0 0 0 0 0 0.56 0.01 0.3 0 0.29 0.02 0.55 0.02 0.39 0 0.36 0.03 0.41 0.56 0.03 0 0.62 0.38 145.476 553.852 395475 4627965 12.967 674 15096.583 +0 1 0 1 "Scrublands" 837 0.09 0 0 0 0 0 0.13 0 0.07 0 0.07 0 0.14 0 0.1 0 0.07 0 0.09 0.91 0 0 0.78 0.22 75.361 2049.214 384540 4626133 11.97 725 15234.5 +0 0 0 0 "Scrublands" 456 0 0 0 0 0 0 0.02 0 0.02 0.02 0.01 0 0.15 0 0.2 0.02 0.15 0.01 0 1 0 0 1 0 98.661 1168.03 394065 4640505 12.985 665 15087 +0 0 0 0 "Scrublands" 428 0 0 0 0 0 0 0.11 0 0.12 0.03 0.08 0.01 0.13 0.01 0.18 0.02 0.12 0.02 0 0.93 0.07 0 0.93 0.07 4.822 750.098 394115 4639855 12.985 665 15087 +0 0 0 0 "Scrublands" 455 0.07 0 0 0 0 0 0.4 0 0.31 0 0.16 0.02 0.4 0.03 0.28 0.06 0.14 0.09 0.07 0.84 0.09 0.09 0 0.91 73.229 649.98 418495 4624515 12.872 691 15094.5 +0 0 0 0 "Scrublands" 367 0.81 0 0.48 0 0.04 0 0.46 0.01 0.17 0 0.11 0.01 0.37 0.01 0.12 0.02 0.08 0.03 0.78 0.07 0.15 0.69 0.07 0.24 37.239 344.395 398310 4610554 13.449 646 15138.5 +0 0 0 0 "Rock outcrops" 977 0 0 0 0 0 0 0 0 0 0 0 0 0 0.01 0 0.01 0 0.02 0.05 0.05 0.9 0.03 0.05 0.92 196.691 1413.648 402525 4604845 12.451 724 15221.333 +0 0 0 0 "Rock outcrops" 888 0 0 0 0 0 0 0 0 0 0 0 0.01 0 0.02 0 0.02 0 0.03 0 0.01 0.99 0 0.01 0.99 143.196 979.071 403205 4604825 14.248 622 15029.25 +0 0 0 0 "Rock outcrops" 601 0 0 0 0 0 0 0 0 0 0 0 0 0.02 0.02 0 0.03 0 0.03 0.01 0.23 0.76 0.14 0.23 0.64 134.645 361.558 404095 4604195 14.545 630 14979 +0 0 0 0 "Rock outcrops" 255 0 0 0 0 0 0 0.02 0 0 0 0 0 0.1 0.01 0.04 0.01 0.03 0.01 0.13 0.56 0.31 0.32 0.15 0.53 59.274 1015.6 405915 4604005 14.545 630 14979 +0 0 0 0 "Rock outcrops" 948 0 0 0 0 0 0 0.09 0.01 0.05 0 0 0.01 0.04 0 0.02 0.01 0 0.02 0.03 0.13 0.84 0.03 0.18 0.79 183.951 2002.618 418335 4609985 12.326 765 15124.75 +0 0 0 0 "Rock outcrops" 748 0 0 0 0 0 0 0.17 0 0.15 0 0.1 0.02 0.2 0.01 0.19 0 0.12 0.02 0.05 0.13 0.83 0.12 0.04 0.84 350.392 1712.338 430635 4623575 11.849 794 15017.583 +0 0 0 0 "Rock outcrops" 1069 0 0 0 0 0 0 0.01 0 0 0 0 0 0.04 0 0.01 0 0.01 0 0.53 0.13 0.33 0.27 0 0.73 109.035 2132.94 448265 4621175 12.823 812 14738.25 +0 0 0 0 "Rock outcrops" 673 0 0 0.19 0 0 0 0.17 0 0.12 0 0.07 0.02 0.33 0 0.26 0 0.21 0.02 0.12 0.41 0.47 0.06 0.22 0.56 176.508 1162.17 439095 4658175 12.39 735 14800.833 +0 0 0 0 "Rock outcrops" 746 0 0 0 0 0 0 0.07 0 0.05 0 0 0 0.08 0 0.06 0 0.03 0 0.81 0 0.19 0.21 0.03 0.76 46.224 2138.555 440245 4660095 10.991 801 14914.667 +0 0 0 0 "Rock outcrops" 555 0.57 0 0.93 0 0.14 0.22 0.37 0.05 0.36 0.07 0.23 0.14 0.41 0.03 0.39 0.07 0.27 0.16 0.47 0.39 0.14 0.61 0.22 0.16 103.515 433.251 438575 4657305 12.39 735 14800.833 +0 0 0 0 "Rock outcrops" 1269 0 0 0 0 0 0 0 0 0 0 0 0 0.04 0 0.02 0 0.02 0 0.02 0.64 0.34 0.21 0.25 0.53 246.676 1911.683 409465 4666945 10.521 803 15021.333 +0 0 0 0 "Rock outcrops" 611 0 0 0 0 0 0 0.27 0.01 0.21 0 0.17 0 0.27 0.01 0.27 0 0.24 0.01 0.2 0 0.8 0.07 0.01 0.92 210.336 1095.446 421275 4653015 12.139 704 14931 +0 0 0 0 "Rock outcrops" 1082 0 0 0 0 0 0 0.07 0 0.02 0.01 0 0.04 0.25 0.02 0.2 0.02 0.12 0.06 0.49 0 0.51 0.77 0 0.23 164.877 15.14 403065 4662535 9.807 877 14878.917 +0 0 0 0 "Rock outcrops" 1436 0 0 0 0 0 0 0.04 0 0.03 0 0 0 0.05 0 0.02 0 0 0 0.08 0.67 0.25 0.43 0.33 0.24 144.464 2301.062 395135 4666005 8.045 961 15171.583 +0 0 0 0 "Rock outcrops" 477 0.51 0 0 0 0.09 0 0.14 0 0.06 0.02 0.07 0 0.1 0 0.06 0.02 0.07 0 0.15 0.3 0.25 0.06 0.5 0.44 273.914 1538.325 403565 4645835 13.035 660 15046.583 +0 0 0 0 "Rock outcrops" 2236 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 89.772 1441.67 412525 4683295 4.679 1279 15123.5 +0 0 0 0 "Rock outcrops" 1942 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 123.688 1382.562 414155 4683915 4.679 1279 15123.5 +0 0 0 0 "Rock outcrops" 1314 0 0 0 0 0 0 0.14 0 0.07 0.01 0.06 0.01 0.12 0 0.07 0.01 0.06 0.01 0.95 0.01 0.04 0.4 0.27 0.33 113.273 128.964 393985 4674445 7.915 970 15036.667 +0 0 0 0 "Rock outcrops" 1319 0 0 0 0.13 0 0.13 0.08 0 0.03 0.02 0.03 0.02 0.15 0 0.11 0.03 0.08 0.01 0.26 0.23 0.52 0.26 0.09 0.65 224.348 89.551 394085 4674735 7.915 970 15036.667 +0 0 0 0 "Rock outcrops" 1652 0 0 0 0 0 0 0.03 0 0 0 0 0 0.02 0 0 0 0 0 0.64 0 0.36 0.05 0.06 0.89 22.932 4477.135 395839 4680393 5.607 1188 15015.667 +0 0 0 0 "Rock outcrops" 1302 0.84 0 0.98 0 0.76 0 0.44 0.01 0.55 0.03 0.24 0.04 0.25 0.01 0.27 0.02 0.14 0.01 0.24 0 0.76 0.22 0 0.78 107.144 56.191 392458 4674807 6.635 1083 15109.333 +0 0 0 0 "Rock outcrops" 1245 0.87 0 1 0 1 0 0.47 0.01 0.53 0.01 0.31 0.01 0.32 0.01 0.3 0.01 0.17 0.01 0 0.13 0.87 0 0 1 323.595 185.642 392595 4674327 6.635 1083 15109.333 +0 0 0 0 "Rock outcrops" 1783 0 0 0 0 0 0 0 0 0 0.11 0 0 0 0 0 0.08 0 0 0.22 0 0.78 0.07 0 0.93 78.291 2757.334 398986 4672721 5.423 1168 15189.667 +0 0 0 0 "Rock outcrops" 1434 0 0 0 0 0 0 0.1 0 0.02 0.23 0.01 0.01 0.12 0 0.11 0.14 0.01 0 0.81 0 0.19 0.29 0.01 0.7 49.922 1727.451 400795 4672455 7.994 951 15166.083 +0 0 0 0 "Rock outcrops" 1392 0 0 0 0 0 0 0.12 0 0.04 0.22 0.01 0.01 0.13 0 0.12 0.13 0.01 0 0.88 0 0.12 0.65 0.01 0.34 65.922 1644.209 400885 4672455 7.994 951 15166.083 +0 0 0 0 "Rock outcrops" 1166 0.32 0 0 0.09 0 0 0.28 0.02 0 0.32 0.04 0 0.19 0.02 0.11 0.14 0.07 0.01 0.35 0.12 0.53 0.32 0.07 0.6 293.819 633.039 403645 4670035 7.994 951 15166.083 +0 0 0 0 "Rock outcrops" 1685 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.17 0.22 0.61 0.2 0.15 0.65 85.649 1961.617 403945 4683765 6.316 1121 15196.917 +0 0 0 0 "Rock outcrops" 1160 0.36 0 0 0 0 0 0.26 0.01 0.23 0 0.15 0.01 0.34 0.01 0.36 0.01 0.16 0.02 0.9 0.08 0.02 0.43 0.04 0.53 532.781 737.556 404405 4673855 10.073 812 15098.833 +0 0 0 0 "Rock outcrops" 1718 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.23 0.3 0.47 0.08 0.49 0.42 163.513 1497.367 404435 4683935 6.316 1121 15196.917 +0 0 0 0 "Rock outcrops" 1122 0 0 0 0.07 0 0.07 0.3 0.01 0.27 0.03 0.13 0.03 0.4 0.02 0.39 0.03 0.16 0.03 0.8 0.14 0.06 0.02 0.08 0.9 332.859 87.983 402951 4674614 9.202 868 15112.583 +0 0 0 0 "Rock outcrops" 990 0.63 0 0.08 0 0.03 0 0.37 0.02 0.3 0.03 0.11 0.03 0.39 0.02 0.38 0.03 0.17 0.03 0.71 0.04 0.15 0.64 0.04 0.32 249.894 108.127 403241 4674379 9.202 868 15112.583 +0 0 0 0 "Rock outcrops" 833 0.36 0 0 0 0 0 0.51 0.09 0.39 0.07 0.25 0.11 0.53 0.06 0.39 0.08 0.21 0.13 1 0 0 0 0 1 73.54 311.004 405785 4678525 9.488 872 15027.417 +0 0 0 0 "Rock outcrops" 756 0.31 0.12 0.31 0.14 0.27 0.18 0.09 0.01 0.06 0.03 0.08 0.05 0.16 0.01 0.16 0.02 0.06 0.03 0.19 0.1 0.61 0.02 0.07 0.81 50.715 53.493 408836 4676685 10.074 812 15113.417 +0 0 0 0 "Rock outcrops" 851 0 0 0 0 0 0 0.08 0.01 0.01 0.01 0 0.02 0.1 0.01 0.03 0.01 0.01 0.02 0.29 0 0.71 0.11 0 0.89 168.06 394.223 411965 4677565 9.342 871 15104.083 +0 0 0 0 "Rock outcrops" 1876 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.2 0.18 0.62 0.04 0.91 0.05 108.588 1407.292 412105 4682595 8.367 985 14955.667 +0 0 0 0 "Rock outcrops" 1227 0 0 0 0 0 0 0.08 0 0 0 0 0 0.05 0.01 0.03 0.01 0.01 0.01 0.31 0.09 0.59 0.12 0.13 0.75 34.988 554.552 416595 4681495 8.686 953 14941.083 +0 0 0 0 "Rock outcrops" 1419 0 0 0 0 0 0 0.32 0 0.18 0 0.18 0 0.15 0 0.11 0 0.11 0 0.55 0.03 0.42 0 0.01 0.99 255.236 2976.867 413495 4681215 8.367 985 14955.667 +0 0 0 0 "Rock outcrops" 839 0.61 0 0.64 0 0.61 0 0.66 0.02 0.47 0.02 0.42 0.02 0.43 0.01 0.32 0.01 0.28 0.02 0.34 0.06 0.6 0.04 0 0.96 178.965 161.289 429736 4630133 11.414 803 15063.167 +0 0 0 0 "Rock outcrops" 535 0 0 0 0 0 0 0.64 0 0.68 0 0.59 0.01 0.69 0.01 0.73 0.01 0.64 0.04 0.15 0.35 0.49 0.15 0.18 0.67 117.259 579.728 439345 4637885 12.733 732 14898.417 +0 0 0 0 "Rock outcrops" 593 0.44 0 0.38 0 0.4 0 0.39 0.01 0.44 0 0.39 0 0.48 0.02 0.52 0.01 0.43 0.04 0.16 0.01 0.82 0.01 0.08 0.89 99.767 885.012 439475 4633035 12.409 748 14981.75 +0 0 0 0 "Rock outcrops" 582 0.31 0 0.37 0 0.04 0.28 0.45 0 0.43 0.07 0.34 0.09 0.54 0.02 0.46 0.14 0.33 0.22 0.26 0.28 0.46 0.13 0.29 0.58 61.766 576.453 439795 4635845 12.409 748 14981.75 +0 0 0 0 "Rock outcrops" 533 0.01 0 0.06 0 0 0 0.53 0.02 0.68 0 0.54 0.06 0.57 0.02 0.68 0 0.57 0.06 0.32 0.1 0.58 0.1 0.01 0.87 232.674 147.485 439905 4653635 12.756 721 14824.833 +0 0 0 0 "Rock outcrops" 514 0.67 0 0.51 0 0.53 0 0.66 0.02 0.81 0 0.61 0.03 0.64 0.02 0.75 0 0.61 0.08 0.2 0.18 0.43 0.05 0.15 0.72 111.831 624.482 440665 4652505 12.756 721 14824.833 +0 0 0 0 "Rock outcrops" 514 0.03 0 0.22 0 0 0.19 0.61 0 0.58 0.04 0.37 0.25 0.73 0.04 0.71 0.11 0.55 0.22 0.04 0.25 0.69 0.03 0.23 0.58 45.43 568.616 441235 4641445 12.961 723 14852.083 +0 0 0 0 "Rock outcrops" 579 0.28 0 0.57 0 0.4 0 0.62 0 0.74 0 0.65 0 0.52 0.01 0.66 0 0.56 0.04 0 0.34 0.64 0 0 0.83 83.138 755.834 441565 4654165 12.665 731 14823.75 +0 0 0 0 "Rock outcrops" 711 0 0 0 0 0 0 0.01 0 0.02 0.01 0.01 0 0.07 0.01 0.1 0.02 0.05 0.04 0.51 0.04 0.46 0.51 0.08 0.41 84.221 667.386 441945 4633475 11.776 786 15034.833 +0 0 0 0 "Rock outcrops" 866 0.01 0 0.05 0 0.06 0 0.1 0 0.16 0 0.09 0 0.21 0.01 0.18 0.02 0.11 0.04 0.12 0.22 0.66 0.01 0.09 0.91 222.387 908.065 452697 4649445 12.24 765 14862.75 +0 0 0 0 "Rock outcrops" 765 0 0 0 0 0 0 0.02 0.01 0 0 0 0.01 0.06 0.01 0.03 0 0.02 0.01 0 0 0.99 0 0 1 77.258 105.815 447297 4642473 11.885 771 14962.5 +0 0 0 0 "Rock outcrops" 772 0 0 0 0 0 0 0.07 0 0 0 0 0 0.11 0.01 0.09 0 0.09 0.01 0.03 0.06 0.91 0 0.01 0.99 202.265 757.741 447235 4641605 11.885 771 14962.5 +0 0 0 0 "Rock outcrops" 854 0 0 0 0 0 0 0.02 0 0 0 0.03 0 0.05 0 0.04 0 0.04 0 0.13 0 0.87 0 0.96 0.04 126.696 893.761 450862 4648514 12.265 755 14910 +0 0 0 0 "Rock outcrops" 1157 0 0 0 0 0 0 0.15 0 0.2 0.01 0.04 0.01 0.19 0 0.29 0 0.15 0 0.39 0.02 0.59 0 0 0.99 84.972 1843.846 452565 4656895 10.232 869 14929.667 +0 0 0 0 "Rock outcrops" 1179 0 0 0 0 0 0 0.01 0 0 0 0 0 0.06 0 0.03 0 0 0 0.19 0.19 0.62 0.56 0.03 0.4 24.753 2867.267 394635 4666675 8.045 961 15171.583 +0 1 0 1 "Rock outcrops" 727 0 0 0 0 0 0 0.11 0.02 0.14 0 0.07 0.02 0.06 0.01 0.08 0 0.03 0.01 0.46 0.16 0.38 0.87 0.13 0 177.767 56.461 443175 4632705 11.776 786 15034.833 +0 0 0 0 "Rock outcrops" 1554 0 0 0 0 0 0 0 0 0 0 0 0 0.02 0 0.01 0.01 0.01 0.02 0.34 0.06 0.6 0.05 0.02 0.93 154.287 323.971 406405 4684075 6.316 1121 15196.917 +0 0 0 0 "Rock outcrops" 2224 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.01 0 0.01 0 0.85 0.15 0 0 1 51.462 723.566 411865 4684875 4.679 1279 15123.5 +0 0 0 0 "Rock outcrops" 2022 0 0 0 0 0 0 0 0 0 0.02 0 0.03 0.02 0 0 0.01 0 0.01 0.15 0.03 0.82 0.03 0.27 0.7 675.384 221.178 410345 4683385 4.657 1289 15117.583 +0 0 0 0 "Rock outcrops" 1179 0 0 0 0 0 0 0.01 0 0 0 0 0 0.05 0 0.03 0 0 0 0.12 0.13 0.75 0.23 0.04 0.73 52.338 2885.214 394585 4666505 8.045 961 15171.583 diff --git a/examples/discovery/data/demo/introduction_pathways_non-native_plants/invasion_success_pathways.csv b/examples/discovery/data/demo/introduction_pathways_non-native_plants/invasion_success_pathways.csv new file mode 100644 index 000000000..a4e4cf2fd --- /dev/null +++ b/examples/discovery/data/demo/introduction_pathways_non-native_plants/invasion_success_pathways.csv @@ -0,0 +1,82 @@ +Scientific_name,family,number.of.invaded.plots,area.of.occupancy.10km,habitat.range,invaded.niche.breadth,mrt,native.niche.breadth,intro.pathway,growth.form,dispersal,vegetative,height,genus,species,genus.relative,duplicated,species.relative,phylo.tip.label +Acer negundo L.,Sapindaceae,3,3,3,0.86,119,0.966,Gardening,Shrub/tree,Wind,No,20,Acer,Acer negundo,NA,FALSE,NA,Acer_negundo +Agave americana L.,Asparagaceae,3,3,3,0.785,250,0.567,Gardening,Shrub/tree,Unspecific,Yes,10,Agave,Agave americana,NA,FALSE,NA,Agave_americana +Ailanthus altissima (Mill.) Swingle,Simaroubaceae,3,2,2,0.968,110,0.706,Gardening,Shrub/tree,Wind,Yes,20,Ailanthus,Ailanthus altissima,NA,FALSE,NA,Ailanthus_altissima +Amaranthus blitoides S. Watson,Amaranthaceae,11,9,3,0.898,100,0.839,Unintentional,Annual fg,Unspecific,No,0.5,Amaranthus,Amaranthus blitoides,NA,FALSE,NA,Amaranthus_blitoides +Amaranthus deflexus L.,Amaranthaceae,5,4,2,1.382,145,0.743,Unintentional,Perennial fg,Unspecific,No,0.6,Amaranthus,Amaranthus deflexus,NA,FALSE,NA,Amaranthus_deflexus +Amaranthus hybridus L.,Amaranthaceae,1,1,1,0,132,1.138,Unintentional,Annual fg,Unspecific,No,2.2,Amaranthus,Amaranthus hybridus,NA,FALSE,NA,Amaranthus_hybridus +Amaranthus muricatus Moq.,Amaranthaceae,1,1,1,0,103,0.71,Unintentional,Perennial fg,Unspecific,Yes,0.5,Amaranthus,Amaranthus muricatus,NA,FALSE,NA,Amaranthus_muricatus +Amaranthus retroflexus L.,Amaranthaceae,35,25,5,1.279,165,0.574,Unintentional,Annual fg,Unspecific,No,1,Amaranthus,Amaranthus retroflexus,NA,FALSE,NA,Amaranthus_retroflexus +Amaranthus viridis L.,Amaranthaceae,2,1,1,0.088,108,1.03,Unintentional,Annual fg,Unspecific,No,0.6,Amaranthus,Amaranthus viridis,NA,FALSE,NA,Amaranthus_viridis +Araujia sericifera Brot.,Apocynaceae,11,6,4,0.391,50,0.588,Gardening,Perennial fg,Wind,Yes,5,Araujia,Araujia sericifera,NA,FALSE,NA,Araujia_sericifera +Artemisia verlotiorum Lamotte,Asteraceae,9,9,4,1.577,99,0.98,Unintentional,Perennial fg,Unspecific,Yes,1.5,Artemisia,Artemisia verlotiorum,NA,FALSE,NA,Artemisia_verlotiorum +Bidens frondosus L.,Asteraceae,2,1,1,0.085,68,0.858,Unintentional,Annual fg,Animal,No,1.5,Bidens,Bidens frondosus,NA,FALSE,NA,Bidens_frondosus +Bidens subalternans DC.,Asteraceae,2,2,2,1.199,77,0.776,Unintentional,Annual fg,Animal,No,2.2,Bidens,Bidens subalternans,NA,FALSE,NA,Bidens_subalternans +Bromus catharticus Vahl,Poaceae,1,1,1,0,100,1.125,Unintentional,Perennial fg,Unspecific,Yes,1.5,Bromus,Bromus catharticus,NA,FALSE,NA,Bromus_catharticus +Canna indica L.,Cannaceae,1,1,1,0,23,1.056,Gardening,Perennial fg,Unspecific,Yes,2,Canna,Canna indica,NA,FALSE,NA,Canna_indica +Carpobrotus edulis (L.) N.E. Br.,Aizoaceae,2,2,1,0.531,100,0.373,Gardening,Perennial fg,Animal,Yes,0.05,Carpobrotus,Carpobrotus edulis,NA,FALSE,NA,Carpobrotus_edulis +Cenchrus longisetus M.C.Johnst. [Pennisetum villosum Fresen.],Poaceae,2,1,1,0,108,0.711,Gardening,Perennial fg,Unspecific,Yes,0.6,Cenchrus,Cenchrus longisetus,NA,FALSE,NA,Cenchrus_longisetus +Cenchrus spinifex Cav.,Poaceae,2,1,1,0.391,40,0.854,Unintentional,Annual fg,Animal,Yes,0.6,Cenchrus,Cenchrus spinifex,NA,FALSE,NA,Cenchrus_spinifex +Cercis siliquastrum L.,Fabaceae,2,2,2,1.39,166,0.54,Gardening,Shrub/tree,Unspecific,No,10,Cercis,Cercis siliquastrum,NA,FALSE,NA,Cercis_siliquastrum +Convolvulus sabatius Viv.,Convolvulaceae,5,4,3,0.943,23,0.754,Gardening,Perennial fg,Unspecific,Yes,0.8,Convolvulus,Convolvulus sabatius,NA,FALSE,NA,Convolvulus_sabatius +Cortaderia selloana (Schult. & Schult. f.) Asch. & Graebn.,Poaceae,5,2,2,0.221,28,1.014,Gardening,Perennial fg,Wind,No,4,Cortaderia,Cortaderia selloana,NA,FALSE,NA,Cortaderia_selloana +Cuscuta campestris Yunck.,Convolvulaceae,3,3,3,1.759,47,1.054,Unintentional,Annual fg,Unspecific,No,1,Cuscuta,Cuscuta campestris,NA,FALSE,NA,Cuscuta_campestris +Cyperus eragrostis Lam.,Cyperaceae,7,5,4,1.096,75,0.831,Unintentional,Perennial fg,Unspecific,Yes,1,Cyperus,Cyperus eragrostis,NA,FALSE,NA,Cyperus_eragrostis +Datura stramonium L.,Solanaceae,1,1,1,0,166,0.784,Gardening,Annual fg,Unspecific,No,2,Datura,Datura stramonium,NA,FALSE,NA,Datura_stramonium +Dichondra micrantha Urb.,Convolvulaceae,1,1,1,0,23,0.794,Gardening,Perennial fg,Unspecific,Yes,0.04,Dichondra,Dichondra micrantha,NA,FALSE,NA,Dichondra_micrantha +Dysphania ambrosioides (L.) Mosyakin & Clemants,Amaranthaceae,15,12,4,1.152,248,1.092,Gardening,Annual fg,Unspecific,No,0.8,Dysphania,Dysphania ambrosioides,NA,FALSE,NA,Dysphania_ambrosioides +Eleusine indica (L.) Gaertn. subsp. indica,Poaceae,1,1,1,0,104,1.248,Unintentional,Annual fg,Unspecific,Yes,0.8,Eleusine,Eleusine indica,NA,FALSE,NA,Eleusine_indica +Eleusine tristachya (Lam.) Lam.,Poaceae,3,3,1,0.701,135,0.683,Unintentional,Perennial fg,Unspecific,No,0.4,Eleusine,Eleusine tristachya,NA,FALSE,NA,Eleusine_tristachya +Erigeron bonariensis L. [Conyza bonariensis (L.) Cronq.],Asteraceae,15,10,4,1.295,142,1.092,Unintentional,Annual fg,Wind,No,0.6,Erigeron,Erigeron bonariensis,NA,FALSE,NA,Erigeron_bonariensis +Erigeron canadensis L. [Conyza canadensis (L.) Cronq.],Asteraceae,29,17,7,1.12,228,1.215,Unintentional,Annual fg,Wind,No,0.8,Erigeron,Erigeron canadensis,NA,FALSE,NA,Erigeron_canadensis +Erigeron sumatrensis Retz. [Conyza sumatrensis (Retz.) E. Walker],Asteraceae,53,25,6,1.005,108,1.147,Unintentional,Annual fg,Wind,No,2.5,Erigeron,Erigeron sumatrensis,NA,FALSE,NA,Erigeron_sumatrensis +Eriobotrya japonica (Thunb.) Lindl.,Rosaceae,2,2,2,0.863,97,0.705,AgriForest,Shrub/tree,Animal,No,10,Eriobotrya,Eriobotrya japonica,NA,FALSE,NA,Eriobotrya_japonica +Eucalyptus camaldulensis Dehnh.,Myrtaceae,1,1,1,0,35,0.454,AgriForest,Shrub/tree,Unspecific,Yes,20,Eucalyptus,Eucalyptus camaldulensis,NA,FALSE,NA,Eucalyptus_camaldulensis +Eucalyptus camaldulensis Dehnh.,Myrtaceae,1,1,1,0,35,0.454,Gardening,Shrub/tree,Unspecific,Yes,20,Eucalyptus,Eucalyptus camaldulensis2,NA,TRUE,Eucalyptus camaldulensis,Eucalyptus_camaldulensis2 +Euphorbia prostrata Aiton,Euphorbiaceae,4,3,1,0.871,100,1.093,Unintentional,Annual fg,Unspecific,No,0.3,Euphorbia,Euphorbia prostrata,NA,FALSE,NA,Euphorbia_prostrata +Galinsoga parviflora Cav.,Asteraceae,2,2,2,1.303,40,0.777,Unintentional,Annual fg,Unspecific,No,0.8,Galinsoga,Galinsoga parviflora,NA,FALSE,NA,Galinsoga_parviflora +Gomphocarpus fruticosus (L.) W.T. Aiton,Apocynaceae,2,2,2,1.046,166,0.67,Gardening,Shrub/tree,Wind,No,2,Gomphocarpus,Gomphocarpus fruticosus,NA,FALSE,NA,Gomphocarpus_fruticosus +Helianthus tuberosus L.,Asteraceae,2,2,2,0.774,228,0.628,AgriForest,Perennial fg,Unspecific,Yes,3,Helianthus,Helianthus tuberosus,NA,FALSE,NA,Helianthus_tuberosus +Iris Γ—germanica L.,Iridaceae,1,1,1,0,135,0.529,Gardening,Perennial fg,Unspecific,Yes,0.9,Iris,Iris x germanica,NA,FALSE,NA,Iris_x_germanica +Jarava plumosa (Spreng.) S.W.L. Jacobs & J. Everet,Poaceae,1,1,1,0,29,0.526,Unintentional,Perennial fg,Animal,Yes,0.5,Jarava,Jarava plumosa,Calamagrostis,FALSE,NA,Jarava_plumosa +Lepidium didymum L. [Coronopus didymus (L.) Sm.],Brassicaceae,3,2,3,0.277,166,0.969,Unintentional,Annual fg,Unspecific,No,0.4,Lepidium,Lepidium didymum,NA,FALSE,NA,Lepidium_didymum +Ligustrum lucidum W.T. Aiton,Oleaceae,5,5,3,0.61,19,0.653,Gardening,Shrub/tree,Animal,No,10,Ligustrum,Ligustrum lucidum,NA,FALSE,NA,Ligustrum_lucidum +Lonicera japonica Thunb.,Caprifoliaceae,7,5,4,0.68,51,0.683,Gardening,Perennial fg,Animal,Yes,5,Lonicera,Lonicera japonica,NA,FALSE,NA,Lonicera_japonica +Lunaria annua L. subsp. annua,Brassicaceae,1,1,1,0,166,0.38,Gardening,Perennial fg,Wind,No,1,Lunaria,Lunaria annua,NA,FALSE,NA,Lunaria_annua +Malcolmia maritima (L.) R. Br.,Brassicaceae,1,1,1,0,138,0.262,Gardening,Annual fg,Unspecific,No,0.35,Malcolmia,Malcolmia maritima,NA,FALSE,NA,Malcolmia_maritima +Mirabilis jalapa L.,Nyctaginaceae,5,4,1,0.729,127,0.988,Gardening,Perennial fg,Unspecific,Yes,1,Mirabilis,Mirabilis jalapa,NA,FALSE,NA,Mirabilis_jalapa +Myoporum laetum G. Forst.,Scrophulariaceae,2,1,1,0,6,0.497,Gardening,Shrub/tree,Animal,No,8,Myoporum,Myoporum laetum,NA,FALSE,NA,Myoporum_laetum +Nicotiana glauca Graham,Solanaceae,1,1,1,0,104,0.873,Gardening,Shrub/tree,Wind,Yes,5,Nicotiana,Nicotiana glauca,NA,FALSE,NA,Nicotiana_glauca +Opuntia ficus-indica (L.) Mill.,Cactaceae,2,2,2,0.625,228,0.441,AgriForest,Shrub/tree,Animal,Yes,5,Opuntia,Opuntia ficus-indica,NA,FALSE,NA,Opuntia_ficus-indica +Opuntia ficus-indica (L.) Mill.,Cactaceae,2,2,2,0.625,228,0.441,Gardening,Shrub/tree,Animal,Yes,5,Opuntia,Opuntia ficus-indica2,NA,TRUE,Opuntia ficus-indica,Opuntia_ficus-indica2 +Oxalis articulata Savigny,Oxalidaceae,3,2,3,0.849,95,0.713,Gardening,Perennial fg,Unspecific,Yes,0.3,Oxalis,Oxalis articulata,NA,FALSE,NA,Oxalis_articulata +Oxalis corniculata L.,Oxalidaceae,3,3,2,1.071,100,0.92,Unintentional,Perennial fg,Unspecific,Yes,0.3,Oxalis,Oxalis corniculata,NA,FALSE,NA,Oxalis_corniculata +Oxalis debilis Kunth,Oxalidaceae,12,7,4,0.762,86,0.881,Gardening,Perennial fg,Unspecific,Yes,0.3,Oxalis,Oxalis debilis,NA,FALSE,NA,Oxalis_debilis +Oxalis debilis Kunth,Oxalidaceae,12,7,4,0.762,86,0.881,Unintentional,Perennial fg,Unspecific,Yes,0.3,Oxalis,Oxalis debilis2,NA,TRUE,Oxalis debilis,Oxalis_debilis2 +Parthenocissus inserta (A. Kern.) Fritsch,Vitaceae,2,2,2,0.363,36,0.82,Gardening,Perennial fg,Animal,Yes,3,Parthenocissus,Parthenocissus inserta,NA,FALSE,NA,Parthenocissus_inserta +Phoenix canariensis Chabaud,Arecaceae,4,2,2,0.767,23,0.305,Gardening,Shrub/tree,Animal,No,20,Phoenix,Phoenix canariensis,NA,FALSE,NA,Phoenix_canariensis +Phytolacca americana L.,Phytolaccaceae,12,8,3,1.307,166,0.781,Gardening,Perennial fg,Animal,No,3,Phytolacca,Phytolacca americana,NA,FALSE,NA,Phytolacca_americana +Pinus radiata D. Don,Pinaceae,1,1,1,0,62,0.578,AgriForest,Shrub/tree,Wind,No,40,Pinus,Pinus radiata,NA,FALSE,NA,Pinus_radiata +Pittosporum tobira (Thunb.) W.T. Aiton,Pittosporaceae,6,2,2,0.122,23,0.726,Gardening,Shrub/tree,Animal,No,6,Pittosporum,Pittosporum tobira,NA,FALSE,NA,Pittosporum_tobira +Populus deltoides Marshall,Salicaceae,5,5,1,0.952,48,0.92,AgriForest,Shrub/tree,Wind,No,10,Populus,Populus deltoides,NA,FALSE,NA,Populus_deltoides +Populus deltoides Marshall,Salicaceae,5,5,1,0.952,48,0.92,Gardening,Shrub/tree,Wind,No,10,Populus,Populus deltoides2,NA,TRUE,Populus deltoides,Populus_deltoides2 +Portulaca grandiflora Hook.,Portulacaceae,3,1,2,0.147,23,0.844,Gardening,Annual fg,Unspecific,No,0.2,Portulaca,Portulaca grandiflora,NA,FALSE,NA,Portulaca_grandiflora +Prunus armeniaca L.,Rosaceae,1,1,1,0,28,0.715,AgriForest,Shrub/tree,Animal,No,10,Prunus,Prunus armeniaca,NA,FALSE,NA,Prunus_armeniaca +Prunus cerasifera Ehrh.,Rosaceae,1,1,1,0,36,0.671,Gardening,Shrub/tree,Animal,No,8,Prunus,Prunus cerasifera,NA,FALSE,NA,Prunus_cerasifera +Ricinus communis L.,Euphorbiaceae,1,1,1,0,109,0.663,Gardening,Shrub/tree,Unspecific,No,10,Ricinus,Ricinus communis,NA,FALSE,NA,Ricinus_communis +Robinia pseudoacacia L.,Fabaceae,20,13,5,1.076,140,0.31,Gardening,Shrub/tree,Unspecific,Yes,25,Robinia,Robinia pseudoacacia,NA,FALSE,NA,Robinia_pseudoacacia +Rumex cristatus DC.,Polygonaceae,2,2,2,0.586,16,0.398,Unintentional,Perennial fg,Unspecific,No,2,Rumex,Rumex cristatus,Rheum,FALSE,NA,Rumex_cristatus +Salix babylonica L.,Salicaceae,1,1,1,0,48,0.681,Gardening,Shrub/tree,Wind,No,15,Salix,Salix babylonica,NA,FALSE,NA,Salix_babylonica +Senecio angulatus L. f.,Asteraceae,2,2,2,0.695,23,0.408,Gardening,Perennial fg,Wind,Yes,2,Senecio,Senecio angulatus,NA,FALSE,NA,Senecio_angulatus +Senecio pterophorus DC.,Asteraceae,1,1,1,0,30,0.405,Unintentional,Shrub/tree,Wind,No,2,Senecio,Senecio pterophorus,NA,FALSE,NA,Senecio_pterophorus +Solanum lycopersicum L.,Solanaceae,4,4,3,1.513,139,1.216,AgriForest,Annual fg,Animal,No,2,Solanum,Solanum lycopersicum,NA,FALSE,NA,Solanum_lycopersicum +Sorghum halepense (L.) Pers.,Poaceae,16,11,5,0.799,166,0.947,Unintentional,Perennial fg,Unspecific,Yes,2.2,Sorghum,Sorghum halepense,NA,FALSE,NA,Sorghum_halepense +Stenotaphrum secundatum (Walter) Kuntze,Poaceae,3,2,2,0.117,31,1.02,Gardening,Perennial fg,Unspecific,Yes,0.4,Stenotaphrum,Stenotaphrum secundatum,NA,FALSE,NA,Stenotaphrum_secundatum +Symphyotrichum pilosum (Willd.) G.L. Nesom,Asteraceae,1,1,1,0,80,0.521,Gardening,Perennial fg,Wind,Yes,2,Symphyotrichum,Symphyotrichum pilosum,NA,FALSE,NA,Symphyotrichum_pilosum +Symphyotrichum squamatum (Spreng.) G.L. Nesom,Asteraceae,34,17,6,1.041,100,0.783,Unintentional,Annual fg,Wind,No,1,Symphyotrichum,Symphyotrichum squamatum,NA,FALSE,NA,Symphyotrichum_squamatum +Tamarix parviflora DC.,Tamaricaceae,1,1,1,0,23,0.44,Gardening,Shrub/tree,Unspecific,No,3,Tamarix,Tamarix parviflora,NA,FALSE,NA,Tamarix_parviflora +Washingtonia filifera (AndrΓ©) de Bary [incl. W. robusta H. Wendl.],Arecaceae,1,1,1,0,23,0.445,Gardening,Shrub/tree,Animal,No,25,Washingtonia,Washingtonia filifera,NA,FALSE,NA,Washingtonia_filifera +Xanthium orientale L.,Asteraceae,13,8,4,0.789,135,1.02,Unintentional,Annual fg,Animal,No,0.8,Xanthium,Xanthium orientale,NA,FALSE,NA,Xanthium_orientale +Xanthium spinosum L.,Asteraceae,15,11,4,1.1,166,0.873,Unintentional,Annual fg,Animal,No,1,Xanthium,Xanthium spinosum,NA,FALSE,NA,Xanthium_spinosum +Yucca aloifolia L.,Asparagaceae,3,2,2,1.015,23,0.456,Gardening,Shrub/tree,Unspecific,Yes,8,Yucca,Yucca aloifolia,NA,FALSE,NA,Yucca_aloifolia +Zea mays L. subsp. mays,Poaceae,3,2,1,0.881,93,0.818,AgriForest,Annual fg,Unspecific,No,4,Zea,Zea mays,NA,FALSE,NA,Zea_mays diff --git a/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_0.json b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_0.json new file mode 100644 index 000000000..39b325fb8 --- /dev/null +++ b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_0.json @@ -0,0 +1,305 @@ +{ + "id": 0, + "domain": "biology", + "workflow_tags": "statistical tests", + "domain_knowledge": "", + "datasets": [ + { + "name": "temporal_trends_contingency_table.csv", + "description": "Dataset contains temporal trends in the introduction pathways of non-native flora (plants) in the region of Catalonia.", + "columns": { + "raw": [ + { + "name": "introduction.period", + "description": "This column represents different time periods, related to when non-native plant species were introduced into the region." + }, + { + "name": "pathway", + "description": "This column represents different modes or routes through which non-native plant species were introduced, such as \"AgriForest\" (agriculture and forestry - plants introduced for cultivation to provide food or timber), \"Gardening\" (plants introduced for cultivation as ornamentals or for medicinal use), and \"Unintentional\" (plants introduced accidentally with the sowing of contaminated seed lots, global trade and tourism)." + }, + { + "name": "n", + "description": "This column represents the frequency or count of non-native plant species introductions for each combination of introduction period and pathway." + } + ] + } + }, + { + "name": "phylogenetic_tree.txt", + "description": "This file contains a phylogenetic tree in Newick format, illustrating the evolutionary relationships among diverse plant species. It features species from multiple families, with branch lengths denoting genetic distances or evolutionary times.", + "columns": { + "raw": [ + { + "name": "", + "description": "" + }] + } + }, + { + "name": "invasion_success_pathways.csv", + "description": "This dataset provides comprehensive information on various non-native plant species that have invaded plots in the Barcelona province. The dataset includes detailed information on the ecological and biological characteristics of each species, the extent of their invasion, and the pathways through which they were introduced.", + "columns": { + "raw": [ + { + "name": "Scientific_name", + "description": "The full scientific name of the non-native plant species, including the author citation." + }, + { + "name": "family", + "description": "The botanical family to which the species belongs." + }, + { + "name": "number.of.invaded.plots", + "description": "The number of different plots in the Barcelona province where the species has been observed to invade." + }, + { + "name": "area.of.occupancy.10km", + "description": "The area of occupancy at a 10 km scale, indicating how widespread the species is within the studied region." + }, + { + "name": "habitat.range", + "description": "The number of different habitat types the species can occupy." + }, + { + "name": "invaded.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species within the invaded region, indicating its ecological versatility." + }, + { + "name": "mrt", + "description": "The mean residence time of the species, indicating how long the species has been present in the new region (in years)." + }, + { + "name": "native.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species in its native range." + }, + { + "name": "intro.pathway", + "description": "The pathway through which the species was introduced to the new region." + }, + { + "name": "growth.form", + "description": "The growth form of the plant species, describing its general structural category." + }, + { + "name": "dispersal", + "description": "The primary method of dispersal for the species." + }, + { + "name": "vegetative", + "description": "Indicates whether the species can reproduce vegetatively (Yes or No)." + }, + { + "name": "height", + "description": "The typical height of the species in meters." + }, + { + "name": "genus", + "description": "The genus to which the species belongs." + }, + { + "name": "species", + "description": "The specific epithet of the species." + }, + { + "name": "genus.relative", + "description": "Data relative to the genus, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "duplicated", + "description": "Indicates if the species data is duplicated within the dataset (TRUE or FALSE)." + }, + { + "name": "species.relative", + "description": "Data relative to the species, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "phylo.tip.label", + "description": "The label used for the species in the phylogenetic tree included in the dataset." + } + ] + } + }, + { + "name": "invaded_niche_pathways.csv", + "description": "The data for the analysis of pathway-specific differences in the invaded niche.", + "columns": { + "raw": [ + { + "name": "n.gard", + "description": "The count or frequency of non-native plant species introduced through the \"Gardening\" pathway." + }, + { + "name": "n.unint", + "description": "The count or frequency of non-native plant species introduced through the \"Unintentional\" pathway" + }, + { + "name": "n.agfo", + "description": "The count or frequency of non-native plant species introduced through the \"AgriForest\" (Agriculture and Forestry) pathway" + }, + { + "name": "n.total", + "description": "The total count or frequency of non-native plant species across all introduction pathways." + }, + { + "name": "habitat", + "description": "A categorical variable representing the habitat type, selected from the ten most widespread habitat types in the Barcelona province" + }, + { + "name": "elevation", + "description": "The elevation (in meters) of the sampled plot or location" + }, + { + "name": "cropland.1956.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.50m", + "description": "This variable represent the percentage of urban land cover in a buffer (within a 50-meter radius) of 50 meters around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "cropland.2009.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "urban.2009.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "cropland.1956.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.500m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.500m", + "description": "This variable represents the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "cropland.1956.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "urban.1956.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "cropland.1993.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "urban.1993.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "progressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "progressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "distance.stream", + "description": "The distance (in meters) from the sampled plot or location to the nearest main stream or waterway." + }, + { + "name": "distance.road", + "description": "The distance (in meters) from the sampled plot or location to the nearest main road." + }, + { + "name": "longitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "latitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "annual.temperature", + "description": "The mean annual temperature of the sampled plot or location." + }, + { + "name": "annual.rainfall", + "description": "The annual precipitation or rainfall of the sampled plot or location." + }, + { + "name": "annual.radiation", + "description": "The mean annual solar radiation (in kJ/m^2/day) of the sampled plot or location." + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "Over which time period has gardening surpassed agriculture as the main contributor to the non-native flora?" + }, + { + "qid": 1, + "question_type": "variables", + "question": "What activity has replaced agriculture as the main contributor to the non-native flora over the past millennium?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "In what way has gardening impacted agriculture as a contributor to non-native flora over the past millennium?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_1.json b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_1.json new file mode 100644 index 000000000..8e01299ca --- /dev/null +++ b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_1.json @@ -0,0 +1,299 @@ +{ + "id": 0, + "domain": "biology", + "workflow_tags": "regression", + "domain_knowledge": "", + "datasets": [ + { + "name": "temporal_trends_contingency_table.csv", + "description": "Dataset contains temporal trends in the introduction pathways of non-native flora (plants) in the region of Catalonia.", + "columns": { + "raw": [ + { + "name": "introduction.period", + "description": "This column represents different time periods, related to when non-native plant species were introduced into the region." + }, + { + "name": "pathway", + "description": "This column represents different modes or routes through which non-native plant species were introduced, such as \"AgriForest\" (agriculture and forestry - plants introduced for cultivation to provide food or timber), \"Gardening\" (plants introduced for cultivation as ornamentals or for medicinal use), and \"Unintentional\" (plants introduced accidentally with the sowing of contaminated seed lots, global trade and tourism)." + }, + { + "name": "n", + "description": "This column represents the frequency or count of non-native plant species introductions for each combination of introduction period and pathway." + } + ] + } + }, + { + "name": "phylogenetic_tree.txt", + "description": "This file contains a phylogenetic tree in Newick format, illustrating the evolutionary relationships among diverse plant species. It features species from multiple families, with branch lengths denoting genetic distances or evolutionary times.", + "columns": { + "raw": [{ + "name": "", + "description": "" + }] + } + }, + { + "name": "invasion_success_pathways.csv", + "description": "This dataset provides comprehensive information on various non-native plant species that have invaded plots in the Barcelona province. The dataset includes detailed information on the ecological and biological characteristics of each species, the extent of their invasion, and the pathways through which they were introduced.", + "columns": { + "raw": [ + { + "name": "Scientific_name", + "description": "The full scientific name of the non-native plant species, including the author citation." + }, + { + "name": "family", + "description": "The botanical family to which the species belongs." + }, + { + "name": "number.of.invaded.plots", + "description": "The number of different plots in the Barcelona province where the species has been observed to invade." + }, + { + "name": "area.of.occupancy.10km", + "description": "The area of occupancy at a 10 km scale, indicating how widespread the species is within the studied region." + }, + { + "name": "habitat.range", + "description": "The number of different habitat types the species can occupy." + }, + { + "name": "invaded.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species within the invaded region, indicating its ecological versatility." + }, + { + "name": "mrt", + "description": "The mean residence time of the species, indicating how long the species has been present in the new region (in years)." + }, + { + "name": "native.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species in its native range." + }, + { + "name": "intro.pathway", + "description": "The pathway through which the species was introduced to the new region." + }, + { + "name": "growth.form", + "description": "The growth form of the plant species, describing its general structural category." + }, + { + "name": "dispersal", + "description": "The primary method of dispersal for the species." + }, + { + "name": "vegetative", + "description": "Indicates whether the species can reproduce vegetatively (Yes or No)." + }, + { + "name": "height", + "description": "The typical height of the species in meters." + }, + { + "name": "genus", + "description": "The genus to which the species belongs." + }, + { + "name": "species", + "description": "The specific epithet of the species." + }, + { + "name": "genus.relative", + "description": "Data relative to the genus, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "duplicated", + "description": "Indicates if the species data is duplicated within the dataset (TRUE or FALSE)." + }, + { + "name": "species.relative", + "description": "Data relative to the species, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "phylo.tip.label", + "description": "The label used for the species in the phylogenetic tree included in the dataset." + } + ] + } + }, + { + "name": "invaded_niche_pathways.csv", + "description": "The data for the analysis of pathway-specific differences in the invaded niche.", + "columns": { + "raw": [ + { + "name": "n.gard", + "description": "The count or frequency of non-native plant species introduced through the \"Gardening\" pathway." + }, + { + "name": "n.unint", + "description": "The count or frequency of non-native plant species introduced through the \"Unintentional\" pathway" + }, + { + "name": "n.agfo", + "description": "The count or frequency of non-native plant species introduced through the \"AgriForest\" (Agriculture and Forestry) pathway" + }, + { + "name": "n.total", + "description": "The total count or frequency of non-native plant species across all introduction pathways." + }, + { + "name": "habitat", + "description": "A categorical variable representing the habitat type, selected from the ten most widespread habitat types in the Barcelona province" + }, + { + "name": "elevation", + "description": "The elevation (in meters) of the sampled plot or location" + }, + { + "name": "cropland.1956.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.50m", + "description": "This variable represent the percentage of urban land cover in a buffer (within a 50-meter radius) of 50 meters around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "cropland.2009.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "urban.2009.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "cropland.1956.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.500m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.500m", + "description": "This variable represents the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "cropland.1956.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "urban.1956.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "cropland.1993.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "urban.1993.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "progressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "progressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "distance.stream", + "description": "The distance (in meters) from the sampled plot or location to the nearest main stream or waterway." + }, + { + "name": "distance.road", + "description": "The distance (in meters) from the sampled plot or location to the nearest main road." + }, + { + "name": "longitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "latitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "annual.temperature", + "description": "The mean annual temperature of the sampled plot or location." + }, + { + "name": "annual.rainfall", + "description": "The annual precipitation or rainfall of the sampled plot or location." + }, + { + "name": "annual.radiation", + "description": "The mean annual solar radiation (in kJ/m^2/day) of the sampled plot or location." + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "variables", + "question": "What are the variables between which a positive relationship is quantified by a coefficient of 0.22?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What is the nature of the relationship between the degree of urban land use and the proportion of gardening-introduced non-native plants?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_2.json b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_2.json new file mode 100644 index 000000000..1b9297ddd --- /dev/null +++ b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_2.json @@ -0,0 +1,299 @@ +{ + "id": 0, + "domain": "biology", + "workflow_tags": "statistical tests, ecological modelling", + "domain_knowledge": "", + "datasets": [ + { + "name": "temporal_trends_contingency_table.csv", + "description": "Dataset contains temporal trends in the introduction pathways of non-native flora (plants) in the region of Catalonia.", + "columns": { + "raw": [ + { + "name": "introduction.period", + "description": "This column represents different time periods, related to when non-native plant species were introduced into the region." + }, + { + "name": "pathway", + "description": "This column represents different modes or routes through which non-native plant species were introduced, such as \"AgriForest\" (agriculture and forestry - plants introduced for cultivation to provide food or timber), \"Gardening\" (plants introduced for cultivation as ornamentals or for medicinal use), and \"Unintentional\" (plants introduced accidentally with the sowing of contaminated seed lots, global trade and tourism)." + }, + { + "name": "n", + "description": "This column represents the frequency or count of non-native plant species introductions for each combination of introduction period and pathway." + } + ] + } + }, + { + "name": "phylogenetic_tree.txt", + "description": "This file contains a phylogenetic tree in Newick format, illustrating the evolutionary relationships among diverse plant species. It features species from multiple families, with branch lengths denoting genetic distances or evolutionary times.", + "columns": { + "raw": [{ + "name": "", + "description": "" + }] + } + }, + { + "name": "invasion_success_pathways.csv", + "description": "This dataset provides comprehensive information on various non-native plant species that have invaded plots in the Barcelona province. The dataset includes detailed information on the ecological and biological characteristics of each species, the extent of their invasion, and the pathways through which they were introduced.", + "columns": { + "raw": [ + { + "name": "Scientific_name", + "description": "The full scientific name of the non-native plant species, including the author citation." + }, + { + "name": "family", + "description": "The botanical family to which the species belongs." + }, + { + "name": "number.of.invaded.plots", + "description": "The number of different plots in the Barcelona province where the species has been observed to invade." + }, + { + "name": "area.of.occupancy.10km", + "description": "The area of occupancy at a 10 km scale, indicating how widespread the species is within the studied region." + }, + { + "name": "habitat.range", + "description": "The number of different habitat types the species can occupy." + }, + { + "name": "invaded.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species within the invaded region, indicating its ecological versatility." + }, + { + "name": "mrt", + "description": "The mean residence time of the species, indicating how long the species has been present in the new region (in years)." + }, + { + "name": "native.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species in its native range." + }, + { + "name": "intro.pathway", + "description": "The pathway through which the species was introduced to the new region." + }, + { + "name": "growth.form", + "description": "The growth form of the plant species, describing its general structural category." + }, + { + "name": "dispersal", + "description": "The primary method of dispersal for the species." + }, + { + "name": "vegetative", + "description": "Indicates whether the species can reproduce vegetatively (Yes or No)." + }, + { + "name": "height", + "description": "The typical height of the species in meters." + }, + { + "name": "genus", + "description": "The genus to which the species belongs." + }, + { + "name": "species", + "description": "The specific epithet of the species." + }, + { + "name": "genus.relative", + "description": "Data relative to the genus, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "duplicated", + "description": "Indicates if the species data is duplicated within the dataset (TRUE or FALSE)." + }, + { + "name": "species.relative", + "description": "Data relative to the species, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "phylo.tip.label", + "description": "The label used for the species in the phylogenetic tree included in the dataset." + } + ] + } + }, + { + "name": "invaded_niche_pathways.csv", + "description": "The data for the analysis of pathway-specific differences in the invaded niche.", + "columns": { + "raw": [ + { + "name": "n.gard", + "description": "The count or frequency of non-native plant species introduced through the \"Gardening\" pathway." + }, + { + "name": "n.unint", + "description": "The count or frequency of non-native plant species introduced through the \"Unintentional\" pathway" + }, + { + "name": "n.agfo", + "description": "The count or frequency of non-native plant species introduced through the \"AgriForest\" (Agriculture and Forestry) pathway" + }, + { + "name": "n.total", + "description": "The total count or frequency of non-native plant species across all introduction pathways." + }, + { + "name": "habitat", + "description": "A categorical variable representing the habitat type, selected from the ten most widespread habitat types in the Barcelona province" + }, + { + "name": "elevation", + "description": "The elevation (in meters) of the sampled plot or location" + }, + { + "name": "cropland.1956.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.50m", + "description": "This variable represent the percentage of urban land cover in a buffer (within a 50-meter radius) of 50 meters around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "cropland.2009.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "urban.2009.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "cropland.1956.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.500m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.500m", + "description": "This variable represents the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "cropland.1956.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "urban.1956.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "cropland.1993.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "urban.1993.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "progressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "progressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "distance.stream", + "description": "The distance (in meters) from the sampled plot or location to the nearest main stream or waterway." + }, + { + "name": "distance.road", + "description": "The distance (in meters) from the sampled plot or location to the nearest main road." + }, + { + "name": "longitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "latitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "annual.temperature", + "description": "The mean annual temperature of the sampled plot or location." + }, + { + "name": "annual.rainfall", + "description": "The annual precipitation or rainfall of the sampled plot or location." + }, + { + "name": "annual.radiation", + "description": "The mean annual solar radiation (in kJ/m^2/day) of the sampled plot or location." + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "How does the prevalence of non-native plants introduced via gardening vary based on habitat type?" + }, + { + "qid": 1, + "question_type": "variables", + "question": "In what context does the gardening pathway for introducing non-native plants show different prevalence styles across habitat types?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_3.json b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_3.json new file mode 100644 index 000000000..62ee3e125 --- /dev/null +++ b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_3.json @@ -0,0 +1,304 @@ +{ + "id": 0, + "domain": "biology", + "workflow_tags": "statistical tests, interaction analysis, regression", + "domain_knowledge": "", + "datasets": [ + { + "name": "temporal_trends_contingency_table.csv", + "description": "Dataset contains temporal trends in the introduction pathways of non-native flora (plants) in the region of Catalonia.", + "columns": { + "raw": [ + { + "name": "introduction.period", + "description": "This column represents different time periods, related to when non-native plant species were introduced into the region." + }, + { + "name": "pathway", + "description": "This column represents different modes or routes through which non-native plant species were introduced, such as \"AgriForest\" (agriculture and forestry - plants introduced for cultivation to provide food or timber), \"Gardening\" (plants introduced for cultivation as ornamentals or for medicinal use), and \"Unintentional\" (plants introduced accidentally with the sowing of contaminated seed lots, global trade and tourism)." + }, + { + "name": "n", + "description": "This column represents the frequency or count of non-native plant species introductions for each combination of introduction period and pathway." + } + ] + } + }, + { + "name": "phylogenetic_tree.txt", + "description": "This file contains a phylogenetic tree in Newick format, illustrating the evolutionary relationships among diverse plant species. It features species from multiple families, with branch lengths denoting genetic distances or evolutionary times.", + "columns": { + "raw": [{ + "name": "", + "description": "" + }] + } + }, + { + "name": "invasion_success_pathways.csv", + "description": "This dataset provides comprehensive information on various non-native plant species that have invaded plots in the Barcelona province. The dataset includes detailed information on the ecological and biological characteristics of each species, the extent of their invasion, and the pathways through which they were introduced.", + "columns": { + "raw": [ + { + "name": "Scientific_name", + "description": "The full scientific name of the non-native plant species, including the author citation." + }, + { + "name": "family", + "description": "The botanical family to which the species belongs." + }, + { + "name": "number.of.invaded.plots", + "description": "The number of different plots in the Barcelona province where the species has been observed to invade." + }, + { + "name": "area.of.occupancy.10km", + "description": "The area of occupancy at a 10 km scale, indicating how widespread the species is within the studied region." + }, + { + "name": "habitat.range", + "description": "The number of different habitat types the species can occupy." + }, + { + "name": "invaded.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species within the invaded region, indicating its ecological versatility." + }, + { + "name": "mrt", + "description": "The mean residence time of the species, indicating how long the species has been present in the new region (in years)." + }, + { + "name": "native.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species in its native range." + }, + { + "name": "intro.pathway", + "description": "The pathway through which the species was introduced to the new region." + }, + { + "name": "growth.form", + "description": "The growth form of the plant species, describing its general structural category." + }, + { + "name": "dispersal", + "description": "The primary method of dispersal for the species." + }, + { + "name": "vegetative", + "description": "Indicates whether the species can reproduce vegetatively (Yes or No)." + }, + { + "name": "height", + "description": "The typical height of the species in meters." + }, + { + "name": "genus", + "description": "The genus to which the species belongs." + }, + { + "name": "species", + "description": "The specific epithet of the species." + }, + { + "name": "genus.relative", + "description": "Data relative to the genus, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "duplicated", + "description": "Indicates if the species data is duplicated within the dataset (TRUE or FALSE)." + }, + { + "name": "species.relative", + "description": "Data relative to the species, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "phylo.tip.label", + "description": "The label used for the species in the phylogenetic tree included in the dataset." + } + ] + } + }, + { + "name": "invaded_niche_pathways.csv", + "description": "The data for the analysis of pathway-specific differences in the invaded niche.", + "columns": { + "raw": [ + { + "name": "n.gard", + "description": "The count or frequency of non-native plant species introduced through the \"Gardening\" pathway." + }, + { + "name": "n.unint", + "description": "The count or frequency of non-native plant species introduced through the \"Unintentional\" pathway" + }, + { + "name": "n.agfo", + "description": "The count or frequency of non-native plant species introduced through the \"AgriForest\" (Agriculture and Forestry) pathway" + }, + { + "name": "n.total", + "description": "The total count or frequency of non-native plant species across all introduction pathways." + }, + { + "name": "habitat", + "description": "A categorical variable representing the habitat type, selected from the ten most widespread habitat types in the Barcelona province" + }, + { + "name": "elevation", + "description": "The elevation (in meters) of the sampled plot or location" + }, + { + "name": "cropland.1956.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.50m", + "description": "This variable represent the percentage of urban land cover in a buffer (within a 50-meter radius) of 50 meters around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "cropland.2009.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "urban.2009.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "cropland.1956.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.500m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.500m", + "description": "This variable represents the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "cropland.1956.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "urban.1956.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "cropland.1993.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "urban.1993.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "progressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "progressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "distance.stream", + "description": "The distance (in meters) from the sampled plot or location to the nearest main stream or waterway." + }, + { + "name": "distance.road", + "description": "The distance (in meters) from the sampled plot or location to the nearest main road." + }, + { + "name": "longitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "latitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "annual.temperature", + "description": "The mean annual temperature of the sampled plot or location." + }, + { + "name": "annual.rainfall", + "description": "The annual precipitation or rainfall of the sampled plot or location." + }, + { + "name": "annual.radiation", + "description": "The mean annual solar radiation (in kJ/m^2/day) of the sampled plot or location." + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "What factors interact significantly to affect the proportion of gardening-introduced non-native plants?" + }, + { + "qid": 1, + "question_type": "variables", + "question": "What is interacting significantly to affect the proportion of non-native plants introduced through gardening?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "How do urban land use and elevation interact in relation to the proportion of non-native plants introduced through gardening?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_4.json b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_4.json new file mode 100644 index 000000000..3689da718 --- /dev/null +++ b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_4.json @@ -0,0 +1,304 @@ +{ + "id": 0, + "domain": "biology", + "workflow_tags": "interaction analysis, ecological modelling, statistical tests", + "domain_knowledge": "", + "datasets": [ + { + "name": "temporal_trends_contingency_table.csv", + "description": "Dataset contains temporal trends in the introduction pathways of non-native flora (plants) in the region of Catalonia.", + "columns": { + "raw": [ + { + "name": "introduction.period", + "description": "This column represents different time periods, related to when non-native plant species were introduced into the region." + }, + { + "name": "pathway", + "description": "This column represents different modes or routes through which non-native plant species were introduced, such as \"AgriForest\" (agriculture and forestry - plants introduced for cultivation to provide food or timber), \"Gardening\" (plants introduced for cultivation as ornamentals or for medicinal use), and \"Unintentional\" (plants introduced accidentally with the sowing of contaminated seed lots, global trade and tourism)." + }, + { + "name": "n", + "description": "This column represents the frequency or count of non-native plant species introductions for each combination of introduction period and pathway." + } + ] + } + }, + { + "name": "phylogenetic_tree.txt", + "description": "This file contains a phylogenetic tree in Newick format, illustrating the evolutionary relationships among diverse plant species. It features species from multiple families, with branch lengths denoting genetic distances or evolutionary times.", + "columns": { + "raw": [{ + "name": "", + "description": "" + }] + } + }, + { + "name": "invasion_success_pathways.csv", + "description": "This dataset provides comprehensive information on various non-native plant species that have invaded plots in the Barcelona province. The dataset includes detailed information on the ecological and biological characteristics of each species, the extent of their invasion, and the pathways through which they were introduced.", + "columns": { + "raw": [ + { + "name": "Scientific_name", + "description": "The full scientific name of the non-native plant species, including the author citation." + }, + { + "name": "family", + "description": "The botanical family to which the species belongs." + }, + { + "name": "number.of.invaded.plots", + "description": "The number of different plots in the Barcelona province where the species has been observed to invade." + }, + { + "name": "area.of.occupancy.10km", + "description": "The area of occupancy at a 10 km scale, indicating how widespread the species is within the studied region." + }, + { + "name": "habitat.range", + "description": "The number of different habitat types the species can occupy." + }, + { + "name": "invaded.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species within the invaded region, indicating its ecological versatility." + }, + { + "name": "mrt", + "description": "The mean residence time of the species, indicating how long the species has been present in the new region (in years)." + }, + { + "name": "native.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species in its native range." + }, + { + "name": "intro.pathway", + "description": "The pathway through which the species was introduced to the new region." + }, + { + "name": "growth.form", + "description": "The growth form of the plant species, describing its general structural category." + }, + { + "name": "dispersal", + "description": "The primary method of dispersal for the species." + }, + { + "name": "vegetative", + "description": "Indicates whether the species can reproduce vegetatively (Yes or No)." + }, + { + "name": "height", + "description": "The typical height of the species in meters." + }, + { + "name": "genus", + "description": "The genus to which the species belongs." + }, + { + "name": "species", + "description": "The specific epithet of the species." + }, + { + "name": "genus.relative", + "description": "Data relative to the genus, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "duplicated", + "description": "Indicates if the species data is duplicated within the dataset (TRUE or FALSE)." + }, + { + "name": "species.relative", + "description": "Data relative to the species, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "phylo.tip.label", + "description": "The label used for the species in the phylogenetic tree included in the dataset." + } + ] + } + }, + { + "name": "invaded_niche_pathways.csv", + "description": "The data for the analysis of pathway-specific differences in the invaded niche.", + "columns": { + "raw": [ + { + "name": "n.gard", + "description": "The count or frequency of non-native plant species introduced through the \"Gardening\" pathway." + }, + { + "name": "n.unint", + "description": "The count or frequency of non-native plant species introduced through the \"Unintentional\" pathway" + }, + { + "name": "n.agfo", + "description": "The count or frequency of non-native plant species introduced through the \"AgriForest\" (Agriculture and Forestry) pathway" + }, + { + "name": "n.total", + "description": "The total count or frequency of non-native plant species across all introduction pathways." + }, + { + "name": "habitat", + "description": "A categorical variable representing the habitat type, selected from the ten most widespread habitat types in the Barcelona province" + }, + { + "name": "elevation", + "description": "The elevation (in meters) of the sampled plot or location" + }, + { + "name": "cropland.1956.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.50m", + "description": "This variable represent the percentage of urban land cover in a buffer (within a 50-meter radius) of 50 meters around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "cropland.2009.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "urban.2009.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "cropland.1956.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.500m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.500m", + "description": "This variable represents the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "cropland.1956.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "urban.1956.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "cropland.1993.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "urban.1993.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "progressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "progressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "distance.stream", + "description": "The distance (in meters) from the sampled plot or location to the nearest main stream or waterway." + }, + { + "name": "distance.road", + "description": "The distance (in meters) from the sampled plot or location to the nearest main road." + }, + { + "name": "longitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "latitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "annual.temperature", + "description": "The mean annual temperature of the sampled plot or location." + }, + { + "name": "annual.rainfall", + "description": "The annual precipitation or rainfall of the sampled plot or location." + }, + { + "name": "annual.radiation", + "description": "The mean annual solar radiation (in kJ/m^2/day) of the sampled plot or location." + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "How do introduction pathways interact with minimum residence time in affecting the success of non-native plant species in Catalonia?" + }, + { + "qid": 1, + "question_type": "variables", + "question": "Which factors influence the success of unintentionally introduced plants compared to intentionally introduced ones over time in Catalonia?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "What is the relationship between introduction pathways and the success of non-native plants over time in Catalonia?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_5.json b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_5.json new file mode 100644 index 000000000..1659b8b80 --- /dev/null +++ b/examples/discovery/data/demo/introduction_pathways_non-native_plants/metadata_5.json @@ -0,0 +1,304 @@ +{ + "id": 0, + "domain": "biology", + "workflow_tags": "statistical tests, glm, spatial analysis, ecological modelling", + "domain_knowledge": "", + "datasets": [ + { + "name": "temporal_trends_contingency_table.csv", + "description": "Dataset contains temporal trends in the introduction pathways of non-native flora (plants) in the region of Catalonia.", + "columns": { + "raw": [ + { + "name": "introduction.period", + "description": "This column represents different time periods, related to when non-native plant species were introduced into the region." + }, + { + "name": "pathway", + "description": "This column represents different modes or routes through which non-native plant species were introduced, such as \"AgriForest\" (agriculture and forestry - plants introduced for cultivation to provide food or timber), \"Gardening\" (plants introduced for cultivation as ornamentals or for medicinal use), and \"Unintentional\" (plants introduced accidentally with the sowing of contaminated seed lots, global trade and tourism)." + }, + { + "name": "n", + "description": "This column represents the frequency or count of non-native plant species introductions for each combination of introduction period and pathway." + } + ] + } + }, + { + "name": "phylogenetic_tree.txt", + "description": "This file contains a phylogenetic tree in Newick format, illustrating the evolutionary relationships among diverse plant species. It features species from multiple families, with branch lengths denoting genetic distances or evolutionary times.", + "columns": { + "raw": [{ + "name": "", + "description": "" + }] + } + }, + { + "name": "invasion_success_pathways.csv", + "description": "This dataset provides comprehensive information on various non-native plant species that have invaded plots in the Barcelona province. The dataset includes detailed information on the ecological and biological characteristics of each species, the extent of their invasion, and the pathways through which they were introduced.", + "columns": { + "raw": [ + { + "name": "Scientific_name", + "description": "The full scientific name of the non-native plant species, including the author citation." + }, + { + "name": "family", + "description": "The botanical family to which the species belongs." + }, + { + "name": "number.of.invaded.plots", + "description": "The number of different plots in the Barcelona province where the species has been observed to invade." + }, + { + "name": "area.of.occupancy.10km", + "description": "The area of occupancy at a 10 km scale, indicating how widespread the species is within the studied region." + }, + { + "name": "habitat.range", + "description": "The number of different habitat types the species can occupy." + }, + { + "name": "invaded.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species within the invaded region, indicating its ecological versatility." + }, + { + "name": "mrt", + "description": "The mean residence time of the species, indicating how long the species has been present in the new region (in years)." + }, + { + "name": "native.niche.breadth", + "description": "A measure (index) of the ecological niche breadth of the species in its native range." + }, + { + "name": "intro.pathway", + "description": "The pathway through which the species was introduced to the new region." + }, + { + "name": "growth.form", + "description": "The growth form of the plant species, describing its general structural category." + }, + { + "name": "dispersal", + "description": "The primary method of dispersal for the species." + }, + { + "name": "vegetative", + "description": "Indicates whether the species can reproduce vegetatively (Yes or No)." + }, + { + "name": "height", + "description": "The typical height of the species in meters." + }, + { + "name": "genus", + "description": "The genus to which the species belongs." + }, + { + "name": "species", + "description": "The specific epithet of the species." + }, + { + "name": "genus.relative", + "description": "Data relative to the genus, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "duplicated", + "description": "Indicates if the species data is duplicated within the dataset (TRUE or FALSE)." + }, + { + "name": "species.relative", + "description": "Data relative to the species, if applicable (specific data not provided, thus marked as NA)." + }, + { + "name": "phylo.tip.label", + "description": "The label used for the species in the phylogenetic tree included in the dataset." + } + ] + } + }, + { + "name": "invaded_niche_pathways.csv", + "description": "The data for the analysis of pathway-specific differences in the invaded niche.", + "columns": { + "raw": [ + { + "name": "n.gard", + "description": "The count or frequency of non-native plant species introduced through the \"Gardening\" pathway." + }, + { + "name": "n.unint", + "description": "The count or frequency of non-native plant species introduced through the \"Unintentional\" pathway" + }, + { + "name": "n.agfo", + "description": "The count or frequency of non-native plant species introduced through the \"AgriForest\" (Agriculture and Forestry) pathway" + }, + { + "name": "n.total", + "description": "The total count or frequency of non-native plant species across all introduction pathways." + }, + { + "name": "habitat", + "description": "A categorical variable representing the habitat type, selected from the ten most widespread habitat types in the Barcelona province" + }, + { + "name": "elevation", + "description": "The elevation (in meters) of the sampled plot or location" + }, + { + "name": "cropland.1956.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.50m", + "description": "This variable represent the percentage of urban land cover in a buffer (within a 50-meter radius) of 50 meters around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "cropland.2009.50m", + "description": "This variable represent the percentage of cropland in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "urban.2009.50m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 2009" + }, + { + "name": "cropland.1956.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "urban.1956.500m", + "description": "This variable represent the percentage of urban land cover in a buffer of 50 meters (within a 50-meter radius) around the sampled plot or location for the year 1956" + }, + { + "name": "cropland.1993.500m", + "description": "This variable represent the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993" + }, + { + "name": "urban.1993.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.500m", + "description": "This variable represents the percentage of cropland in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.500m", + "description": "This variable represents the percentage of urban land cover in a buffer of 500 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "cropland.1956.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "urban.1956.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1956." + }, + { + "name": "cropland.1993.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "urban.1993.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 1993." + }, + { + "name": "cropland.2009.1000m", + "description": "This variable represents the percentage of cropland in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "urban.2009.1000m", + "description": "This variable represents the percentage of urban land cover in a buffer of 1000 meters (within a 50-meter radius) around the sampled plot or location for the year 2009." + }, + { + "name": "progressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1956.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1956 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "progressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as progressive, indicating an increase in urban or cropland areas over this period." + }, + { + "name": "regressive.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes changes as regressive, indicating a decrease in urban or cropland areas over this period." + }, + { + "name": "no.changes.1993.2009.50m", + "description": "This variable represents the historical landscape changes in a 50-meter buffer around the sampled plot or location between 1993 and 2009. It categorizes areas where there have been no significant changes in urban or cropland cover over this period." + }, + { + "name": "distance.stream", + "description": "The distance (in meters) from the sampled plot or location to the nearest main stream or waterway." + }, + { + "name": "distance.road", + "description": "The distance (in meters) from the sampled plot or location to the nearest main road." + }, + { + "name": "longitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "latitude", + "description": "The geographic coordinates of the sampled plot or location." + }, + { + "name": "annual.temperature", + "description": "The mean annual temperature of the sampled plot or location." + }, + { + "name": "annual.rainfall", + "description": "The annual precipitation or rainfall of the sampled plot or location." + }, + { + "name": "annual.radiation", + "description": "The mean annual solar radiation (in kJ/m^2/day) of the sampled plot or location." + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In what scenario did urban land use promote a specific type of plant invasion?" + }, + { + "qid": 1, + "question_type": "variables", + "question": "What types of plant introductions were influenced by urban land use?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "How did urban land use affect the invasion of different types of introduced plants?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/introduction_pathways_non-native_plants/phylogenetic_tree.txt b/examples/discovery/data/demo/introduction_pathways_non-native_plants/phylogenetic_tree.txt new file mode 100644 index 000000000..ef1cb99e4 --- /dev/null +++ b/examples/discovery/data/demo/introduction_pathways_non-native_plants/phylogenetic_tree.txt @@ -0,0 +1 @@ +(((((((((Galinsoga_parviflora:17.695347,(Helianthus_tuberosus:7.022053,(Xanthium_spinosum:2.561524,Xanthium_orientale:2.561525):4.460528):10.673294):0.462661,(Bidens_subalternans:5.015196,Bidens_frondosus:5.015196):13.142811):4.831452,((((Symphyotrichum_squamatum:1.01914,Symphyotrichum_pilosum:1.019141):3.273597,(Erigeron_sumatrensis:2.164648,(Erigeron_bonariensis:1.329532,Erigeron_canadensis:1.329532):0.835117):2.128089):14.650364,Artemisia_verlotiorum:18.943101):2.505677,(Senecio_pterophorus:4.400593,Senecio_angulatus:4.400594):17.048185):1.540681):70.740752,(Pittosporum_tobira:85.640655,Lonicera_japonica:85.640655)mrcaott1673ott2128:8.089557)mrcaott320ott1673:13.011199,((Myoporum_laetum:71.020608,Ligustrum_lucidum:71.020609):18.730459,((Araujia_sericifera:12.207297,Gomphocarpus_fruticosus:12.207297):73.70584,(((Solanum_lycopersicum:18.773214,Datura_stramonium:18.773214):11.139496,Nicotiana_glauca:29.91271):36.737115,((Convolvulus_sabatius:46.983204,Cuscuta_campestris:46.983204):0.603677,Dichondra_micrantha:47.58688):19.062944):19.263312)mrcaott1191ott2192:3.837931)mrcaott248ott1191:16.990343)mrcaott248ott320:13.132823,(((((Opuntia_ficus-indica2:0.836242,Opuntia_ficus-indica:0.836243):27.966925,Portulaca_grandiflora:28.803168):54.152271,(Carpobrotus_edulis:74.876157,(Mirabilis_jalapa:71.440815,Phytolacca_americana:71.440815):3.435342):8.079282):4.895165,(Dysphania_ambrosioides:43.661326,(((Amaranthus_retroflexus:3.994735,Amaranthus_hybridus:3.994735):0.354,Amaranthus_blitoides:4.348735):0.118536,(Amaranthus_deflexus:2.435929,(Amaranthus_muricatus:1.516463,Amaranthus_viridis:1.516463):0.919466):2.031342):39.194055):44.189278):19.200704,(Rumex_cristatus:93.255548,Tamarix_parviflora:93.255549):13.795759)Caryophyllales.rn.d8s.tre:12.822926)mrcaott248ott557:3.860003,(((((Robinia_pseudoacacia:84.026445,Cercis_siliquastrum:84.026445):28.674751,(Eriobotrya_japonica:50.733268,(Prunus_cerasifera:10.267824,Prunus_armeniaca:10.267824):40.465444):61.967928)mrcaott371ott579:3.084369,(((Salix_babylonica:38.038662,(Populus_deltoides2:2.955636,Populus_deltoides:2.955637):35.083025):64.569097,(Euphorbia_prostrata:61.896507,Ricinus_communis:61.896507):40.711252):8.818668,(Oxalis_debilis2:22.8346585,Oxalis_debilis:22.8346585,Oxalis_corniculata:22.8346585,Oxalis_articulata:22.8346595):88.5917675)mrcaott2ott345:4.359138)mrcaott2ott371:2.793039,((((Lunaria_annua:27.449165,Lepidium_didymum:27.449165):0.604057,Malcolmia_maritima:28.053222):76.204837,(Ailanthus_altissima:79.148402,Acer_negundo:79.148402):25.109657)mrcaott96ott378:12.735137,(Eucalyptus_camaldulensis2:12.651451,Eucalyptus_camaldulensis:12.651452):104.341744)mrcaott96ott607:1.585408)mrcaott2ott96:2.70865,Parthenocissus_inserta:121.287254)mrcaott2ott8384:2.446983)Pentapetalae:12.023828,(((Agave_americana:15.881341,Yucca_aloifolia:15.881341):64.776305,Iris_x_germanica:80.657645):33.942917,(((((((Cenchrus_longisetus:3.364914,Cenchrus_spinifex:3.364915):2.816717,Stenotaphrum_secundatum:6.181632):14.637477,(Sorghum_halepense:14.875097,Zea_mays:14.875097):5.944012):4.846477,((Eleusine_indica:1.830099,Eleusine_tristachya:1.830099):22.226607,Cortaderia_selloana:24.056706):1.60888):14.085492,(Jarava_plumosa:19.529615,Bromus_catharticus:19.529616):20.221462):54.070429,Cyperus_eragrostis:93.821507):14.413646,(Canna_indica:98.209005,(Washingtonia_filifera:31.3008955,Phoenix_canariensis:31.3008955):66.9081095)mrcaott121ott4575:10.026148)mrcaott121ott252:6.36541)mrcaott121ott334:21.157502)mrcaott2ott121:189.291963,Pinus_radiata:325.050028)Spermatophyta; diff --git a/examples/discovery/data/demo/introduction_pathways_non-native_plants/temporal_trends_contingency_table.csv b/examples/discovery/data/demo/introduction_pathways_non-native_plants/temporal_trends_contingency_table.csv new file mode 100644 index 000000000..1e3699ae6 --- /dev/null +++ b/examples/discovery/data/demo/introduction_pathways_non-native_plants/temporal_trends_contingency_table.csv @@ -0,0 +1,13 @@ +"introduction.period" "pathway" "n" +"Before 1500" "AgriForest" 55 +"Before 1500" "Gardening" 18 +"Before 1500" "Unintentional" 34 +"1501-1900" "AgriForest" 36 +"1501-1900" "Gardening" 54 +"1501-1900" "Unintentional" 31 +"1901-1984" "AgriForest" 38 +"1901-1984" "Gardening" 142 +"1901-1984" "Unintentional" 110 +"1985-2019" "AgriForest" 32 +"1985-2019" "Gardening" 397 +"1985-2019" "Unintentional" 99 diff --git a/examples/discovery/data/demo/meta_regression/meta-regression_joined_data_heterogeneity_in_replication_projects.csv b/examples/discovery/data/demo/meta_regression/meta-regression_joined_data_heterogeneity_in_replication_projects.csv new file mode 100644 index 000000000..873b63ad6 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/meta-regression_joined_data_heterogeneity_in_replication_projects.csv @@ -0,0 +1,87 @@ +"study","project.x","ro","rr","fiso","fisr","se_fiso","se_fisr","po","pr","po1","pr1","pm_belief","nr","no","authors.o","pub_year","journal","discipline","length","citations","n.r","power.o","power.r","power_planned.r","experiment_country.o","experiment_country.r","experiment_language.o","experiment_language.r","online.o","online.r","compensation.o","compensation.r","subjects.o","subjects.r","endprice","transactions","trading_volume","replicated","project.y","relative_es","n_authors.o","n_authors.r","author_citations_avg.o","author_citations_max.o","authors_male.o","seniority.o","author_citations_avg.r","author_citations_max.r","authors_male.r","seniority.r","aggregated","lab_id","es_80power","same_country","same_language","same_online","same_subjects","us_lab.o","us_lab.r" +"A Roelofs","Psychology",0.594605285,0.148340453,0.684760158473966,0.149443120256283,0.288675134594813,0.192450089729875,0.0176883192346041,0.437436568193535,0.00884415961730206,0.218718284096767,NA,30,15,"Ardi Roelofs",2008,"JEPLMC","Cognitive",16,46,28,0.608309222353551,0.117559205357017,0.925885903067168,"Netherlands","Netherlands","Dutch","Dutch",0,0,"cash","credit","community","students",NA,NA,NA,0,"rpp",0.249477185524848,1,2,5062,5062,1,"Professor",568.5,713,0.5,"Associate Professor",NA,"",0.510256214851446,1,1,1,0,0,0 +"AL Morris, ML Still","Psychology",0.611120054,0.229604631,0.710707113969394,0.233772054589965,0.21320071635561,0.21320071635561,0.000857570191410346,0.272865215514357,0.000428785095705173,0.136432607757179,NA,25,25,"Alison L. Morris and Mary L. Still",2008,"JEPLMC","Cognitive",21,6,23,0.898827183416202,0.186118243971358,0.911871048336865,"United States","Australia","English","English",0,0,"credit","cash","students","students",0.7995,51,421.532308,0,"rpp",0.375711170820128,2,1,184,331,0,"Associate Professor",96,96,1,"Researcher",NA,"",0.537653446800911,0,1,1,1,1,0 +"B Liefooghe, P Barrouillet, A Vandierendonck, V Camos","Psychology",0.424979665,-0.215243835,0.453753708518043,-0.218663485593077,0.208514414057075,0.182574185835055,0.0295458149206969,0.231045810573275,0.0147729074603485,0.884477094713362,NA,33,26,"Baptist Liefooghe and Pierre Barrouillet and AndrΓ© Vandierendonck and ValΓ©rie Camos",2008,"JEPLMC","Cognitive",17,55,31,0.562129379790831,0.216135299865856,0.697196475029823,"Netherlands","Germany","Dutch","German",0,0,"credit","mixed","students","students",0.7968,35,388.617158,0,"rpp",-0.506480315946411,4,3,1711.75,2395,0.75,"Professor",762.666666666667,1939,0.666666666666667,"Professor",NA,"",0.472582641843102,0,0,1,1,0,0 +"BC Storm, EL Bjork, RA Bjork","Psychology",0.228739615,-0.006108358,0.232859089732106,-0.00610843397346167,0.0727392967453308,0.0611990061362105,0.00136816982783159,0.920493078013187,0.000684084913915796,0.539753460993406,NA,270,192,"Benjamin C. Storm and Elizabeth Ligon Bjork and Robert A. Bjork",2008,"JEPLMC","Cognitive",7,39,268,0.891005171966745,0.0510771071851892,0.900160733204935,"United States","United States","English","English",0,0,"credit","credit","students","students",NA,NA,NA,0,"rpp",-0.0267044167229188,3,1,2557.33333333333,5282,0.666666666666667,"Professor",50,50,0,"Assistant Professor",NA,"",0.198555348433242,1,1,1,1,1,1 +"C Mitchell, S Nash, G Hall","Psychology",0.461300742,0.134811802,0.498962392476004,0.135637524870972,0.182574185835055,0.147441956154897,0.00627737525620875,0.357604923558456,0.00313868762810438,0.178802461779228,NA,49,33,"Chris Mitchell and Scott Nash and Geoffrey Hall",2008,"JEPLMC","Cognitive",6,36,47,0.763224459726542,0.148118639208091,0.921801160282869,"Australia","Netherlands","English","Dutch",0,0,"credit","cash","students","students",NA,NA,NA,0,"rpp",0.292242759930354,3,1,1930,4652,1,"Professor",622,622,1,"Assistant Professor",NA,"",0.391767027527364,0,0,1,1,0,0 +"CJ Berry, DR Shanks, RN Henson","Psychology",0.594960462,0.395843475,0.685309769582024,0.418710420698763,0.21320071635561,0.182574185835055,0.00130723017695369,0.0218266303226357,0.000653615088476846,0.0109133151613179,NA,33,25,"Christopher J. Berry and David R. Shanks and Richard N. A. Henson",2008,"JEPLMC","Cognitive",15,37,31,0.876909361266149,0.612681195420325,0.950518895266713,"United Kingdom","Germany","English","German",0,0,"cash","cash","students","students",0.7614,43,376.1699899,1,"rpp",0.665327362543294,3,2,8008.33333333333,17884,1,"Professor",35,35,0.5,"Assistant Professor",NA,"",0.486647389970205,0,0,1,1,0,0 +"CP Beaman, I Neath, AM Surprenant","Psychology",0.715143728,0.131411981,0.897633709106095,0.132176372921613,0.101015254455221,0.277350098112615,6.32798053722595e-19,0.633669283281907,3.16399026861297e-19,0.316834641640954,NA,16,101,"C. Philip Beaman and Ian Neath and AimΓ©e M. Surprenant",2008,"JEPLMC","Cognitive",11,12,14,0.999999999996776,0.0727864633484799,0.890641082552758,"United States","Netherlands","English","Dutch",0,0,"credit","credit","students","students",NA,NA,NA,0,"rpp",0.183756042114097,3,2,1112.66666666667,1994,0.666666666666667,"Professor",35.5,36,1,"Researcher",NA,"",0.657058779759921,0,0,1,1,1,0 +"CS Dodson, J Darragh, A Williams","Psychology",0.561422534,-0.110614348,0.634908053973584,-0.111068831072181,0.166666666666667,0.182574185835055,0.000139277211756085,0.542956000908511,6.96386058780426e-05,0.728521999545745,NA,33,39,"Chad S. Dodson and James Darragh and Allison Williams",2008,"JEPLMC","Cognitive",18,14,31,0.962789138789341,0.09105819137415,0.933675694628179,"United States","United States","English","English",0,0,"credit","mixed","students","students",0.3055,47,489.7627094,0,"rpp",-0.197025130451924,3,1,529.333333333333,1525,0.666666666666667,"Associate Professor",35,35,0,"Associate Professor",NA,"",0.472582641843102,1,1,1,1,1,1 +"D Ganor-Stern, J Tzelgov","Psychology",0.698586719,0.780725807,0.864534746467758,1.04722667369848,0.192450089729875,0.188982236504614,7.04731955042914e-06,3.00058304149717e-08,3.52365977521457e-06,1.50029152074859e-08,NA,31,30,"Dana Ganor-Stern and Joseph Tzelgov",2008,"JEPLMC","Cognitive",8,51,29,0.992297032558239,0.999715950466223,0.952937391061236,"Israel","Israel","Arabic","Arabic",0,0,"mixed","mixed","students","students",NA,NA,NA,1,"rpp",1.11757894298016,2,1,1013,1826,0.5,"Professor",599,599,1,"Associate Professor",NA,"",0.582147807147936,1,1,1,1,0,0 +"D Mirman, JS Magnuson","Psychology",0.672084167,0.466443576,0.814534581372408,0.505515292648616,0.223606797749979,0.188982236504614,0.000269783364662946,0.00747436294308648,0.000134891682331473,0.00373718147154324,NA,31,23,"Daniel Mirman and James S. Magnuson",2008,"JEPLMC","Cognitive",15,56,29,0.94073699042936,0.743958643247644,0.99005156291934,"United States","United States","English","English",0,0,"credit","credit","students","students",0.839,59,684.8234996,1,"rpp",0.694025538619778,2,2,1294.5,1855,1,"Professor",135.5,233,1,"Assistant Professor",NA,"",0.486647389970205,1,1,1,1,1,1 +"JR Schmidt, D Besner","Psychology",0.195136354,0.24687812,0.197671341200697,0.252085564199444,0.103695169473043,0.0645497224367903,0.0566147460949143,9.41113781028231e-05,0.0283073730474571,4.70556890514116e-05,NA,243,96,"James R. Schmidt and Derek Besner",2008,"JEPLMC","Cognitive",10,92,241,0.473849373181194,0.973589954260019,0.926722652921322,"Canada","United States","English","English",0,0,"credit","mixed","students","students",0.8159,52,487.4677456,1,"rpp",1.26515697838651,2,2,2079.5,3781,1,"Professor",35,35,0.5,"Professor",NA,"",0.160911817192179,0,1,1,1,0,1 +"K Oberauer","Psychology",0.560499145,0.401939168,0.63356067564677,0.425959601340054,0.182574185835055,0.235702260395516,0.000520158609726401,0.0707321084979368,0.000260079304863201,0.0353660542489684,NA,21,33,"Klaus Oberauer",2008,"JEPLMC","Cognitive",21,11,19,0.92473807877139,0.413765735768296,0.915886389595029,"Germany","Netherlands","German","German",0,0,"cash","credit","students","students",NA,NA,NA,0,"rpp",0.71710933296785,1,7,4368,4368,1,"Professor",38,56,0.285714285714286,"Assistant Professor",NA,"",0.486647389970205,0,1,1,1,0,0 +"L Sahakyan, PF Delaney, ER Waldum","Psychology",0.224092029,0.019422053,0.227960364129573,0.0194244956568418,0.103695169473043,0.0975900072948533,0.0279227398534142,0.842230007417763,0.0139613699267071,0.421115003708881,NA,108,96,"Lili Sahakyan and Peter F. Delaney and Emily R. Waldum",2008,"JEPLMC","Cognitive",7,21,106,0.588903812336217,0.0543479059182908,0.624736156996657,"United States","Germany","English","German",0,0,"credit","cash","students","students",0.5582,64,981.5794743,0,"rpp",0.0866699859279689,3,2,393,641,0.333333333333333,"Associate Professor",141.5,248,0.5,"Assistant Professor",NA,"",0.273229674690716,0,0,1,1,1,0 +"M Bassok, SF Pedigo, AT Oskarsson","Psychology",0.363732358,0.284274426,0.381180652550176,0.292326185936339,0.0813788458771159,0.145864991497895,2.81293921929081e-06,0.045060703026347,1.4064696096454e-06,0.0225303515131735,NA,50,154,"Miriam Bassok and Samuel F. Pedigo and An T. Oskarsson",2008,"JEPLMC","Cognitive",10,50,48,0.99660263692206,0.507420161596179,0.937074167273003,"United States","Hong Kong","English","English",0,0,"credit","cash","students","students",0.4289,48,527.0967429,1,"rpp",0.78154835484832,3,3,646.666666666667,1835,0.333333333333333,"Associate Professor",23.3333333333333,35,0.333333333333333,"Assistant Professor",NA,"",0.29674387257627,0,1,1,1,1,0 +"MJ Yap, DA Balota, CS Tse, D Besner","Psychology",0.378068955,0.379573968,0.397804631884721,0.399561810599662,0.182574185835055,0.120385853085769,0.029341646224431,0.000903373008396583,0.0146708231122155,0.000451686504198291,NA,72,33,"Melvin J. Yap and David A. Balota and Chi-Shing Tse and Derek Besner",2008,"JEPLMC","Cognitive",19,50,70,0.569123371781351,0.90843276002441,0.897599537983423,"United States","United States","English","English",0,0,"mixed","credit","students","students",0.8032,50,491.6204493,1,"rpp",1.00398078969483,4,2,4103.5,10101,1,"Professor",48.5,62,0.5,"Assistant Professor",NA,"",0.332167942429203,1,1,1,1,1,1 +"N Janssen, W Schirm, BZ Mahon, A Caramazza","Psychology",0.341773488,0.103261105,0.356099200575971,0.103630490542844,0.182574185835055,0.105999788000636,0.0511242163875682,0.328248402716705,0.0255621081937841,0.164124201358352,NA,92,33,"Niels Janssen and Walter Schirm and Bradford Z. Mahon and Alfonso Caramazza",2008,"JEPLMC","Cognitive",8,72,90,0.480195557882869,0.162893648465884,0.940955316283638,"United States","United States","English","English",0,0,"cash","cash","students","students",0.7566,51,820.9410736,0,"rpp",0.302133163120014,4,1,5175.75,18006,1,"Professor",265,265,1,"Associate Professor",NA,"",0.275865977707637,1,1,1,1,1,1 +"NB Turk-Browne, PJ Isola, BJ Scholl, TA Treat","Psychology",0.73782079,0.703904904,0.945679455083732,0.87499861266107,0.408248290463863,0.277350098112615,0.0205346864110139,0.00160579340650055,0.010267343205507,0.000802896703250276,NA,16,9,"Nicholas B. Turk-Browne and Phillip J. Isola and Brian J. Scholl and Teresa A. Treat",2008,"JEPLMC","Cognitive",9,61,14,0.519065761193176,0.846939958463064,0.919121277485814,"United States","Germany","English","German",0,0,"mixed","mixed","students","students",0.7535,40,297.0801541,1,"rpp",0.954032352490366,4,2,1722.75,4044,0.75,"Professor",182,328,1,"Professor",NA,"",0.657058779759921,0,0,1,1,1,0 +"PA White","Psychology",0.623367433,0.480717115,0.730493922145522,0.523916497045395,0.169030850945703,0.166666666666667,1.54859465492401e-05,0.00166940996242807,7.74297327462004e-06,0.000834704981214036,NA,39,38,"Peter A. White",2008,"JEPLMC","Cognitive",15,11,37,0.988809983155229,0.870823082366717,0.99052174572788,"United Kingdom","Germany","English","German",0,0,"credit","mixed","students","students",0.6737,28,168.955465,1,"rpp",0.771161741136387,1,2,891,891,1,"Professor",153,271,0.5,"Assistant Professor",NA,"",0.442173224120335,0,0,1,1,0,0 +"S Farrell","Psychology",0.516812354,0.315999964,0.571980598502644,0.327197057137756,0.162221421130763,0.162221421130763,0.000422006124616039,0.0436977966209845,0.00021100306230802,0.0218488983104923,NA,41,41,"Simon Farrell",2008,"JEPLMC","Cognitive",18,27,39,0.934387575473364,0.509905380426121,0.940171248683558,"United Kingdom","United States","English","English",0,0,"credit","cash","students","community",NA,NA,NA,1,"rpp",0.61144042233944,1,2,1496,1496,1,"Researcher",3091.5,6148,0,"Associate Professor",NA,"",0.426537652439599,0,1,1,0,0,1 +"S Pacton, P Perruchet","Psychology",0.713921386,0.681560537,0.895136292560066,0.832022572221689,0.229415733870562,0.229415733870562,9.54766243931683e-05,0.000287063920195273,4.77383121965842e-05,0.000143531960097637,NA,22,22,"SΓ©bastien Pacton and AmΓ©lie Sobaco and Pierre Perruchet",2015,"JEPLMC","Cognitive",5,72,20,0.964216644568933,0.937991454931454,0.986369795868394,"France","Germany","French","German",0,0,"credit","credit","students","students",0.6749,33,343.3752978,1,"rpp",0.954671691260976,3,1,1123,2731,0.666666666666667,"Professor",229,229,1,"Assistant Professor",NA,"",0.537653446800911,0,0,1,1,0,0 +"T Makovski, R Sussman, YV Jiang","Psychology",0.551178255,0.349791589,0.620072140594967,0.365206268595272,0.316227766016838,0.25,0.0498976578719934,0.144063452747625,0.0249488289359967,0.0720317263738124,NA,19,13,"Tal Makovski and Rachel Sussman and Yuhong V. Jiang",2008,"JEPLMC","Cognitive",12,116,17,0.445231792735256,0.288143777090402,0.690542502935302,"United States","United States","English","English",0,0,"mixed","cash","community","community",NA,NA,NA,0,"rpp",0.634625161328253,3,1,1598,4084,0.333333333333333,"Professor",858,858,0,"Assistant Professor",NA,"",0.608957415620332,1,1,1,1,1,1 +"BK Payne, MA Burkley, MB Stokes","Psychology",0.352175067,0.150236955,0.367924620459263,0.151382853963031,0.123091490979333,0.0755928946018454,0.00279861219592578,0.0452194963425392,0.00139930609796289,0.0226097481712696,NA,178,69,"B. Keith Payne and Melissa A. Burkley and Mark B. Stokes",2008,"JPSP","Social",16,165,176,0.842010220684075,0.514410604560066,0.988251621411283,"United States","Italy","English","Italian",0,0,"credit","credit","students","students",NA,NA,NA,1,"rpp",0.426597363293751,3,1,747.666666666667,1944,0.666666666666667,"Professor",409,409,1,"Professor",NA,"",0.23923782169819,0,0,1,1,1,0 +"CR Cox, J Arndt, T Pyszczynski, J Greenberg, A Abdollahi, S Solomon","Psychology",0.225490086,-0.052290282,0.229432826442352,-0.0523380189858188,0.104828483672192,0.0723574605292422,0.0286223125439699,0.469479737446759,0.0143111562719849,0.765260131276621,NA,194,94,"Cathy R. Cox and Jamie Arndt and Tom Pyszczynski and Jeff Greenberg and Abdolhossein Abdollahi and Sheldon Solomon",2008,"JPSP","Social",22,74,192,0.585020308311045,0.111281650974134,0.621556276169374,"United States","Netherlands","English","Dutch",0,0,"credit","credit","students","students",NA,NA,NA,0,"rpp",-0.231896146423041,6,6,6314,12106,0.833333333333333,"Professor",785.833333333333,4540,0.833333333333333,"Professor",NA,"",0.275865977707637,0,0,1,1,1,0 +"D Albarracin, IM Handley, K Noguchi, KC McCulloch, H Li, J Leeper, RD Brown, A Earl, WP Hart","Psychology",0.378345294,-0.030485538,0.398127096417791,-0.0304949873644915,0.174077655955698,0.108465228909328,0.0221919683295681,0.778595435683191,0.011095984164784,0.610702282158404,NA,88,36,"Dolores AlbarracΓ­n and Ian M. Handley and Kenji Noguchi and Kathleen C. McCulloch and Hong Li and Joshua Leeper and Rick D. Brown and Allison Earl and William P. Hart",2008,"JPSP","Social",14,46,86,0.612082384643667,0.0587885930692275,0.954086657761347,"United States","United States","English","English",0,1,"credit","cash","students","online",NA,NA,NA,0,"rpp",-0.0805759672010087,9,3,526.222222222222,2700,0.555555555555556,"Professor",361.333333333333,1014,0.333333333333333,"Associate Professor",NA,"",0.29674387257627,1,1,0,0,1,1 +"DB Centerbar, S Schnall, GL Clore, ED Garvin","Psychology",0.20573686,0.094021673,0.208715672093056,0.0943002047242515,0.0877058019307029,0.0953462589245592,0.0173255716521825,0.322649006531586,0.00866278582609123,0.161324503265793,NA,113,133,"David B. Centerbar and Simone Schnall and Gerald L. Clore and Erika D. Garvin",2008,"JPSP","Social",19,35,111,0.658845397439266,0.165956597085643,0.558192492384353,"United States","United States","English","English",0,0,"credit","credit","students","students",NA,NA,NA,0,"rpp",0.456999649941192,4,7,2035.25,6921,0.5,"Professor",30,35,0.285714285714286,"Assistant Professor",NA,"",0.270667517835744,1,1,1,1,1,1 +"DM Amodio, PG Devine, E Harmon-Jones","Psychology",0.377126878,0.076700677,0.396705896912039,0.0768516200278652,0.182574185835055,0.117851130197758,0.0297917830391096,0.514331748713048,0.0148958915195548,0.257165874356524,NA,75,33,"David M. Amodio and Patricia G. Devine and Eddie Harmon-Jones",2008,"JPSP","Social",1,136,73,0.5668055248264,0.0988957444357503,0.386134795359013,"United States","United States","English","English",0,0,"credit","credit","students","students",NA,NA,NA,0,"rpp",0.203381624260682,3,3,5951.33333333333,7716,0.666666666666667,"Professor",674,1952,0.666666666666667,"Assistant Professor",NA,"",0.582147807147936,1,1,1,1,1,1 +"E van Dijk, GA van Kleef, W Steinel, I van Beest","Psychology",0.379139495,-0.042140104,0.399054301120861,-0.0421650745796795,0.101015254455221,0.164398987305357,7.80089452471955e-05,0.797580116963371,3.90044726235978e-05,0.601209941518314,NA,40,101,"Eric van Dijk and Gerben A. van Kleef and Wolfgang Steinel and Ilja van Beest",2008,"JPSP","Social",15,97,38,0.975438918563721,0.056907201026436,0.979712178321226,"Netherlands","Austria","Dutch","German",0,0,"cash","cash","community","students",NA,NA,NA,0,"rpp",-0.111146700767748,4,2,1100.5,3094,1,"Professor",2194.5,4354,0.5,"Associate Professor",NA,"",0.271939546678411,0,0,1,0,0,0 +"EP Lemay, MS Clark","Psychology",0.167288593,0.03675175,0.168875882264062,0.0367683101769054,0.0743294146247166,0.060084176812611,0.0230869554386351,0.540573061549528,0.0115434777193176,0.270286530774764,NA,280,184,"Edward P. Lemay and Margaret S. Clark",2008,"JPSP","Social",22,40,278,0.619780087349576,0.0936344012183919,0.830149566475905,"United States","United States","English","English",0,0,"credit","mixed","students","students",0.6335,41,397.2947766,0,"rpp",0.219690711368467,2,1,1900,3211,0.5,"Professor",46,46,0,"Assistant Professor",NA,"",0.160911817192179,1,1,1,1,1,1 +"H Ersner-Hershfield, JA Mikels, SJ Sullivan, LL Carstensen","Psychology",0.219666896,-0.004719346,0.223306089724669,-0.00471938103724949,0.0966736489045664,0.0675737378399486,0.0208939040954606,0.944320642152142,0.0104469520477303,0.527839678923929,NA,222,110,"Hal Ersner-Hershfield and Joseph A. Mikels and Sarah J. Sullivan and Laura L. Carstensen",2008,"JPSP","Social",10,80,220,0.632139327102614,0.0504831226834457,0.896122118188971,"United States","United States","English","English",0,0,"nothing","nothing","students","students",NA,NA,NA,0,"rpp",-0.0214841020014231,4,3,3645,11519,0.5,"Professor",61,96,0.333333333333333,"Assistant Professor",NA,"",0.191897420404617,1,1,1,1,1,1 +"J Correll","Psychology",0.274010238,0.073797292,0.281194479704791,0.0739316991417271,0.122169444356305,0.0833333333333333,0.0213534498494471,0.374981788609135,0.0106767249247236,0.187490894304568,NA,147,70,"Joshua Correll",2008,"JPSP","Social",12,42,145,0.626128934161457,0.142993279495696,0.917654001042725,"United States","Canada","English","English",0,0,"credit","credit","students","students",0.305,53,514.0265139,0,"rpp",0.269323119233231,1,1,1418,1418,1,"Associate Professor",320,320,1,"Researcher",NA,"",0.230945343241393,0,1,1,1,1,0 +"JJ Exline, RF Baumeister, AL Zell, AJ Kraft, CV Witvliet","Psychology",0.431865646,0.011721165,0.462187795505302,0.0117217018177725,0.158113883008419,0.0877058019307029,0.00346529263741485,0.893680954965587,0.00173264631870742,0.446840477482793,NA,133,43,"Julie Juola Exline and Roy F. Baumeister and Anne L. Zell and Amy J. Kraft and Charlotte V. O. Witvliet",2008,"JPSP","Social",21,81,131,0.820994828388779,0.0519077829877793,0.999736860909135,"United States","United States","English","English",0,0,"credit","cash","students","community",NA,NA,NA,0,"rpp",0.0271407672931688,5,2,7169.8,33557,0.2,"Professor",524.5,1014,0.5,"Associate Professor",NA,"",0.234161005773579,1,1,1,0,1,1 +"JL Risen, T Gilovich","Psychology",0.185849827,0.003001488,0.188035056711107,0.00300149701344736,0.0932504808240314,0.0672672793996312,0.0437528019522977,0.96440983342415,0.0218764009761488,0.482204916712075,NA,224,118,"Jane L. Risen and Thomas Gilovich",2008,"JPSP","Social",15,59,222,0.518496177409217,0.0501547751813145,0.810188425541801,"United States","United States","English","English",0,1,"credit","cash","students","online",NA,NA,NA,0,"rpp",0.016150071530602,2,2,2908.5,5276,0.5,"Professor",524.5,1014,0.5,"Associate Professor",NA,"",0.183487803594318,1,1,0,0,1,1 +"KE Stanovich, RF West","Psychology",0.222131584,0.073353737,0.225897217190119,0.0734857299440754,0.0518475847365213,0.0758098043578903,1.3188874164648e-05,0.332373905561989,6.59443708232398e-06,0.166186952780994,NA,177,375,"Keith E. Stanovich and Richard F. West",2008,"JPSP","Social",24,291,175,0.991590275909163,0.161766395818398,0.910493950352143,"United States","United States","English","English",0,0,"credit","credit","students","students",0.8786,57,788.1228737,0,"rpp",0.330226506645719,2,1,6544.5,9283,1,"Professor",46,46,0,"Assistant Professor",NA,"",0.189238857749047,1,1,1,1,1,1 +"KL Blankenship, DT Wegener","Psychology",0.207643072,0.04449582,0.210706953837398,0.044525220365058,0.0625,0.0637576713063338,0.00074811280429647,0.48495784098406,0.000374056402148235,0.24247892049203,NA,249,259,"Kevin L. Blankenship and Duane T. Wegener",2008,"JPSP","Social",18,37,247,0.919852903104656,0.107123694809727,0.89661586466733,"United States","United States","English","English",0,0,"credit","credit","students","students",NA,NA,NA,0,"rpp",0.214289933063599,2,1,2688.5,5172,1,"Professor",178,178,0,"Associate Professor",NA,"",0.181176025662725,1,1,1,1,1,1 +"N Shnabel, A Nadler","Psychology",0.268278965,-0.101637251,0.275008386572745,-0.101989411675437,0.105999788000636,0.0857492925712544,0.0094749578078274,0.234285956184889,0.0047374789039137,0.882857021907555,NA,139,92,"Nurit Shnabel and Arie Nadler",2008,"JPSP","Social",17,138,137,0.731612425595625,0.219695518707564,0.804060803587268,"Israel","United States","Hebrew","English",0,0,"mixed","credit","students","students",0.4177,44,530.7945301,0,"rpp",-0.37884912445521,2,1,792.5,1224,0.5,"Professor",38,38,0,"Assistant Professor",NA,"",0.26695589846776,0,0,1,1,0,1 +"NO Rule, N Ambady","Psychology",0.3,0.27,0.309519604203112,0.2768638226551,0.158113883008419,0.158113883008419,0.0502800371952371,0.079938923243933,0.0251400185976185,0.0399694616219665,NA,43,43,"Nicholas O. Rule and Nalini Ambady",2008,"PS","Social",3,141,41,0.544584576420136,0.40693454987122,0.571555841860208,"United States","United States","English","English",0,0,"credit","credit","students","students",0.2995,79,969.9425562,0,"rpp",0.9,2,3,4459,7721,0.5,"Professor",61,96,0.333333333333333,"Assistant Professor",NA,"",0.384318619638178,1,1,1,1,1,1 +"PA Goff, CM Steele, PG Davies","Psychology",0.395559206,0.013045451,0.418373377644792,0.013046191117014,0.14142135623731,0.147441956154897,0.00309294216539201,0.929492339829872,0.001546471082696,0.464746169914936,NA,49,53,"Phillip Atiba Goff and Claude M. Steele and Paul G. Davies",2008,"JPSP","Social",17,165,47,0.832319963027853,0.0505395567507966,0.934659034891888,"United States","United States","English","English",0,0,"mixed","credit","students","students",NA,NA,NA,0,"rpp",0.0329797683940138,3,5,5996,12700,1,"Professor",28,35,0.4,"Professor",NA,"",0.325378636177841,1,1,1,1,1,1 +"SL Murray, JL Derrick, S Leder, JG Holmes","Psychology",0.317462994,-0.1351,0.328823222570441,-0.135931069224414,0.110431526074847,0.122169444356305,0.00290495027395403,0.265861460893431,0.00145247513697701,0.867069269553285,NA,70,85,"Sandra L. Murray and Jaye L. Derrick and Sadie Leder and John G. Holmes",2008,"JPSP","Social",31,117,68,0.840568405513763,0.196553160170575,0.79038467926301,"United States","United States","English","English",0,0,"mixed","mixed","community","community",NA,NA,NA,0,"rpp",-0.425561412049179,4,4,2261,5375,0.25,"Professor",77,203,0,"Associate Professor",NA,"",0.32107711524284,1,1,1,1,1,1 +"SM McCrea","Psychology",0.344312999,0.28970838,0.358977418835944,0.298247895841039,0.2,0.131306432859723,0.0726716905742514,0.0231234818155492,0.0363358452871257,0.0115617409077746,NA,61,28,"Sean M. McCrea",2008,"JPSP","Social",19,51,59,0.416785762526042,0.613459572932229,0.818140393111894,"Germany","United States","German","English",0,0,"mixed","credit","students","students",0.4329,47,584.829525,1,"rpp",0.841409940494288,1,2,455,455,1,"Associate Professor",48.5,62,0.5,"Assistant Professor",NA,"",0.336938941206028,0,0,1,1,0,1 +"V Purdie-Vaughns, CM Steele, PG Davies, R Ditlmann, JR Crosby","Psychology",0.378274916,-0.036935346,0.398044963704019,-0.0369521557380355,0.117851130197758,0.0259499648053841,0.000731416986422988,0.154453099981121,0.000365708493211494,0.922773450009439,NA,1488,75,"Valerie Purdie-Vaughns and Claude M. Steele and Paul G. Davies and Ruth Ditlmann and Jennifer Randall Crosby",2008,"JPSP","Social",16,240,1486,0.917801906552723,0.296143685545875,0.939689840638925,"United States","United States","English","English",0,1,"mixed","cash","anyone","online",0.4228,43,364.890918,0,"rpp",-0.0976415417405049,5,1,3707.4,12700,0.4,"Professor",161,161,0,"Assistant Professor",NA,"",0.307264434989798,1,1,0,0,1,1 +"B Dessalegn, B Landau","Psychology",0.381744212,-0.222920955,0.402099816773692,-0.226727712667986,0.174077655955698,0.150755672288882,0.0208943784338003,0.132596428904428,0.0104471892169002,0.933701785547786,NA,47,36,"Banchiamlack Dessalegn and Barbara Landau",2008,"PS","Cognitive",7,52,45,0.620636757528144,0.31690159161954,0.748912924081406,"United States","Canada","English","English",0,0,"nothing","nothing","anyone","anyone",NA,NA,NA,0,"rpp",-0.583953726061995,2,1,1530,2931,0,"Professor",189,189,0,"Associate Professor",NA,"",0.403805773745115,0,1,1,1,1,0 +"B Eitam, RR Hassin, Y Schul","Psychology",0.222330235,0.104941786,0.226106188572231,0.105329585238058,0.10976425998969,0.0803219328902499,0.0394056718374882,0.189741987689252,0.0197028359187441,0.0948709938446262,NA,158,86,"Baruch Eitam and Ran R. Hassin and Yaacov Schul",2008,"PS","Social",7,66,156,0.534069092528967,0.257162625835535,0.805927982843049,"Israel","United States","Hebrew","English",0,0,"mixed","mixed","students","students",NA,NA,NA,0,"rpp",-0.47200861367326,3,2,935,1356,1,"Professor",171,240,1,"Associate Professor",NA,"",0.220698594028647,0,0,1,1,0,1 +"C Farris, TA Treat, RJ Viken, RM McFall","Psychology",0.554427187,0.091162691,0.624750856813492,0.0914164977721764,0.060084176812611,0.0842151921066519,2.53386760393824e-25,0.277695497591929,1.26693380196912e-25,0.138847748795964,NA,144,280,"Coreen Farris and Teresa A. Treat and Richard J. Viken and Richard M. McFall",2008,"PS","Social",7,37,142,1,0.190784023756395,0.999999978807044,"United States","United Kingdom","English","English",0,0,"credit","cash","students","students",0.6825,58,647.3161446,0,"rpp",0.164426805065748,4,5,1600.5,3167,0.5,"Professor",2978.2,9651,0.6,"Professor",NA,"",0.230945343241393,0,1,1,1,1,0 +"C Janiszewski, D Uy","Psychology",0.333441051,0.225764222,0.34669477755082,0.229721666607801,0.136082763487954,0.0932504808240314,0.0108443165751358,0.0137591570547646,0.00542215828756792,0.00687957852738228,NA,118,57,"Chris Janiszewski and Dan Uy",2008,"PS","Cognitive",7,53,116,0.712227796486529,0.688386789283739,0.96439452901424,"United States","United States","English","English",0,0,"credit","cash","students","students",NA,NA,NA,1,"rpp",0.67707386754848,2,1,1209.5,2382,1,"Professor",1182,1182,1,"Researcher",NA,"",0.25248505382702,1,1,1,1,1,1 +"C McKinstry, R Dale, MJ Spivey","Psychology",0.701088742,0.750404422,0.869438514085779,0.973880109255998,0.353553390593274,0.353553390593274,0.0139268947409689,0.00587731034191647,0.00696344737048445,0.00293865517095824,NA,11,11,"Chris McKinstry and Rick Dale and Michael J. Spivey",2008,"PS","Cognitive",3,103,9,0.605320465392755,0.702117047800843,0.999988471023109,"United States","United States","English","English",0,0,"cash","cash","students","students",NA,NA,NA,1,"rpp",1.0703415659754,3,3,1863,4176,1,"Professor",2072.66666666667,6148,0,"Associate Professor",NA,"",0.373917726825712,1,1,1,1,1,1 +"DA Armor, C Massey, AM Sackett","Psychology",0.681152182,0.763508789,0.831260365276445,1.00457484452595,0.0901669634667432,0.0758098043578903,2.99538839894271e-20,4.43736400251912e-40,1.49769419947135e-20,2.21868200125956e-40,NA,177,126,"David A. Armor and Cade Massey and Aaron M. Sackett",2008,"PS","Social",3,32,175,0.999999999999727,1,0.999999999433941,"United States","Netherlands","English","Dutch",0,0,"cash","mixed","students","students",NA,NA,NA,1,"rpp",1.12090779296659,3,4,423.666666666667,1054,1,"Professor",324,740,0.5,"Assistant Professor",NA,"",0.281375882936472,0,0,1,1,1,0 +"DR Addis, AT Wong, DL Schacter","Psychology",0.570693584,0.652730428,0.648550826829565,0.780041330582545,0.185695338177052,0.185695338177052,0.000478426484546996,2.66148446924835e-05,0.000239213242273498,1.33074223462417e-05,NA,32,32,"Donna Rose Addis and Alana T. Wong and Daniel L. Schacter",2008,"PS","Social",9,248,30,0.927404096531052,0.984116126311401,0.94271865119267,"United States","Uruguay","English","Spanish",0,0,"cash","cash","anyone","anyone",NA,NA,NA,1,"rpp",1.14374937146656,3,1,14466.6666666667,37476,0.333333333333333,"Professor",56,56,1,"Associate Professor",NA,"",0.472582641843102,0,0,1,1,1,0 +"E Nurmsoo, P Bloom","Psychology",0.501799075,-0.450187899,0.551707796113811,-0.484935913612849,0.182574185835055,0.377964473009227,0.00251253226950747,0.199485129088897,0.00125626613475374,0.900257435455551,NA,10,33,"Erika Nurmsoo and Paul Bloom",2008,"PS","Cognitive",5,22,8,0.841195517463324,0.209027377045275,0.73001487657636,"United States","United States","English","English",0,0,"nothing","nothing","community","community",0.8069,41,616.9839362,0,"rpp",-0.897147725910017,2,10,2644.5,5139,0.5,"Professor",31.6,36,0.3,"Assistant Professor",NA,"",0.537653446800911,1,1,1,1,1,1 +"E Vul, H Pashler","Psychology",0.28771881,0.322727228,0.296077402194826,0.334688430685667,0.0764719112901873,0.0851256530758749,0.000108072525717394,8.43476929254931e-05,5.40362628586968e-05,4.21738464627466e-05,NA,141,174,"Edward Vul and Harold Pashler",2008,"PS","Cognitive",3,122,139,0.971229817698038,0.97475524775114,0.517547929195961,"United States","Netherlands","English","Dutch",0,1,"cash","mixed","anyone","students",NA,NA,NA,1,"rpp",1.12167580562425,2,4,5255.5,9150,1,"Professor",473.75,1519,0.5,"Professor",NA,"",0.391767027527364,0,0,0,0,1,0 +"EJ Masicampo, RF Baumeister","Psychology",0.21368683,-0.048918238,0.217031412373752,-0.0489573144713754,0.0953462589245592,0.0798086884467622,0.0228313705069474,0.539589812341313,0.0114156852534737,0.730205093829344,NA,160,113,"E.J. Masicampo and Roy F. Baumeister",2008,"PS","Social",6,202,158,0.619584035001245,0.0936595586606222,0.89976783513493,"United States","United States","English","English",0,0,"credit","cash","students","students",0.1322,100,1464.378793,0,"rpp",-0.228924908474706,2,3,17103.5,33557,1,"Professor",35,35,0.333333333333333,"Researcher",NA,"",0.185485691964494,1,1,1,1,1,1 +"G Hajcak, D Foti","Psychology",0.38,0.25,0.400059650056057,0.255412811882995,0.188982236504614,0.158113883008419,0.0342669191586874,0.106230060871415,0.0171334595793437,0.0531150304357076,NA,43,31,"Greg Hajcak and Dan Foti",2008,"PS","Social",6,172,40,0.528260690889257,0.348396680693637,0.713678669534678,"United States","United States","English","English",0,0,"credit","credit","students","students",NA,NA,NA,0,"rpp",0.657894736842105,2,2,4343.5,7484,1,"Associate Professor",135.5,236,0.5,"Professor",NA,"",0.416985971128854,1,1,1,1,1,1 +"GA Alvarez, A Oliva","Psychology",0.721890032,0.922934494,0.911580609250946,1.60847594653146,0.408248290463863,0.258198889747161,0.0255550546585833,4.67625936336409e-10,0.0127775273292917,2.33812968168204e-10,NA,18,9,"George A. Alvarez and Aude Oliva",2008,"PS","Cognitive",7,134,16,0.490819433311135,0.999959719895599,0.920119123101395,"United States","Germany","English","German",0,0,"cash","cash","anyone","students",NA,NA,NA,1,"rpp",1.27849735151905,2,2,5104,7443,0.5,"Professor",18.5,37,0.5,"Assistant Professor",NA,"",0.639793007944246,0,0,1,0,1,0 +"GP Lau, AC Kay, SJ Spencer","Psychology",0.384260399,-0.034279559,0.405048559767843,-0.0342929956428471,0.174077655955698,0.122169444356305,0.0199744840018644,0.778940303990297,0.00998724200093222,0.610529848004851,NA,70,36,"Grace P. Lau and Aaron C. Kay and Steven J. Spencer",2008,"PS","Social",2,54,68,0.626948359502404,0.0586763276834715,0.799853091367056,"Canada","Germany","English","German",0,0,"credit","cash","students","students",0.411,47,602.4906923,0,"rpp",-0.0892091901460811,3,2,2834.66666666667,6080,0.666666666666667,"Professor",355,673,1,"Researcher",NA,"",0.384318619638178,0,0,1,1,0,0 +"J Winawer, AC Huk, L Boroditsky","Psychology",0.685347348,0.527307869,0.83912881221931,0.58640880552693,0.192450089729875,0.204124145231932,1.29919150726939e-05,0.00406845527232685,6.49595753634693e-06,0.00203422763616343,NA,27,30,"Jonathan Winawer and Alexander C. Huk and Lera Boroditsky",2008,"PS","Cognitive",8,47,25,0.989117311748619,0.798660675127801,0.995529016381458,"United States","United States","English","English",0,0,"cash","cash","anyone","anyone",NA,NA,NA,1,"rpp",0.769402362960628,3,3,1832.33333333333,2131,0.666666666666667,"Associate Professor",157,401,0.666666666666667,"Associate Professor",NA,"",0.472582641843102,1,1,1,1,1,1 +"JA Richeson, S Trawalter","Psychology",0.371775163,0.017674908,0.390481398198824,0.0176767489061405,0.2,0.125988157669742,0.0508899036986226,0.888419136608458,0.0254449518493113,0.444209568304229,NA,66,28,"Jennifer A. Richeson and Sophie Trawalter",2008,"PS","Social",5,75,64,0.477462894679522,0.0519673333486299,0.92518906195642,"United States","United States","English","English",0,0,"credit","credit","students","students",NA,NA,NA,0,"rpp",0.0475419279151792,2,1,2182.5,3175,0,"Professor",35,35,1,"Researcher",NA,"",0.311027943561939,1,1,1,1,1,1 +"JS Nairne, JN Pandeirada, SR Thompson","Psychology",0.445652636,0.423363806,0.479262324352725,0.451783336020072,0.21320071635561,0.166666666666667,0.0245800442144559,0.00671413399059717,0.012290022107228,0.00335706699529858,NA,39,25,"James S. Nairne and Josefa N.S. Pandeirada and Sarah R. Thompson",2008,"PS","Cognitive",5,191,37,0.588258709412486,0.759609049633919,0.608458353744122,"United States","Germany","English","German",0,0,"credit","mixed","students","students",0.646,68,779.1670431,1,"rpp",0.949986091858324,3,2,1236.66666666667,2600,0.333333333333333,"Professor",153,271,0.5,"Assistant Professor",NA,"",0.537653446800911,0,0,1,1,1,0 +"JT Larsen, AR McKibban","Psychology",0.20996855,0.500179963,0.213138445861106,0.549546123796944,0.0936585811581694,0.0655121782080418,0.0228642190248385,4.92561993965075e-17,0.0114321095124193,2.46280996982538e-17,NA,236,117,"Jeff T. Larsen and Amie R. McKibban",2008,"PS","Social",7,22,234,0.619539508905307,0.999999999926348,0.800233314327382,"United States","Netherlands","English","Dutch",0,1,"credit","credit","students","students",NA,NA,NA,1,"rpp",2.38216610535244,2,6,1236,2468,0.5,"Associate Professor",35,35,0.333333333333333,"Assistant Professor",NA,"",0.209907693523954,0,0,0,1,1,0 +"KD Vohs, JW Schooler","Psychology",0.498149178,0.102355162,0.546841417850272,0.102714869812965,0.192450089729875,0.134839972492648,0.00449058854906171,0.446206921825165,0.00224529427453085,0.223103460912582,NA,58,30,"Kathleen D. Vohs and Jonathan W. Schooler",2008,"PS","Social",6,341,56,0.792655858874911,0.116851149399564,0.98323148473343,"United States","United Kingdom","English","English",0,0,"credit","credit","students","students",0.3954,108,2563.7719,0,"rpp",0.205470904139482,2,3,7347,9318,0.5,"Professor",380.666666666667,1072,0.333333333333333,"Professor",NA,"",0.358329719257349,0,1,1,1,1,0 +"N Halevy, G Bornstein, L Sagiv","Psychology",0.768651993,0.652982567,1.01702492443777,0.780480766803962,0.115470053837925,0.169030850945703,1.27746356363167e-18,3.88603124555268e-06,6.38731781815835e-19,1.94301562277634e-06,NA,38,78,"Nir Halevy and Gary Bornstein and Lilach Sagiv",2008,"PS","Social",7,126,36,0.999999999993627,0.994965319644377,0.999997950997909,"Israel","United Kingdom","Hebrew","English",0,0,"cash","cash","students","students",0.7655,47,645.7433215,1,"rpp",0.849516521061047,3,3,1054,1445,0.666666666666667,"Professor",57,80,0.333333333333333,"Researcher",NA,"",0.408040586090251,0,0,1,1,0,0 +"N Janssen, FX Alario, A Caramazza","Psychology",0.649955571,0.497082522,0.775221776719086,0.545423705409234,0.277350098112615,0.316227766016838,0.0051883364095681,0.0845669112237942,0.00259416820478405,0.0422834556118971,NA,13,16,"Niels Janssen and F.-Xavier Alario and Alfonso Caramazza",2008,"PS","Cognitive",5,15,11,0.753516515286822,0.36083290130928,0.918016595802745,"United States","United Kingdom","English","English",0,0,"credit","credit","students","students",NA,NA,NA,0,"rpp",0.764794617015445,3,1,6635.33333333333,18006,1,"Professor",360,360,0,"Researcher",NA,"",0.570001690869039,0,1,1,1,1,0 +"P Bressan, D Stranieri","Psychology",0.188662549,0.018283411,0.190949992796085,0.0182854486872586,0.0719815750748694,0.0565233418944221,0.00798361228028981,0.746314630432584,0.00399180614014491,0.373157315216292,NA,316,196,"Paola Bressan and Debora Stranieri",2008,"PS","Social",7,20,314,0.753351082649504,0.0619754627148572,0.585505936729258,"Italy","United States","Italian","English",0,1,"cash","cash","students","students",NA,NA,NA,0,"rpp",0.0969106539528415,2,2,433.5,851,0,"Researcher",242,334,0.5,"Researcher",NA,"",0.241001711283855,0,0,0,1,0,1 +"S Forti, GW Humphreys","Psychology",0.722536382,0.207514339,0.91293165300265,0.210572424096053,0.288675134594813,0.242535625036333,0.00156427175174326,0.385278096558336,0.000782135875871629,0.192639048279168,NA,20,15,"Sara Forti and Glyn W. Humphreys",2008,"PS","Cognitive",7,7,18,0.844537185963727,0.132218289715573,0.900403259192616,"United Kingdom","Germany","English","German",0,0,"cash","cash","students","students",NA,NA,NA,0,"rpp",0.287202615909243,2,2,2374.5,4654,0.5,"Professor",141.5,248,0,"Researcher",NA,"",0.657058779759921,0,0,1,1,0,0 +"S Schnall, J Benton, S Harvey","Psychology",0.400010244,0.002839798,0.423661125491188,0.00283980563384247,0.158113883008419,0.0901669634667432,0.00737392014248134,0.974874805067713,0.00368696007124067,0.487437402533856,NA,126,43,"Simone Schnall and Jennifer Benton and Sophie Harvey",2008,"PS","Social",4,246,124,0.751500148895341,0.0499839152960572,0.98716433984191,"United Kingdom","United States","English","English",0,0,"credit","credit","students","students",NA,NA,NA,0,"rpp",0.00709931318658929,3,3,410,943,0,"Researcher",1390,4049,1,"Professor",NA,"",0.275865977707637,0,1,1,1,0,1 +"SE Palmer, T Ghose","Psychology",0.859552403,0.120073568,1.29162832504362,0.120655671934318,0.408248290463863,0.408248290463863,0.00155707538638029,0.76757771086608,0.000778537693190146,0.38378885543304,NA,9,9,"Stephen E. Palmer and Tandra Ghose",2008,"PS","Cognitive",8,38,7,0.776346912326851,0.0570134653754987,0.907500479518165,"United States","United States","English","English",0,0,"credit","cash","students","community",NA,NA,NA,0,"rpp",0.139693132822293,2,3,1502.5,2940,0.5,"Professor",2072.66666666667,6148,0.333333333333333,"Associate Professor",NA,"",0.799713352675318,1,1,1,0,1,1 +"SK Moeller, MD Robinson, DL Zabelina","Psychology",0.31,-0.034,0.320545409301946,-0.0340131104279282,0.14142135623731,0.120385853085769,0.023414774454383,0.777533991849339,0.0117073872271915,0.611233004075331,NA,72,53,"Sara K. Moeller and Michael D. Robinson and Darya L. Zabelina",2008,"PS","Social",7,33,69,0.60126059524309,0.0586680084331144,0.763910991680809,"United States","United States","English","English",0,0,"credit","mixed","students","students",NA,NA,NA,0,"rpp",-0.109677419354839,3,2,1484,4121,0.333333333333333,"Professor",218,401,0,"Associate Professor",NA,"",0.323206406832095,1,1,1,1,1,1 +"T Goschke, G Dreisbach","Psychology",0.374787884,0.411336583,0.393981877130233,0.437218946739661,0.164398987305357,0.104257207028537,0.0165525747937481,2.74493164235645e-05,0.00827628739687404,1.37246582117822e-05,NA,95,40,"Thomas Goschke and Gesine Dreisbach",2008,"PS","Social",8,45,93,0.654437907458993,0.986363997165243,0.958450433478,"Germany","Italy","German","Italian",0,0,"mixed","credit","students","students",NA,NA,NA,1,"rpp",1.09751835787733,2,2,1536,2032,0.5,"Professor",1526,2989,1,"Professor",NA,"",0.290297003545121,0,0,1,1,0,0 +"V Lobue, JS DeLoache","Psychology",0.483442318,0.177837588,0.527466797836033,0.179748763345337,0.152498570332605,0.152498570332605,0.000542524997916142,0.238521055287891,0.000271262498958071,0.119260527643946,NA,46,46,"Vanessa LoBue and Judy S. DeLoache",2008,"PS","Cognitive",6,143,44,0.926528236257896,0.213078788855479,0.855484679781282,"United States","United States","English","English",0,0,"nothing","nothing","anyone","anyone",0.8309,69,1155.332901,0,"rpp",0.367856890840077,2,5,1860.5,3266,0,"Professor",44.2,116,0.2,"Professor",NA,"",0.453597235153017,1,1,1,1,1,1 +"Z Estes, M Verges, LW Barsalou","Psychology",0.595271914,0.25414164,0.685792015328455,0.259835474573725,0.25,0.223606797749979,0.00608495338227784,0.245227516439382,0.00304247669113892,0.122613758219691,NA,23,19,"Zachary Estes and Michelle Verges and Lawrence W. Barsalou",2008,"PS","Cognitive",5,91,21,0.748039466859184,0.202355341611864,0.822736588190895,"United Kingdom","Germany","English","German",0,0,"credit","credit","students","students",0.4209,75,877.1563848,0,"rpp",0.426933698739901,3,2,3845.33333333333,10614,0.666666666666667,"Professor",153,271,0.5,"Assistant Professor",NA,"",0.582147807147936,0,0,1,1,0,0 +"Abeler et al. (AER 2011)","Experimental Economics",0.182821975588,0.0790703532018,0.184900697919314,0.0792357599310821,0.0924500327042048,0.0563436169819011,0.045499525126659,0.159636475304643,0.0227497625633295,0.0798182376523215,0.696,318,120,"Johannes Abeler and Armin Falk and Lorenz Goette and David Huffman",2011,"The American Economic Review","Economics",23,311,318,0.518864645035727,0.290890403272648,0.907643887957432,"Germany","Austria","German","German",0,0,"cash","cash","students","students",0.6438279,124,1840.86240082979,0,"ee",0.432499179392936,4,4,1520.5,4182,1,"Professor",129.25,258,1,"Professor",NA,"",0.156337494342257,0,1,1,1,0,0 +"Ambrus and Greiner (AER 2012)","Experimental Economics",0.310518647505,0.229536356959,0.321119300033783,0.233699982142729,0.166666666666667,0.0928476690885259,0.0540150576876717,0.0118350149503858,0.0270075288438358,0.00591750747519289,0.692,119,39,"Attila Ambrus and Ben Greiner",2012,"The American Economic Review","Economics",16,58,357,0.930800055311097,0.992712209889608,0.999959766855466,"Australia","Singapore","English","English",0,0,"cash","cash","students","students",0.6917935,101,1514.69622662291,1,"ee",0.739203104225785,2,2,109,130,1,"Professor",1669.5,3339,1,"Professor",NA,"",0.151243059015666,0,1,1,1,0,0 +"Bartling et al. (AER 2012)","Experimental Economics",0.719849875686,0.657411288278,0.907333333334142,0.788240781367703,0.333333333333333,0.242535625036333,0.00648881424644106,0.00115405008478788,0.00324440712322053,0.000577025042393941,0.805,20,12,"BjΓΆrn Bartling and Ernst Fehr and Klaus M. Schmidt",2012,"The American Economic Review","Economics",31,72,360,1,1,1,"Switzerland","Austria","German","German",0,0,"cash","cash","students","students",0.805131,95,1401.34383332729,1,"ee",0.913261637273594,3,4,8886.33333333333,22239,1,"Professor",129.25,258,1,"Professor",NA,"",0.147019296182102,0,1,1,1,0,0 +"Charness and Dufwenberg (AER 2011)","Experimental Economics",0.383943377571,0.363002684809,0.404676672172116,0.380339980340638,0.158113883008419,0.127000127000191,0.0104853017515998,0.00274624846486566,0.0052426508757999,0.00137312423243283,0.695,65,43,"Gary Charness and Martin Dufwenberg",2011,"The American Economic Review","Economics",27,63,264,0.99920300841706,0.999986371256698,0.999997773778529,"United States","Singapore","English","English",0,0,"cash","cash","community","students",0.6952792,84,1179.50625985861,1,"ee",0.945458911155077,2,2,2470,3080,1,"Professor",1669.5,3339,1,"Professor",NA,"",0.171411332315411,0,1,1,0,1,0 +"Chen and Chen (AER 2011)","Experimental Economics",0.842508557739,0.170189166838,1.22975607337423,0.171861466409585,0.577350269189626,0.301511344577764,0.0331716133671628,0.568677698092065,0.0165858066835814,0.284338849046032,0.778,14,6,"Roy Chen and Yan Chen",2011,"The American Economic Review","Economics",28,114,168,1,0.600022757587298,1,"United States","Singapore","English","English",0,0,"cash","cash","students","students",0.7781491,119,1826.34612164274,0,"ee",0.202002899555144,2,2,363.5,693,0.5,"Professor",1669.5,3339,1,"Professor",NA,"",0.214156820187213,0,1,1,1,1,0 +"de Clippel et al. (AER 2014)","Experimental Economics",0.117768981826,0.266538269195,0.118318025033561,0.273133631334902,0.035646150289976,0.0358748001667088,0.000902637488494951,2.6672296555751e-14,0.000451318744247476,1.33361482778755e-14,0.759,780,790,"Geoffroy De Clippel and Kfir Eliaz and Brian G. Knight",2014,"The American Economic Review","Economics",25,7,156,0.314800287089614,0.923393074510633,0.31144040857597,"United States","Singapore","English","English",0,0,"cash","cash","students","students",0.7590009,81,735.444808721542,1,"ee",2.26322975710582,3,2,315.333333333333,437,1,"Professor",1669.5,3339,1,"Professor",NA,"",0.222079964538651,0,1,1,1,1,0 +"Duffy and Puzzello (AER 2014)","Experimental Economics",0.761510174904,-0.11596300548,0.99980006334502,-0.116487041206908,0.408248290463863,0.277350098112615,0.0143253448140312,0.674485453697526,0.0071626724070156,0.662757273151237,0.806,16,9,"John Duffy and Daniela Puzzello",2014,"The American Economic Review","Economics",42,8,96,0.999999914711349,0.203664356811666,0.999999999999994,"United States","Austria","English","German",0,0,"cash","cash","students","students",0.8063298,115,1166.89113223553,0,"ee",-0.152280299755411,2,4,588.5,1051,0.5,"Professor",129.25,258,1,"Professor",NA,"",0.281375882936472,0,0,1,1,1,0 +"Dulleck et al. (AER 2011)","Experimental Economics",0.722509234548,0.731605727911,0.912874854511016,0.932173679757267,0.235702260395516,0.277350098112615,0.000107503827691344,0.000776608255431059,5.37519138456719e-05,0.00038830412771553,0.738,16,21,"Uwe Dulleck and Rudolf Kerschbamer and Matthias Sutter",2011,"The American Economic Review","Economics",30,115,128,1,1,1,"Austria","Germany","German","German",0,0,"cash","cash","students","students",0.7378578,75,775.856934577227,1,"ee",1.01259015092693,3,3,753.333333333333,1786,1,"Professor",3601.66666666667,9699,0.333333333333333,"Professor",NA,"",0.244649774891448,0,1,1,1,0,0 +"Fehr et al. (AER 2013)","Experimental Economics",0.453281944406,0.311199311719,0.488823227914378,0.321872775072724,0.192450089729875,0.144337567297406,0.0110852468860838,0.0257474428774305,0.00554262344304188,0.0128737214387152,0.629,51,30,"Ernst Fehr and Holger Herz and Tom Wilkening",2013,"The American Economic Review","Economics",35,82,102,0.960567998425193,0.89555475641947,0.998276368400413,"Switzerland","Austria","German","German",0,0,"cash","cash","students","students",0.6289412,193,2849.37293988466,1,"ee",0.686546898382936,3,4,7435.33333333333,22239,1,"Professor",129.25,258,1,"Professor",NA,"",0.273229674690716,0,1,1,1,0,0 +"Friedman and Oprea (AER 2012)","Experimental Economics",0.642590125822,0.437953607707,0.76257319491392,0.469695940515872,0.115470053837925,0.164398987305357,4.00000000001255e-11,0.00427599999997553,2.00000000000628e-11,0.00213799999998777,0.833,40,78,"Daniel Friedman and Ryan Oprea",2012,"The American Economic Review","Economics",27,52,40,0.999998555141462,0.8232682054637,0.996788338227856,"United States","Singapore","English","English",0,0,"cash","cash","students","students",0.8329033,188,2247.26402801275,1,"ee",0.681544289562751,2,3,1132.5,2140,1,"Professor",6063,18185,1,"Professor",NA,"",0.426537652439599,0,1,1,1,1,0 +"Fudenberg et al. (AER 2012)","Experimental Economics",0.303741608956,0.326573539422,0.313636363636516,0.338987905389151,0.0909090909090909,0.0894427190999916,0.000560586553628869,0.000150647284756123,0.000280293276814434,7.53236423780617e-05,0.933,128,124,"Drew Fudenberg and David G. Rand and Anna Dreber",2012,"The American Economic Review","Economics",30,122,128,0.933518139734671,0.967360586346617,0.925798977579971,"United States","United States","English","English",0,0,"cash","cash","students","students",0.9331694,84,884.62292265892,1,"ee",1.07516899232769,3,3,1702.33333333333,3979,0.666666666666667,"Professor",6063,18185,1,"Professor",NA,"",0.25248505382702,1,1,1,1,1,1 +"Huck et al. (AER 2011)","Experimental Economics",0.832065702534,0.367593525514,1.19481329601125,0.385637790157604,0.333333333333333,0.277350098112615,0.00033780235064149,0.164396270288325,0.000168901175320745,0.0821981351441625,0.92,16,12,"Steffen Huck and Andrew J. Seltzer and Brian Wallace",2011,"The American Economic Review","Economics",25,14,160,1,0.998047234794973,1,"United Kingdom","United Kingdom","English","English",0,0,"cash","cash","students","students",0.9204491,74,733.880657434464,0,"ee",0.441784260545772,3,3,597.333333333333,1599,1,"Professor",6063,18185,1,"Professor",NA,"",0.219342541352358,1,1,1,1,0,0 +"Ifcher and Zarghamee (AER 2011)","Experimental Economics",0.282103220856,-0.00701629144787,0.289965676326035,-0.00701640658474442,0.134839972492648,0.0883883476483184,0.0315201893322661,0.936729120378959,0.0157600946661331,0.531635439810521,0.588,131,58,"John Ifcher and Homa Zarghamee",2011,"The American Economic Review","Economics",21,91,131,0.581817191829917,0.0506048355309016,0.908546253759074,"United States","United Kingdom","English","English",0,0,"cash","cash","students","students",0.587827,165,2371.09555201232,0,"ee",-0.0248713632149028,2,3,40,54,0.5,"Assistant Professor",6063,18185,1,"Professor",NA,"",0.241898411765474,0,1,1,1,1,0 +"Kessler and Roth (AER 2012)","Experimental Economics",0.486223364603,0.34463841128,0.531102458556899,0.359346644766374,0.0592348877759092,0.149071198499986,3.07362367504119e-19,0.0159275894040055,1.5368118375206e-19,0.00796379470200273,0.937,48,288,"Judd B. Kessler and Alvin E. Roth",2012,"The American Economic Review","Economics",30,37,48,0.99999999999889,0.681683002988294,0.948941107532118,"United States","United States","English","English",0,0,"cash","cash","students","students",0.9369977,144,1928.54822969064,1,"ee",0.708806756861563,2,5,3210,6401,1,"Professor",2168,9699,0.4,"Professor",NA,"",0.391767027527364,1,1,1,1,1,1 +"Kirchler et al (AER 2012)","Experimental Economics",0.664409308738,0.533556821497,0.800666666666098,0.595104413660859,0.333333333333333,0.229415733870562,0.0163057084315163,0.00948664869157146,0.00815285421575816,0.00474332434578573,0.712,22,12,"Michael Kirchler and JΓΌrgen Huber and Thomas StΓΆckl",2012,"The American Economic Review","Economics",19,102,220,0.999999999991495,0.999999999995567,1,"Germany","Germany","German","German",0,0,"cash","cash","students","students",0.7121724,78,865.73511326313,1,"ee",0.803054436566569,3,3,188.333333333333,258,1,"Professor",3601.66666666667,9699,0.333333333333333,"Professor",NA,"",0.187575889817031,1,1,1,1,0,0 +"Kogan et al. (AER 2011)","Experimental Economics",0.323896842962,0.304223231247,0.335994578360668,0.31416702155206,0.0798086884467622,0.0957826285221151,2.55370688277548e-05,0.00103807086643231,1.27685344138774e-05,0.000519035433216156,0.802,112,160,"Shimon Kogan and Anthony M. Kwasnica and Roberto A. Weber",2011,"The American Economic Review","Economics",21,20,90,0.962435923367311,0.837644076137814,0.882867728993477,"United States","Austria","English","German",0,0,"cash","cash","students","students",0.8020024,72,947.360323607922,1,"ee",0.939259614928036,3,4,458.666666666667,1052,1,"Professor",129.25,258,1,"Professor",NA,"",0.290297003545121,0,0,1,1,1,0 +"Kuziemko et al. (QJE 2014)","Experimental Economics",0.282261740933,-0.119848901099,0.290137910919302,-0.120427724712724,0.160128153805087,0.0842151921066519,0.0700000000003192,0.152717019072755,0.0350000000001596,0.923641490463622,0.632,144,42,"Ilyana Kuziemko and Ryan W. Buell and Taly Reich and Michael I. Norton",2014,"The Quarterly Journal of Economics","Economics",45,48,144,0.448454931064687,0.299807891736904,0.922503277030528,"United States","United States","English","English",0,0,"cash","cash","students","students",0.6315749,166,2684.01664584875,0,"ee",-0.424602017425458,4,4,1005.5,3487,0.5,"Professor",2709.75,9699,0.5,"Professor",NA,"",0.235819735848147,1,1,1,1,1,1 +"Ericson and Fuster (QJE 2011)","Experimental Economics",0.212921823047,0.122871403263,0.216229939365633,0.123495411116135,0.0995037190209989,0.063887656499994,0.0297739959884483,0.0532350966055683,0.0148869979942242,0.0266175483027842,0.622,248,104,"Keith M. Marzilli Ericson and Andreas Fuster",2011,"The Quarterly Journal of Economics","Economics",29,129,262,0.620338048136283,0.512313004378123,0.925704754055452,"United States","United States","English","English",0,0,"cash","cash","students","students",0.6224774,122,1787.08544743061,0,"ee",0.577072862843608,2,4,63,115,1,"Assistant Professor",2709.75,9699,0.5,"Professor",NA,"",0.176117007593052,1,1,1,1,1,1 diff --git a/examples/discovery/data/demo/meta_regression/metadata_0.json b/examples/discovery/data/demo/meta_regression/metadata_0.json new file mode 100644 index 000000000..576550ce1 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_0.json @@ -0,0 +1,287 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering, data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "For which domains do the effect size estimates tend to be larger in original studies compared to replication studies?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "In Experimental Economics, what is the average effect estimate in original studies as compared to that in replication studies?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "In Psychology, what is the average effect estimate in original studies as compared to that in replication studies?" + }, + { + "qid": 3, + "question_type": "variable", + "question": "Which factor in Experimental Economics has a value of 0.57 on the Fisher-z scale in original studies compared to 0.31 in replication studies?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_1.json b/examples/discovery/data/demo/meta_regression/metadata_1.json new file mode 100644 index 000000000..6052f02bd --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_1.json @@ -0,0 +1,277 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "In Experimental Economics, what were the average observed power in original studies and the planned power in replication studies?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "In Psychology, what were the average observed power in original studies and the planned power in replication studies?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_10.json b/examples/discovery/data/demo/meta_regression/metadata_10.json new file mode 100644 index 000000000..021f596f7 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_10.json @@ -0,0 +1,277 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "What is the proportion of junior professors in replicated studies in Psychology?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What is the proportion of senior professors in original studies in Psychology?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_11.json b/examples/discovery/data/demo/meta_regression/metadata_11.json new file mode 100644 index 000000000..2c03bf38b --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_11.json @@ -0,0 +1,277 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In which domain do all original and replication studies use cash compensation for participants?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What type of compensation is used for participants in all original and replication studies in Experimental Economics?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_12.json b/examples/discovery/data/demo/meta_regression/metadata_12.json new file mode 100644 index 000000000..13fa15436 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_12.json @@ -0,0 +1,277 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In which field a mix of compensation types (cash, credit, or mixed) was used in both original and replication studies (original: cash: 20.6%, credit or mixed: 73.5%, nothing: 5.9%; replication: cash: 36.8%, credit or mixed: 57.4%, nothing: 5.9%).?" + }, + { + "qid": 1, + "question_type": "context", + "question": "What type of compensation is used in original studies in Psychology?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_13.json b/examples/discovery/data/demo/meta_regression/metadata_13.json new file mode 100644 index 000000000..c75d5766d --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_13.json @@ -0,0 +1,287 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "What proportion of subjects were students in original studies in Experimental Economics?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What proportion of subjects were students in replication studies in Experimental Economics?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "What proportion of subjects were from the community in original studies in Experimental Economics?" + }, + { + "qid": 3, + "question_type": "relationship", + "question": "What type of subjects were used in all replication studies in Experimental Economics?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_14.json b/examples/discovery/data/demo/meta_regression/metadata_14.json new file mode 100644 index 000000000..47d64e150 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_14.json @@ -0,0 +1,272 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In which domain did both original and replication studies primarily use student subjects (original: 82.4%, replication: 80.9%)?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_15.json b/examples/discovery/data/demo/meta_regression/metadata_15.json new file mode 100644 index 000000000..6b5c9328b --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_15.json @@ -0,0 +1,282 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In which country were the majority of original studies in Experimental Economics conducted?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What proportion of replication studies in Experimental Economics were conducted in the United States?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "What proportion of original studies in Experimental Economics were conducted in the United Kingdom?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_16.json b/examples/discovery/data/demo/meta_regression/metadata_16.json new file mode 100644 index 000000000..2eac5dc66 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_16.json @@ -0,0 +1,277 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "What proportion of original studies in Psychology were conducted in the United States?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What proportion of replication studies in Psychology were conducted in the United States?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_17.json b/examples/discovery/data/demo/meta_regression/metadata_17.json new file mode 100644 index 000000000..bbb26f7cf --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_17.json @@ -0,0 +1,282 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In which country's labs were the majority of original studies in both Experimental Economics and Psychology conducted?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What proportion of original studies in Experimental Economics were conducted in the labs of United States?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "What proportion of original studies in Psychology were conducted in the labs of United States?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_18.json b/examples/discovery/data/demo/meta_regression/metadata_18.json new file mode 100644 index 000000000..3230594b7 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_18.json @@ -0,0 +1,282 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In which domain were all studies, both original and replication, conducted in a lab setting?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What proportion of studies in Experimental Economics were conducted in a lab setting?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "What proportion of replication studies in Psychology were conducted online?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_19.json b/examples/discovery/data/demo/meta_regression/metadata_19.json new file mode 100644 index 000000000..34ec7651d --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_19.json @@ -0,0 +1,292 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In which domains, a significant proportion of replication studies were conducted in a different country or language setting compared to the original study?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "In Experimental Economics, what is the proportion of replication studies conducted in a different country compared to the original study?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "In Psychology, what is the proportion of replication studies conducted in a different country compared to the original study?" + }, + { + "qid": 3, + "question_type": "relationship", + "question": "In Psychology, what is the proportion of replication studies conducted in a different language compared to the original study?" + }, + { + "qid": 4, + "question_type": "relationship", + "question": "In Experimental Economics, what is the proportion of replication studies conducted in a different language compared to the original study?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_2.json b/examples/discovery/data/demo/meta_regression/metadata_2.json new file mode 100644 index 000000000..3ae0a2a45 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_2.json @@ -0,0 +1,277 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "Which domain tend to have longer original papers?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What is the average length of original papers in Experimental Economics and that in Psychology?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_3.json b/examples/discovery/data/demo/meta_regression/metadata_3.json new file mode 100644 index 000000000..72eb7528f --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_3.json @@ -0,0 +1,282 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In which domain do original studies tend to receive comparetively more citations?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "In psychology, what is the average number of citations received by the original studies?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "In experimental economics, what is the average number of citations received by the original studies?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_4.json b/examples/discovery/data/demo/meta_regression/metadata_4.json new file mode 100644 index 000000000..57a61f6d3 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_4.json @@ -0,0 +1,272 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "What is the average number of authors for original studies compared to replication studies in the field of Experimental Economics?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_5.json b/examples/discovery/data/demo/meta_regression/metadata_5.json new file mode 100644 index 000000000..66f71f2f3 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_5.json @@ -0,0 +1,282 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "Fow which domain do authors of original studies tend to have higher citation metrics compared to authors of replication studies?" + }, + { + "qid": 1, + "question_type": "context", + "question": "What are the average author citations for original studies and replication studies in Psychology?" + }, + { + "qid": 2, + "question_type": "context", + "question": "What are the maximum author citations for original studies and replication studies in Psychology?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_6.json b/examples/discovery/data/demo/meta_regression/metadata_6.json new file mode 100644 index 000000000..e5e004b3f --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_6.json @@ -0,0 +1,282 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In which domain do authors of replication studies tend to have higher citation metrics compared to authors of original studies?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What is the average number of author citations for replication studies in Experimental Economics?" + }, + { + "qid": 2, + "question_type": "relationship", + "question": "What is the maximum number of author citations for original studies in Experimental Economics compared to replication studies?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_7.json b/examples/discovery/data/demo/meta_regression/metadata_7.json new file mode 100644 index 000000000..5fb9ef023 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_7.json @@ -0,0 +1,277 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In which domain do the majority of authors tend to be male for both original and replication studies?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What is the ratio of male authors in replication studies in Experimental Economics?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_8.json b/examples/discovery/data/demo/meta_regression/metadata_8.json new file mode 100644 index 000000000..a2604d0e6 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_8.json @@ -0,0 +1,277 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In which domain is there a more balanced gender representation of authors, particularly in replication studies?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What is the ratio of male authors in original studies compared to replication studies in Psychology?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/meta_regression/metadata_9.json b/examples/discovery/data/demo/meta_regression/metadata_9.json new file mode 100644 index 000000000..1e1f06a42 --- /dev/null +++ b/examples/discovery/data/demo/meta_regression/metadata_9.json @@ -0,0 +1,272 @@ +{ + "id": 0, + "domain": "meta science", + "workflow_tags": "data selection, meta regression, summary statistics, feature engineering,data standardization", + "domain_knowledge": "1. When working with multiple studies that are original and replicated, a preprocess & joining step is needed to gather the data required for metaregression analysis. 2. Combining such study data has to account for study & author names and may need multiple heuristics for the ideal join depending on the line of inquiry. 3. An extra step may also be required to handle duplicate values & missing pairs.", + "datasets": [ + { + "name": "meta-regression_joined_data_heterogeneity_in_replication_projects.csv", + "description": "Data created by merging data.csv and RProjects.csv", + "columns": { + "raw": [ + { + "name": "study", + "description": "Study identifier, usually names of authors from original study" + }, + { + "name": "project.x", + "description": "The replication project that the study was on (Psychology or Experimental Economics)" + }, + { + "name": "ro", + "description": "Effect estimate of original study on correlation scale" + }, + { + "name": "rr", + "description": "Effect estimate of replication study on correlation scale" + }, + { + "name": "fiso", + "description": "Effect estimate of original study transformed to Fisher-z scale" + }, + { + "name": "fisr", + "description": "Effect estimate of replication study transformed to Fisher-z scale" + }, + { + "name": "se_fiso", + "description": "Standard error of Fisher-z transformed effect estimate of original study" + }, + { + "name": "se_fisr", + "description": "Standard error of Fisher-z transformed effect estimate of replication study" + }, + { + "name": "po", + "description": "Two-sided p-value from significance test of effect estimate from original study" + }, + { + "name": "pr", + "description": "Two-sided p-value from significance test of effect estimate from replication study" + }, + { + "name": "po1", + "description": "One-sided p-value from significance test of effect estimate from original study (in the direction of the original effect estimate)" + }, + { + "name": "pr1", + "description": "One-sided p-value from significance test of effect estimate from replication study (in the direction of the original effect estimate)" + }, + { + "name": "pm_belief", + "description": "Peer belief about whether replication effect estimate will achieve statistical significance elicited through prediction market (only available for EERP and SSRP)" + }, + { + "name": "no", + "description": "Sample size in original study" + }, + { + "name": "nr", + "description": "Sample size in replication study" + }, + { + "name": "authors.o", + "description": "Names of Original paper's authors" + }, + { + "name": "pub_year", + "description": "Year of Publication of the study" + }, + { + "name": "journal", + "description": "Journal in which the study was published" + }, + { + "name": "discipline", + "description": "Discipline of original paper. One of the following: Social, Cognitive or Economics" + }, + { + "name": "length", + "description": "Number of pages of original paper" + }, + { + "name": "citations", + "description": "Number of citations of original paper" + }, + { + "name": "n.r", + "description": "Sample size of replication" + }, + { + "name": "power.o", + "description": "Post hoc power based on original effect size" + }, + { + "name": "power.r", + "description": "Post hoc power based on replication effect size" + }, + { + "name": "power_planned.r", + "description": "Planned power of the replication based on planned N and original ES" + }, + { + "name": "experiment_country.o", + "description": "Country where original experiment was conducted" + }, + { + "name": "experiment_country.r", + "description": "Country where replication is to be conducted" + }, + { + "name": "experiment_language.o", + "description": "Language used with subjects in original experiment (English, German, Dutch, Polish, Hebrew, French, Italian, Arabic, Spanish, Korean)" + }, + { + "name": "experiment_language.r", + "description": "Language to be used with subjects in replication (English, Polish, German, Dutch, Italian, Portuguese, Malay, Turkish, Czech, Arabic, Spanish)" + }, + { + "name": "online.o", + "description": "If the original experiment was conducted online (1: yes, 0: no)" + }, + { + "name": "online.r", + "description": "If the replication was conducted online (1: yes, 0: no)" + }, + { + "name": "compensation.o", + "description": "Compensation in original experiment (credit, cash, nothing, mixed)" + }, + { + "name": "compensation.r", + "description": "Compensation in replication (credit, cash, nothing, mixed)" + }, + { + "name": "subjects.o", + "description": "Type of subjects used in original experiment (students, online, anyone, community)" + }, + { + "name": "subjects.r", + "description": "Type of subjects used in replication (students, online, anyone, community)" + }, + { + "name": "endprice", + "description": "Final market price in prediction market" + }, + { + "name": "transactions", + "description": "Number of transactions in prediction market" + }, + { + "name": "trading_volume", + "description": "Total volume of traded stocks in prediction market" + }, + { + "name": "replicated", + "description": "Binary outcome variable; study is replicated if p <= 0.05 and effect goes in the same direction as the original" + }, + { + "name": "project.y", + "description": "The replication project that the study was on (rpp: Psychology or ee: Experimental Economics)" + }, + { + "name": "relative_es", + "description": "The continuous outcome variable; the standardized replication effect size to the original effect | relative effect size = (replication effect size / original effect size)" + }, + { + "name": "n_authors.o", + "description": "Number of authors in original study" + }, + { + "name": "n_authors.r", + "description": "Number of authors in replication" + }, + { + "name": "author_citations_avg.o", + "description": "Average number of citations of authors in original study" + }, + { + "name": "author_citations_max.o", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.o", + "description": "Ratio of male authors in original study" + }, + { + "name": "seniority.o", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "author_citations_avg.r", + "description": "Average number of citations of authors in replication study" + }, + { + "name": "author_citations_max.r", + "description": "Number of citations of the author in original study with the highest citation count" + }, + { + "name": "authors_male.r", + "description": "Ratio of male authors in replication" + }, + { + "name": "seniority.r", + "description": "Most senior author in the original paper (Professor, Associate Professor, Assistant, Researcher, Assistant Professor)" + }, + { + "name": "aggregated", + "description": "Aggregated column" + }, + { + "name": "lab_id", + "description": "Unique id for each replication lab" + }, + { + "name": "es_80power", + "description": "Standardized effect size required in replication to achieve 80% power" + }, + { + "name": "same_country", + "description": "Original study and replication are in the same country" + }, + { + "name": "same_language", + "description": "Original study and replication are in the same language" + }, + { + "name": "same_online", + "description": "Original study and replication are both conducted online" + }, + { + "name": "same_subjects", + "description": "Original study and replication use same type of subjects" + }, + { + "name": "us_lab.o", + "description": "Original experiment lab in the US" + }, + { + "name": "us_lab.r", + "description": "Replication experiment lab in the US" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "How are the studies in Psychology divided compared to those in Experimental Economics?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/worldbank_education_gdp/metadata_0.json b/examples/discovery/data/demo/worldbank_education_gdp/metadata_0.json new file mode 100644 index 000000000..1b2e1c1b6 --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp/metadata_0.json @@ -0,0 +1,216 @@ +{ + "id": 0, + "domain": "economics", + "workflow_tags": "data selection, econometric modelling, mixture models", + "datasets": [ + { + "name": "worldbank_education_gdp.csv", + "description": "This is the panel data created based on the information provided in the paper. The dataset contains information of two groups of countries- Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015, sourced from the World Development Indicators (WDI) database. It focuses on the relationship between government expenditure on education (% of total expenditure) and per capita GDP (in 2010 US$), using variables like total labor force, gross primary and secondary enrollment, and exports (annual % growth) as key indicators.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "Series Name", + "description": "The name of the indicator or variable being measured. Description of the different indicators in the series is as follows:-GNI per capita (constant 2015 US dollar) - Gross National Income (GNI) per capita adjusted for inflation to 2015 US dollars. Adjusted savings: education expenditure (percentage of GNI) - The percentage of GNI that is saved for future education expenditures. Exports of goods and services (annual percentage growth) - The annual percentage growth rate of exports of goods and services. School enrollment, primary (percentage gross) - The gross enrollment ratio for primary school, representing the percentage of children of official primary school age who are enrolled in primary school. School enrollment, secondary (percentage gross) - The gross enrollment ratio for secondary school, representing the percentage of children of official secondary school age who are enrolled in secondary school. Labor force participation rate, total (percentage of total population ages 15+) (modeled ILO estimate) - The percentage of the total population ages 15 and older that is economically active." + }, + { + "name": "Series Code", + "description": "The code representing the indicator or variable" + }, + { + "name": "1975 [YR1975]", + "description": "The value of each indicator or variable for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of each indicator or variable for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of each indicator or variable for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of each indicator or variable for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of each indicator or variable for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of each indicator or variable for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of each indicator or variable for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of each indicator or variable for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of each indicator or variable for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of each indicator or variable for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of each indicator or variable for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of each indicator or variable for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of each indicator or variable for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of each indicator or variable for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of each indicator or variable for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of each indicator or variable for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of each indicator or variable for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of each indicator or variable for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of each indicator or variable for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of each indicator or variable for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of each indicator or variable for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of each indicator or variable for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of each indicator or variable for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of each indicator or variable for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of each indicator or variable for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of each indicator or variable for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of each indicator or variable for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of each indicator or variable for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of each indicator or variable for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of each indicator or variable for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of each indicator or variable for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of each indicator or variable for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of each indicator or variable for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of each indicator or variable for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of each indicator or variable for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of each indicator or variable for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of each indicator or variable for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of each indicator or variable for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of each indicator or variable for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of each indicator or variable for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of each indicator or variable for the year 2015" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In what regions does increased education spending positively impact per capita GDP?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What is the impact of increased education expenditure on per capita GDP in developing countries?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/worldbank_education_gdp/metadata_1.json b/examples/discovery/data/demo/worldbank_education_gdp/metadata_1.json new file mode 100644 index 000000000..375623d35 --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp/metadata_1.json @@ -0,0 +1,212 @@ +{ + "id": 0, + "domain": "economics", + "workflow_tags": "data selection, econometric modelling, mixture models", + "domain_knowledge": "Managing endogeneity and autoregression can be important in econometrics data to ensure accurate and reliable estimates, enabling valid causal inferences from the data. This will be relevant to extracting correct inferences from the data. 1. Government’s expenditure on education could be a proxy for the investment in human capital. 2. Exports of goods and services could be a proxy for economic output. 3. Terminology: Adjusted Savings: Education Expenditure (percentage of GNI) - This metric indicates the proportion of Gross National Income (GNI) allocated for future educational spending. GNI Per Capita (constant 2015 US dollars) - Represents the Gross National Income per capita adjusted for inflation to the year 2015, expressed in US dollars. School Enrollment, Primary (percentage gross) - The gross enrollment ratio for primary education, which measures the percentage of children of official primary school age who are enrolled in primary school. School Enrollment, Secondary (percentage gross) - The gross enrollment ratio for secondary education, indicating the percentage of children of official secondary school age who are enrolled in secondary school. Labor Force Participation Rate, Total (percentage of total population ages 15+) (modeled ILO estimate) - This figure represents the percentage of the total population aged 15 and older who are economically active, as estimated by the International Labour Organization (ILO). Exports of Goods and Services (annual percentage growth) - Reflects the annual percentage growth rate of exports of goods and services. Lower middle income countries could be assumed to be coming under the label of developing countries.", + "datasets": [ + { + "name": "worldbank_education_gdp.csv", + "description": "This is the panel data created based on the information provided in the paper. The dataset contains information of two groups of countries- Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015, sourced from the World Development Indicators (WDI) database. It focuses on the relationship between government expenditure on education (% of total expenditure) and per capita GDP (in 2010 US$), using variables like total labor force, gross primary and secondary enrollment, and exports (annual % growth) as key indicators.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "Series Name", + "description": "The name of the indicator or variable being measured. Description of the different indicators in the series is as follows:-GNI per capita (constant 2015 US dollar) - Gross National Income (GNI) per capita adjusted for inflation to 2015 US dollars. Adjusted savings: education expenditure (percentage of GNI) - The percentage of GNI that is saved for future education expenditures. Exports of goods and services (annual percentage growth) - The annual percentage growth rate of exports of goods and services. School enrollment, primary (percentage gross) - The gross enrollment ratio for primary school, representing the percentage of children of official primary school age who are enrolled in primary school. School enrollment, secondary (percentage gross) - The gross enrollment ratio for secondary school, representing the percentage of children of official secondary school age who are enrolled in secondary school. Labor force participation rate, total (percentage of total population ages 15+) (modeled ILO estimate) - The percentage of the total population ages 15 and older that is economically active." + }, + { + "name": "Series Code", + "description": "The code representing the indicator or variable" + }, + { + "name": "1975 [YR1975]", + "description": "The value of each indicator or variable for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of each indicator or variable for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of each indicator or variable for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of each indicator or variable for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of each indicator or variable for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of each indicator or variable for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of each indicator or variable for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of each indicator or variable for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of each indicator or variable for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of each indicator or variable for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of each indicator or variable for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of each indicator or variable for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of each indicator or variable for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of each indicator or variable for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of each indicator or variable for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of each indicator or variable for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of each indicator or variable for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of each indicator or variable for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of each indicator or variable for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of each indicator or variable for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of each indicator or variable for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of each indicator or variable for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of each indicator or variable for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of each indicator or variable for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of each indicator or variable for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of each indicator or variable for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of each indicator or variable for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of each indicator or variable for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of each indicator or variable for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of each indicator or variable for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of each indicator or variable for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of each indicator or variable for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of each indicator or variable for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of each indicator or variable for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of each indicator or variable for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of each indicator or variable for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of each indicator or variable for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of each indicator or variable for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of each indicator or variable for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of each indicator or variable for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of each indicator or variable for the year 2015" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "What relationship exists between education expenditure and per capita GDP in developing countries and how does it affect economic output?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/worldbank_education_gdp/metadata_2.json b/examples/discovery/data/demo/worldbank_education_gdp/metadata_2.json new file mode 100644 index 000000000..99a207eb0 --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp/metadata_2.json @@ -0,0 +1,212 @@ +{ + "id": 0, + "domain": "economics", + "workflow_tags": "data selection, econometric modelling, mixture models", + "domain_knowledge": "Managing endogeneity and autoregression can be important in econometrics data to ensure accurate and reliable estimates, enabling valid causal inferences from the data. This will be relevant to extracting correct inferences from the data. 1. Government’s expenditure on education could be a proxy for the investment in human capital. 2. Exports of goods and services could be a proxy for economic output. 3. Terminology: Adjusted Savings: Education Expenditure (percentage of GNI) - This metric indicates the proportion of Gross National Income (GNI) allocated for future educational spending. GNI Per Capita (constant 2015 US dollars) - Represents the Gross National Income per capita adjusted for inflation to the year 2015, expressed in US dollars. School Enrollment, Primary (percentage gross) - The gross enrollment ratio for primary education, which measures the percentage of children of official primary school age who are enrolled in primary school. School Enrollment, Secondary (percentage gross) - The gross enrollment ratio for secondary education, indicating the percentage of children of official secondary school age who are enrolled in secondary school. Labor Force Participation Rate, Total (percentage of total population ages 15+) (modeled ILO estimate) - This figure represents the percentage of the total population aged 15 and older who are economically active, as estimated by the International Labour Organization (ILO). Exports of Goods and Services (annual percentage growth) - Reflects the annual percentage growth rate of exports of goods and services. Lower middle income countries could be assumed to be coming under the label of developing countries.", + "datasets": [ + { + "name": "worldbank_education_gdp.csv", + "description": "This is the panel data created based on the information provided in the paper. The dataset contains information of two groups of countries- Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015, sourced from the World Development Indicators (WDI) database. It focuses on the relationship between government expenditure on education (% of total expenditure) and per capita GDP (in 2010 US$), using variables like total labor force, gross primary and secondary enrollment, and exports (annual % growth) as key indicators.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "Series Name", + "description": "The name of the indicator or variable being measured. Description of the different indicators in the series is as follows:-GNI per capita (constant 2015 US dollar) - Gross National Income (GNI) per capita adjusted for inflation to 2015 US dollars. Adjusted savings: education expenditure (percentage of GNI) - The percentage of GNI that is saved for future education expenditures. Exports of goods and services (annual percentage growth) - The annual percentage growth rate of exports of goods and services. School enrollment, primary (percentage gross) - The gross enrollment ratio for primary school, representing the percentage of children of official primary school age who are enrolled in primary school. School enrollment, secondary (percentage gross) - The gross enrollment ratio for secondary school, representing the percentage of children of official secondary school age who are enrolled in secondary school. Labor force participation rate, total (percentage of total population ages 15+) (modeled ILO estimate) - The percentage of the total population ages 15 and older that is economically active." + }, + { + "name": "Series Code", + "description": "The code representing the indicator or variable" + }, + { + "name": "1975 [YR1975]", + "description": "The value of each indicator or variable for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of each indicator or variable for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of each indicator or variable for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of each indicator or variable for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of each indicator or variable for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of each indicator or variable for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of each indicator or variable for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of each indicator or variable for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of each indicator or variable for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of each indicator or variable for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of each indicator or variable for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of each indicator or variable for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of each indicator or variable for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of each indicator or variable for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of each indicator or variable for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of each indicator or variable for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of each indicator or variable for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of each indicator or variable for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of each indicator or variable for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of each indicator or variable for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of each indicator or variable for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of each indicator or variable for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of each indicator or variable for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of each indicator or variable for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of each indicator or variable for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of each indicator or variable for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of each indicator or variable for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of each indicator or variable for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of each indicator or variable for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of each indicator or variable for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of each indicator or variable for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of each indicator or variable for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of each indicator or variable for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of each indicator or variable for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of each indicator or variable for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of each indicator or variable for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of each indicator or variable for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of each indicator or variable for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of each indicator or variable for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of each indicator or variable for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of each indicator or variable for the year 2015" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "How does the effect of increasing education expenditure on per capita GDP compare between developing countries and countries in Sub-Saharan Africa?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/worldbank_education_gdp/metadata_3.json b/examples/discovery/data/demo/worldbank_education_gdp/metadata_3.json new file mode 100644 index 000000000..cda0df2c3 --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp/metadata_3.json @@ -0,0 +1,212 @@ +{ + "id": 0, + "domain": "economics", + "workflow_tags": "data selection, econometric modelling, mixture models", + "domain_knowledge": "Managing endogeneity and autoregression can be important in econometrics data to ensure accurate and reliable estimates, enabling valid causal inferences from the data. This will be relevant to extracting correct inferences from the data. 1. Government’s expenditure on education could be a proxy for the investment in human capital. 2. Exports of goods and services could be a proxy for economic output. 3. Terminology: Adjusted Savings: Education Expenditure (percentage of GNI) - This metric indicates the proportion of Gross National Income (GNI) allocated for future educational spending. GNI Per Capita (constant 2015 US dollars) - Represents the Gross National Income per capita adjusted for inflation to the year 2015, expressed in US dollars. School Enrollment, Primary (percentage gross) - The gross enrollment ratio for primary education, which measures the percentage of children of official primary school age who are enrolled in primary school. School Enrollment, Secondary (percentage gross) - The gross enrollment ratio for secondary education, indicating the percentage of children of official secondary school age who are enrolled in secondary school. Labor Force Participation Rate, Total (percentage of total population ages 15+) (modeled ILO estimate) - This figure represents the percentage of the total population aged 15 and older who are economically active, as estimated by the International Labour Organization (ILO). Exports of Goods and Services (annual percentage growth) - Reflects the annual percentage growth rate of exports of goods and services. Lower middle income countries could be assumed to be coming under the label of developing countries.", + "datasets": [ + { + "name": "worldbank_education_gdp.csv", + "description": "This is the panel data created based on the information provided in the paper. The dataset contains information of two groups of countries- Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015, sourced from the World Development Indicators (WDI) database. It focuses on the relationship between government expenditure on education (% of total expenditure) and per capita GDP (in 2010 US$), using variables like total labor force, gross primary and secondary enrollment, and exports (annual % growth) as key indicators.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "Series Name", + "description": "The name of the indicator or variable being measured. Description of the different indicators in the series is as follows:-GNI per capita (constant 2015 US dollar) - Gross National Income (GNI) per capita adjusted for inflation to 2015 US dollars. Adjusted savings: education expenditure (percentage of GNI) - The percentage of GNI that is saved for future education expenditures. Exports of goods and services (annual percentage growth) - The annual percentage growth rate of exports of goods and services. School enrollment, primary (percentage gross) - The gross enrollment ratio for primary school, representing the percentage of children of official primary school age who are enrolled in primary school. School enrollment, secondary (percentage gross) - The gross enrollment ratio for secondary school, representing the percentage of children of official secondary school age who are enrolled in secondary school. Labor force participation rate, total (percentage of total population ages 15+) (modeled ILO estimate) - The percentage of the total population ages 15 and older that is economically active." + }, + { + "name": "Series Code", + "description": "The code representing the indicator or variable" + }, + { + "name": "1975 [YR1975]", + "description": "The value of each indicator or variable for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of each indicator or variable for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of each indicator or variable for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of each indicator or variable for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of each indicator or variable for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of each indicator or variable for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of each indicator or variable for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of each indicator or variable for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of each indicator or variable for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of each indicator or variable for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of each indicator or variable for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of each indicator or variable for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of each indicator or variable for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of each indicator or variable for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of each indicator or variable for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of each indicator or variable for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of each indicator or variable for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of each indicator or variable for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of each indicator or variable for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of each indicator or variable for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of each indicator or variable for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of each indicator or variable for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of each indicator or variable for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of each indicator or variable for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of each indicator or variable for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of each indicator or variable for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of each indicator or variable for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of each indicator or variable for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of each indicator or variable for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of each indicator or variable for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of each indicator or variable for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of each indicator or variable for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of each indicator or variable for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of each indicator or variable for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of each indicator or variable for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of each indicator or variable for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of each indicator or variable for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of each indicator or variable for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of each indicator or variable for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of each indicator or variable for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of each indicator or variable for the year 2015" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "How does increased education expenditure influence human capital and economic output?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/worldbank_education_gdp/metadata_4.json b/examples/discovery/data/demo/worldbank_education_gdp/metadata_4.json new file mode 100644 index 000000000..fc08a1a14 --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp/metadata_4.json @@ -0,0 +1,212 @@ +{ + "id": 0, + "domain": "economics", + "workflow_tags": "data selection, econometric modelling, mixture models", + "domain_knowledge": "Managing endogeneity and autoregression can be important in econometrics data to ensure accurate and reliable estimates, enabling valid causal inferences from the data. This will be relevant to extracting correct inferences from the data. 1. Government’s expenditure on education could be a proxy for the investment in human capital. 2. Exports of goods and services could be a proxy for economic output. 3. Terminology: Adjusted Savings: Education Expenditure (percentage of GNI) - This metric indicates the proportion of Gross National Income (GNI) allocated for future educational spending. GNI Per Capita (constant 2015 US dollars) - Represents the Gross National Income per capita adjusted for inflation to the year 2015, expressed in US dollars. School Enrollment, Primary (percentage gross) - The gross enrollment ratio for primary education, which measures the percentage of children of official primary school age who are enrolled in primary school. School Enrollment, Secondary (percentage gross) - The gross enrollment ratio for secondary education, indicating the percentage of children of official secondary school age who are enrolled in secondary school. Labor Force Participation Rate, Total (percentage of total population ages 15+) (modeled ILO estimate) - This figure represents the percentage of the total population aged 15 and older who are economically active, as estimated by the International Labour Organization (ILO). Exports of Goods and Services (annual percentage growth) - Reflects the annual percentage growth rate of exports of goods and services. Lower middle income countries could be assumed to be coming under the label of developing countries.", + "datasets": [ + { + "name": "worldbank_education_gdp.csv", + "description": "This is the panel data created based on the information provided in the paper. The dataset contains information of two groups of countries- Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015, sourced from the World Development Indicators (WDI) database. It focuses on the relationship between government expenditure on education (% of total expenditure) and per capita GDP (in 2010 US$), using variables like total labor force, gross primary and secondary enrollment, and exports (annual % growth) as key indicators.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "Series Name", + "description": "The name of the indicator or variable being measured. Description of the different indicators in the series is as follows:-GNI per capita (constant 2015 US dollar) - Gross National Income (GNI) per capita adjusted for inflation to 2015 US dollars. Adjusted savings: education expenditure (percentage of GNI) - The percentage of GNI that is saved for future education expenditures. Exports of goods and services (annual percentage growth) - The annual percentage growth rate of exports of goods and services. School enrollment, primary (percentage gross) - The gross enrollment ratio for primary school, representing the percentage of children of official primary school age who are enrolled in primary school. School enrollment, secondary (percentage gross) - The gross enrollment ratio for secondary school, representing the percentage of children of official secondary school age who are enrolled in secondary school. Labor force participation rate, total (percentage of total population ages 15+) (modeled ILO estimate) - The percentage of the total population ages 15 and older that is economically active." + }, + { + "name": "Series Code", + "description": "The code representing the indicator or variable" + }, + { + "name": "1975 [YR1975]", + "description": "The value of each indicator or variable for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of each indicator or variable for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of each indicator or variable for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of each indicator or variable for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of each indicator or variable for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of each indicator or variable for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of each indicator or variable for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of each indicator or variable for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of each indicator or variable for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of each indicator or variable for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of each indicator or variable for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of each indicator or variable for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of each indicator or variable for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of each indicator or variable for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of each indicator or variable for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of each indicator or variable for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of each indicator or variable for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of each indicator or variable for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of each indicator or variable for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of each indicator or variable for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of each indicator or variable for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of each indicator or variable for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of each indicator or variable for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of each indicator or variable for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of each indicator or variable for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of each indicator or variable for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of each indicator or variable for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of each indicator or variable for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of each indicator or variable for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of each indicator or variable for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of each indicator or variable for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of each indicator or variable for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of each indicator or variable for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of each indicator or variable for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of each indicator or variable for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of each indicator or variable for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of each indicator or variable for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of each indicator or variable for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of each indicator or variable for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of each indicator or variable for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of each indicator or variable for the year 2015" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "How do labor productivity and education levels relate to economic output, particularly in terms of export growth?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/worldbank_education_gdp/worldbank_education_gdp.csv b/examples/discovery/data/demo/worldbank_education_gdp/worldbank_education_gdp.csv new file mode 100644 index 000000000..b202b3821 --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp/worldbank_education_gdp.csv @@ -0,0 +1,13 @@ +Country Group,Country Code,Series Name,Series Code,1975 [YR1975],1976 [YR1976],1977 [YR1977],1978 [YR1978],1979 [YR1979],1980 [YR1980],1981 [YR1981],1982 [YR1982],1983 [YR1983],1984 [YR1984],1985 [YR1985],1986 [YR1986],1987 [YR1987],1988 [YR1988],1989 [YR1989],1990 [YR1990],1991 [YR1991],1992 [YR1992],1993 [YR1993],1994 [YR1994],1995 [YR1995],1996 [YR1996],1997 [YR1997],1998 [YR1998],1999 [YR1999],2000 [YR2000],2001 [YR2001],2002 [YR2002],2003 [YR2003],2004 [YR2004],2005 [YR2005],2006 [YR2006],2007 [YR2007],2008 [YR2008],2009 [YR2009],2010 [YR2010],2011 [YR2011],2012 [YR2012],2013 [YR2013],2014 [YR2014],2015 [YR2015] +Sub-Saharan Africa,SSF,GNI per capita (constant 2015 USdollar),NY.GNP.PCAP.KD,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1555.253067,1556.395539,1578.260044,1620.904978,1634.258291 +Sub-Saharan Africa,SSF,Adjusted savings: education expenditure (percentage of GNI),NY.ADJ.AEDU.GN.ZS,4.300054875,4.309288287,4.22636904,4.357860281,4.429892995,4.479610035,4.430887088,4.391518711,4.435321374,4.302502613,4.280350991,4.370817544,4.327706901,4.307132606,4.170064016,4.212590679,4.559711157,5.49420621,5.336775358,5.304095753,5.932380859,5.482556972,6.096690035,4.578321059,4.461234601,4.043158992,4.030061307,4.020684079,4.14149817,4.285403602,4.227454729,4.1132321,3.992515875,3.098683936,3.35135401,3.465260559,3.650844042,3.633341627,3.377043176,3.254962989,3.153822596 +Sub-Saharan Africa,SSF,Exports of goods and services (annual percentage growth),NE.EXP.GNFS.KD.ZG,,,,,,,,,,,,,,,,,-2.454921874,5.731948742,5.356991615,4.887844216,7.359975856,7.741045105,3.892442104,10.52091472,30.04133886,-2.12893807,4.824344274,8.606579828,10.490787,6.582601133,6.275344203,9.610609527,10.35673672,7.840506769,-2.516281624,7.063128748,-43.29252094,-10.33887014,2.868318905,6.5130378,-1.549814673 +Sub-Saharan Africa,SSF,"School enrollment, primary (percentage gross)",SE.PRM.ENRR,60.60010147,61.75576019,65.11662292,68.88246918,73.33525085,76.39717102,78.95310974,81.02980042,81.0493927,80.25372314,77.87875366,75.23184204,73.42900848,72.20050812,72.66378784,71.65241241,72.52577972,72.32569122,73.29406738,73.44699097,74.04862976,72.22676086,76.10172272,77.36666107,77.87535095,80.26200104,82.29016113,84.68801117,87.46829224,89.69049072,92.25982666,94.17098999,95.14208221,96.49623108,96.90567017,96.70926666,97.98725891,98.6880188,98.82795715,97.96379089,96.97496796 +Sub-Saharan Africa,SSF,"School enrollment, secondary (percentage gross)",SE.SEC.ENRR,14.78637981,15.27048969,15.75590038,16.53960037,17.41189957,18.54446983,19.71949005,21.06694984,22.32876968,23.49694061,24.05698013,23.73015022,23.73509026,23.50935936,22.77041054,22.98447037,23.21471024,23.03520966,23.14027023,23.41647911,23.92342949,24.1364193,24.18354034,24.72714996,24.99921989,25.5554409,26.85679054,27.84757996,28.95659065,30.48270035,31.39464951,32.42610931,33.35705185,35.29730988,37.07627869,38.99422836,40.18935013,41.43228149,44.04343033,42.84716034,43.37297058 +Sub-Saharan Africa,SSF,"Labor force participation rate, total (percentage of total population ages 15+) (modeled ILO estimate)",SL.TLF.CACT.ZS,,,,,,,,,,,,,,,,,69.42964501,69.42501007,69.35421301,69.30190233,69.29758175,69.26307039,69.19699664,69.13637472,69.07736893,69.02503951,69.00455891,69.01008811,69.00717143,69.00755285,68.99940304,68.89034492,68.74238833,68.55562154,68.24515774,67.75495899,67.74850718,67.46582821,67.2828816,67.06038892,67.0351976 +Lower middle income,LMC,GNI per capita (constant 2015 USdollar),NY.GNP.PCAP.KD,880.766431,934.500597,939.0693809,919.155717,878.0473809,840.4903638,844.2360062,884.5842433,918.6554826,901.8426083,906.0616701,897.0908752,909.9363169,929.5696074,954.1128768,986.8567154,1001.900102,1007.737394,994.1098994,991.8233531,1012.349859,1052.539314,1066.127258,1092.515828,1129.053979,1157.995847,1187.052521,1213.979916,1269.706182,1331.80148,1390.881557,1460.679779,1542.179634,1571.87283,1611.409495,1682.27428,1726.782904,1768.70889,1821.097219,1894.302198,1965.288164 +Lower middle income,LMC,Adjusted savings: education expenditure (percentage of GNI),NY.ADJ.AEDU.GN.ZS,2.694685433,3.771469358,3.48811821,3.765355538,3.813484079,3.634042597,3.640844295,3.686585495,3.508542505,3.210604605,3.252193646,3.408056802,3.514179399,3.268386282,3.558447946,3.564863634,3.623879425,3.730229771,3.758408951,3.721667048,3.763498753,3.683770988,3.668857318,3.753211228,3.873222604,3.741925129,3.67550405,3.641694172,3.577680711,3.475960423,3.524511687,3.519232431,3.510348478,3.265790023,3.259990887,3.289517623,3.227041217,3.219500592,3.20751584,3.106095732,3.060663041 +Lower middle income,LMC,Exports of goods and services (annual percentage growth),NE.EXP.GNFS.KD.ZG,-5.569420334,10.45802151,-3.623539788,-13.08727097,-6.300238971,-23.54957475,3.265488371,13.84350722,10.08879292,-2.501570702,-3.898022239,0.746649826,12.66508558,8.545190067,8.832839695,9.62702111,-1.400414229,-11.43366862,3.140851622,6.515998852,5.564533495,7.869439963,1.38340055,1.910801281,5.660292069,10.96472501,2.932501247,8.519005051,9.414156267,15.57978692,10.86589177,9.87465734,6.264558844,8.009968404,-6.439527256,10.94035872,8.165342325,2.727892399,4.733821517,2.921528391,0.260325532 +Lower middle income,LMC,"School enrollment, primary (percentage gross)",SE.PRM.ENRR,79.07386017,79.52011871,81.8299408,81.29871368,82.04834747,83.76357269,85.0454483,86.49360657,87.7649231,88.36634827,87.44582367,87.47873688,86.36930084,87.43833923,88.60176086,88.38516998,88.69302368,89.19795227,90.54950714,91.45050812,90.75634003,90.45321655,90.94722748,90.76187897,91.36080933,92.78392029,92.65696716,93.49974823,97.50721741,99.10671997,100.4120102,101.0586014,101.6256027,101.0495377,100.6591034,100.7808914,100.7460175,101.8817902,102.8081207,101.6483307,101.7831116 +Lower middle income,LMC,"School enrollment, secondary (percentage gross)",SE.SEC.ENRR,27.81277084,28.17565918,28.91234016,29.54128075,30.37379074,31.46453094,32.13985062,33.28364182,34.80886841,35.84233856,36.56085968,37.14120865,37.34346008,37.34712982,37.51271057,38.88959122,40.41157913,41.85649872,43.21253967,43.9512291,44.24134064,44.4618187,45.12900162,44.88694,44.59563828,45.82976913,46.3972702,47.68738937,49.10155106,50.12987137,51.1917305,51.93574905,53.08201981,54.90346909,55.25709152,57.56341171,59.62149811,61.43220139,62.24372864,64.28119659,64.53296661 +Lower middle income,LMC,"Labor force participation rate, total (percentage of total population ages 15+) (modeled ILO estimate)",SL.TLF.CACT.ZS,,,,,,,,,,,,,,,,,57.12102868,57.06694953,57.03711693,57.0493771,57.16016067,57.3802261,57.57408143,57.77990857,58.02985553,58.13264008,57.82351234,57.46268401,57.19583225,56.94141435,56.70706604,56.68895467,56.67971969,56.56786086,56.63037128,56.65415124,56.24325692,55.85628973,55.77785543,55.57102194,55.4697902 diff --git a/examples/discovery/data/demo/worldbank_education_gdp_indicators/Adjusted_savings_education_expenditure_percentage_of_GNI.csv b/examples/discovery/data/demo/worldbank_education_gdp_indicators/Adjusted_savings_education_expenditure_percentage_of_GNI.csv new file mode 100644 index 000000000..f44ce1561 --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp_indicators/Adjusted_savings_education_expenditure_percentage_of_GNI.csv @@ -0,0 +1,3 @@ +Country Group,Country Code,1975 [YR1975],1976 [YR1976],1977 [YR1977],1978 [YR1978],1979 [YR1979],1980 [YR1980],1981 [YR1981],1982 [YR1982],1983 [YR1983],1984 [YR1984],1985 [YR1985],1986 [YR1986],1987 [YR1987],1988 [YR1988],1989 [YR1989],1990 [YR1990],1991 [YR1991],1992 [YR1992],1993 [YR1993],1994 [YR1994],1995 [YR1995],1996 [YR1996],1997 [YR1997],1998 [YR1998],1999 [YR1999],2000 [YR2000],2001 [YR2001],2002 [YR2002],2003 [YR2003],2004 [YR2004],2005 [YR2005],2006 [YR2006],2007 [YR2007],2008 [YR2008],2009 [YR2009],2010 [YR2010],2011 [YR2011],2012 [YR2012],2013 [YR2013],2014 [YR2014],2015 [YR2015] +Sub-Saharan Africa,SSF,4.300054875,4.309288287,4.22636904,4.357860281,4.429892995,4.479610035,4.430887088,4.391518711,4.435321374,4.302502613,4.280350991,4.370817544,4.327706901,4.307132606,4.170064016,4.212590679,4.559711157,5.49420621,5.336775358,5.304095753,5.932380859,5.482556972,6.096690035,4.578321059,4.461234601,4.043158992,4.030061307,4.020684079,4.14149817,4.285403602,4.227454729,4.1132321,3.992515875,3.098683936,3.35135401,3.465260559,3.650844042,3.633341627,3.377043176,3.254962989,3.153822596 +Lower middle income,LMC,2.694685433,3.771469358,3.48811821,3.765355538,3.813484079,3.634042597,3.640844295,3.686585495,3.508542505,3.210604605,3.252193646,3.408056802,3.514179399,3.268386282,3.558447946,3.564863634,3.623879425,3.730229771,3.758408951,3.721667048,3.763498753,3.683770988,3.668857318,3.753211228,3.873222604,3.741925129,3.67550405,3.641694172,3.577680711,3.475960423,3.524511687,3.519232431,3.510348478,3.265790023,3.259990887,3.289517623,3.227041217,3.219500592,3.20751584,3.106095732,3.060663041 diff --git a/examples/discovery/data/demo/worldbank_education_gdp_indicators/Exports_of_goods_and_services_annual_percentage_growth.csv b/examples/discovery/data/demo/worldbank_education_gdp_indicators/Exports_of_goods_and_services_annual_percentage_growth.csv new file mode 100644 index 000000000..0ea4b4665 --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp_indicators/Exports_of_goods_and_services_annual_percentage_growth.csv @@ -0,0 +1,3 @@ +Country Group,Country Code,1975 [YR1975],1976 [YR1976],1977 [YR1977],1978 [YR1978],1979 [YR1979],1980 [YR1980],1981 [YR1981],1982 [YR1982],1983 [YR1983],1984 [YR1984],1985 [YR1985],1986 [YR1986],1987 [YR1987],1988 [YR1988],1989 [YR1989],1990 [YR1990],1991 [YR1991],1992 [YR1992],1993 [YR1993],1994 [YR1994],1995 [YR1995],1996 [YR1996],1997 [YR1997],1998 [YR1998],1999 [YR1999],2000 [YR2000],2001 [YR2001],2002 [YR2002],2003 [YR2003],2004 [YR2004],2005 [YR2005],2006 [YR2006],2007 [YR2007],2008 [YR2008],2009 [YR2009],2010 [YR2010],2011 [YR2011],2012 [YR2012],2013 [YR2013],2014 [YR2014],2015 [YR2015] +Sub-Saharan Africa,SSF,,,,,,,,,,,,,,,,,-2.454921874,5.731948742,5.356991615,4.887844216,7.359975856,7.741045105,3.892442104,10.52091472,30.04133886,-2.12893807,4.824344274,8.606579828,10.490787,6.582601133,6.275344203,9.610609527,10.35673672,7.840506769,-2.516281624,7.063128748,-43.29252094,-10.33887014,2.868318905,6.5130378,-1.549814673 +Lower middle income,LMC,-5.569420334,10.45802151,-3.623539788,-13.08727097,-6.300238971,-23.54957475,3.265488371,13.84350722,10.08879292,-2.501570702,-3.898022239,0.746649826,12.66508558,8.545190067,8.832839695,9.62702111,-1.400414229,-11.43366862,3.140851622,6.515998852,5.564533495,7.869439963,1.38340055,1.910801281,5.660292069,10.96472501,2.932501247,8.519005051,9.414156267,15.57978692,10.86589177,9.87465734,6.264558844,8.009968404,-6.439527256,10.94035872,8.165342325,2.727892399,4.733821517,2.921528391,0.260325532 diff --git a/examples/discovery/data/demo/worldbank_education_gdp_indicators/GNI_per_capita_constant_2015_USdollar.csv b/examples/discovery/data/demo/worldbank_education_gdp_indicators/GNI_per_capita_constant_2015_USdollar.csv new file mode 100644 index 000000000..c0a4928bf --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp_indicators/GNI_per_capita_constant_2015_USdollar.csv @@ -0,0 +1,3 @@ +Country Group,Country Code,1975 [YR1975],1976 [YR1976],1977 [YR1977],1978 [YR1978],1979 [YR1979],1980 [YR1980],1981 [YR1981],1982 [YR1982],1983 [YR1983],1984 [YR1984],1985 [YR1985],1986 [YR1986],1987 [YR1987],1988 [YR1988],1989 [YR1989],1990 [YR1990],1991 [YR1991],1992 [YR1992],1993 [YR1993],1994 [YR1994],1995 [YR1995],1996 [YR1996],1997 [YR1997],1998 [YR1998],1999 [YR1999],2000 [YR2000],2001 [YR2001],2002 [YR2002],2003 [YR2003],2004 [YR2004],2005 [YR2005],2006 [YR2006],2007 [YR2007],2008 [YR2008],2009 [YR2009],2010 [YR2010],2011 [YR2011],2012 [YR2012],2013 [YR2013],2014 [YR2014],2015 [YR2015] +Sub-Saharan Africa,SSF,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1555.253067,1556.395539,1578.260044,1620.904978,1634.258291 +Lower middle income,LMC,880.766431,934.500597,939.0693809,919.155717,878.0473809,840.4903638,844.2360062,884.5842433,918.6554826,901.8426083,906.0616701,897.0908752,909.9363169,929.5696074,954.1128768,986.8567154,1001.900102,1007.737394,994.1098994,991.8233531,1012.349859,1052.539314,1066.127258,1092.515828,1129.053979,1157.995847,1187.052521,1213.979916,1269.706182,1331.80148,1390.881557,1460.679779,1542.179634,1571.87283,1611.409495,1682.27428,1726.782904,1768.70889,1821.097219,1894.302198,1965.288164 diff --git a/examples/discovery/data/demo/worldbank_education_gdp_indicators/Labor_force_participation_rate_total_percentage_of_total_population_ages_15+_modeled_ILO_estimate.csv b/examples/discovery/data/demo/worldbank_education_gdp_indicators/Labor_force_participation_rate_total_percentage_of_total_population_ages_15+_modeled_ILO_estimate.csv new file mode 100644 index 000000000..23ff3cfe3 --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp_indicators/Labor_force_participation_rate_total_percentage_of_total_population_ages_15+_modeled_ILO_estimate.csv @@ -0,0 +1,3 @@ +Country Group,Country Code,1975 [YR1975],1976 [YR1976],1977 [YR1977],1978 [YR1978],1979 [YR1979],1980 [YR1980],1981 [YR1981],1982 [YR1982],1983 [YR1983],1984 [YR1984],1985 [YR1985],1986 [YR1986],1987 [YR1987],1988 [YR1988],1989 [YR1989],1990 [YR1990],1991 [YR1991],1992 [YR1992],1993 [YR1993],1994 [YR1994],1995 [YR1995],1996 [YR1996],1997 [YR1997],1998 [YR1998],1999 [YR1999],2000 [YR2000],2001 [YR2001],2002 [YR2002],2003 [YR2003],2004 [YR2004],2005 [YR2005],2006 [YR2006],2007 [YR2007],2008 [YR2008],2009 [YR2009],2010 [YR2010],2011 [YR2011],2012 [YR2012],2013 [YR2013],2014 [YR2014],2015 [YR2015] +Sub-Saharan Africa,SSF,,,,,,,,,,,,,,,,,69.42964501,69.42501007,69.35421301,69.30190233,69.29758175,69.26307039,69.19699664,69.13637472,69.07736893,69.02503951,69.00455891,69.01008811,69.00717143,69.00755285,68.99940304,68.89034492,68.74238833,68.55562154,68.24515774,67.75495899,67.74850718,67.46582821,67.2828816,67.06038892,67.0351976 +Lower middle income,LMC,,,,,,,,,,,,,,,,,57.12102868,57.06694953,57.03711693,57.0493771,57.16016067,57.3802261,57.57408143,57.77990857,58.02985553,58.13264008,57.82351234,57.46268401,57.19583225,56.94141435,56.70706604,56.68895467,56.67971969,56.56786086,56.63037128,56.65415124,56.24325692,55.85628973,55.77785543,55.57102194,55.4697902 diff --git a/examples/discovery/data/demo/worldbank_education_gdp_indicators/School_enrollment_primary_percentage_gross.csv b/examples/discovery/data/demo/worldbank_education_gdp_indicators/School_enrollment_primary_percentage_gross.csv new file mode 100644 index 000000000..65e419e9f --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp_indicators/School_enrollment_primary_percentage_gross.csv @@ -0,0 +1,3 @@ +Country Group,Country Code,1975 [YR1975],1976 [YR1976],1977 [YR1977],1978 [YR1978],1979 [YR1979],1980 [YR1980],1981 [YR1981],1982 [YR1982],1983 [YR1983],1984 [YR1984],1985 [YR1985],1986 [YR1986],1987 [YR1987],1988 [YR1988],1989 [YR1989],1990 [YR1990],1991 [YR1991],1992 [YR1992],1993 [YR1993],1994 [YR1994],1995 [YR1995],1996 [YR1996],1997 [YR1997],1998 [YR1998],1999 [YR1999],2000 [YR2000],2001 [YR2001],2002 [YR2002],2003 [YR2003],2004 [YR2004],2005 [YR2005],2006 [YR2006],2007 [YR2007],2008 [YR2008],2009 [YR2009],2010 [YR2010],2011 [YR2011],2012 [YR2012],2013 [YR2013],2014 [YR2014],2015 [YR2015] +Sub-Saharan Africa,SSF,60.60010147,61.75576019,65.11662292,68.88246918,73.33525085,76.39717102,78.95310974,81.02980042,81.0493927,80.25372314,77.87875366,75.23184204,73.42900848,72.20050812,72.66378784,71.65241241,72.52577972,72.32569122,73.29406738,73.44699097,74.04862976,72.22676086,76.10172272,77.36666107,77.87535095,80.26200104,82.29016113,84.68801117,87.46829224,89.69049072,92.25982666,94.17098999,95.14208221,96.49623108,96.90567017,96.70926666,97.98725891,98.6880188,98.82795715,97.96379089,96.97496796 +Lower middle income,LMC,79.07386017,79.52011871,81.8299408,81.29871368,82.04834747,83.76357269,85.0454483,86.49360657,87.7649231,88.36634827,87.44582367,87.47873688,86.36930084,87.43833923,88.60176086,88.38516998,88.69302368,89.19795227,90.54950714,91.45050812,90.75634003,90.45321655,90.94722748,90.76187897,91.36080933,92.78392029,92.65696716,93.49974823,97.50721741,99.10671997,100.4120102,101.0586014,101.6256027,101.0495377,100.6591034,100.7808914,100.7460175,101.8817902,102.8081207,101.6483307,101.7831116 diff --git a/examples/discovery/data/demo/worldbank_education_gdp_indicators/School_enrollment_secondary_percentage_gross.csv b/examples/discovery/data/demo/worldbank_education_gdp_indicators/School_enrollment_secondary_percentage_gross.csv new file mode 100644 index 000000000..ac9e9c25e --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp_indicators/School_enrollment_secondary_percentage_gross.csv @@ -0,0 +1,3 @@ +Country Group,Country Code,1975 [YR1975],1976 [YR1976],1977 [YR1977],1978 [YR1978],1979 [YR1979],1980 [YR1980],1981 [YR1981],1982 [YR1982],1983 [YR1983],1984 [YR1984],1985 [YR1985],1986 [YR1986],1987 [YR1987],1988 [YR1988],1989 [YR1989],1990 [YR1990],1991 [YR1991],1992 [YR1992],1993 [YR1993],1994 [YR1994],1995 [YR1995],1996 [YR1996],1997 [YR1997],1998 [YR1998],1999 [YR1999],2000 [YR2000],2001 [YR2001],2002 [YR2002],2003 [YR2003],2004 [YR2004],2005 [YR2005],2006 [YR2006],2007 [YR2007],2008 [YR2008],2009 [YR2009],2010 [YR2010],2011 [YR2011],2012 [YR2012],2013 [YR2013],2014 [YR2014],2015 [YR2015] +Sub-Saharan Africa,SSF,14.78637981,15.27048969,15.75590038,16.53960037,17.41189957,18.54446983,19.71949005,21.06694984,22.32876968,23.49694061,24.05698013,23.73015022,23.73509026,23.50935936,22.77041054,22.98447037,23.21471024,23.03520966,23.14027023,23.41647911,23.92342949,24.1364193,24.18354034,24.72714996,24.99921989,25.5554409,26.85679054,27.84757996,28.95659065,30.48270035,31.39464951,32.42610931,33.35705185,35.29730988,37.07627869,38.99422836,40.18935013,41.43228149,44.04343033,42.84716034,43.37297058 +Lower middle income,LMC,27.81277084,28.17565918,28.91234016,29.54128075,30.37379074,31.46453094,32.13985062,33.28364182,34.80886841,35.84233856,36.56085968,37.14120865,37.34346008,37.34712982,37.51271057,38.88959122,40.41157913,41.85649872,43.21253967,43.9512291,44.24134064,44.4618187,45.12900162,44.88694,44.59563828,45.82976913,46.3972702,47.68738937,49.10155106,50.12987137,51.1917305,51.93574905,53.08201981,54.90346909,55.25709152,57.56341171,59.62149811,61.43220139,62.24372864,64.28119659,64.53296661 diff --git a/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_0.json b/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_0.json new file mode 100644 index 000000000..b5e81e61a --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_0.json @@ -0,0 +1,1109 @@ +{ + "id": 0, + "domain": "economics", + "workflow_tags": "data selection, econometric modelling, mixture models", + "domain_knowledge": "Managing endogeneity and autoregression can be important in econometrics data to ensure accurate and reliable estimates, enabling valid causal inferences from the data. This will be relevant to extracting correct inferences from the data. 1. Government’s expenditure on education could be a proxy for the investment in human capital. 2. Exports of goods and services could be a proxy for economic output. 3. Terminology: Adjusted Savings: Education Expenditure (percentage of GNI) - This metric indicates the proportion of Gross National Income (GNI) allocated for future educational spending. GNI Per Capita (constant 2015 US dollars) - Represents the Gross National Income per capita adjusted for inflation to the year 2015, expressed in US dollars. School Enrollment, Primary (percentage gross) - The gross enrollment ratio for primary education, which measures the percentage of children of official primary school age who are enrolled in primary school. School Enrollment, Secondary (percentage gross) - The gross enrollment ratio for secondary education, indicating the percentage of children of official secondary school age who are enrolled in secondary school. Labor Force Participation Rate, Total (percentage of total population ages 15+) (modeled ILO estimate) - This figure represents the percentage of the total population aged 15 and older who are economically active, as estimated by the International Labour Organization (ILO). Exports of Goods and Services (annual percentage growth) - Reflects the annual percentage growth rate of exports of goods and services. Lower middle income countries could be assumed to be coming under the label of developing countries.", + "datasets": [ + { + "name": "Adjusted_savings_education_expenditure_percentage_of_GNI.csv", + "description": "This dataset contains information on adjusted savings: education expenditure (percentage of gni) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "Exports_of_goods_and_services_annual_percentage_growth.csv", + "description": "This dataset contains information on exports of goods and services (annual percentage growth) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "GNI_per_capita_constant_2015_USdollar.csv", + "description": "This dataset contains information on gni per capita (constant 2015 usdollar) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "Labor_force_participation_rate_total_percentage_of_total_population_ages_15+_modeled_ILO_estimate.csv", + "description": "This dataset contains information on labor force participation rate, total (percentage of total population ages 15+) (modeled ilo estimate) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "School_enrollment_primary_percentage_gross.csv", + "description": "This dataset contains information on school enrollment, primary (percentage gross) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "School_enrollment_secondary_percentage_gross.csv", + "description": "This dataset contains information on school enrollment, secondary (percentage gross) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "context", + "question": "In what regions does increased education spending positively impact per capita GDP?" + }, + { + "qid": 1, + "question_type": "relationship", + "question": "What is the impact of increased education expenditure on per capita GDP in developing countries?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_1.json b/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_1.json new file mode 100644 index 000000000..2a7ada4eb --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_1.json @@ -0,0 +1,1104 @@ +{ + "id": 0, + "domain": "economics", + "workflow_tags": "data selection, econometric modelling, mixture models", + "domain_knowledge": "Managing endogeneity and autoregression can be important in econometrics data to ensure accurate and reliable estimates, enabling valid causal inferences from the data. This will be relevant to extracting correct inferences from the data. 1. Government’s expenditure on education could be a proxy for the investment in human capital. 2. Exports of goods and services could be a proxy for economic output. 3. Terminology: Adjusted Savings: Education Expenditure (percentage of GNI) - This metric indicates the proportion of Gross National Income (GNI) allocated for future educational spending. GNI Per Capita (constant 2015 US dollars) - Represents the Gross National Income per capita adjusted for inflation to the year 2015, expressed in US dollars. School Enrollment, Primary (percentage gross) - The gross enrollment ratio for primary education, which measures the percentage of children of official primary school age who are enrolled in primary school. School Enrollment, Secondary (percentage gross) - The gross enrollment ratio for secondary education, indicating the percentage of children of official secondary school age who are enrolled in secondary school. Labor Force Participation Rate, Total (percentage of total population ages 15+) (modeled ILO estimate) - This figure represents the percentage of the total population aged 15 and older who are economically active, as estimated by the International Labour Organization (ILO). Exports of Goods and Services (annual percentage growth) - Reflects the annual percentage growth rate of exports of goods and services. Lower middle income countries could be assumed to be coming under the label of developing countries.", + "datasets": [ + { + "name": "Adjusted_savings_education_expenditure_percentage_of_GNI.csv", + "description": "This dataset contains information on adjusted savings: education expenditure (percentage of gni) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "Exports_of_goods_and_services_annual_percentage_growth.csv", + "description": "This dataset contains information on exports of goods and services (annual percentage growth) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "GNI_per_capita_constant_2015_USdollar.csv", + "description": "This dataset contains information on gni per capita (constant 2015 usdollar) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "Labor_force_participation_rate_total_percentage_of_total_population_ages_15+_modeled_ILO_estimate.csv", + "description": "This dataset contains information on labor force participation rate, total (percentage of total population ages 15+) (modeled ilo estimate) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "School_enrollment_primary_percentage_gross.csv", + "description": "This dataset contains information on school enrollment, primary (percentage gross) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "School_enrollment_secondary_percentage_gross.csv", + "description": "This dataset contains information on school enrollment, secondary (percentage gross) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "What relationship exists between education expenditure and per capita GDP in developing countries and how does it affect economic output?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_2.json b/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_2.json new file mode 100644 index 000000000..1b5350cd0 --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_2.json @@ -0,0 +1,1104 @@ +{ + "id": 0, + "domain": "economics", + "workflow_tags": "data selection, econometric modelling, mixture models", + "domain_knowledge": "Managing endogeneity and autoregression can be important in econometrics data to ensure accurate and reliable estimates, enabling valid causal inferences from the data. This will be relevant to extracting correct inferences from the data. 1. Government’s expenditure on education could be a proxy for the investment in human capital. 2. Exports of goods and services could be a proxy for economic output. 3. Terminology: Adjusted Savings: Education Expenditure (percentage of GNI) - This metric indicates the proportion of Gross National Income (GNI) allocated for future educational spending. GNI Per Capita (constant 2015 US dollars) - Represents the Gross National Income per capita adjusted for inflation to the year 2015, expressed in US dollars. School Enrollment, Primary (percentage gross) - The gross enrollment ratio for primary education, which measures the percentage of children of official primary school age who are enrolled in primary school. School Enrollment, Secondary (percentage gross) - The gross enrollment ratio for secondary education, indicating the percentage of children of official secondary school age who are enrolled in secondary school. Labor Force Participation Rate, Total (percentage of total population ages 15+) (modeled ILO estimate) - This figure represents the percentage of the total population aged 15 and older who are economically active, as estimated by the International Labour Organization (ILO). Exports of Goods and Services (annual percentage growth) - Reflects the annual percentage growth rate of exports of goods and services. Lower middle income countries could be assumed to be coming under the label of developing countries.", + "datasets": [ + { + "name": "Adjusted_savings_education_expenditure_percentage_of_GNI.csv", + "description": "This dataset contains information on adjusted savings: education expenditure (percentage of gni) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "Exports_of_goods_and_services_annual_percentage_growth.csv", + "description": "This dataset contains information on exports of goods and services (annual percentage growth) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "GNI_per_capita_constant_2015_USdollar.csv", + "description": "This dataset contains information on gni per capita (constant 2015 usdollar) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "Labor_force_participation_rate_total_percentage_of_total_population_ages_15+_modeled_ILO_estimate.csv", + "description": "This dataset contains information on labor force participation rate, total (percentage of total population ages 15+) (modeled ilo estimate) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "School_enrollment_primary_percentage_gross.csv", + "description": "This dataset contains information on school enrollment, primary (percentage gross) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "School_enrollment_secondary_percentage_gross.csv", + "description": "This dataset contains information on school enrollment, secondary (percentage gross) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "How does the effect of increasing education expenditure on per capita GDP compare between developing countries and countries in Sub-Saharan Africa?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_3.json b/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_3.json new file mode 100644 index 000000000..c58c40d2e --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_3.json @@ -0,0 +1,1104 @@ +{ + "id": 0, + "domain": "economics", + "workflow_tags": "data selection, econometric modelling, mixture models", + "domain_knowledge": "Managing endogeneity and autoregression can be important in econometrics data to ensure accurate and reliable estimates, enabling valid causal inferences from the data. This will be relevant to extracting correct inferences from the data. 1. Government’s expenditure on education could be a proxy for the investment in human capital. 2. Exports of goods and services could be a proxy for economic output. 3. Terminology: Adjusted Savings: Education Expenditure (percentage of GNI) - This metric indicates the proportion of Gross National Income (GNI) allocated for future educational spending. GNI Per Capita (constant 2015 US dollars) - Represents the Gross National Income per capita adjusted for inflation to the year 2015, expressed in US dollars. School Enrollment, Primary (percentage gross) - The gross enrollment ratio for primary education, which measures the percentage of children of official primary school age who are enrolled in primary school. School Enrollment, Secondary (percentage gross) - The gross enrollment ratio for secondary education, indicating the percentage of children of official secondary school age who are enrolled in secondary school. Labor Force Participation Rate, Total (percentage of total population ages 15+) (modeled ILO estimate) - This figure represents the percentage of the total population aged 15 and older who are economically active, as estimated by the International Labour Organization (ILO). Exports of Goods and Services (annual percentage growth) - Reflects the annual percentage growth rate of exports of goods and services. Lower middle income countries could be assumed to be coming under the label of developing countries.", + "datasets": [ + { + "name": "Adjusted_savings_education_expenditure_percentage_of_GNI.csv", + "description": "This dataset contains information on adjusted savings: education expenditure (percentage of gni) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "Exports_of_goods_and_services_annual_percentage_growth.csv", + "description": "This dataset contains information on exports of goods and services (annual percentage growth) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "GNI_per_capita_constant_2015_USdollar.csv", + "description": "This dataset contains information on gni per capita (constant 2015 usdollar) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "Labor_force_participation_rate_total_percentage_of_total_population_ages_15+_modeled_ILO_estimate.csv", + "description": "This dataset contains information on labor force participation rate, total (percentage of total population ages 15+) (modeled ilo estimate) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "School_enrollment_primary_percentage_gross.csv", + "description": "This dataset contains information on school enrollment, primary (percentage gross) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "School_enrollment_secondary_percentage_gross.csv", + "description": "This dataset contains information on school enrollment, secondary (percentage gross) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "relationship", + "question": "How does increased education expenditure influence human capital and economic output?" + } + ] + ] +} diff --git a/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_4.json b/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_4.json new file mode 100644 index 000000000..8f02ce226 --- /dev/null +++ b/examples/discovery/data/demo/worldbank_education_gdp_indicators/metadata_4.json @@ -0,0 +1,1104 @@ +{ + "id": 0, + "domain": "economics", + "workflow_tags": "data selection, econometric modelling, mixture models", + "domain_knowledge": "Managing endogeneity and autoregression can be important in econometrics data to ensure accurate and reliable estimates, enabling valid causal inferences from the data. This will be relevant to extracting correct inferences from the data. 1. Government’s expenditure on education could be a proxy for the investment in human capital. 2. Exports of goods and services could be a proxy for economic output. 3. Terminology: Adjusted Savings: Education Expenditure (percentage of GNI) - This metric indicates the proportion of Gross National Income (GNI) allocated for future educational spending. GNI Per Capita (constant 2015 US dollars) - Represents the Gross National Income per capita adjusted for inflation to the year 2015, expressed in US dollars. School Enrollment, Primary (percentage gross) - The gross enrollment ratio for primary education, which measures the percentage of children of official primary school age who are enrolled in primary school. School Enrollment, Secondary (percentage gross) - The gross enrollment ratio for secondary education, indicating the percentage of children of official secondary school age who are enrolled in secondary school. Labor Force Participation Rate, Total (percentage of total population ages 15+) (modeled ILO estimate) - This figure represents the percentage of the total population aged 15 and older who are economically active, as estimated by the International Labour Organization (ILO). Exports of Goods and Services (annual percentage growth) - Reflects the annual percentage growth rate of exports of goods and services. Lower middle income countries could be assumed to be coming under the label of developing countries.", + "datasets": [ + { + "name": "Adjusted_savings_education_expenditure_percentage_of_GNI.csv", + "description": "This dataset contains information on adjusted savings: education expenditure (percentage of gni) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "Exports_of_goods_and_services_annual_percentage_growth.csv", + "description": "This dataset contains information on exports of goods and services (annual percentage growth) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "GNI_per_capita_constant_2015_USdollar.csv", + "description": "This dataset contains information on gni per capita (constant 2015 usdollar) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "Labor_force_participation_rate_total_percentage_of_total_population_ages_15+_modeled_ILO_estimate.csv", + "description": "This dataset contains information on labor force participation rate, total (percentage of total population ages 15+) (modeled ilo estimate) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "School_enrollment_primary_percentage_gross.csv", + "description": "This dataset contains information on school enrollment, primary (percentage gross) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + }, + { + "name": "School_enrollment_secondary_percentage_gross.csv", + "description": "This dataset contains information on school enrollment, secondary (percentage gross) for Sub-Saharan Africa and Lower Middle Income Countries (LMC) from 1975 to 2015.", + "columns": { + "raw": [ + { + "name": "Country Group", + "description": "The name given to the group of countries- Sub-Saharan Africa and Lower middle income" + }, + { + "name": "Country Code", + "description": "The code name assigned to each group of countries: Sub-Saharan Africa-SSA and Lower Middle Income Countries-LMC" + }, + { + "name": "1975 [YR1975]", + "description": "The value of indicator for the year 1975" + }, + { + "name": "1976 [YR1976]", + "description": "The value of indicator for the year 1976" + }, + { + "name": "1977 [YR1977]", + "description": "The value of indicator for the year 1977" + }, + { + "name": "1978 [YR1978]", + "description": "The value of indicator for the year 1978" + }, + { + "name": "1979 [YR1979]", + "description": "The value of indicator for the year 1979" + }, + { + "name": "1980 [YR1980]", + "description": "The value of indicator for the year 1980" + }, + { + "name": "1981 [YR1981]", + "description": "The value of indicator for the year 1981" + }, + { + "name": "1982 [YR1982]", + "description": "The value of indicator for the year 1982" + }, + { + "name": "1983 [YR1983]", + "description": "The value of indicator for the year 1983" + }, + { + "name": "1984 [YR1984]", + "description": "The value of indicator for the year 1984" + }, + { + "name": "1985 [YR1985]", + "description": "The value of indicator for the year 1985" + }, + { + "name": "1986 [YR1986]", + "description": "The value of indicator for the year 1986" + }, + { + "name": "1987 [YR1987]", + "description": "The value of indicator for the year 1987" + }, + { + "name": "1988 [YR1988]", + "description": "The value of indicator for the year 1988" + }, + { + "name": "1989 [YR1989]", + "description": "The value of indicator for the year 1989" + }, + { + "name": "1990 [YR1990]", + "description": "The value of indicator for the year 1990" + }, + { + "name": "1991 [YR1991]", + "description": "The value of indicator for the year 1991" + }, + { + "name": "1992 [YR1992]", + "description": "The value of indicator for the year 1992" + }, + { + "name": "1993 [YR1993]", + "description": "The value of indicator for the year 1993" + }, + { + "name": "1994 [YR1994]", + "description": "The value of indicator for the year 1994" + }, + { + "name": "1995 [YR1995]", + "description": "The value of indicator for the year 1995" + }, + { + "name": "1996 [YR1996]", + "description": "The value of indicator for the year 1996" + }, + { + "name": "1997 [YR1997]", + "description": "The value of indicator for the year 1997" + }, + { + "name": "1998 [YR1998]", + "description": "The value of indicator for the year 1998" + }, + { + "name": "1999 [YR1999]", + "description": "The value of indicator for the year 1999" + }, + { + "name": "2000 [YR2000]", + "description": "The value of indicator for the year 2000" + }, + { + "name": "2001 [YR2001]", + "description": "The value of indicator for the year 2001" + }, + { + "name": "2002 [YR2002]", + "description": "The value of indicator for the year 2002" + }, + { + "name": "2003 [YR2003]", + "description": "The value of indicator for the year 2003" + }, + { + "name": "2004 [YR2004]", + "description": "The value of indicator for the year 2004" + }, + { + "name": "2005 [YR2005]", + "description": "The value of indicator for the year 2005" + }, + { + "name": "2006 [YR2006]", + "description": "The value of indicator for the year 2006" + }, + { + "name": "2007 [YR2007]", + "description": "The value of indicator for the year 2007" + }, + { + "name": "2008 [YR2008]", + "description": "The value of indicator for the year 2008" + }, + { + "name": "2009 [YR2009]", + "description": "The value of indicator for the year 2009" + }, + { + "name": "2010 [YR2010]", + "description": "The value of indicator for the year 2010" + }, + { + "name": "2011 [YR2011]", + "description": "The value of indicator for the year 2011" + }, + { + "name": "2012 [YR2012]", + "description": "The value of indicator for the year 2012" + }, + { + "name": "2013 [YR2013]", + "description": "The value of indicator for the year 2013" + }, + { + "name": "2014 [YR2014]", + "description": "The value of indicator for the year 2014" + }, + { + "name": "2015 [YR2015]", + "description": "The value of indicator for the year 2015" + } + ] + } + } + ], + "intermediate": [], + "hypotheses": { + "main": [ + {} + ], + "intermediate": [] + }, + "queries": [ + [ + { + "qid": 0, + "question_type": "variables", + "question": "How do labor productivity and education levels relate to economic output, particularly in terms of export growth?" + } + ] + ] +} diff --git a/examples/discovery/demo.py b/examples/discovery/demo.py new file mode 100644 index 000000000..75e50cefd --- /dev/null +++ b/examples/discovery/demo.py @@ -0,0 +1,155 @@ +import asyncio +import os + +import streamlit as st +from agentic_db import AgenticDB +from atypes import Dataset, Question +from pandas import DataFrame +from transductions import answer_question_from_data + +DISCOVERY_TASK_PATH = "/Users/gliozzo/Code/agentics911/agentics/sandbox/discoverybench/discoverybench/real/demo" + +st.header("Discovery Demo") +if "databases" not in st.session_state: + st.session_state.databases = [] +if "dataset" not in st.session_state: + st.session_state.dataset = None +if "selected_question" not in st.session_state: + st.session_state.selected_question = None + +if "selected_question_object" not in st.session_state: + st.session_state.selected_question_object = None + + +st.session_state.discovery_tasks = os.listdir(DISCOVERY_TASK_PATH) + + +with st.sidebar: + st.markdown("### Select Sample Discovery Tasks") + with st.form("Select Discovery Tasks"): + selected_task = st.selectbox( + "Task Name", options=st.session_state.discovery_tasks, index=None + ) + select_task_button = st.form_submit_button("Select Task") + + if select_task_button: + with st.spinner("Loading dataset, wait a few secs ..."): + st.session_state.dataset = Dataset.import_from_discovery_bench_metadata( + selected_task, metadata_path=DISCOVERY_TASK_PATH + ) + st.session_state.databases = st.session_state.dataset.dbs + st.rerun() + + if st.session_state.dataset: + with st.form("Select question"): + st.session_state.selected_question = st.selectbox( + "Select Question", + options=[ + question.question for question in st.session_state.dataset.questions + ], + index=None, + ) + selected_question_button = st.form_submit_button("Select Question") + + if selected_question_button: + st.session_state.selected_question_object = st.session_state.dataset.get_questions_as_ag().filter_by_attribute_value( + "question", st.session_state.selected_question + )[ + 0 + ] + + if st.session_state.selected_question_object: + with st.popover("Question Details"): + st.write(st.session_state.selected_question_object) + + if st.session_state.databases: + for db in st.session_state.databases: + st.markdown( + f"""### {db.name if db.name else ''} +N rows: {len(DataFrame(db.df)) if db.df else ''} + +""" + ) + st.dataframe(DataFrame(db.df).head(10)) + with st.popover("Dataset Info"): + st.write(db) + + st.markdown("### Import data") + uploaded_files = st.file_uploader( + "Upload one or more CSV files", type=["csv"], accept_multiple_files=True + ) + # if st.session_state.dataset: + # for question in st.session_state.dataset.questions[:3]: + # st.write(question.question) + + if uploaded_files: + st.success(f"{len(uploaded_files)} file(s) uploaded successfully.") + st.session_state.databases = [] + for file in uploaded_files: + + try: + db = AgenticDB() + df = db.import_db_from_csv(file) # βœ… pass the buffer + st.session_state.databases.append( + db + ) # show a preview of the first rows + + except Exception as e: + st.error(f"❌ Error importing {file.name}: {e}") + + +with st.form("Discovery Task"): + question = st.text_area( + "Research Question", + value=( + st.session_state.selected_question_object.question + if st.session_state.selected_question_object + else None + ), + ) + domain_knowledge = st.text_area( + "Domain Knowledge", + value=( + st.session_state.selected_question_object.domain_knowledge + if st.session_state.selected_question_object + else None + ), + ) + execute_query_button = st.form_submit_button("Submit Discovery Task") + + +if execute_query_button: + with st.spinner( + "Agentics is reading your documents and generating intermediate evidence before answering your question. This might take some time ..." + ): + + question = Question( + question=question, + dbs=st.session_state.databases, + domain_knowledge=domain_knowledge, + ) + answer = asyncio.run(answer_question_from_data(question)) + st.markdown( + f""" +#### Short Answer: + +{question.full_answer.short_answer} + +#### Full Answer + +{question.full_answer.full_answer} + +##### evidence +""" + ) + if question.full_answer.selected_evidence: + for evidence in question.full_answer.selected_evidence: + if evidence.evidence_found: + st.markdown( + f""" +{evidence.evidence} + +=================================================== + +""" + ) diff --git a/examples/discovery/discovery_playground.ipynb b/examples/discovery/discovery_playground.ipynb new file mode 100644 index 000000000..f14f5ca05 --- /dev/null +++ b/examples/discovery/discovery_playground.ipynb @@ -0,0 +1,64 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "0", + "metadata": {}, + "outputs": [], + "source": [ + "from transductions import answer_question_from_data\n", + "from atypes import IntermediateEvidence,Answer,Question\n", + "from agentic_db import AgenticDB\n", + "from dotenv import load_dotenv\n", + "load_dotenv()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1", + "metadata": {}, + "outputs": [], + "source": [ + "db = AgenticDB()\n", + "db.import_db_from_csv(\"/Users/gliozzo/Code/agentics911/agentics/sandbox/discoverybench/discoverybench/real/test/nls_incarceration/nls_incarceration_processed.csv\") \n", + "states=[Question(question=\"In what year were gender disparities highest in median wealth among individuals who were ever incarcerated?\", dbs=[db]) , \n", + " Question(question=\"In what year were gender disparities highest in median wealth among individuals who were ever incarcerated?\", dbs=[db])]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2", + "metadata": {}, + "outputs": [], + "source": [ + "answer = await answer_question_from_data(states)\n", + "for ans in answer:\n", + " print (ans)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.14" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/examples/discovery/my_sql_db.py b/examples/discovery/my_sql_db.py new file mode 100644 index 000000000..9d20e0281 --- /dev/null +++ b/examples/discovery/my_sql_db.py @@ -0,0 +1,149 @@ +import asyncio +import json +import os +import sqlite3 +from pathlib import Path +from typing import IO, Any, Dict, List, Literal, Optional, Union + +import aiosqlite +import pandas as pd +from httpx import AsyncClient +from loguru import logger +from pydantic import BaseModel, ConfigDict, PrivateAttr + + +async def _endpoint_call(call: Literal["GET", "POST"], endpoint: str, **kwargs): + api_key = os.getenv("ENDPOINT_API_KEY") + url = f"{os.getenv('ENDPOINT_URL')}{endpoint}" + async with AsyncClient(verify=False) as client: + headers = { + "Authorization": f"Bearer {api_key}", + "content-type": "application/json", + } + match call: + case "GET": + response = await client.get(url, headers=headers, **kwargs) + case "POST": + response = await client.post( + url, headers=headers, timeout=None, **kwargs + ) + return response.json() + + +async def execute_sql_on_endpoint(sql: str, db_id: str) -> str: + payload = { + "sql": sql, + "dataSourceId": db_id, + "includeCount": True, + "timeout": 10, + "verify": False, + } + endpoint = "/sql" + try: + sample = await _endpoint_call("POST", endpoint, json=payload) + if "error" in sample and sample["error"]: + return str(sample) + return json.dumps(sample.get("results")) + except Exception as e: + logger.debug( + f"Error executing the payload {payload} {e.__class__.__name__, str(e)}, retrying..." + ) + return None + + +class SqliteDB(BaseModel): + model_config = ConfigDict(arbitrary_types_allowed=True) + name: Optional[str] = None + _conn: sqlite3.Connection = PrivateAttr(default=None) + dataset_description: Optional[str] = None + name: Optional[str] = None + sqlite_schema: Optional[Any] = None + db_path: Optional[str] = None + columns: Optional[list] = None + metadata: Optional[dict] = None + + async def _get_sqlite_schema(self, db_path: str = None): + """ + Asynchronously return a simple schema description: + { + "table1": [ + {"name": "...", "type": "...", "nullable": True/False, + "default": "...", "pk": 0/1}, + ... + ] + } + """ + self.db_path = db_path or self.db_path + + schema = {} + + async with aiosqlite.connect(self.db_path) as db: + + # Get user tables + async with db.execute( + """ + SELECT name FROM sqlite_master + WHERE type='table' AND name NOT LIKE 'sqlite_%'; + """ + ) as cursor: + tables = [row[0] async for row in cursor] + + # For each table, read PRAGMA info + for table in tables: + async with db.execute(f"PRAGMA table_info({table});") as cursor: + columns_info = [row async for row in cursor] + + columns = [] + for cid, name, col_type, notnull, default, pk in columns_info: + columns.append( + { + "name": name, + "type": col_type, + "nullable": (notnull == 0), + "default": default, + "pk": pk, + } + ) + + schema[table] = columns + self.sqlite_schema = schema + + return schema + + async def import_db(self, db_path: str = None): + self.db_path = db_path or self.db_path + self._conn = sqlite3.connect(self.db_path) + self.sqlite_schema = await self._get_sqlite_schema() + return self.sqlite_schema + + async def async_execute_sql(self, sql_query: str) -> str: + """DB id could be a path or a Endpoint connection string""" + + if self.db_path: + try: + async with aiosqlite.connect(self.db_path) as db: + async with db.execute(sql_query) as cursor: + columns = [description[0] for description in cursor.description] + rows = await asyncio.wait_for(cursor.fetchall(), timeout=10) + df = pd.DataFrame(rows, columns=columns) + return df.to_json() + except Exception as e: + pass + + try: + async with aiosqlite.connect(self.db_path) as db: + async with db.execute(sql_query.replace('"', "'")) as cursor: + columns = [description[0] for description in cursor.description] + rows = await asyncio.wait_for(cursor.fetchall(), timeout=10) + df = pd.DataFrame(rows, columns=columns) + return df.to_json() + except Exception as e: + return f"Error: {str(e)}" + + +# def get_description(self) -> str: +# return f"""=============================== +# Dataset Path: {self.path} +# Dataset Description: {self.dataset_description} +# Columns: {self.columns} +# """ diff --git a/examples/discovery/transductions.py b/examples/discovery/transductions.py new file mode 100644 index 000000000..4310c1eba --- /dev/null +++ b/examples/discovery/transductions.py @@ -0,0 +1,85 @@ +from agentic_db import AgenticDB +from atypes import Answer, IntermediateEvidence, Question +from pandas import DataFrame + +from agentics import AG +from agentics.core.transducible_functions import ( + Transduce, + transducible, +) + +# @transducible() +# async def describe_dataset(state: AgenticDB) -> str: +# """Provide a paragraph long description for the following dataset""" +# return Transduce(state) +# # # new_ag = AgenticDB(df=state.df) +# # # return ParameterWrapper(new_ag) + + +@transducible() +async def answer_question_from_data(state: Question) -> Question: + + if state.question: + source_data = [AG.from_dataframe(DataFrame(db.df)) for db in state.dbs] + + intermediate_evidence = AG(atype=IntermediateEvidence) + + for data in source_data: + dataset_descriptions = [ + f""" +================== + +Name: {db.name} +Description: +{db.dataset_description} +Columns: +{db.columns} + +""" + for db in state.dbs + ] + + intermediate_evidence_for_source = await ( + AG( + atype=IntermediateEvidence, + transduction_type="areduce", + areduce_batch_size=10000, + instructions=f""" +You have been provided with a CSV file which might contain relevant information to answer a given QUESTION. +QUESTION: {state.question} + +Your task is to collect intermediate evidence needed to answer the question from the provided data at a later stage +DATASET descriptions: +{"".join(dataset_descriptions)} +DOMAIN_KNOWLEDGE: {state.domain_knowledge} +QUESTION: {state.question} +""", + ) + << data + ) + + intermediate_evidence.states += intermediate_evidence_for_source.states + + final_answer = await ( + AG( + atype=Answer, + instructions=f""" +You previously collected intermediate evidence to answer a given QUESTION after inspecting several data sources. +Your task is to proivide a single answer to the question taking into account the provided evidence. +DATASET descriptions: +{"".join(dataset_descriptions)} +DOMAIN_KNOWLEDGE: {state.domain_knowledge} +QUESTION: {state.question} +""", + transduction_type="areduce", + areduce_batch_size=10000, + ) + << intermediate_evidence + ) + + if len(final_answer) > 0: + state.generated_hypothesis = final_answer[0].short_answer + state.full_answer = final_answer[0] + state.intermediate_evidence = intermediate_evidence.states + + return state diff --git a/src/agentics/core/semantic_operators.py b/src/agentics/core/semantic_operators.py index 8f61845d5..844ce846b 100644 --- a/src/agentics/core/semantic_operators.py +++ b/src/agentics/core/semantic_operators.py @@ -15,11 +15,19 @@ from agentics.core.atype import create_pydantic_model +def normalize_input_data(source: AG | pd.DataFrame) -> AG: + if type(source) is pd.DataFrame: + return AG.from_dataframe(source) + elif type(source) is AG: + return source.clone() + else: + raise ValueError("source must be of type AG or pd.DataFrame") + + async def sem_map( source: AG | pd.DataFrame, target_type: Type[BaseModel] | str, - instructions: str = None, - source_fields: list[str] = None, + instructions: str, merge_output: bool = True, ## Target, Merged **kwargs, ) -> AG | pd.DataFrame: @@ -69,12 +77,7 @@ async def sem_map( ----- - The semantic mapping is executed asynchronously via `await (target_ag << source)`. """ - if type(source) is pd.DataFrame: - ag_source = AG.from_dataframe(source) - elif type(source) is AG: - ag_source = source.clone() - else: - raise ValueError("source must be of type AG or pd.DataFrame") + ag_source = normalize_input_data(source) target_ag = AG( atype=( create_pydantic_model( @@ -86,7 +89,7 @@ async def sem_map( **kwargs, ) - ag_source.transduce_fields = source_fields + ag_source.prompt_template = instructions map_out = await (target_ag << ag_source) output_ag = None @@ -142,12 +145,7 @@ async def sem_filter( - Default classifier settings include `amap_batch_size=20` for batched evaluation. """ - if type(source) is pd.DataFrame: - ag_source = AG.from_dataframe(source) - elif type(source) is AG: - ag_source = source.clone() - else: - raise ValueError("source must be of type AG or pd.DataFrame") + ag_source = normalize_input_data(source) target_ag = AG( atype=create_pydantic_model( @@ -171,3 +169,32 @@ async def sem_filter( return target.to_dataframe() else: return target + + +async def sem_agg( + source: AG | pd.DataFrame, + target_type: Type[BaseModel] | str, + instructions: str = None, + # merge_output: bool = True, ## Target, Merged + **kwargs, +) -> AG | pd.DataFrame: + """ """ + ag_source = normalize_input_data(source) + target_ag = AG( + atype=( + create_pydantic_model( + [(target_type, "str", instructions, False)], target_type + ) + if isinstance(target_type, str) + else target_type + ), + **kwargs, + ) + + ag_source.prompt_template = instructions + ag_source.transduction_type = "areduce" + + output_ag = await (target_ag << ag_source) + if type(source) is pd.DataFrame: + return output_ag.to_dataframe() + return output_ag From afc42c2460ee47b75a5df15b0b9cd6bb8a7688d8 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Fri, 16 Jan 2026 08:18:18 -0500 Subject: [PATCH 09/27] fixex --- src/agentics/core/agentics.py | 28 ++++++++------ src/agentics/core/llm_connections.py | 26 +++++++++++++ .../agentics1.0/logical_transduction.ipynb | 1 + tutorials/atypes.ipynb | 38 ++++++++++++++++--- 4 files changed, 75 insertions(+), 18 deletions(-) diff --git a/src/agentics/core/agentics.py b/src/agentics/core/agentics.py index b82255c7f..0af42007b 100644 --- a/src/agentics/core/agentics.py +++ b/src/agentics/core/agentics.py @@ -1065,24 +1065,28 @@ def merge_states(self, other: AG) -> AG: Merge states of two AGs pairwise """ - merged = self.clone() - merged.states = [] - merged.explanations = [] - merged.atype = merge_pydantic_models( - self.atype, - other.atype, - name=f"Merged{self.atype.__name__}#{other.atype.__name__}", - ) - for self_state in self: - for other_state in other: + if len(self) == len(other): + merged = self.clone() + merged.states = [] + merged.explanations = [] + merged.atype = merge_pydantic_models( + self.atype, + other.atype, + name=f"Merged{self.atype.__name__}#{other.atype.__name__}", + ) + for self_state, other_state in zip(self, other): merged.states.append( merged.atype(**other_state.model_dump(), **self_state.model_dump()) ) - return merged + return merged + else: + raise ValueError( + f"Cannot merge states of AGs with different lengths: {len(self)} != {len(other)}" + ) def compose_states(self, other: AG) -> AG: """ - compose states of two AGs pairwise, + compose states of two AGs, """ merged = self.clone() diff --git a/src/agentics/core/llm_connections.py b/src/agentics/core/llm_connections.py index 3fe0d76cb..35c704fb2 100644 --- a/src/agentics/core/llm_connections.py +++ b/src/agentics/core/llm_connections.py @@ -12,6 +12,32 @@ verbose = False +from typing import Any, Dict, List + +from crewai.llm import BaseLLM + + +class LangChainLLMAdapter(BaseLLM): + """ + CrewAI LLM adapter that delegates calls to a LangChain LLM. + """ + + def __init__(self, lc_llm, model_name: str = "langchain-wrapped"): + super().__init__(model=model_name) + self.lc_llm = lc_llm + + def call(self, messages: List[Dict[str, str]], **kwargs) -> str: + """ + messages: [{"role": "system|user|assistant", "content": "..."}] + """ + # LangChain expects a string or messages depending on LLM type. + # For chat models, concatenate safely: + prompt = "\n".join(f"{m['role']}: {m['content']}" for m in messages) + + result = self.lc_llm.invoke(prompt) + return result.content if hasattr(result, "content") else str(result) + + def get_llm_provider(provider_name: str = None) -> LLM: """ Retrieve the LLM instance based on the provider name. If no provider name is given, diff --git a/tutorials/agentics1.0/logical_transduction.ipynb b/tutorials/agentics1.0/logical_transduction.ipynb index e1df9cfea..f2f5dc36f 100644 --- a/tutorials/agentics1.0/logical_transduction.ipynb +++ b/tutorials/agentics1.0/logical_transduction.ipynb @@ -213,6 +213,7 @@ " atype=Answer,\n", " verbose_transduction=True, # Set to verbose to see transduction timings and other logs\n", " transduction_logs_path=\"/tmp/answers.jsonl\",\n", + " amap_batch_size=2,\n", ") # Optionally write longs of transductions on the specified path\n", "questions = [\n", " \"Where is Paris?\",\n", diff --git a/tutorials/atypes.ipynb b/tutorials/atypes.ipynb index 1c0833bde..ab18b724f 100644 --- a/tutorials/atypes.ipynb +++ b/tutorials/atypes.ipynb @@ -192,6 +192,32 @@ "cell_type": "markdown", "id": "8", "metadata": {}, + "source": [ + "## Create type from definition" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9", + "metadata": {}, + "outputs": [], + "source": [ + "from agentics.core.atype import create_pydantic_model\n", + "Answer = create_pydantic_model([(\"answer\", \"str\", \"the answer for the question\",False), (\"confidence\", \"float\", None,False)])\n", + "\n", + "schema = [{'name': 'name', 'description': 'the name of the person', 'type': 'str', 'multiple': 'False'},\n", + " {'name': 'age', 'description': 'the age of the person', 'type': 'int', 'multiple': 'False'}]\n", + "new_schema = [(field[\"name\"] , field[\"description\"], field[\"type\"], False) for field in schema]\n", + "Person = create_pydantic_model(new_schema)\n", + "print(Answer(confidence=0.95, answer=\"Paris is the capital of France.\"))\n", + "print(Person(name=\"Alice\"))" + ] + }, + { + "cell_type": "markdown", + "id": "10", + "metadata": {}, "source": [ "### Synthetic Data Generation πŸ§ͺ\n", "\n", @@ -219,7 +245,7 @@ { "cell_type": "code", "execution_count": null, - "id": "9", + "id": "11", "metadata": {}, "outputs": [], "source": [ @@ -232,7 +258,7 @@ }, { "cell_type": "markdown", - "id": "10", + "id": "12", "metadata": {}, "source": [ "### Dynamic Type Generation ✨\n", @@ -257,7 +283,7 @@ { "cell_type": "code", "execution_count": null, - "id": "11", + "id": "13", "metadata": {}, "outputs": [], "source": [ @@ -270,7 +296,7 @@ }, { "cell_type": "markdown", - "id": "12", + "id": "14", "metadata": {}, "source": [ "## Logical Proximity (LP) 🧭\n", @@ -295,7 +321,7 @@ { "cell_type": "code", "execution_count": null, - "id": "13", + "id": "15", "metadata": {}, "outputs": [], "source": [ @@ -305,7 +331,7 @@ }, { "cell_type": "markdown", - "id": "14", + "id": "16", "metadata": {}, "source": [ "### Logical Proximity - Key Properties\n", From eeae2aae00deba6e26012634dd247a7e8883d55f Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Thu, 22 Jan 2026 09:07:52 -0500 Subject: [PATCH 10/27] merge fix for langflow integration --- src/agentics/core/agentics.py | 40 +++++++++++---------- src/agentics/core/atype.py | 8 ++--- src/agentics/core/transducible_functions.py | 4 ++- tutorials/transducible_functions.ipynb | 2 +- 4 files changed, 30 insertions(+), 24 deletions(-) diff --git a/src/agentics/core/agentics.py b/src/agentics/core/agentics.py index 0af42007b..4ca8e891c 100644 --- a/src/agentics/core/agentics.py +++ b/src/agentics/core/agentics.py @@ -689,15 +689,18 @@ async def llm_call(input: AGString) -> AGString: if self.provide_explanations and isinstance(other, AG): target_explanation = AG(atype=Explanation) + output.prompt_template = None + output.transduce_fields = None target_explanation.instructions = f""" - You have been presented with two Pydantic Objects: - a left object that was logically derived from a right object. - Your task is to provide a detailed explanation of how the left object was derived from the right object.""" - target_explanation = await ( - target_explanation << output.compose_states(other) - ) - - self.explanations = target_explanation.states + You have previously transduced an object of type {other.atype.__name__} (source) from an object of type {self.atype.__name__} (target). + Now look back at both objects and provide a detailed explanation on how each field of the target object was logically derived from the source object. + Provide short and concise, data grounded explanations, field by field, avoiding redundancy. + If you think that transduction was wrong or not logically supported by the source object, say it clearly in the explanation and provide low confidence score (0.0). + Provide high confidence score (1.0) only if you are certain that the transduction is logically correct and fully supported by the source object. + """ + explanation = await (target_explanation << output.compose_states(other)) + + self.explanations = explanation.states self.states = output.states return self else: @@ -983,7 +986,7 @@ def to_dataframe(self) -> DataFrame: Returns: DataFrame: A pandas DataFrame representing the current states. """ - data = [state.model_dump() for state in self.states] + data = [state.model_dump(mode="json") for state in self.states] return pd.DataFrame(data) ######################################## @@ -1089,15 +1092,16 @@ def compose_states(self, other: AG) -> AG: compose states of two AGs, """ - merged = self.clone() - merged.states = [] - merged.explanations = [] - merged.atype = self.atype @ other.atype - - for self_state in self: - for other_state in other: - merged.states.append(merged.atype(right=other_state, left=self_state)) - return merged + composed = self.clone() + composed.states = [] + composed.explanations = [] + composed.atype = self.atype @ other.atype + + for self_state, other_state in zip(self.states, other.states): + composed.states.append( + composed.atype(source=other_state, target=self_state) + ) + return composed async def map_atypes(self, other: AG) -> ATypeMapping: if self.verbose_agent: diff --git a/src/agentics/core/atype.py b/src/agentics/core/atype.py index 7ea397490..3607e8268 100644 --- a/src/agentics/core/atype.py +++ b/src/agentics/core/atype.py @@ -636,8 +636,8 @@ def compose_types(A, B, *, name=None): Composite = create_model( name, - left=(Optional[A], None), - right=(Optional[B], None), + target=(Optional[A], None), + source=(Optional[B], None), __base__=BaseModel, ) @@ -670,7 +670,7 @@ def _istype_matmul(A, B): def _instance_matmul(a: BaseModel, b: BaseModel): """ INSTANCE composition: - a @ b β†’ Composite(left=a, right=b) + a @ b β†’ Composite(target=a, surce=b) """ if not isinstance(b, BaseModel): raise TypeError(f"Cannot compose instance {a} with {b}") @@ -681,7 +681,7 @@ def _instance_matmul(a: BaseModel, b: BaseModel): CompositeModel = A @ B # Build structural composite - return CompositeModel(left=a, right=b) + return CompositeModel(target=a, source=b) BaseModel.__matmul__ = _instance_matmul diff --git a/src/agentics/core/transducible_functions.py b/src/agentics/core/transducible_functions.py index 3d09f0b88..6ac6bc6de 100644 --- a/src/agentics/core/transducible_functions.py +++ b/src/agentics/core/transducible_functions.py @@ -135,6 +135,7 @@ def transducible( post_processing_function: Optional[Callable[[BaseModel], BaseModel]] = None, persist_output: str = None, transduce_fields: list[str] = None, + prompt_template: str = None, ): if tools is None: tools = [] @@ -169,6 +170,7 @@ def _transducible(fn: Callable): transduction_timeout=timeout, save_amap_batches_to_path=persist_output, provide_explanations=provide_explanation, + prompt_template=prompt_template, ) source_ag_template = AG( atype=SourceModel, @@ -228,7 +230,7 @@ async def wrap_single(input_obj): if provide_explanation and len(target_ag.explanations) == 1: return TransductionResult(out, target_ag.explanations[0]) - return out + return TransductionResult(out, None) raise RuntimeError("Transduction returned no output.") diff --git a/tutorials/transducible_functions.ipynb b/tutorials/transducible_functions.ipynb index 1e7374435..8aaea046d 100644 --- a/tutorials/transducible_functions.ipynb +++ b/tutorials/transducible_functions.ipynb @@ -254,7 +254,7 @@ "from agentics.core.transducible_functions import Transduce, transducible\n", "\n", "\n", - "@transducible(provide_explanation=True, llm=llm)\n", + "@transducible(provide_explanation=False, llm=llm, prompt_template=\"{movie_name}: {description}\")\n", "async def classify_genre(state:Movie)-> Genre:\n", " \"\"\"Classify the genre of the source Movie \"\"\"\n", " return Transduce(state)\n", From 80c3cddbce593efadf4ebce69afdeeb5f00fab58 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Thu, 29 Jan 2026 08:52:20 -0500 Subject: [PATCH 11/27] Small fixes --- src/agentics/core/agentics.py | 6 +++--- src/agentics/core/atype.py | 1 - tutorials/logical_transduction_algebra.ipynb | 2 +- tutorials/transducible_functions.ipynb | 2 +- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a/src/agentics/core/agentics.py b/src/agentics/core/agentics.py index 4ca8e891c..62799e8ab 100644 --- a/src/agentics/core/agentics.py +++ b/src/agentics/core/agentics.py @@ -602,8 +602,8 @@ async def llm_call(input: AGString) -> AGString: # Perform Transduction transducer_class = ( PydanticTransducerCrewAI - if type(self.llm) == LLM - else PydanticTransducerMellea if type(self.llm) == str else None + # if type(self.llm) == LLM + # else PydanticTransducerMellea if type(self.llm) == str else None ) if not transducer_class: raise TypeError( @@ -986,7 +986,7 @@ def to_dataframe(self) -> DataFrame: Returns: DataFrame: A pandas DataFrame representing the current states. """ - data = [state.model_dump(mode="json") for state in self.states] + data = [state.model_dump() for state in self.states] return pd.DataFrame(data) ######################################## diff --git a/src/agentics/core/atype.py b/src/agentics/core/atype.py index 3607e8268..f30fa088b 100644 --- a/src/agentics/core/atype.py +++ b/src/agentics/core/atype.py @@ -325,7 +325,6 @@ def create_pydantic_model( model_name = name field_definitions = {} - print(fields) for field_name, type_name, description, required in fields: ptype = type_mapping[type_name] if type_name in type_mapping else Any if required: diff --git a/tutorials/logical_transduction_algebra.ipynb b/tutorials/logical_transduction_algebra.ipynb index a0ea4054a..f45d6d18e 100644 --- a/tutorials/logical_transduction_algebra.ipynb +++ b/tutorials/logical_transduction_algebra.ipynb @@ -112,7 +112,7 @@ "write_mail_to_alfio = Email<< With(\n", " GenericInput,\n", " instructions=\"Write an email to Alfio Gliozzo\",\n", - ")\n", + " prompyt_template=\"{content}\" )\n", "news = GenericInput(content=\"Zoran Mandani won the Election in NYC\")\n", "mail = await write_mail_to_alfio(news)\n", "print(mail)" diff --git a/tutorials/transducible_functions.ipynb b/tutorials/transducible_functions.ipynb index 8aaea046d..314879069 100644 --- a/tutorials/transducible_functions.ipynb +++ b/tutorials/transducible_functions.ipynb @@ -259,7 +259,7 @@ " \"\"\"Classify the genre of the source Movie \"\"\"\n", " return Transduce(state)\n", "\n", - "genre, explanation = await classify_genre(Movie(\n", + "genre = await classify_genre(Movie(\n", " movie_name=\"The Godfather\",\n", " description=\"The aging patriarch of an organized crime dynasty transfers control of his clandestine empire to his reluctant son.\",\n", " year=1972\n", From 74ad03ea32de491862157e3d7d10cc27393ef685 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Wed, 4 Feb 2026 17:45:01 -0500 Subject: [PATCH 12/27] semantic operator fix for langflow --- src/agentics/core/semantic_operators.py | 6 ++++-- src/agentics/core/transducible_functions.py | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/src/agentics/core/semantic_operators.py b/src/agentics/core/semantic_operators.py index 844ce846b..a59a3284a 100644 --- a/src/agentics/core/semantic_operators.py +++ b/src/agentics/core/semantic_operators.py @@ -156,8 +156,10 @@ async def sem_filter( amap_batch_size=20, **kwargs, ) - - ag_source.prompt_template = predicate_template + if "{" in predicate_template: + ag_source.prompt_template = predicate_template + else: + target_ag.instructions += f"\n\nPredicate: {predicate_template}" map_out = await (target_ag << ag_source) target = ag_source.clone() target.states = [] diff --git a/src/agentics/core/transducible_functions.py b/src/agentics/core/transducible_functions.py index 6ac6bc6de..9f5817293 100644 --- a/src/agentics/core/transducible_functions.py +++ b/src/agentics/core/transducible_functions.py @@ -558,7 +558,7 @@ async def generate_prototypical_instances( instructions=f""" Generate list of {n_instances} random instances of the following type {type.model_json_schema()}. - Try to fill most of the attributed for each generated instance as possible + fill all attributed for each generated instance """, llm=llm, ) From 62ff40dc6e7ebbaef8d15550f243235ae3c1c979 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Wed, 11 Feb 2026 18:46:25 -0500 Subject: [PATCH 13/27] New Documentation --- README.md | 67 +++--- docs/agentics.md | 31 ++- docs/core_concepts.md | 34 +-- docs/getting_started.md | 126 +++++++++-- docs/index.md | 79 ++++++- docs/map_reduce.md | 381 +++++++++++++++++++++++++++++++ docs/optimization.md | 323 ++++++++++++++++++++++++++ docs/tool_integration.md | 402 +++++++++++++++++++++++++++++++++ docs/transducible_functions.md | 209 ++++++++++++++--- examples/hello_world.py | 35 ++- mkdocs.yml | 13 +- tutorials/data/movies.jsonl | 20 -- 12 files changed, 1568 insertions(+), 152 deletions(-) create mode 100644 docs/map_reduce.md create mode 100644 docs/optimization.md create mode 100644 docs/tool_integration.md delete mode 100644 tutorials/data/movies.jsonl diff --git a/README.md b/README.md index 73a7b98a1..5126b6a96 100644 --- a/README.md +++ b/README.md @@ -52,38 +52,40 @@ uv run python examples/hello_world.py ## πŸ§ͺ Example Usage ```python -from typing import Optional from pydantic import BaseModel, Field +from agentics.core.transducible_functions import transducible, Transduce + +class ProductDescription(BaseModel): + name: str + features: str + price: float + +class ViralTweet(BaseModel): + tweet: str = Field(..., description="Engaging tweet under 280 characters") + hashtags: list[str] = Field(..., description="3-5 relevant hashtags") + hook: str = Field(..., description="Attention-grabbing opening line") + +@transducible() +async def generate_viral_tweet(product: ProductDescription) -> ViralTweet: + """Transform boring product descriptions into viral social media content.""" + return Transduce(product) + +# Transform a product into viral content +product = ProductDescription( + name="Agentics Framework", + features="Type-safe AI workflows with LLM-powered transductions", + price=0.0 # Open source! +) -from agentics.core.transducible_functions import Transduce, transducible - - -class Movie(BaseModel): - movie_name: Optional[str] = None - description: Optional[str] = None - year: Optional[int] = None - - -class Genre(BaseModel): - genre: Optional[str] = Field(None, description="e.g., comedy, drama, action") - - -@transducible(provide_explanation=True) -async def classify_genre(state: Movie) -> Genre: - """Classify the genre of the source Movie.""" - return Transduce(state) - +tweet = await generate_viral_tweet(product) +print(f"πŸ”₯ {tweet.tweet}") +print(f"πŸ“± {' '.join(tweet.hashtags)}") +``` -genre, explanation = await classify_genre( - Movie( - movie_name="The Godfather", - description=( - "The aging patriarch of an organized crime dynasty transfers control " - "of his clandestine empire to his reluctant son." - ), - year=1972, - ) -) +**Output:** +``` +πŸ”₯ Stop wrestling with unstructured LLM outputs! 🎯 Agentics gives you type-safe AI workflows that just work. Build production-ready agents in minutes, not weeks. And it's FREE! πŸš€ +πŸ“± #AI #OpenSource #Python #LLM #DevTools ``` --- @@ -124,12 +126,15 @@ Apache 2.0 - Alfio Massimiliano Gliozzo (IBM Research) β€” gliozzo@us.ibm.com **Core Contributors** -- Nahuel Defosse (IBM Research) β€” nahuel.defosse@ibm.com - Junkyu Lee (IBM Research) β€” Junkyu.Lee@ibm.com +- Nahuel Defosse (IBM Research) β€” nahuel.defosse@ibm.com - Naweed Aghmad Khan (IBM Research) β€” naweed.khan@ibm.com + +**Additional Contributors** - Christodoulos Constantinides (IBM Watson) β€” Christodoulos.Constantinides@ibm.com +- Nandana Mihindukulasooriya (IBM Research) β€” nandana@ibm.com - Mustafa Eyceoz (Red Hat) β€” Mustafa.Eyceoz@partner.ibm.com - +- Gaetano Rossiello (IBM Research) β€” gaetano.rossiello@ibm.com --- diff --git a/docs/agentics.md b/docs/agentics.md index ea4388bff..2c622ab0f 100644 --- a/docs/agentics.md +++ b/docs/agentics.md @@ -1,11 +1,20 @@ -# Agentics +# AG (Agentics) -Agentics objects are wrappers around list of objects having the same Pydantic Type. +**AG** (short for "Agentics") objects are wrappers around lists of objects having the same Pydantic type. They are designed to enable async logical transduction among their instances. -Agentics enable us to think about AI workflows in terms of structured data transformations rather than agent behaviours, knowledge and tasks. +AG containers enable us to think about AI workflows in terms of structured data transformations rather than agent behaviors, knowledge, and tasks. -## The Agentics class -Agentics is a Python class that wraps a list of Pydantic objects and enables structured, type-driven logical transduction between them. +## The AG Class + +AG (Agentics) is a Python class that wraps a list of Pydantic objects and enables structured, type-driven logical transduction between them. + +**Import and Usage:** +```python +from agentics import AG # Recommended: use AG alias + +# Create a typed container +movies = AG(atype=Movie) +``` Internally, Agentics is implemented as a Pydantic model. It holds: β€’ atype: a reference to the Pydantic class shared by all objects in the list. @@ -109,11 +118,12 @@ You can also modify and rebind an exiting Agentic. Similarly can also remove att ```python movies = AG.from_csv("data/orders.csv") +print(movies[0]) movies.add_attribute("review",str) movies.add_attribute("quality_score",int,description="The quality of the movies in a scale 0 to 10") print(movies[0]) movies.subset_atype("title","genre","description") -print(movies[0]) ## note that movies[0] is equivalent to +print(movies[0]) ## note that movies[0] is a shorthand for movies.states[0] ``` @@ -149,9 +159,12 @@ asyncio.run(main()) ## Reference code -[explore this example](src/agentics/examples/agentics_basics.py) +See the [examples directory](../examples/) for practical demonstrations of AG usage, including: +- `hello_world.py` - Basic transduction example +- `generate_tweets.py` - Content generation +- `emotion_extractor.py` - Text analysis -## See Next: Transduction +## See Next: Transducible Functions -Wrapping pydantic types into Agentics provides them with the ability to perform transduction, as described in the [next section](transduction.md) +Wrapping pydantic types into Agentics provides them with the ability to perform transduction, as described in the [Transducible Functions](transducible_functions.md) documentation. diff --git a/docs/core_concepts.md b/docs/core_concepts.md index 0068e764b..40e01dcf8 100644 --- a/docs/core_concepts.md +++ b/docs/core_concepts.md @@ -64,11 +64,11 @@ Example: from pydantic import BaseModel class Review(BaseModel): - text: str + text: Optional[str] = None class ReviewSummary(BaseModel): - sentiment: str - summary: str + sentiment: Optional[str] = None + summary: Optional[str] = None ``` A transducible function might be: @@ -92,14 +92,14 @@ You don’t call the LLM directly; you **call the transducible function**, which --- -## 3. Typed State Containers (AGs): Working with Collections πŸ—‚οΈ +## 3. Typed State Containers (AG): Working with Collections πŸ—‚οΈ Transformations rarely happen on a single object. You typically work with **collections** of items (rows, documents, events, etc.). -Agentics introduces **typed state containers** (AG) to: +Agentics introduces **typed state containers** (called **AG**, short for "Agentics") to: -- Hold a collection of instances of a given Pydantic type -- Preserve that type information across operations +- Hold a collection of instances of a given Pydantic type +- Preserve that type information across operations - Provide a uniform interface for Map–Reduce, filtering, joining, etc. @@ -113,18 +113,24 @@ AG[Review] └─ row n: Review(text="…") ``` -Applying a transducible function `(Review) -> ReviewSummary` over an `AG[Review]` conceptually yields an `AG[ReviewSummary]`. +Applying a transducible function `(Review) -> ReviewSummary` over an `AG` with atype `Review` conceptually yields an `AG` of type `ReviewSummary`. Typed state containers give you: -- **Clarity** – you always know what type you’re holding. -- **Safety** – operations can check types and schemas instead of guessing. +- **Clarity** – you always know what type you're holding. +- **Safety** – operations can check types and schemas instead of guessing. - **Composability** – containers can flow between functions and stages. -You can think of state containers as the **data plane** of Agentics. +You can think of state containers (AGs) as the **data plane** of Agentics. -Note: The name Agentics is derived as a legacy from the first version of Agentics, in which data models and transformations were blended into the same object. By introducing transducible functions as first class citizens, Agentics 2.0 uses AGs primarily as a data structure, although it is still possible to use them directly for transformations. See agentics v1.0 documentation to learn more. +```python +from agentics import AG # Recommended alias + +movies = AG(atype=Movie) # Create a typed container +``` + +**Historical Note:** In Agentics 1.0, data models and transformations were blended into the same object. Agentics 2.0 separates concerns by introducing transducible functions as first-class citizens, while AG containers focus on data management. The v1.0 API is still supported for backward compatibility. --- @@ -251,5 +257,5 @@ A typical workflow looks like this: From here, you can explore: - πŸ‘‰ [Transducible Functions](transducible_functions.md) for concrete examples of defining and using transducible functions -- πŸ‘‰ `types_and_states.md` for data modeling patterns -- πŸ‘‰ `mapreduce.md` to see how large-scale execution works in practice +- πŸ‘‰ [Agentics (AG)](agentics.md) for data modeling patterns and typed state containers +- πŸ‘‰ [Map-Reduce Tutorial](../tutorials/map_reduce.ipynb) to see how large-scale execution works in practice diff --git a/docs/getting_started.md b/docs/getting_started.md index 157a2f095..c29e798b5 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -19,7 +19,7 @@ Agentics is a lightweight, Python-native framework for building structured, agen curl -LsSf https://astral.sh/uv/install.sh | sh ``` - Other installation options [here](curl -LsSf https://astral.sh/uv/install.sh | sh) + Other installation options [here](https://docs.astral.sh/uv/getting-started/installation/) * Install the dependencies @@ -59,11 +59,11 @@ ollama pull ollama/deepseek-r1:latest - `MODEL` - watsonx/meta-llama/llama-3-3-70b-instruct (or alternative supporting function call) -#### Google Gemini (offer free API key) +#### Google Gemini (offers free API key) -- `WATSONX_APIKEY` - WatsonX API key +- `GEMINI_API_KEY` - Your Google Gemini API key (get it from [Google AI Studio](https://aistudio.google.com/)) -- `MODEL` - watsonx/meta-llama/llama-3-3-70b-instruct (or alternative supporting function call) +- `MODEL` - `gemini/gemini-1.5-pro` or `gemini/gemini-1.5-flash` (or other Gemini models supporting function calling) #### VLLM (Need dedicated GPU server): @@ -87,28 +87,128 @@ python examples/agentics_web_search_report.py ## Hello World +Transform boring product descriptions into viral tweets in just a few lines: + ```python -from typing import Optional from pydantic import BaseModel, Field +from agentics.core.transducible_functions import transducible, Transduce + +from typing import Optional + +class ProductDescription(BaseModel): + name: Optional[str] = None + features: Optional[str] = None + price: Optional[float] = None + +class ViralTweet(BaseModel): + tweet: Optional[str] = Field(None, description="Engaging tweet under 280 characters") + hashtags: Optional[list[str]] = Field(None, description="3-5 relevant hashtags") + hook: Optional[str] = Field(None, description="Attention-grabbing opening line") + +@transducible() +async def generate_viral_tweet(product: ProductDescription) -> ViralTweet: + """Transform boring product descriptions into viral social media content.""" + return Transduce(product) + +# Transform a product into viral content +product = ProductDescription( + name="Agentics Framework", + features="Type-safe AI workflows with LLM-powered transductions", + price=0.0 # Open source! +) + +tweet = await generate_viral_tweet(product) +print(f"πŸ”₯ {tweet.tweet}") +print(f"πŸ“± {' '.join(tweet.hashtags)}") +``` + +**Output:** +``` +πŸ”₯ Stop wrestling with unstructured LLM outputs! 🎯 Agentics gives you type-safe AI workflows that just work. Build production-ready agents in minutes, not weeks. And it's FREE! πŸš€ +πŸ“± #AI #OpenSource #Python #LLM #DevTools +``` -from agentics.core.transducible_functions import Transduce, transducible +### Alternative: Using the `<<` Operator +For quick one-off transductions, use the `<<` operator: -class Movie(BaseModel): - movie_name: Optional[str] = None +```python +from pydantic import BaseModel + +from typing import Optional + +class Product(BaseModel): + name: Optional[str] = None description: Optional[str] = None - year: Optional[int] = None +class Tweet(BaseModel): + content: Optional[str] = None + +# Create transduction on the fly +make_tweet = Tweet << Product -class Genre(BaseModel): - genre: Optional[str] = Field(None, description="e.g., comedy, drama, action") +product = Product( + name="Agentics", + description="Type-safe AI framework for Python" +) + +tweet = await make_tweet(product) +print(tweet.content) +``` -movie = Movie(movie_name="The Godfather") +This concise syntax is perfect for exploratory work and rapid prototyping! -genre = await (Genre << Movie)(movie) +### Batch Processing: Multiple Products + +Transducible functions automatically support batch processing. Process multiple products at once in parallel: + +```python +products = [ + ProductDescription( + name="Agentics Framework", + features="Type-safe AI workflows with LLM-powered transductions", + price=0.0 + ), + ProductDescription( + name="Smart Coffee Maker", + features="AI-powered brewing with perfect temperature control", + price=299.99 + ), + ProductDescription( + name="Wireless Earbuds Pro", + features="Active noise cancellation and 30-hour battery life", + price=149.99 + ), +] + +# Automatically processes all products in parallel +tweets = await generate_viral_tweet(products) + +# Display results +for product, tweet in zip(products, tweets): + print(f"\nπŸ“¦ Product: {product.name}") + print(f"πŸ”₯ Tweet: {tweet.tweet}") + print(f"πŸ“± Tags: {' '.join(tweet.hashtags)}") ``` +**Output:** +``` +πŸ“¦ Product: Agentics Framework +πŸ”₯ Tweet: Stop wrestling with unstructured LLM outputs! 🎯 Agentics gives you type-safe AI workflows that just work. Build production-ready agents in minutes, not weeks. And it's FREE! πŸš€ +πŸ“± Tags: #AI #OpenSource #Python #LLM #DevTools + +πŸ“¦ Product: Smart Coffee Maker +πŸ”₯ Tweet: Wake up to perfection! β˜• Our AI-powered coffee maker learns your taste and brews the perfect cup every time. Never settle for mediocre coffee again! πŸ€– +πŸ“± Tags: #SmartHome #Coffee #AI #Tech #MorningRoutine + +πŸ“¦ Product: Wireless Earbuds Pro +πŸ”₯ Tweet: Silence the world, amplify your music! 🎧 30 hours of pure audio bliss with active noise cancellation. Your commute just got an upgrade! πŸ”‹ +πŸ“± Tags: #Audio #Tech #Wireless #Music #Productivity +``` + +The same transducible function works seamlessly for both single items and batchesβ€”no code changes needed! + ### Installation details === "Poetry" diff --git a/docs/index.md b/docs/index.md index ae0e35f00..3ab2f767f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,27 +6,40 @@ Agentics is a lightweight, Python-native framework for building **structured and ## πŸ“š Documentation Overview -- **[Getting Started](getting_started.md)** πŸš€ +### Core Documentation + +- **[Getting Started](getting_started.md)** πŸš€ Install Agentics, set up your environment, and run your first transducible function over a small dataset. -- **[Core Concepts](core_concepts.md)** 🧠 +- **[Core Concepts](core_concepts.md)** 🧠 The mental model: Pydantic types, transducible functions, typed state containers, Logical Transduction Algebra (LTA), and Map–Reduce. - -- **[Transducible Functions](transducible_functions.md)** βš™οΈ +- **[Transducible Functions](transducible_functions.md)** βš™οΈ How to define, configure, and invoke transducible functions; specifying instructions; controlling temperature, retries, and structured decoding. -- **[Agentics](agentics.md)** 🧬 - Defining Pydantic models for inputs/outputs, working with `AG` containers, loading data from JSON/CSV/DataFrames, and preserving type information across the pipeline. +- **[Map-Reduce Operations](map_reduce.md)** πŸ” + Scaling transducible functions with map and reduce operations, batch processing patterns, and hierarchical reduction strategies. + +- **[Agentics (AG)](agentics.md)** 🧬 + Working with `AG` typed state containers, loading data from JSON/CSV/DataFrames, and preserving type information across the pipeline. + +### Advanced Topics + +- **[Performance Optimization](optimization.md)** ⚑ + Batch size tuning, persisting intermediate results, error handling, retries, and performance benchmarking. + +- **[Tool Integration](tool_integration.md)** πŸ”Œ + Using MCP tools, web search, databases, custom tools, and best practices for tool orchestration. -- **[Logical Transduction Algebra](../tutorials/logical_transduction_algebra.ipynb)** πŸ” - Chaining transducible functions, branching, fan-in/fan-out patterns, and building reusable pipeline components. +### Tutorials & Examples -- **[Async Map–Reduce Execution](../tutorials/map_reduce.ipynb)** πŸš€ - Using `amap` and `areduce` for large-scale runs, batching strategies, handling failures, and performance considerations. +- **[Logical Transduction Algebra](../tutorials/logical_transduction_algebra.ipynb)** πŸ” + Interactive tutorial: Chaining transducible functions, branching, fan-in/fan-out patterns, and building reusable pipeline components. +- **[Map-Reduce Tutorial](../tutorials/map_reduce.ipynb)** πŸš€ + Interactive tutorial: Using `amap` and `areduce` for large-scale runs, batching strategies, handling failures, and performance considerations. -- **[Examples & Use Cases](../examples)** πŸ“˜ +- **[Examples & Use Cases](../examples)** πŸ“˜ End-to-end examples: text-to-SQL, data extraction and enrichment, classification, document workflows, evaluation pipelines, and more. @@ -141,4 +154,48 @@ The framework is intentionally small: - 🚫 No custom DSL to learn - 🐍 Just Python functions, Pydantic models, and a few core primitives + +--- + +## πŸ“– Glossary + +**AG (Agentics)** +Short for "Agentics". A typed state container that wraps a list of Pydantic objects, enabling structured transductions. Used as `AG[Type]` or simply `AG(atype=Type)`. The recommended way to work with collections of typed data. + +**Agentics** +The full name of the framework and the class name for typed state containers. In code, typically imported and used as `AG` for brevity. + +**Transducible Function** +A typed, explainable function that maps inputs of type `Source` to outputs of type `Target`. Defined using the `@transducible()` decorator or dynamically with the `<<` operator. Guarantees totality, local evidence, and slot-level provenance. + +**Transduction** +The process of transforming data from one typed structure to another using LLM-powered reasoning. Unlike simple mapping, transduction preserves semantic relationships and provides explainability. + +**Logical Transduction Algebra (LTA)** +The formal mathematical framework underlying Agentics. Treats transductions as morphisms between types, enabling composition, explainability, and stability guarantees. + +**`<<` Operator (Left Shift)** +The transduction operator. `Target << Source` creates a transducible function that maps `Source` to `Target`. Can be used with types, instances, or existing functions for composition. + +**`With()` Function** +A helper that wraps a source type with configuration parameters. Used as `Target << With(Source, instructions="...", tools=[...])` to create configured transducible functions dynamically. + +**TransductionResult** +A wrapper object returned when `provide_explanation=True`. Supports automatic unpacking into `(value, explanation)` tuples or single value assignment. + +**AType** +Short for "Agentics Type". The Pydantic model class that defines the schema for all instances in an AG container. Accessed via `ag.atype`. + +**Map-Reduce** +The execution pattern for scaling transductions. `amap` applies a function to each element in parallel; `areduce` aggregates results into a summary. + +**MCP (Model Context Protocol)** +A standard protocol for exposing tools (web search, databases, APIs) to LLMs. Agentics supports MCP tools via the `tools` parameter. + +**Evidence** +The subset of input fields that contributed to generating a specific output field. Tracked automatically to enable explainability and provenance. + +**Slot** +A field in a Pydantic model. "Slot-level provenance" means tracking which input slots contributed to each output slot. + - πŸŒ‰ Easy to embed into existing stacks (LangFlow nodes, CrewAI agents, MCPs, etc.) diff --git a/docs/map_reduce.md b/docs/map_reduce.md new file mode 100644 index 000000000..9048cc700 --- /dev/null +++ b/docs/map_reduce.md @@ -0,0 +1,381 @@ +# πŸ” Map-Reduce Operations + +Map-Reduce is the execution pattern for scaling transducible functions to large datasets. Agentics provides built-in support for both **map** (parallel transformation) and **reduce** (aggregation) operations over typed collections. + +--- + +## Overview + +When you define a transducible function, it automatically supports both single-item and batch processing: + +```python +from pydantic import BaseModel + +class UserMessage(BaseModel): + content: str + +class Email(BaseModel): + to: str + subject: str + body: str + +@transducible() +async def write_email(message: UserMessage) -> Email: + """Convert a message into a professional email.""" + return Transduce(message) + +# Single item +email = await write_email(UserMessage(content="Hi John, great progress!")) + +# Batch processing (automatic map) +messages = [ + UserMessage(content="Hi John, I made great progress with Agentics."), + UserMessage(content="Hi, I fixed the last blocking bug in the pipeline."), +] +emails = await write_email(messages) # Returns list[Email] +``` + +--- + +## The Map Operation + +The **map** operation applies a transducible function to each element independently, enabling concurrency and parallelism. + +### How Map Works + +```python +# Conceptually: +# amap(write_email, messages) -> list[Email] + +# Each element is processed independently +# Results maintain the same order as inputs +``` + +### Map Characteristics + +| Aspect | Description | +|--------|-------------| +| **Input** | Single item or list of items | +| **Output** | List of transformed items (one per input) | +| **Operation** | Independent transformation of each element | +| **Parallelization** | Fully parallel - elements processed concurrently | +| **Use Cases** | Enrichment, extraction, classification, normalization | + +### Map Examples + +**Example 1: Data Enrichment** + +```python +class Product(BaseModel): + name: str + category: str + +class EnrichedProduct(BaseModel): + name: str + category: str + description: str + keywords: list[str] + +@transducible() +async def enrich_product(product: Product) -> EnrichedProduct: + """Add description and keywords to product.""" + return Transduce(product) + +# Process entire catalog +products = load_products() # list[Product] +enriched = await enrich_product(products) # Parallel processing +``` + +**Example 2: Text Classification** + +```python +class Document(BaseModel): + text: str + +class ClassifiedDocument(BaseModel): + text: str + category: str + confidence: float + tags: list[str] + +@transducible(batch_size=20) +async def classify_document(doc: Document) -> ClassifiedDocument: + """Classify document into categories.""" + return Transduce(doc) + +documents = load_documents(1000) +classified = await classify_document(documents) # Processes in batches of 20 +``` + +--- + +## The Reduce Operation + +The **reduce** operation aggregates a collection of items into a single summary or consolidated result. + +### Using `transduction_type="areduce"` + +Specify the transduction type to create a reduce operation: + +```python +from typing import List + +class Review(BaseModel): + text: str + rating: int + +class ReviewSummary(BaseModel): + overall_sentiment: str + average_rating: float + key_themes: List[str] + total_reviews: int + +@transducible(transduction_type="areduce") +async def summarize_reviews(reviews: List[Review]) -> ReviewSummary: + """Aggregate multiple reviews into a single summary.""" + return Transduce(reviews) + +# Use it +reviews = [ + Review(text="Great product!", rating=5), + Review(text="Good value for money", rating=4), + Review(text="Not bad, could be better", rating=3), +] + +summary = await summarize_reviews(reviews) +print(f"Overall: {summary.overall_sentiment}") +print(f"Average: {summary.average_rating}") +``` + +### Reduce Characteristics + +| Aspect | Description | +|--------|-------------| +| **Input** | List of items | +| **Output** | Single aggregated result | +| **Operation** | Aggregation across all elements | +| **Parallelization** | Sequential or hierarchical | +| **Use Cases** | Summarization, statistics, consolidation, consensus | + +### Common Reduce Patterns + +**Pattern 1: Summarization** + +```python +class Document(BaseModel): + title: str + content: str + +class ExecutiveSummary(BaseModel): + main_points: List[str] + conclusion: str + word_count: int + +@transducible(transduction_type="areduce") +async def create_executive_summary(docs: List[Document]) -> ExecutiveSummary: + """Summarize multiple documents into key insights.""" + return Transduce(docs) +``` + +**Pattern 2: Statistical Aggregation** + +```python +class DataPoint(BaseModel): + value: float + category: str + timestamp: str + +class Statistics(BaseModel): + mean: float + median: float + categories: List[str] + trend: str # "increasing", "decreasing", "stable" + +@transducible(transduction_type="areduce") +async def analyze_data(points: List[DataPoint]) -> Statistics: + """Compute statistics and identify trends.""" + return Transduce(points) +``` + +**Pattern 3: Consensus Building** + +```python +class Opinion(BaseModel): + author: str + stance: str + reasoning: str + +class Consensus(BaseModel): + majority_view: str + key_arguments: List[str] + dissenting_views: List[str] + confidence: float + +@transducible(transduction_type="areduce") +async def build_consensus(opinions: List[Opinion]) -> Consensus: + """Find consensus across multiple opinions.""" + return Transduce(opinions) +``` + +--- + +## Dynamic Map-Reduce with `<<` Operator + +Create map and reduce operations on the fly: + +### Dynamic Map + +```python +# Create a map function dynamically +enrich = EnrichedProduct << Product + +products = [Product(name="Widget", category="Tools"), ...] +enriched = await enrich(products) # Automatic map +``` + +### Dynamic Reduce + +```python +from agentics import With + +# Create a reduce function on the fly +summarize = ReviewSummary << With( + List[Review], + transduction_type="areduce", + instructions="Analyze all reviews and provide comprehensive summary" +) + +summary = await summarize(reviews) +``` + +--- + +## Combining Map and Reduce + +Build complete Map-Reduce pipelines by chaining operations: + +```python +# Step 1: Map - Extract insights from each document +class Document(BaseModel): + text: str + +class Insight(BaseModel): + key_point: str + importance: int + +@transducible() +async def extract_insight(doc: Document) -> Insight: + """Extract key insight from a document.""" + return Transduce(doc) + +# Step 2: Reduce - Consolidate all insights +class Report(BaseModel): + top_insights: List[str] + overall_theme: str + +@transducible(transduction_type="areduce") +async def consolidate_insights(insights: List[Insight]) -> Report: + """Consolidate insights into a final report.""" + return Transduce(insights) + +# Execute the pipeline +documents = [Document(text="..."), Document(text="..."), ...] +insights = await extract_insight(documents) # Map phase +report = await consolidate_insights(insights) # Reduce phase +``` + +### Multi-Stage Pipeline Example + +```python +# Stage 1: Map - Clean and normalize +@transducible() +async def clean_data(raw: RawData) -> CleanData: + return Transduce(raw) + +# Stage 2: Map - Extract features +@transducible() +async def extract_features(clean: CleanData) -> Features: + return Transduce(clean) + +# Stage 3: Reduce - Aggregate statistics +@transducible(transduction_type="areduce") +async def compute_stats(features: List[Features]) -> Statistics: + return Transduce(features) + +# Execute pipeline +raw_data = load_raw_data() +clean = await clean_data(raw_data) +features = await extract_features(clean) +stats = await compute_stats(features) +``` + +--- + + +## Best Practices + +### For Map Operations + +1. **Use appropriate batch sizes** - Balance throughput and memory (see [Optimization](optimization.md)) +2. **Handle failures gracefully** - Individual items can fail without stopping the batch +3. **Monitor progress** - Use `verbose_transduction=True` for long-running operations +4. **Consider rate limits** - Adjust batch size for API rate limits + +### For Reduce Operations + +1. **Keep reduce operations focused** - Each reduce should have a clear aggregation goal +2. **Handle empty lists** - Consider what happens when the input list is empty +3. **Use hierarchical reduction** - For very large collections, reduce in stages +4. **Provide clear instructions** - Help the LLM understand the aggregation logic +5. **Consider token limits** - Large collections may exceed context windows +6. **Test with representative data** - Ensure reduce logic works across different input sizes + +### General Best Practices + +```python +# Good: Clear separation of concerns +@transducible() +async def extract(item: Raw) -> Processed: + """Map: Extract and normalize.""" + return Transduce(item) + +@transducible(transduction_type="areduce") +async def summarize(items: List[Processed]) -> Summary: + """Reduce: Aggregate results.""" + return Transduce(items) + +# Execute +processed = await extract(raw_items) +summary = await summarize(processed) +``` + +--- + +## Performance Considerations + +### Batch Size Tuning + +```python +# Small batches for complex operations +@transducible(batch_size=5) +async def complex_analysis(item: Data) -> Analysis: + return Transduce(item) + +# Large batches for simple operations +@transducible(batch_size=30) +async def simple_extraction(item: Data) -> Extract: + return Transduce(item) +``` + +### Parallel Execution + +Map operations are automatically parallelized based on `batch_size`. For more control, see [Optimization](optimization.md). + +--- + +## See Also + +- πŸ‘‰ [Transducible Functions](transducible_functions.md) - Core concepts and basic usage +- πŸ‘‰ [Performance Optimization](optimization.md) - Batch processing and performance tuning +- πŸ‘‰ [Map-Reduce Tutorial](../tutorials/map_reduce.ipynb) - Interactive examples +- πŸ‘‰ [Core Concepts](core_concepts.md) - Understanding the theoretical foundation diff --git a/docs/optimization.md b/docs/optimization.md new file mode 100644 index 000000000..a84726020 --- /dev/null +++ b/docs/optimization.md @@ -0,0 +1,323 @@ +# ⚑ Performance Optimization + +Efficient batch processing and performance optimization are crucial for large-scale transductions. This guide covers strategies to maximize throughput, manage resources, and handle large datasets effectively. + +--- + +## Understanding Batch Size + +The `batch_size` parameter controls how many items are processed concurrently. Choosing the right batch size is critical for balancing throughput, memory usage, and reliability. + +```python +# Small batches - lower memory, more overhead +@transducible(batch_size=5) +async def conservative_process(state: Item) -> Result: + return Transduce(state) + +# Large batches - higher throughput, more memory +@transducible(batch_size=25) +async def aggressive_process(state: Item) -> Result: + return Transduce(state) +``` + +### Choosing the Right Batch Size + +| Scenario | Recommended Batch Size | Reason | +|----------|----------------------|---------| +| Simple transformations (< 1s each) | 20-30 | Maximize throughput | +| Complex reasoning (> 5s each) | 5-10 | Avoid timeout issues | +| Large input/output objects | 10-15 | Manage memory usage | +| Rate-limited APIs | 5-15 | Stay within limits | +| Local LLM (Ollama) | 1-5 | Limited by GPU memory | + +--- + +## Persisting Intermediate Results + +Use `persist_output` to save results incrementally, enabling recovery from failures: + +```python +@transducible( + batch_size=20, + persist_output="./output/processed_batches" +) +async def process_large_dataset(state: DataItem) -> ProcessedItem: + """Results saved after each batch completes.""" + return Transduce(state) + +# Process 10,000 items +large_dataset = load_items(10000) +results = await process_large_dataset(large_dataset) + +# If interrupted, previously completed batches are saved +# Resume by loading saved batches and processing remaining items +``` + +### File Structure + +``` +output/processed_batches/ +β”œβ”€β”€ batch_0000.jsonl # First 20 items +β”œβ”€β”€ batch_0001.jsonl # Next 20 items +β”œβ”€β”€ batch_0002.jsonl # And so on... +└── ... +``` + +--- + +## Monitoring Progress + +Enable verbose logging to track batch processing: + +```python +@transducible( + batch_size=25, + verbose_transduction=True, # Show progress + verbose_agent=False # Hide detailed agent logs +) +async def monitored_process(state: Item) -> Result: + return Transduce(state) + +# Output shows: +# Processing batch 1/40 (25 items)... +# Processing batch 2/40 (25 items)... +# ... +``` + +--- + +## Performance Optimization Strategies + +### 1. Adaptive Batch Sizing + +Tune batch size based on item complexity: + +```python +# Adaptive batching based on input size +def get_batch_size(items): + avg_size = sum(len(str(item)) for item in items) / len(items) + if avg_size < 500: + return 25 # Small items + elif avg_size < 2000: + return 15 # Medium items + else: + return 5 # Large items + +batch_size = get_batch_size(dataset) +process_fn = Result << With(Item, batch_size=batch_size) +``` + +### 2. Field-Specific Transduction + +Only transduce the fields you need: + +```python +@transducible( + transduce_fields=["summary", "category"], # Only these fields + batch_size=30 +) +async def focused_transform(state: FullData) -> PartialResult: + """Faster by ignoring unnecessary fields.""" + return Transduce(state) +``` + +### 3. Reduce Token Usage with Prompt Templates + +```python +# Custom template to reduce token count +compact_template = """ +Input: {input_data} +Task: {instructions} +Output format: {output_schema} +""" + +@transducible( + prompt_template=compact_template, + batch_size=40 +) +async def efficient_transform(state: Item) -> Result: + return Transduce(state) +``` + +### 4. Parallel Processing with Multiple Workers + +For extremely large datasets, consider splitting work across multiple processes: + +```python +import asyncio +from concurrent.futures import ProcessPoolExecutor + +async def process_chunk(chunk, process_fn): + """Process a chunk of data.""" + return await process_fn(chunk) + +async def parallel_process(dataset, process_fn, num_workers=4): + """Split dataset across multiple workers.""" + chunk_size = len(dataset) // num_workers + chunks = [dataset[i:i+chunk_size] for i in range(0, len(dataset), chunk_size)] + + tasks = [process_chunk(chunk, process_fn) for chunk in chunks] + results = await asyncio.gather(*tasks) + + # Flatten results + return [item for chunk_result in results for item in chunk_result] +``` + +--- + +## Performance Benchmarking + +Measure throughput for your specific use case: + +```python +import time + +async def benchmark_transduction(): + test_items = generate_test_data(100) + + start = time.time() + results = await process_fn(test_items) + elapsed = time.time() - start + + print(f"Processed {len(results)} items in {elapsed:.2f}s") + print(f"Throughput: {len(results)/elapsed:.2f} items/sec") + print(f"Average time per item: {elapsed/len(results):.2f}s") + +await benchmark_transduction() +``` + +### Profiling Memory Usage + +```python +import tracemalloc + +async def profile_memory(): + tracemalloc.start() + + # Your transduction + results = await process_fn(large_dataset) + + current, peak = tracemalloc.get_traced_memory() + print(f"Current memory: {current / 1024 / 1024:.2f} MB") + print(f"Peak memory: {peak / 1024 / 1024:.2f} MB") + + tracemalloc.stop() +``` + +--- + +## Error Handling & Retries + +### Automatic Retries + +Configure retry behavior for transient failures: + +```python +@transducible( + max_retries=3, # Retry up to 3 times + retry_delay=2.0, # Wait 2 seconds between retries + batch_size=20 +) +async def resilient_process(state: Item) -> Result: + return Transduce(state) +``` + +### Graceful Degradation + +Use optional fields to handle partial failures: + +```python +class RobustResult(BaseModel): + required_field: str + optional_field: Optional[str] = None # May be None if extraction fails + confidence: Optional[float] = None + +@transducible( + batch_size=25, + allow_partial=True # Continue even if some fields fail +) +async def robust_transform(state: Item) -> RobustResult: + return Transduce(state) +``` + +### Batch-Level Error Handling + +```python +async def process_with_error_handling(items): + results = [] + failed = [] + + for batch in chunk_items(items, batch_size=20): + try: + batch_results = await process_fn(batch) + results.extend(batch_results) + except Exception as e: + print(f"Batch failed: {e}") + failed.extend(batch) + + # Retry failed items with smaller batch size + if failed: + print(f"Retrying {len(failed)} failed items...") + retry_fn = Result << With(Item, batch_size=5) + retry_results = await retry_fn(failed) + results.extend(retry_results) + + return results +``` + +--- + +## Best Practices + +1. **Start with conservative batch sizes** - Increase gradually based on benchmarks +2. **Monitor memory usage** - Especially with large input/output objects +3. **Use persist_output for long-running jobs** - Protect against interruptions +4. **Profile before optimizing** - Measure to identify actual bottlenecks +5. **Consider API rate limits** - Adjust batch size and concurrency accordingly +6. **Test with representative data** - Performance varies with input complexity +7. **Use field-specific transduction** - Only process what you need +8. **Enable progress monitoring** - Track long-running operations + +--- + +## Common Performance Issues + +### Issue: High Memory Usage + +**Symptoms:** Process crashes or slows down with large datasets + +**Solutions:** +- Reduce batch size +- Use field-specific transduction +- Process in chunks with persistence +- Stream results instead of loading all at once + +### Issue: Slow Throughput + +**Symptoms:** Processing takes much longer than expected + +**Solutions:** +- Increase batch size (if memory allows) +- Reduce prompt complexity +- Use faster LLM models +- Optimize prompt templates +- Consider parallel processing + +### Issue: Frequent Timeouts + +**Symptoms:** Many requests timeout or fail + +**Solutions:** +- Reduce batch size +- Increase timeout value +- Simplify the transduction task +- Use faster models +- Check network connectivity + +--- + +## See Also + +- πŸ‘‰ [Transducible Functions](transducible_functions.md) - Core concepts and basic usage +- πŸ‘‰ [Tool Integration](tool_integration.md) - Using external tools +- πŸ‘‰ [Map-Reduce Tutorial](../tutorials/map_reduce.ipynb) - Large-scale execution patterns diff --git a/docs/tool_integration.md b/docs/tool_integration.md new file mode 100644 index 000000000..8797aed15 --- /dev/null +++ b/docs/tool_integration.md @@ -0,0 +1,402 @@ +# πŸ”Œ Tool Integration + +Transducible functions can use external tools to enhance their capabilities through the Model Context Protocol (MCP). This enables LLM-powered workflows to access real-time data, execute code, query databases, and interact with external services. + +--- + +## What Are Tools? + +Tools extend transducible functions with external capabilities: + +- **Web search** - Retrieve real-time information from the internet +- **Database queries** - Access structured data from SQL/NoSQL databases +- **API calls** - Integrate with external services and APIs +- **Code execution** - Run computations and scripts +- **File operations** - Read/write files and process documents +- **Custom functions** - Any Python function you define + +--- + +## Using MCP Tools + +MCP (Model Context Protocol) provides a standard way to expose tools to LLMs, making them discoverable and callable during transductions. + +### Defining an MCP Server + +First, create an MCP server with your tools (see `examples/mcp_server_example.py`): + +```python +from ddgs import DDGS +from mcp.server.fastmcp import FastMCP + +mcp = FastMCP("Search") + +@mcp.tool() +def web_search(query: str, max_results: int = 5) -> list[str]: + """Search the web using DuckDuckGo. + + Args: + query: Search query with optional operators + max_results: Number of results to return (5-20) + + Returns: + List of search result snippets with titles and URLs + """ + results = DDGS().text(query, max_results=max_results) + return [f"{r['title']}\n{r['body']}\n{r['href']}" for r in results] + +if __name__ == "__main__": + mcp.run(transport="stdio") +``` + +### Using Tools in Transductions + +There are several ways to import and use MCP tools in your transductions: + +#### Option 1: Connect to Remote/External MCP Server + +Use `MCPServerAdapter` from `crewai_tools` to connect to a remote or external MCP server (e.g., a server provided by a third party or running on another machine): + +```python +import os +from pydantic import BaseModel +from typing import Optional +from mcp import StdioServerParameters +from crewai_tools import MCPServerAdapter +from agentics.core.transducible_functions import transducible, Transduce + +class ResearchQuery(BaseModel): + topic: Optional[str] = None + focus_area: Optional[str] = None + +class ResearchReport(BaseModel): + summary: Optional[str] = None + key_findings: Optional[list[str]] = None + sources: Optional[list[str]] = None + +# Configure connection to remote MCP server +# The server could be: +# - A third-party MCP server (e.g., from a service provider) +# - An MCP server running on another machine +# - A pre-existing MCP server script +server_params = StdioServerParameters( + command="python3", + args=["path/to/remote/mcp_server.py"], # Path to the MCP server + env={"UV_PYTHON": "3.12", **os.environ}, +) + +# Connect to the remote MCP server +with MCPServerAdapter(server_params) as server_tools: + print(f"Available tools from remote server: {[tool.name for tool in server_tools]}") + + @transducible( + tools=server_tools, # Use tools from remote MCP server + reasoning=True, + max_iter=5 + ) + async def research_topic(state: ResearchQuery) -> ResearchReport: + """Research a topic using tools from remote MCP server.""" + return Transduce(state) + + # Execute + query = ResearchQuery( + topic="Agentics framework", + focus_area="practical applications" + ) + report = await research_topic(query) +``` + +**Example: Connecting to a Third-Party MCP Server** + +```python +# Example: Connect to a hypothetical weather MCP server +weather_server_params = StdioServerParameters( + command="npx", # MCP servers can be in any language + args=["-y", "@weather/mcp-server"], # npm package +) + +with MCPServerAdapter(weather_server_params) as weather_tools: + print(f"Weather tools: {[tool.name for tool in weather_tools]}") + + @transducible(tools=weather_tools, reasoning=True) + async def get_weather_report(location: Location) -> WeatherReport: + return Transduce(location) +``` + +**Note:** The `MCPServerAdapter` handles the connection lifecycle - it starts the server process when entering the context and stops it when exiting. You don't manage the server manually. + +#### Option 2: Use CrewAI Tools + +Agentics supports CrewAI tools, which provide a standardized interface for tool integration: + +```python +from pydantic import BaseModel +from typing import Optional +from agentics.core.transducible_functions import transducible, Transduce +from crewai_tools import tool + +class ResearchQuery(BaseModel): + topic: Optional[str] = None + focus_area: Optional[str] = None + +class ResearchReport(BaseModel): + summary: Optional[str] = None + key_findings: Optional[list[str]] = None + sources: Optional[list[str]] = None + +# Define CrewAI tool +@tool("Web Search Tool") +def web_search(query: str, max_results: int = 5) -> list[str]: + """Search the web using DuckDuckGo. + + Args: + query: Search query + max_results: Number of results to return + + Returns: + List of search results with title, snippet, and URL + """ + from ddgs import DDGS + results = DDGS().text(query, max_results=max_results) + return [f"{r['title']}\n{r['body']}\n{r['href']}" for r in results] + +@transducible( + tools=[web_search], # Pass CrewAI tool + reasoning=True, + max_iter=5 +) +async def research_topic(state: ResearchQuery) -> ResearchReport: + """Research a topic using web search and synthesize findings.""" + return Transduce(state) + +# Use it +query = ResearchQuery( + topic="Agentics framework", + focus_area="practical applications" +) +report = await research_topic(query) +``` + +You can also use pre-built CrewAI tools: + +```python +from crewai_tools import SerperDevTool, WebsiteSearchTool + +# Use existing CrewAI tools +search_tool = SerperDevTool() +website_tool = WebsiteSearchTool() + +@transducible( + tools=[search_tool, website_tool], + reasoning=True, + max_iter=5 +) +async def research_with_multiple_tools(state: ResearchQuery) -> ResearchReport: + """Research using multiple CrewAI tools.""" + return Transduce(state) +``` + +#### Option 3: Launch MCP Server as Subprocess + +You can launch the MCP server as a separate subprocess and connect to it. This approach starts the server automatically when needed: + +```python +from mcp import ClientSession, StdioServerParameters +from mcp.client.stdio import stdio_client + +# Launch MCP server as subprocess +# The server_params specify how to start the server process +server_params = StdioServerParameters( + command="python", # Command to run + args=["mcp_server.py"], # Server script to execute +) + +# stdio_client launches the server and manages the connection +async with stdio_client(server_params) as (read, write): + async with ClientSession(read, write) as session: + # Initialize connection and discover available tools + await session.initialize() + tools = await session.list_tools() + + # Use tools in transduction + @transducible( + tools=tools, + reasoning=True, + max_iter=5 + ) + async def research_topic(state: ResearchQuery) -> ResearchReport: + return Transduce(state) + + # Execute + query = ResearchQuery(topic="Agentics framework") + report = await research_topic(query) + +# Server process is automatically terminated when exiting the context +``` + +**When to use each option:** +- **Option 1** (Remote MCP Server): Best for connecting to third-party or external MCP servers; use when you want to leverage existing MCP services or servers running elsewhere +- **Option 2** (CrewAI tools): Best for leveraging the CrewAI ecosystem and pre-built tools; simplest for getting started with existing tools +- **Option 3** (MCP subprocess with ClientSession): Lower-level approach for advanced use cases; gives more control over server lifecycle and communication; use for custom server management + +--- + +## Tool Usage Patterns + +### Pattern 1: Information Retrieval + +Enrich data with external sources: + +```python +@transducible(tools=[web_search, database_query]) +async def enrich_data(state: BasicInfo) -> EnrichedInfo: + """Enrich basic info with external data sources.""" + return Transduce(state) +``` + +### Pattern 2: Verification + +Verify claims against external sources: + +```python +@transducible(tools=[fact_checker, web_search]) +async def verify_claims(state: Claims) -> VerifiedClaims: + """Verify claims against external sources.""" + return Transduce(state) +``` + +### Pattern 3: Multi-Step Reasoning + +Solve complex problems requiring multiple tools: + +```python +@transducible( + tools=[web_search, calculator, code_executor], + reasoning=True, + max_iter=10, + verbose_agent=True # See tool calls +) +async def solve_complex_problem(state: Problem) -> Solution: + """Solve problems requiring multiple tools and reasoning steps.""" + return Transduce(state) +``` + +--- + +## Creating Custom Tools + +Define your own tools following the MCP pattern: + +```python +from mcp.server.fastmcp import FastMCP + +mcp = FastMCP("CustomTools") + +@mcp.tool() +def calculate_metrics(data: dict) -> dict: + """Calculate statistical metrics from data. + + Args: + data: Dictionary with numeric values + + Returns: + Dictionary with mean, median, std, etc. + """ + import statistics + values = list(data.values()) + return { + "mean": statistics.mean(values), + "median": statistics.median(values), + "stdev": statistics.stdev(values) if len(values) > 1 else 0 + } + +@mcp.tool() +def fetch_database_record(record_id: str) -> dict: + """Fetch a record from the database. + + Args: + record_id: Unique identifier for the record + + Returns: + Record data as dictionary + """ + # Your database logic here + return db.get(record_id) +``` + +--- + +## Tool Configuration + +Control how tools are used in your transductions: + +### Limiting Tool Usage + +```python +# Limit tool usage +@transducible( + tools=[expensive_api_tool], + max_iter=3, # Maximum 3 tool calls + timeout=120 # 2 minute timeout including tool calls +) +async def controlled_tool_usage(state: Input) -> Output: + return Transduce(state) +``` + +### Verbose Tool Logging + +```python +# Verbose tool logging +@transducible( + tools=[web_search], + verbose_agent=True, # Log each tool call + provide_explanation=True # Include tool usage in explanation +) +async def logged_tool_usage(state: Input) -> Output: + return Transduce(state) + +result, explanation = await logged_tool_usage(input_data) +print(f"Tools used: {explanation.tools_called}") +``` + +--- + +## Best Practices + +1. **Provide clear tool descriptions** - Help the LLM understand when to use each tool +2. **Limit tool iterations** - Prevent infinite loops with `max_iter` +3. **Handle tool failures gracefully** - Tools may timeout or return errors +4. **Use reasoning mode** - Enable `reasoning=True` for complex tool orchestration +5. **Monitor tool usage** - Use `verbose_agent=True` during development +6. **Cache tool results** - Avoid redundant API calls for the same queries + +### Example: Robust Tool Usage + +```python +@transducible( + tools=[web_search, database_query], + reasoning=True, + max_iter=5, + timeout=300, + verbose_agent=True, + provide_explanation=True +) +async def robust_research(state: Query) -> Report: + """Research with fallback strategies if tools fail.""" + return Transduce(state) + +try: + report, explanation = await robust_research(query) + print(f"Used tools: {explanation.tools_called}") + print(f"Tool call count: {len(explanation.tool_calls)}") +except TimeoutError: + print("Research timed out - try simpler query or increase timeout") +``` + +--- + +## See Also + +- πŸ‘‰ [Transducible Functions](transducible_functions.md) - Core concepts and basic usage +- πŸ‘‰ [Optimization](optimization.md) - Performance tuning and batch processing +- πŸ‘‰ [Examples](../examples/mcp_server_example.py) - Complete MCP server example diff --git a/docs/transducible_functions.md b/docs/transducible_functions.md index 08340a8c6..e007a41fd 100644 --- a/docs/transducible_functions.md +++ b/docs/transducible_functions.md @@ -14,15 +14,12 @@ This document explains what transducible functions are, how they work in Agentic Formally, a **transducible function** \(T : X \to Y\) is an *explainable* function that satisfies: -1. **Totality** - For every valid input \(x \in \llbracket X \rrbracket\), the function produces a valid output of type \(Y\). - > No silent failures: the function always returns some well-typed `Y`. -2. **Local Evidence** +1. **Local Evidence** Each output slot \(y_i\) is computed only from its *evidence subset* \(\mathcal{E}_i(x)\). > No field is generated β€œfrom nowhere”: if `subject` appears in the output, we know which inputs and instructions it depended on. -3. **Slot-Level Provenance** +2. **Slot-Level Provenance** The mapping between input and output slots is explicit: \[ \mathcal{T}(y_i) = \mathcal{E}_i @@ -38,7 +35,7 @@ Transducible functions extend normal functions with **structural transparency at --- -## 2. Source and Target Types (X and Y) πŸ“ +## 2. Source and Target Types πŸ“ Agentics uses **Pydantic models** to represent the input type `X` and the output type `Y`. @@ -393,29 +390,177 @@ This pattern generalizes: - `With(..., provide_explanation=True)` can be used with other source/target pairs. - Explanations can be logged, inspected, or surfaced in UI as **transparent justification** for the model’s decision. +### 6.5 `With()` Function Reference + +The `With()` function creates a `TransductionConfig` object that wraps a source model with configuration parameters. It's used with the `<<` operator to create configured transducible functions dynamically. + +**Signature:** +```python +def With(model: Type[BaseModel], **kwargs) -> TransductionConfig +``` + +**Parameters:** + +All parameters from the `@transducible()` decorator are supported: + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `instructions` | `str` | `""` | Custom instructions for the LLM on how to perform the transduction | +| `tools` | `list[Any]` | `[]` | List of tools (MCP, CrewAI, or LangChain) available during transduction | +| `enforce_output_type` | `bool` | `False` | If `True`, raises `TypeError` if output doesn't match target type | +| `llm` | `Any` | `AG.get_llm_provider()` | LLM provider to use (OpenAI, WatsonX, Ollama, etc.) | +| `reasoning` | `bool` | `False` | Enable reasoning mode for complex transductions | +| `max_iter` | `int` | `10` | Maximum iterations for agentic reasoning loops | +| `verbose_transduction` | `bool` | `True` | Print detailed transduction logs | +| `verbose_agent` | `bool` | `False` | Print agent-level execution logs | +| `batch_size` | `int` | `10` | Number of items to process in parallel batches | +| `provide_explanation` | `bool` | `False` | Return explanation alongside result (see Section 6.6) | +| `timeout` | `int` | `300` | Timeout in seconds for each transduction | +| `post_processing_function` | `Callable` | `None` | Function to apply to outputs after transduction | +| `persist_output` | `str` | `None` | Path to save intermediate batch results | +| `transduce_fields` | `list[str]` | `None` | Specific fields to use for transduction | +| `prompt_template` | `str` | `None` | Custom prompt template for the LLM | +| `areduce` | `bool` | `False` | Use reduce mode instead of map (for aggregations) | + +**Usage Patterns:** + +```python +# Basic usage with instructions +classify = Genre << With(Movie, instructions="Classify the movie genre") + +# Multiple parameters +enrich = EnrichedData << With( + RawData, + instructions="Enrich with external data", + tools=[web_search_tool], + batch_size=20, + timeout=600, + provide_explanation=True +) + +# Comparison: With() vs @transducible() +# These are equivalent: + +# Using With() +fn1 = TargetType << With(SourceType, instructions="Transform data") + +# Using decorator +@transducible(instructions="Transform data") +async def fn2(state: SourceType) -> TargetType: + return Transduce(state) +``` + +**When to use `With()` vs `@transducible()`:** + +- Use `With()` for **dynamic, one-off transductions** where you don't need a named function +- Use `@transducible()` for **reusable functions** that you'll call multiple times or compose into larger workflows +- `With()` is ideal for **exploratory work** and **inline transformations** +- `@transducible()` is better for **production code** with clear function names and documentation + +--- + +### 6.6 Result Unpacking with `TransductionResult` + +When you set `provide_explanation=True` (either in `@transducible()` or `With()`), the transduction returns a `TransductionResult` object that supports automatic unpacking. + +**The `TransductionResult` Class:** + +```python +class TransductionResult: + def __init__(self, value, explanation): + self.value = value # The actual transduced output + self.explanation = explanation # Explanation of how it was derived + + def __iter__(self): + yield self.value + yield self.explanation +``` + +**Automatic Unpacking Behavior:** + +The framework automatically detects how you assign the result: + +```python +# Single assignment - get only the value +result = await classify_genre(movie) +print(result.genre) # Access the Genre object directly + +# Tuple unpacking - get both value and explanation +genre, explanation = await classify_genre(movie) +print(genre.genre) # The Genre object +print(explanation.reasoning) # The explanation object +``` + +**Example with Decorator:** + +```python +@transducible(provide_explanation=True) +async def classify_genre(state: Movie) -> Genre: + """Classify the genre of the source Movie.""" + return Transduce(state) + +movie = Movie( + movie_name="The Godfather", + description="Crime family drama", + year=1972 +) + +# Get both result and explanation +genre, explanation = await classify_genre(movie) + +print(f"Genre: {genre.genre}") +print(f"Reasoning: {explanation.reasoning}") +print(f"Confidence: {explanation.confidence}") +``` + +**Example with `With()`:** + +```python +classify_genre = Genre << With( + Movie, + provide_explanation=True, + instructions="Classify based on plot and themes" +) + +# Tuple unpacking works the same way +genre, explanation = await classify_genre(movie) +``` + +**Batch Processing with Explanations:** + +When processing lists, each item gets its own explanation: + +```python +movies = [movie1, movie2, movie3] + +# Returns list of values and list of explanations +genres, explanations = await classify_genre(movies) + +for genre, explanation in zip(genres, explanations): + print(f"{genre.genre}: {explanation.reasoning}") +``` + +**Note:** If you don't need explanations, simply omit `provide_explanation=True` and the function returns only the transduced value(s). + --- -## 7. Map–Reduce: Scaling Transducible Functions πŸš€ +--- -When wrapped by `@transducible()` **or** created dynamically with `<<`, transducible functions are overloaded to accept **lists** of `X` as well. When called this way, they return a corresponding list of `Y`: +## 7. Batch Processing with Lists + +Transducible functions automatically support batch processing. When you pass a list of items, they are processed efficiently: ```python messages = [ UserMessage(content="Hi John, I made great progress with Agentics."), - UserMessage(content="Hi , I fixed the last blocking bug in the pipeline."), + UserMessage(content="Hi, I fixed the last blocking bug in the pipeline."), ] +# Automatically processes all messages emails = await write_email_with_llm(messages) ``` -Under the hood, Agentics uses an **asynchronous Map** operation: - -- Conceptually: - `amap(write_email_with_llm, messages) -> list[Email]` -- Each element is processed independently, enabling concurrency and parallelism. -- This pattern scales to **batch inference, dataset scans, and large evidence extraction tasks**. - -Later, you can combine this with **Reduce** operations (e.g., summarizing all emails into a single report), forming full Map–Reduce pipelines over typed states. +For detailed information on Map-Reduce operations, scaling to large datasets, and aggregation patterns, see the dedicated [Map-Reduce](map_reduce.md) documentation. --- @@ -423,8 +568,7 @@ Later, you can combine this with **Reduce** operations (e.g., summarizing all em Because transducible functions are defined over explicit types and carry evidence subsets, Agentics can: -- Track which input fields contributed to each output field. -- Represent this as a **bipartite graph** between input and output slots. +- Track which input fields contributed to the output. - Attach this trace as **metadata** to your states (depending on your Agentics configuration). For example, in the email examples: @@ -435,25 +579,24 @@ For example, in the email examples: This is critical when you: -- Need **auditable** LLM behavior. -- Want to debug why a particular field was generated. -- Need to enforce *β€œno hallucination from outside these inputs”* policies. +- Need **auditable** LLM behavior. +- Want to debug why a particular field was generated. +- Need to enforce *"no hallucination from outside these inputs"* policies. --- ## 9. When to Create a New Transducible Function -In a real system, you’ll typically end up with many small, focused transducible functions instead of one giant one. +In a real system, you'll typically end up with many small, focused transducible functions instead of one giant one. Good reasons to define a separate transducible function: -- You’re doing a logically distinct step: +- You're doing a logically distinct step: - e.g., *extract entities*, *normalize names*, *classify intent*, *summarize conversation*. - You want to **test** and **benchmark** that step independently. - You expect to **reuse** it across pipelines. - You need different **instructions, constraints, or safety properties** for that stage. -Think of transducible functions as the **operators** of your Logical Transduction Algebra. --- @@ -473,8 +616,16 @@ Think of transducible functions as the **operators** of your Logical Transductio - Expose structured explainability traces for each output field. - Compose into robust, interpretable, large-scale reasoning pipelines. -From here you can explore: +--- + +## 11. Next Steps + +For advanced topics, see: -- `core_concepts.md` – the broader mental model (types, states, LTA, Map–Reduce). -- `mapreduce.md` – how Agentics orchestrates large-scale transductions over typed state containers. -- `types_and_states.md` – how to design good schemas and manage collections of states. +- πŸ‘‰ **[Map-Reduce Operations](map_reduce.md)** - Scaling with map and reduce, batch processing patterns +- πŸ‘‰ **[Performance Optimization](optimization.md)** - Batch processing, performance tuning, error handling, and retries +- πŸ‘‰ **[Tool Integration](tool_integration.md)** - Using MCP tools, web search, databases, and custom tools +- πŸ‘‰ **[Map-Reduce Tutorial](../tutorials/map_reduce.ipynb)** - Interactive examples and patterns +- πŸ‘‰ **[Core Concepts](core_concepts.md)** - Understanding the theoretical foundation + +--- diff --git a/examples/hello_world.py b/examples/hello_world.py index 29bd03387..3958926a8 100644 --- a/examples/hello_world.py +++ b/examples/hello_world.py @@ -3,8 +3,7 @@ from pydantic import BaseModel, Field -from agentics import AG -from agentics.core.transducible_functions import Transduce, With, transducible +import agentics.core.transducible_functions class Movie(BaseModel): @@ -17,20 +16,18 @@ class Genre(BaseModel): genre: Optional[str] = Field(None, description="Provide one category only") -movie = Movie( - movie_name="The Godfather", - description="The aging patriarch of an organized crime dynasty transfers control of his clandestine empire to his reluctant son.", - year=1972, -) - - -## Using Transducible Decorator -@transducible(provide_explanation=True) -async def classify_genre(state: Movie) -> Genre: - """Classify the genre of the source Movie""" - return Transduce(state) - - -genre, explanation = asyncio.run(classify_genre(movie)) -print(genre.model_dump_json(indent=2)) -print(explanation.model_dump_json(indent=2)) +# Create movie instance +movies = [ + Movie( + movie_name="The Godfather", + description="The aging patriarch of an organized crime dynasty transfers control of his clandestine empire to his reluctant son.", + year=1972, + ), + Movie(movie_name="The Shawshank Redemption"), +] +# Create transduction function using << operator +classify_genre = Genre << Movie + +# Execute transduction +result = asyncio.run(classify_genre(movies)) +print(result) diff --git a/mkdocs.yml b/mkdocs.yml index 4d6365aab..76386733b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -10,12 +10,13 @@ plugins: nav: - πŸ‘‹ Welcome to Agentics: index.md - - Getting Started: getting_started.md - - What is Agentics?: background.md - - Use Cases: use_cases.md - - Agentics: agentics.md - - Transduction: transduction.md - - Memory: memory.md + - πŸš€ Getting Started: getting_started.md + - 🧠 Core Concepts: core_concepts.md + - βš™οΈ Transducible Functions: transducible_functions.md + - πŸ” Map-Reduce Operations: map_reduce.md + - 🧬 Agentics (AG): agentics.md + - ⚑ Performance Optimization: optimization.md + - πŸ”Œ Tool Integration: tool_integration.md markdown_extensions: - pymdownx.tabbed diff --git a/tutorials/data/movies.jsonl b/tutorials/data/movies.jsonl deleted file mode 100644 index 6d157f171..000000000 --- a/tutorials/data/movies.jsonl +++ /dev/null @@ -1,20 +0,0 @@ -{"movie_name":"The Shawshank Redemption","genre":"Drama, Crime","description":"Imprisoned in the 1940s for the double murder of his wife and her lover, upstanding banker Andy Dufresne begins a new life at the Shawshank prison, where he puts his accounting skills to work for an amoral warden. During his long stretch in prison, Dufresne comes to be admired by the other inmates -- including an older prisoner named Red -- for his integrity and unquenchable sense of hope."} -{"movie_name":"The Godfather","genre":"Drama, Crime","description":"Spanning the years 1945 to 1955, a chronicle of the fictional Italian-American Corleone crime family. When organized crime family patriarch, Vito Corleone barely survives an attempt on his life, his youngest son, Michael steps in to take care of the would-be killers, launching a campaign of bloody revenge."} -{"movie_name":"The Godfather Part II","genre":"Drama, Crime","description":"In the continuing saga of the Corleone crime family, a young Vito Corleone grows up in Sicily and in 1910s New York. In the 1950s, Michael Corleone attempts to expand the family business into Las Vegas, Hollywood and Cuba."} -{"movie_name":"Schindler's List","genre":"Drama, History, War","description":"The true story of how businessman Oskar Schindler saved over a thousand Jewish lives from the Nazis while they worked as slaves in his factory during World War II."} -{"movie_name":"12 Angry Men","genre":"Drama","description":"The defense and the prosecution have rested and the jury is filing into the jury room to decide if a young Spanish-American is guilty or innocent of murdering his father. What begins as an open and shut case soon becomes a mini-drama of each of the jurors' prejudices and preconceptions about the trial, the accused, and each other."} -{"movie_name":"Spirited Away","genre":"Animation, Family, Fantasy","description":"A young girl, Chihiro, becomes trapped in a strange new world of spirits. When her parents undergo a mysterious transformation, she must call upon the courage she never knew she had to free her family."} -{"movie_name":"The Dark Knight","genre":"Drama, Action, Crime, Thriller","description":"Batman raises the stakes in his war on crime. With the help of Lt. Jim Gordon and District Attorney Harvey Dent, Batman sets out to dismantle the remaining criminal organizations that plague the streets. The partnership proves to be effective, but they soon find themselves prey to a reign of chaos unleashed by a rising criminal mastermind known to the terrified citizens of Gotham as the Joker."} -{"movie_name":"Dilwale Dulhania Le Jayenge","genre":"Comedy, Drama, Romance","description":"Raj is a rich, carefree, happy-go-lucky second generation NRI. Simran is the daughter of Chaudhary Baldev Singh, who in spite of being an NRI is very strict about adherence to Indian values. Simran has left for India to be married to her childhood fiancΓ©. Raj leaves for India with a mission at his hands, to claim his lady love under the noses of her whole family. Thus begins a saga."} -{"movie_name":"The Green Mile","genre":"Fantasy, Drama, Crime","description":"A supernatural tale set on death row in a Southern prison, where gentle giant John Coffey possesses the mysterious power to heal people's ailments. When the cell block's head guard, Paul Edgecomb, recognizes Coffey's miraculous gift, he tries desperately to help stave off the condemned man's execution."} -{"movie_name":"Parasite","genre":"","description":"All unemployed, Ki-taek's family takes peculiar interest in the wealthy and glamorous Parks for their livelihood until they get entangled in an unexpected incident."} -{"movie_name":"Pulp Fiction","genre":"","description":"A burger-loving hit man, his philosophical partner, a drug-addled gangster's moll and a washed-up boxer converge in this sprawling, comedic crime caper. Their adventures unfurl in three stories that ingeniously trip back and forth in time."} -{"movie_name":"Your Name.","genre":"","description":"High schoolers Mitsuha and Taki are complete strangers living separate lives. But one night, they suddenly switch places. Mitsuha wakes up in Taki’s body, and he in hers. This bizarre occurrence continues to happen randomly, and the two must adjust their lives around each other."} -{"movie_name":"The Lord of the Rings: The Return of the King","genre":"","description":"As armies mass for a final battle that will decide the fate of the world--and powerful, ancient forces of Light and Dark compete to determine the outcome--one member of the Fellowship of the Ring is revealed as the noble heir to the throne of the Kings of Men. Yet, the sole hope for triumph over evil lies with a brave hobbit, Frodo, who, accompanied by his loyal friend Sam and the hideous, wretched Gollum, ventures deep into the very dark heart of Mordor on his seemingly impossible quest to destroy the Ring of Power.​"} -{"movie_name":"Forrest Gump","genre":"","description":"A man with a low IQ has accomplished great things in his life and been present during significant historic eventsβ€”in each case, far exceeding what anyone imagined he could do. But despite all he has achieved, his one true love eludes him."} -{"movie_name":"The Good, the Bad and the Ugly","genre":"","description":"While the Civil War rages on between the Union and the Confederacy, three men – a quiet loner, a ruthless hitman, and a Mexican bandit – comb the American Southwest in search of a strongbox containing $200,000 in stolen gold."} -{"movie_name":"Seven Samurai","genre":"","description":"A samurai answers a village's request for protection after he falls on hard times. The town needs protection from bandits, so the samurai gathers six others to help him teach the people how to defend themselves, and the villagers provide the soldiers with food."} -{"movie_name":"GoodFellas","genre":"","description":"The true story of Henry Hill, a half-Irish, half-Sicilian Brooklyn kid who is adopted by neighbourhood gangsters at an early age and climbs the ranks of a Mafia family under the guidance of Jimmy Conway."} -{"movie_name":"Interstellar","genre":"","description":"The adventures of a group of explorers who make use of a newly discovered wormhole to surpass the limitations on human space travel and conquer the vast distances involved in an interstellar voyage."} -{"movie_name":"Grave of the Fireflies","genre":"","description":"In the final months of World War II, 14-year-old Seita and his sister Setsuko are orphaned when their mother is killed during an air raid in Kobe, Japan. After a falling out with their aunt, they move into an abandoned bomb shelter. With no surviving relatives and their emergency rations depleted, Seita and Setsuko struggle to survive."} -{"movie_name":"Life Is Beautiful","genre":"","description":"A touching story of an Italian book seller of Jewish ancestry who lives in his own little fairy tale. His creative and happy life would come to an abrupt halt when his entire family is deported to a concentration camp during World War II. While locked up he tries to convince his son that the whole thing is just a game."} From d6a7f8501e096b5a685085113c3ecff262b8ad04 Mon Sep 17 00:00:00 2001 From: Junkyu Lee Date: Wed, 11 Feb 2026 21:52:06 -0500 Subject: [PATCH 14/27] reviewed docs and minor edits Signed-off-by: Junkyu Lee --- docs/agentics.md | 11 +-- docs/core_concepts.md | 13 ++-- docs/getting_started.md | 48 ++++-------- docs/index.md | 133 ++++----------------------------- docs/map_reduce.md | 8 +- docs/optimization.md | 1 + docs/tool_integration.md | 1 + docs/transducible_functions.md | 35 +++------ 8 files changed, 58 insertions(+), 192 deletions(-) diff --git a/docs/agentics.md b/docs/agentics.md index 2c622ab0f..0c519ac51 100644 --- a/docs/agentics.md +++ b/docs/agentics.md @@ -114,7 +114,7 @@ print(movies.states[0]) ``` -You can also modify and rebind an exiting Agentic. Similarly can also remove attributes. The following code is equivalent to the code before +You can also modify and rebind an exiting Agentic. Similarly can also remove attributes. The following code is equivalent to the code before. ```python movies = AG.from_csv("data/orders.csv") @@ -148,8 +148,7 @@ async def main(): "What is the best F1 team in history?", ] - answers = await (AG(atype=Answer) \ - << input_questions) + answers = await (AG(atype=Answer) << input_questions) answers.pretty_print() @@ -164,7 +163,5 @@ See the [examples directory](../examples/) for practical demonstrations of AG us - `generate_tweets.py` - Content generation - `emotion_extractor.py` - Text analysis - -## See Next: Transducible Functions - -Wrapping pydantic types into Agentics provides them with the ability to perform transduction, as described in the [Transducible Functions](transducible_functions.md) documentation. +## Go to Index +- πŸ‘‰ [Index](index.md) \ No newline at end of file diff --git a/docs/core_concepts.md b/docs/core_concepts.md index 40e01dcf8..a8d51c555 100644 --- a/docs/core_concepts.md +++ b/docs/core_concepts.md @@ -6,7 +6,7 @@ Agentics is built around a small set of concepts that work together: - **Transducible functions** – LLM-powered, type-safe transformations - **Typed state containers (AGs)** – collections of typed rows/documents - **Logical Transduction Algebra (LTA)** – the formal backbone -- **Map–Reduce** – the execution pattern for large workloads +- **Map–Reduce** – the programming model used to execute large-scale workloads This page gives you the mental model you need before diving into code. @@ -74,7 +74,7 @@ class ReviewSummary(BaseModel): A transducible function might be: ```python -fn: (Review) -> ReviewSummary +fn: Review -> ReviewSummary ``` with instructions like: @@ -105,7 +105,6 @@ Agentics introduces **typed state containers** (called **AG**, short for "Agenti Conceptually, you can think of an `AG[Source]` like a type-aware table: - ```text AG[Review] β”œβ”€ row 0: Review(text="…") @@ -113,7 +112,7 @@ AG[Review] └─ row n: Review(text="…") ``` -Applying a transducible function `(Review) -> ReviewSummary` over an `AG` with atype `Review` conceptually yields an `AG` of type `ReviewSummary`. +Applying a transducible function `Review -> ReviewSummary` over an `AG` with atype `Review` conceptually yields an `AG` of type `ReviewSummary`. Typed state containers give you: @@ -164,7 +163,7 @@ In short: Once you have: -- Typed collections (`AG[Source]`), and +- Typed collections (`AG[Source]`) and - Typed transformations (`Source -> Target`), you need a way to run these at scale. Agentics uses a familiar pattern: **Map–Reduce**. @@ -254,8 +253,6 @@ A typical workflow looks like this: - **Logical Transduction Algebra (LTA)** explains why these transformations compose and remain interpretable. - **Map–Reduce** provides the pattern for scaling these transductions to large datasets. -From here, you can explore: +## Next - πŸ‘‰ [Transducible Functions](transducible_functions.md) for concrete examples of defining and using transducible functions -- πŸ‘‰ [Agentics (AG)](agentics.md) for data modeling patterns and typed state containers -- πŸ‘‰ [Map-Reduce Tutorial](../tutorials/map_reduce.ipynb) to see how large-scale execution works in practice diff --git a/docs/getting_started.md b/docs/getting_started.md index c29e798b5..1eaac5dc7 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -24,7 +24,6 @@ Agentics is a lightweight, Python-native framework for building structured, agen * Install the dependencies ```bash - uv sync # Source the environment (optional, you can skip this and prepend uv run to the later lines) source .venv/bin/activate # bash/zsh 🐚 @@ -32,21 +31,21 @@ Agentics is a lightweight, Python-native framework for building structured, agen ``` -### 🎯 Set Environment Variables +### 🎯 Environment Variables Create a `.env` file in the root directory with your environment variables. See `.env.sample` for an example. -Set Up LLM provider, Chose one of the following: +Set up LLM provider, chose one of the following: #### OpenAI - Obtain API key from [OpenAI](https://platform.openai.com/) - `OPENAI_API_KEY` - Your OpenAI APIKey -- `OPENAI_MODEL_ID` - Your favorute model, default to **openai/gpt-4** +- `OPENAI_MODEL_ID` - Selected model, default to **openai/gpt-4** #### Ollama (local) - Download and install [Ollama](https://ollama.com/) -- Download a Model. You should use a model that support reasoning and fit your GPU. So smaller are preferred. +- Download a model. You should use a model that support reasoning and fit your GPU. So smaller are preferred. ``` ollama pull ollama/deepseek-r1:latest ``` @@ -75,7 +74,7 @@ ollama pull ollama/deepseek-r1:latest ## Test Installation -test hello world example (need to set up llm credentials first) +Test hello world example (need to set up llm credentials first) ```bash python python examples/hello_world.py @@ -128,9 +127,9 @@ print(f"πŸ“± {' '.join(tweet.hashtags)}") πŸ“± #AI #OpenSource #Python #LLM #DevTools ``` -### Alternative: Using the `<<` Operator +### Alternative: Using `<<` Operator -For quick one-off transductions, use the `<<` operator: +For quick one-off transductions, use `<<` operator: ```python from pydantic import BaseModel @@ -211,7 +210,7 @@ The same transducible function works seamlessly for both single items and batche ### Installation details -=== "Poetry" +#### Poetry Install poetry (skip if available) @@ -227,7 +226,7 @@ The same transducible function works seamlessly for both single items and batche source $(poetry env info --path)/bin/activate ``` -=== "Python" +#### Python > Ensure you have Python 3.11+ 🚨. > @@ -243,17 +242,8 @@ The same transducible function works seamlessly for both single items and batche * Activate the virtual environment - ### Bash/Zsh - - `source .venv/bin/activate` - - ### Fish - - `source .venv/bin/activate.fish` - - ### VSCode - - Press `F1` key and start typing `> Select python` and select `Select Python Interpreter` + `source .venv/bin/activate` # Bash/Zsh + `source .venv/bin/activate.fish` # Fish * Install the package ```bash @@ -261,7 +251,7 @@ The same transducible function works seamlessly for both single items and batche ``` -=== "uv" +#### uv * Ensure `uv` is installed. ```bash @@ -273,7 +263,7 @@ The same transducible function works seamlessly for both single items and batche * `uv pip install ./agentics` or `uv add ./agentics` (recommended) -=== "uvx πŸƒπŸ½" +#### uvx πŸƒπŸ½ > This is a way to run agentics temporarily or quick tests @@ -286,7 +276,7 @@ The same transducible function works seamlessly for both single items and batche * uvx --verbose --from ./agentics ipython -=== "Conda" +#### Conda 1. Create a conda environment: ```bash @@ -305,11 +295,5 @@ The same transducible function works seamlessly for both single items and batche pip install ./agentics ``` -## Documentation - -This documentation page is written using Mkdocs. -You can start the server to visualize this interactively. -```bash -mkdocs serve -``` -After started, documentation will be available here [http://127.0.0.1:8000/](http://127.0.0.1:8000/) +## Next +- πŸ‘‰ [Core Concepts](core_concepts.md) - Understanding the theoretical foundation diff --git a/docs/index.md b/docs/index.md index 3ab2f767f..df8e8166a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,6 +1,6 @@ # 🌐 Agentics -Agentics is a lightweight, Python-native framework for building **structured and massively parallel agentic workflows** using Pydantic models and **transducible functions** . +Agentics is a lightweight, Python-native framework for building **structured and massively parallel agentic workflows** using Pydantic models and **transducible functions**. --- @@ -12,13 +12,14 @@ Agentics is a lightweight, Python-native framework for building **structured and Install Agentics, set up your environment, and run your first transducible function over a small dataset. - **[Core Concepts](core_concepts.md)** 🧠 - The mental model: Pydantic types, transducible functions, typed state containers, Logical Transduction Algebra (LTA), and Map–Reduce. + Pydantic types, transducible functions, typed state containers, Logical Transduction Algebra (LTA), and Map–Reduce. - **[Transducible Functions](transducible_functions.md)** βš™οΈ - How to define, configure, and invoke transducible functions; specifying instructions; controlling temperature, retries, and structured decoding. + How to define, configure, and execute transducible functions. + Understanding dynamic generation and composition of transducible functions, batch processing, and provenance of generation. - **[Map-Reduce Operations](map_reduce.md)** πŸ” - Scaling transducible functions with map and reduce operations, batch processing patterns, and hierarchical reduction strategies. + Scaling transducible functions with map and reduce operations, batch processing patterns, and best practices. - **[Agentics (AG)](agentics.md)** 🧬 Working with `AG` typed state containers, loading data from JSON/CSV/DataFrames, and preserving type information across the pipeline. @@ -26,10 +27,10 @@ Agentics is a lightweight, Python-native framework for building **structured and ### Advanced Topics - **[Performance Optimization](optimization.md)** ⚑ - Batch size tuning, persisting intermediate results, error handling, retries, and performance benchmarking. + Batch size tuning, persisting intermediate results, performance optimization strategies, performance benchmarking, error handling, and best practices. - **[Tool Integration](tool_integration.md)** πŸ”Œ - Using MCP tools, web search, databases, custom tools, and best practices for tool orchestration. + Using MCP tools, tool usage patterns, custom tools, and best practices. ### Tutorials & Examples @@ -44,116 +45,6 @@ Agentics is a lightweight, Python-native framework for building **structured and ---- - -## Transducible Functions - -A **transducible function** is an LLM-powered, type-safe transformation between Pydantic models. Agentics lets you: - -- Define these transformations **declaratively** -- Compose them into **pipelines** -- Execute them at scale using an asynchronous **Map–Reduce** execution engine βš™οΈ - -Under the hood, Agentics is grounded in **Logical Transduction Algebra (LTA)**, a logico-mathematical formalism that guarantees: - -- βœ… Composability -- βœ… Explainability -- βœ… Stability of LLM-based transformations - -The result is a way to build agentic systems that are: - -- **Typed** – every step has explicit input/output schemas πŸ“ -- **Composable** – pipelines are built from reusable transducible functions 🧩 -- **Traceable** – outputs carry evidence back to input fields πŸ” -- **Scalable** – async `amap` / `areduce` primitives support large workloads πŸš€ -- **Minimal** – no heavy orchestrators: just types, functions, and data πŸͺΆ - -Agentics code is **simple, predictable, and robust**, and is easy to embed into modern ecosystems (LangFlow, LangChain, CrewAI, MCP, etc.) 🀝. - - - -## πŸ”‘ Key Features - -### βš™οΈ Transducible Functions (Core Abstraction) - -Define LLM-powered transformations as first-class functions: - -- 🧾 Typed input and output via Pydantic models -- πŸ›‘οΈ Automatic schema validation and type-constrained generation -- πŸͺœ Composable into higher-level workflows and chains - ---- - -### 🧱 Typed State Containers - a.k.a. Agentics (AG) - -Wrap data into typed state collections so that every row or document carries a concrete Pydantic type: - -- Safe, batch-level operations βœ… -- Clear semantics over datasets and intermediate states πŸ“Š -- Input/output from DBs, CSV and Json -- Ideal to represent tabular/structured data - ---- - -### πŸš€ Async Map–Reduce Execution - -Run transducible functions over large collections using: - -- ⚑ `amap` for massively parallel application -- πŸ“‰ `areduce` for aggregations and global summaries - -Designed to scale on multi-core or distributed execution backends πŸ–₯️πŸ–₯️πŸ–₯️. - ---- - -### 🧩 Dynamic Type & Function Composition - -Create new workflows on the fly: - -- πŸ”„ Merge or refine types dynamically -- 🧬 Compose transducible functions declaratively -- πŸ”€ Build polymorphic or adaptive pipelines driven by data and instructions - ---- - -### πŸ” Explainable & Traceable Inference - -Each generated attribute can be traced back to: - -- Specific input fields 🧷 -- The specific transducible function or step that produced it 🧠 - -This enables **auditable, debuggable** LLM reasoning across the pipeline. - ---- - -### πŸ›‘οΈ End-to-End Type Safety - -Pydantic models are enforced at every boundary: - -- βœ… Validation on input loading -- βœ… Validation after each transducible function -- βœ… Predictable runtime behavior and clear failure modes - ---- - -### πŸ”Œ Tool Integration - -Agentics is fully compatible with Model Context Protocol (MCP) and expose external tools and knowledge to transducible functions: - -- 🌐 Web / search tools -- πŸ—„οΈ Databases & vector stores -- πŸ’» Code execution backends -- πŸ”— MCP-based tools - ---- - -### ✨ Minimalistic, Pythonic API - -The framework is intentionally small: - -- 🚫 No custom DSL to learn -- 🐍 Just Python functions, Pydantic models, and a few core primitives --- @@ -198,4 +89,12 @@ The subset of input fields that contributed to generating a specific output fiel **Slot** A field in a Pydantic model. "Slot-level provenance" means tracking which input slots contributed to each output slot. -- πŸŒ‰ Easy to embed into existing stacks (LangFlow nodes, CrewAI agents, MCPs, etc.) + +## Documentation + +This documentation page is written using Mkdocs. +You can start the server to visualize this interactively. +```bash +mkdocs serve +``` +After started, documentation will be available here [http://127.0.0.1:8000/](http://127.0.0.1:8000/) \ No newline at end of file diff --git a/docs/map_reduce.md b/docs/map_reduce.md index 9048cc700..e660aff39 100644 --- a/docs/map_reduce.md +++ b/docs/map_reduce.md @@ -373,9 +373,7 @@ Map operations are automatically parallelized based on `batch_size`. For more co --- -## See Also +## Next +- πŸ‘‰ [Agentics (AG)](agentics.md) for data modeling patterns and typed state containers +- πŸ‘‰ [Map-Reduce Tutorial](../tutorials/map_reduce.ipynb) to see how large-scale execution works in practice -- πŸ‘‰ [Transducible Functions](transducible_functions.md) - Core concepts and basic usage -- πŸ‘‰ [Performance Optimization](optimization.md) - Batch processing and performance tuning -- πŸ‘‰ [Map-Reduce Tutorial](../tutorials/map_reduce.ipynb) - Interactive examples -- πŸ‘‰ [Core Concepts](core_concepts.md) - Understanding the theoretical foundation diff --git a/docs/optimization.md b/docs/optimization.md index a84726020..1aae2cca7 100644 --- a/docs/optimization.md +++ b/docs/optimization.md @@ -321,3 +321,4 @@ async def process_with_error_handling(items): - πŸ‘‰ [Transducible Functions](transducible_functions.md) - Core concepts and basic usage - πŸ‘‰ [Tool Integration](tool_integration.md) - Using external tools - πŸ‘‰ [Map-Reduce Tutorial](../tutorials/map_reduce.ipynb) - Large-scale execution patterns +- πŸ‘‰ [Index](index.md) \ No newline at end of file diff --git a/docs/tool_integration.md b/docs/tool_integration.md index 8797aed15..89ceffdc5 100644 --- a/docs/tool_integration.md +++ b/docs/tool_integration.md @@ -400,3 +400,4 @@ except TimeoutError: - πŸ‘‰ [Transducible Functions](transducible_functions.md) - Core concepts and basic usage - πŸ‘‰ [Optimization](optimization.md) - Performance tuning and batch processing - πŸ‘‰ [Examples](../examples/mcp_server_example.py) - Complete MCP server example +- πŸ‘‰ [Index](index.md) \ No newline at end of file diff --git a/docs/transducible_functions.md b/docs/transducible_functions.md index e007a41fd..e6dfcb6f4 100644 --- a/docs/transducible_functions.md +++ b/docs/transducible_functions.md @@ -4,7 +4,7 @@ Transducible functions are the *workhorse* of Agentics. They turn β€œcall this LLM with a prompt” into: > **A typed, explainable transformation** -> `T: X β†’ Y` with guarantees about how each output field was produced. +> `T: X β†’ Y` with explanation about how each output field was produced. This document explains what transducible functions are, how they work in Agentics, and how to use them in practice β€” including **dynamic generation** and **compositional patterns** using the `<<` operator. @@ -12,18 +12,16 @@ This document explains what transducible functions are, how they work in Agentic ## 1. What Is a Transducible Function? -Formally, a **transducible function** \(T : X \to Y\) is an *explainable* function that satisfies: +Formally, a **transducible function** `T: X β†’ Y` is an *explainable* function that satisfies: 1. **Local Evidence** - Each output slot \(y_i\) is computed only from its *evidence subset* \(\mathcal{E}_i(x)\). + Each output slot **yα΅’*** is computed only from its *evidence subset* **Eα΅’(x)**. > No field is generated β€œfrom nowhere”: if `subject` appears in the output, we know which inputs and instructions it depended on. -2. **Slot-Level Provenance** - The mapping between input and output slots is explicit: - \[ - \mathcal{T}(y_i) = \mathcal{E}_i - \] +2. **Slot-Level Provenance** + The mapping between input and output slots is explicit: **T(yα΅’) = Eα΅’** + This induces a bipartite graph between **input slots** and **output slots**, which acts as the *explainability trace* of the transduction. Intuitively: @@ -58,7 +56,7 @@ class Email(BaseModel): > **Recommendation** > In transduction scenarios, it is often useful to declare fields as `Optional[...] = None`. -> This gives the LLM the ability to say *β€œI don’t have enough evidence for this field”* by leaving it `null`, instead of hallucinating content. +> This gives an LLM the ability to say *β€œI don’t have enough evidence for this field”* by leaving it `null`, instead of hallucinating content. The transducible function we will define next will transform exactly **one** `UserMessage` into **one** `Email` (and later, we’ll see how to scale to lists). @@ -76,12 +74,12 @@ They can be defined in two main ways: 1. Using the **`@transducible()` decorator** on an async Python function. 2. **Dynamically generating** them from source and target types (e.g., via builders or the `<<` operator), with instructions and parameters. -This section starts with the decorator pattern and then moves to dynamic generation and composition. --- ## 4. The `@transducible()` Decorator +This section starts with the decorator pattern and then moves to dynamic generation and composition. The decorator turns an ordinary async function into a transducible function. When decorated with `@transducible()`, your function can return either: - A **concrete instance of the target type** `Y` (pure Python logic), or @@ -243,7 +241,7 @@ Usage: ```python input_state = GenericInput( - content="Write a news story on Zoran Mandani winning the election in NYC and send it to Alfio" + content="Write a news story on the winner of Super Bowl in 2025 and send it to Alfio." ) mail = await write_mail(input_state) @@ -272,7 +270,7 @@ class Summary(BaseModel): ```python input_state = GenericInput( - content="Write news story on Zoran Mandani winning the election in NYC and send it to Alfio" + content="Write a news story on the winner of Super Bowl in 2025 and send it to Alfio." ) write_mail = Email << GenericInput # GenericInput β†’ Email @@ -343,7 +341,7 @@ summarize = Summary << With( ) input_state = GenericInput( - content="Zoran Mandani won the election in NYC. Draft a message to the press list." + content="Philadelphia Eagles won Super Bowl 2025. Draft a message to the press list." ) mail = await write_mail(input_state) @@ -618,14 +616,5 @@ Good reasons to define a separate transducible function: --- -## 11. Next Steps - -For advanced topics, see: - +## Next - πŸ‘‰ **[Map-Reduce Operations](map_reduce.md)** - Scaling with map and reduce, batch processing patterns -- πŸ‘‰ **[Performance Optimization](optimization.md)** - Batch processing, performance tuning, error handling, and retries -- πŸ‘‰ **[Tool Integration](tool_integration.md)** - Using MCP tools, web search, databases, and custom tools -- πŸ‘‰ **[Map-Reduce Tutorial](../tutorials/map_reduce.ipynb)** - Interactive examples and patterns -- πŸ‘‰ **[Core Concepts](core_concepts.md)** - Understanding the theoretical foundation - ---- From 4f6a3e458c90562856728d9a09c9d53afd0d3171 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Thu, 12 Feb 2026 14:59:23 -0500 Subject: [PATCH 15/27] Improved Attribitions in Readme --- README.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 5126b6a96..9bcbb5109 100644 --- a/README.md +++ b/README.md @@ -122,19 +122,24 @@ Apache 2.0 ## πŸ‘₯ Authors -**Project Lead** +**Project Lead and Main Contributor** - Alfio Massimiliano Gliozzo (IBM Research) β€” gliozzo@us.ibm.com **Core Contributors** -- Junkyu Lee (IBM Research) β€” Junkyu.Lee@ibm.com -- Nahuel Defosse (IBM Research) β€” nahuel.defosse@ibm.com -- Naweed Aghmad Khan (IBM Research) β€” naweed.khan@ibm.com +- Junkyu Lee (IBM) β€” Junkyu.Lee@ibm.com +- Nahuel Defosse (IBM) β€” nahuel.defosse@ibm.com +- Naweed Aghmad Khan (IBM) β€” naweed.khan@ibm.com -**Additional Contributors** -- Christodoulos Constantinides (IBM Watson) β€” Christodoulos.Constantinides@ibm.com -- Nandana Mihindukulasooriya (IBM Research) β€” nandana@ibm.com +**Community Contributors** +- Christodoulos Constantinides (IBM) β€” Christodoulos.Constantinides@ibm.com +- Nandana Mihindukulasooriya (IBM) β€” nandana@ibm.com - Mustafa Eyceoz (Red Hat) β€” Mustafa.Eyceoz@partner.ibm.com -- Gaetano Rossiello (IBM Research) β€” gaetano.rossiello@ibm.com +- Gaetano Rossiello (IBM) β€” gaetano.rossiello@ibm.com +- Agostino Capponi (Columbia University) β€” ac3827@columbia.edu +- Chunghyun Han (Columbia University) β€” ch4005@columbia.edu +- Abhinav Goel (Columbia University) ag5252@columbia.edu +- Chaitya Shan (Columbia University) β€” cs4621@columbia.edu +- Brian Zi Qi Zhu (Columbia University) β€” bzz2101@columbia.edu --- From a9020267032fed73dc3c105583dcf2b717a1c28f Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Thu, 12 Feb 2026 15:15:51 -0500 Subject: [PATCH 16/27] Update README --- README.md | 86 ++++++++++++++++++++++++++----------------------------- 1 file changed, 41 insertions(+), 45 deletions(-) diff --git a/README.md b/README.md index 9bcbb5109..02d2b110b 100644 --- a/README.md +++ b/README.md @@ -11,6 +11,12 @@ Build AI-powered pipelines as typed data transformationsβ€”combining Pydantic schemas, LLM-powered transduction, and async execution.

+--- + +## ✨ Why Agentics + +Most "agent frameworks" let untyped text flow through a pipeline. Agentics flips that: **types are the interface**. +Workflows are expressed as transformations between structured states, with predictable schemas and composable operators. --- @@ -52,40 +58,38 @@ uv run python examples/hello_world.py ## πŸ§ͺ Example Usage ```python +from typing import Optional from pydantic import BaseModel, Field -from agentics.core.transducible_functions import transducible, Transduce - -class ProductDescription(BaseModel): - name: str - features: str - price: float - -class ViralTweet(BaseModel): - tweet: str = Field(..., description="Engaging tweet under 280 characters") - hashtags: list[str] = Field(..., description="3-5 relevant hashtags") - hook: str = Field(..., description="Attention-grabbing opening line") - -@transducible() -async def generate_viral_tweet(product: ProductDescription) -> ViralTweet: - """Transform boring product descriptions into viral social media content.""" - return Transduce(product) - -# Transform a product into viral content -product = ProductDescription( - name="Agentics Framework", - features="Type-safe AI workflows with LLM-powered transductions", - price=0.0 # Open source! -) -tweet = await generate_viral_tweet(product) -print(f"πŸ”₯ {tweet.tweet}") -print(f"πŸ“± {' '.join(tweet.hashtags)}") -``` +from agentics.core.transducible_functions import Transduce, transducible -**Output:** -``` -πŸ”₯ Stop wrestling with unstructured LLM outputs! 🎯 Agentics gives you type-safe AI workflows that just work. Build production-ready agents in minutes, not weeks. And it's FREE! πŸš€ -πŸ“± #AI #OpenSource #Python #LLM #DevTools + +class Movie(BaseModel): + movie_name: Optional[str] = None + description: Optional[str] = None + year: Optional[int] = None + + +class Genre(BaseModel): + genre: Optional[str] = Field(None, description="e.g., comedy, drama, action") + + +@transducible(provide_explanation=True) +async def classify_genre(state: Movie) -> Genre: + """Classify the genre of the source Movie.""" + return Transduce(state) + + +genre, explanation = await classify_genre( + Movie( + movie_name="The Godfather", + description=( + "The aging patriarch of an organized crime dynasty transfers control " + "of his clandestine empire to his reluctant son." + ), + year=1972, + ) +) ``` --- @@ -122,24 +126,16 @@ Apache 2.0 ## πŸ‘₯ Authors -**Project Lead and Main Contributor** +**Principal Investigator** - Alfio Massimiliano Gliozzo (IBM Research) β€” gliozzo@us.ibm.com **Core Contributors** -- Junkyu Lee (IBM) β€” Junkyu.Lee@ibm.com -- Nahuel Defosse (IBM) β€” nahuel.defosse@ibm.com -- Naweed Aghmad Khan (IBM) β€” naweed.khan@ibm.com - -**Community Contributors** -- Christodoulos Constantinides (IBM) β€” Christodoulos.Constantinides@ibm.com -- Nandana Mihindukulasooriya (IBM) β€” nandana@ibm.com +- Nahuel Defosse (IBM Research) β€” nahuel.defosse@ibm.com +- Junkyu Lee (IBM Research) β€” Junkyu.Lee@ibm.com +- Naweed Aghmad Khan (IBM Research) β€” naweed.khan@ibm.com +- Christodoulos Constantinides (IBM Watson) β€” Christodoulos.Constantinides@ibm.com - Mustafa Eyceoz (Red Hat) β€” Mustafa.Eyceoz@partner.ibm.com -- Gaetano Rossiello (IBM) β€” gaetano.rossiello@ibm.com -- Agostino Capponi (Columbia University) β€” ac3827@columbia.edu -- Chunghyun Han (Columbia University) β€” ch4005@columbia.edu -- Abhinav Goel (Columbia University) ag5252@columbia.edu -- Chaitya Shan (Columbia University) β€” cs4621@columbia.edu -- Brian Zi Qi Zhu (Columbia University) β€” bzz2101@columbia.edu + --- From e968ee497d94f6d9e48ae2b0843efcf20f022e79 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Thu, 12 Feb 2026 15:19:21 -0500 Subject: [PATCH 17/27] Update README from stash --- README.md | 206 +++++++++++++++++++++++++----------------------------- 1 file changed, 94 insertions(+), 112 deletions(-) diff --git a/README.md b/README.md index 02d2b110b..c1ca953a6 100644 --- a/README.md +++ b/README.md @@ -1,183 +1,165 @@

Agentics

-

Transduction is all you need

- +

Transduction is all you need

- Agentics logo + +

-

- Agentics is a Python framework for structured, scalable, and semantically grounded agentic computation.
- Build AI-powered pipelines as typed data transformationsβ€”combining Pydantic schemas, LLM-powered transduction, and async execution. -

+Agentics is a Python framework that provides structured, scalable, and semantically grounded agentic computation. It enables developers to build AI-powered pipelines where all operations are based on typed data transformations, combining the power of Pydantic models and LLMs with the flexibility of asynchronous execution. ---- +## Getting started -## ✨ Why Agentics +Learn how to install Agentic, set up your environment, and run your first logical transduction. [Getting Started](docs/getting_started.md) -Most "agent frameworks" let untyped text flow through a pipeline. Agentics flips that: **types are the interface**. -Workflows are expressed as transformations between structured states, with predictable schemas and composable operators. ---- +## Authors -## πŸš€ Key Features +- **Principal Investigator** + - *Alfio Massimiliano Gliozzo*, IBM Research, gliozzo@us.ibm.com +- **Core Contributors**: + - *Junkyu Lee*, IBM Research, Junkyu.Lee@ibm.com + - *Naweed Aghmad Khan*, IBM Research, naweed.khan@ibm.com + - *Nahuel Defosse*, IBM Research, nahuel.defosse@ibm.com + - *Christodoulos Constantinides*, IBM Watson, Christodoulos.Constantinides@ibm.com + - *Mustafa Eyceoz*, RedHat, Mustafa.Eyceoz@partner.ibm.com -- **Typed agentic computation**: Define workflows over structured types using standard **Pydantic** models. -- **Logical transduction (`<<`)**: Transform data between types using LLMs (few-shot examples, tools, memory). -- **Async mapping & reduction**: Scale out with `amap` and `areduce` over datasets. -- **Batch execution & retry**: Built-in batching, retries, and graceful fallbacks. -- **Tool support (MCP)**: Integrate external tools via MCP. ---- -## πŸ“¦ Getting Started +Agentics is an implementation of **Logical Transduction Algebra**, described in +- Alfio Gliozzo, Naweed Khan, Christodoulos Constantinides, Nandana Mihindukulasooriya, Nahuel Defosse, Junkyu Lee. *Transduction is All You Need for Structured Data Workflows. August 2025*, [arXiv:2508.15610](https://arxiv.org/abs/2508.15610) -Quickstart: -Install Agentics in your current env, set up your environment variable, and run your first logical transduction: +We welcome new AG entusiasts to extend this framework with new applications and extension to the language. -```bash -uv pip install agentics-py -``` -set up your .env using the required parameters for your LLM provider of choice. Use [.env_sample](.env_sample) as a reference. -Find out more -πŸ‘‰ **Getting Started**: [docs/getting_started.md](docs/getting_started.md) -**Examples** -Run scripts in the `examples/` folder (via `uv`): +## πŸš€ Key Features -```bash -uv run python examples/hello_world.py -``` +**Typed Agentic Computation**: Define workflows over structured types using standard Pydantic schemas. +**Logical Transduction (`<<`)**: Transform data between types using LLMs with few-shot examples, tools, and memory. ---- +**Async Mapping and Reduction**: Apply async mapping (`amap`) and aggregation (`areduce`) functions over datasets. -## πŸ§ͺ Example Usage +**Batch Execution & Retry**: Automatically handles batch-based asynchronous execution with graceful fallback. -```python -from typing import Optional -from pydantic import BaseModel, Field +**Domain Customization** +- **Prompt Templates** Customize prompting behavior and add ad-hoc instructions +- **Memory Augmentation**: Use retrieval-augmented memory to inform transduction. -from agentics.core.transducible_functions import Transduce, transducible +**Built-in Support for Tools**: Integrate LangChain tools or custom functions. -class Movie(BaseModel): - movie_name: Optional[str] = None - description: Optional[str] = None - year: Optional[int] = None +## Tutorial +| Notebook | Description | +|----------| --------------- | +| [LLMs](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/llms.ipynb) | Basics | +| [Agentic Basics](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/agentics_basics.ipynb) | Step by step guide illustrating how to make a new AG, access and print its content, import and export it to files | +|[Transduction](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/transduction.ipynb) | Demonstrate the use of logical transduction (`<<`) in Agentics | +| [Amap Reduce](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/amap_reduce.ipynb) | Try out MapReduce in Agentics to scale out | +| [MCP Tools](./tutorials/mcp_tools.ipynb) | | -class Genre(BaseModel): - genre: Optional[str] = Field(None, description="e.g., comedy, drama, action") + +## πŸš€ Documentation -@transducible(provide_explanation=True) -async def classify_genre(state: Movie) -> Genre: - """Classify the genre of the source Movie.""" - return Transduce(state) +πŸ‘‰ [Getting Started](docs/getting_started.md): Learn how to install Agentic, set up your environment, and run your first logical transduction. +🧠 [Agentics](docs/agentics.md): Explore how Agentics wraps `pydantic` models into transduction-ready agents. -genre, explanation = await classify_genre( - Movie( - movie_name="The Godfather", - description=( - "The aging patriarch of an organized crime dynasty transfers control " - "of his clandestine empire to his reluctant son." - ), - year=1972, - ) -) -``` +πŸ” [Transduction](docs/transduction.md): Discover how the `<<` operator implements logical transduction between types and how to control its behavior. ---- +πŸ› οΈ [Tools](docs/tools.md): Learn how to integrate external tools (e.g., LangChain, CrewAI) to provide access to external data necessary for logical transduction. -## πŸ“˜ Documentation and Notebooks +## πŸ“˜ Example Usage +```python +from agentics import AG +from pydantic import BaseModel -Complete documentation available [here](./docs/index.md) +class Answer(BaseModel): + answer: str + justification: str + confidence: float -| Notebook | Description | -|---|---| -| [agentics.ipynb](./tutorials/agentics.ipynb) | Core Agentics concepts: typed states, operators, and workflow structure | -| [atypes.ipynb](./tutorials/atypes.ipynb) | Working with ATypes: schema composition, merging, and type-driven design patterns | -| [logical_transduction_algebra.ipynb](./tutorials/logical_transduction_algebra.ipynb) | Logical Transduction Algebra: principles and examples behind `<<` | -| [map_reduce.ipynb](./tutorials/map_reduce.ipynb) | Scale out workflows with `amap` / `areduce` (MapReduce-style execution) | -| [synthetic_data_generation.ipynb](./tutorials/synthetic_data_generation.ipynb) | Generate structured synthetic datasets using typed transductions | -| [transducible_functions.ipynb](./tutorials/transducible_functions.ipynb) | Build reusable `@transducible` functions, explanations, and transduction control | +# Instantiate an Agentics object with a target type +qa_agent = AG(atype=Answer) -## βœ… Tests +# Perform transduction from text prompts +qa_agent = await (qa_agent << [ + "Who is the president of the US?", + "When is the end of the world predicted?", + "This is a report from the US embassy" +]) -Run all tests: +# Access structured answers +for result in qa_agent.states: + print(result.answer, result.confidence) -```bash -uv run pytest ``` +### 🧠 Conceptual Overview ---- +Agentics models workflows as transformations between typed states. Each instance of Agentics includes: -## πŸ“„ License +`atype`: A Pydantic model representing the schema. -Apache 2.0 +`states`: A list of objects of that type. ---- +Optional `llm`, `tools`, `prompt_template`, `memory`. -## πŸ‘₯ Authors +#### Operations: -**Principal Investigator** -- Alfio Massimiliano Gliozzo (IBM Research) β€” gliozzo@us.ibm.com +`amap`(func): Applies an async function over each state. -**Core Contributors** -- Nahuel Defosse (IBM Research) β€” nahuel.defosse@ibm.com -- Junkyu Lee (IBM Research) β€” Junkyu.Lee@ibm.com -- Naweed Aghmad Khan (IBM Research) β€” naweed.khan@ibm.com -- Christodoulos Constantinides (IBM Watson) β€” Christodoulos.Constantinides@ibm.com -- Mustafa Eyceoz (Red Hat) β€” Mustafa.Eyceoz@partner.ibm.com +`areduce`(func): Reduces a list of states into a single value. ---- +`<<`: Performs logical transduction from source to target Agentics. +#### πŸ”§ Advanced Usage -## 🧠 Conceptual Overview +##### Customizing Prompts -Most β€œagent frameworks” let untyped text flow through a pipeline. Agentics flips that: **types are the interface**. -Workflows are expressed as transformations between structured states, with predictable schemas and composable operators. +agent.prompt_template = """ +You are an assistant that extracts key information. +Please respond using the format {answer}, {justification}, {confidence}. +""" -Because every step is a typed transformation, you can **compose** workflows safely (merge and compose types/instances, chain transductions, and reuse `@transducible` functions) without losing semantic structure. +# πŸ“š Documentation -Agentics makes it natural to **scale out**: apply transformations over collections with async `amap`, and aggregate results with `areduce`. +Full documentation and examples are available at: -Agentics models workflows as transformations between **typed states**. +# πŸ§ͺ Tests -Core operations: +Run all tests using: -- `amap(func)`: apply an async function over each state -- `areduce(func)`: reduce a list of states into a single value -- `<<`: logical transduction from source to target Agentics -- `&`: merge Pydantic types / instances -- `@`: compose Pydantic types / instances +`uv run pytest` +# Examples -## πŸ“œ Reference +Run all scripts in example folder using uv -Agentics implements **Logical Transduction Algebra**, described in: +`uv run python examples/hello_world.py` -- Alfio Gliozzo, Naweed Khan, Christodoulos Constantinides, Nandana Mihindukulasooriya, Nahuel Defosse, Junkyu Lee. - *Transduction is All You Need for Structured Data Workflows* (August 2025). - arXiv:2508.15610 β€” https://arxiv.org/abs/2508.15610 +## $ πŸ“„ License + +Apache 2.0 + +## πŸ‘₯ Authors +Developed by Alfio Gliozzo and contributors. ---- +Contributions welcome! -## 🀝 Contributing +Core team Alfio Gliozzo, Junkyu Lee, Naweed Aghmad, Nahuel Defosse, Christodoulos Constantinides, Mustafa Eyceoz and contributors. -Contributions are welcome! -[CONTRIBUTING.md](CONTRIBUTING.md) +## Contributing - Please ensure your commit messages include: +Your commit messages should include the line: -```text +```shell Signed-off-by: Author Name ``` From 96bf3389b94aa34d7b4b40a4692269a28d5ff14a Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Thu, 12 Feb 2026 15:35:34 -0500 Subject: [PATCH 18/27] Readme attribution fixed --- README.md | 230 ++++++++++++++++++++++++++++++------------------------ 1 file changed, 126 insertions(+), 104 deletions(-) diff --git a/README.md b/README.md index c1ca953a6..9bcbb5109 100644 --- a/README.md +++ b/README.md @@ -1,165 +1,187 @@

Agentics

-

Transduction is all you need

+

Transduction is all you need

+

- - + Agentics logo

-Agentics is a Python framework that provides structured, scalable, and semantically grounded agentic computation. It enables developers to build AI-powered pipelines where all operations are based on typed data transformations, combining the power of Pydantic models and LLMs with the flexibility of asynchronous execution. - -## Getting started - -Learn how to install Agentic, set up your environment, and run your first logical transduction. [Getting Started](docs/getting_started.md) - - -## Authors - -- **Principal Investigator** - - *Alfio Massimiliano Gliozzo*, IBM Research, gliozzo@us.ibm.com -- **Core Contributors**: - - *Junkyu Lee*, IBM Research, Junkyu.Lee@ibm.com - - *Naweed Aghmad Khan*, IBM Research, naweed.khan@ibm.com - - *Nahuel Defosse*, IBM Research, nahuel.defosse@ibm.com - - *Christodoulos Constantinides*, IBM Watson, Christodoulos.Constantinides@ibm.com - - *Mustafa Eyceoz*, RedHat, Mustafa.Eyceoz@partner.ibm.com - - - -Agentics is an implementation of **Logical Transduction Algebra**, described in -- Alfio Gliozzo, Naweed Khan, Christodoulos Constantinides, Nandana Mihindukulasooriya, Nahuel Defosse, Junkyu Lee. *Transduction is All You Need for Structured Data Workflows. August 2025*, [arXiv:2508.15610](https://arxiv.org/abs/2508.15610) - - -We welcome new AG entusiasts to extend this framework with new applications and extension to the language. - +

+ Agentics is a Python framework for structured, scalable, and semantically grounded agentic computation.
+ Build AI-powered pipelines as typed data transformationsβ€”combining Pydantic schemas, LLM-powered transduction, and async execution. +

+--- ## πŸš€ Key Features -**Typed Agentic Computation**: Define workflows over structured types using standard Pydantic schemas. +- **Typed agentic computation**: Define workflows over structured types using standard **Pydantic** models. +- **Logical transduction (`<<`)**: Transform data between types using LLMs (few-shot examples, tools, memory). +- **Async mapping & reduction**: Scale out with `amap` and `areduce` over datasets. +- **Batch execution & retry**: Built-in batching, retries, and graceful fallbacks. +- **Tool support (MCP)**: Integrate external tools via MCP. -**Logical Transduction (`<<`)**: Transform data between types using LLMs with few-shot examples, tools, and memory. +--- -**Async Mapping and Reduction**: Apply async mapping (`amap`) and aggregation (`areduce`) functions over datasets. +## πŸ“¦ Getting Started -**Batch Execution & Retry**: Automatically handles batch-based asynchronous execution with graceful fallback. +Quickstart: -**Domain Customization** -- **Prompt Templates** Customize prompting behavior and add ad-hoc instructions -- **Memory Augmentation**: Use retrieval-augmented memory to inform transduction. +Install Agentics in your current env, set up your environment variable, and run your first logical transduction: -**Built-in Support for Tools**: Integrate LangChain tools or custom functions. - - -## Tutorial +```bash +uv pip install agentics-py +``` +set up your .env using the required parameters for your LLM provider of choice. Use [.env_sample](.env_sample) as a reference. -| Notebook | Description | -|----------| --------------- | -| [LLMs](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/llms.ipynb) | Basics | -| [Agentic Basics](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/agentics_basics.ipynb) | Step by step guide illustrating how to make a new AG, access and print its content, import and export it to files | -|[Transduction](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/transduction.ipynb) | Demonstrate the use of logical transduction (`<<`) in Agentics | -| [Amap Reduce](https://colab.research.google.com/github/IBM/Agentics/blob/main/tutorials/amap_reduce.ipynb) | Try out MapReduce in Agentics to scale out | -| [MCP Tools](./tutorials/mcp_tools.ipynb) | | +Find out more +πŸ‘‰ **Getting Started**: [docs/getting_started.md](docs/getting_started.md) - +**Examples** -## πŸš€ Documentation +Run scripts in the `examples/` folder (via `uv`): -πŸ‘‰ [Getting Started](docs/getting_started.md): Learn how to install Agentic, set up your environment, and run your first logical transduction. +```bash +uv run python examples/hello_world.py +``` -🧠 [Agentics](docs/agentics.md): Explore how Agentics wraps `pydantic` models into transduction-ready agents. -πŸ” [Transduction](docs/transduction.md): Discover how the `<<` operator implements logical transduction between types and how to control its behavior. +--- -πŸ› οΈ [Tools](docs/tools.md): Learn how to integrate external tools (e.g., LangChain, CrewAI) to provide access to external data necessary for logical transduction. +## πŸ§ͺ Example Usage -## πŸ“˜ Example Usage ```python -from agentics import AG -from pydantic import BaseModel +from pydantic import BaseModel, Field +from agentics.core.transducible_functions import transducible, Transduce + +class ProductDescription(BaseModel): + name: str + features: str + price: float + +class ViralTweet(BaseModel): + tweet: str = Field(..., description="Engaging tweet under 280 characters") + hashtags: list[str] = Field(..., description="3-5 relevant hashtags") + hook: str = Field(..., description="Attention-grabbing opening line") + +@transducible() +async def generate_viral_tweet(product: ProductDescription) -> ViralTweet: + """Transform boring product descriptions into viral social media content.""" + return Transduce(product) + +# Transform a product into viral content +product = ProductDescription( + name="Agentics Framework", + features="Type-safe AI workflows with LLM-powered transductions", + price=0.0 # Open source! +) + +tweet = await generate_viral_tweet(product) +print(f"πŸ”₯ {tweet.tweet}") +print(f"πŸ“± {' '.join(tweet.hashtags)}") +``` + +**Output:** +``` +πŸ”₯ Stop wrestling with unstructured LLM outputs! 🎯 Agentics gives you type-safe AI workflows that just work. Build production-ready agents in minutes, not weeks. And it's FREE! πŸš€ +πŸ“± #AI #OpenSource #Python #LLM #DevTools +``` -class Answer(BaseModel): - answer: str - justification: str - confidence: float +--- -# Instantiate an Agentics object with a target type -qa_agent = AG(atype=Answer) +## πŸ“˜ Documentation and Notebooks -# Perform transduction from text prompts -qa_agent = await (qa_agent << [ - "Who is the president of the US?", - "When is the end of the world predicted?", - "This is a report from the US embassy" -]) +Complete documentation available [here](./docs/index.md) -# Access structured answers -for result in qa_agent.states: - print(result.answer, result.confidence) +| Notebook | Description | +|---|---| +| [agentics.ipynb](./tutorials/agentics.ipynb) | Core Agentics concepts: typed states, operators, and workflow structure | +| [atypes.ipynb](./tutorials/atypes.ipynb) | Working with ATypes: schema composition, merging, and type-driven design patterns | +| [logical_transduction_algebra.ipynb](./tutorials/logical_transduction_algebra.ipynb) | Logical Transduction Algebra: principles and examples behind `<<` | +| [map_reduce.ipynb](./tutorials/map_reduce.ipynb) | Scale out workflows with `amap` / `areduce` (MapReduce-style execution) | +| [synthetic_data_generation.ipynb](./tutorials/synthetic_data_generation.ipynb) | Generate structured synthetic datasets using typed transductions | +| [transducible_functions.ipynb](./tutorials/transducible_functions.ipynb) | Build reusable `@transducible` functions, explanations, and transduction control | -``` +## βœ… Tests -### 🧠 Conceptual Overview +Run all tests: -Agentics models workflows as transformations between typed states. Each instance of Agentics includes: +```bash +uv run pytest +``` -`atype`: A Pydantic model representing the schema. -`states`: A list of objects of that type. +--- -Optional `llm`, `tools`, `prompt_template`, `memory`. +## πŸ“„ License -#### Operations: +Apache 2.0 -`amap`(func): Applies an async function over each state. +--- -`areduce`(func): Reduces a list of states into a single value. +## πŸ‘₯ Authors -`<<`: Performs logical transduction from source to target Agentics. +**Project Lead and Main Contributor** +- Alfio Massimiliano Gliozzo (IBM Research) β€” gliozzo@us.ibm.com -#### πŸ”§ Advanced Usage +**Core Contributors** +- Junkyu Lee (IBM) β€” Junkyu.Lee@ibm.com +- Nahuel Defosse (IBM) β€” nahuel.defosse@ibm.com +- Naweed Aghmad Khan (IBM) β€” naweed.khan@ibm.com -##### Customizing Prompts +**Community Contributors** +- Christodoulos Constantinides (IBM) β€” Christodoulos.Constantinides@ibm.com +- Nandana Mihindukulasooriya (IBM) β€” nandana@ibm.com +- Mustafa Eyceoz (Red Hat) β€” Mustafa.Eyceoz@partner.ibm.com +- Gaetano Rossiello (IBM) β€” gaetano.rossiello@ibm.com +- Agostino Capponi (Columbia University) β€” ac3827@columbia.edu +- Chunghyun Han (Columbia University) β€” ch4005@columbia.edu +- Abhinav Goel (Columbia University) ag5252@columbia.edu +- Chaitya Shan (Columbia University) β€” cs4621@columbia.edu +- Brian Zi Qi Zhu (Columbia University) β€” bzz2101@columbia.edu +--- -agent.prompt_template = """ -You are an assistant that extracts key information. -Please respond using the format {answer}, {justification}, {confidence}. -""" -# πŸ“š Documentation +## 🧠 Conceptual Overview -Full documentation and examples are available at: +Most β€œagent frameworks” let untyped text flow through a pipeline. Agentics flips that: **types are the interface**. +Workflows are expressed as transformations between structured states, with predictable schemas and composable operators. -# πŸ§ͺ Tests +Because every step is a typed transformation, you can **compose** workflows safely (merge and compose types/instances, chain transductions, and reuse `@transducible` functions) without losing semantic structure. -Run all tests using: +Agentics makes it natural to **scale out**: apply transformations over collections with async `amap`, and aggregate results with `areduce`. -`uv run pytest` +Agentics models workflows as transformations between **typed states**. +Core operations: -# Examples +- `amap(func)`: apply an async function over each state +- `areduce(func)`: reduce a list of states into a single value +- `<<`: logical transduction from source to target Agentics +- `&`: merge Pydantic types / instances +- `@`: compose Pydantic types / instances -Run all scripts in example folder using uv -`uv run python examples/hello_world.py` -## $ πŸ“„ License +## πŸ“œ Reference -Apache 2.0 +Agentics implements **Logical Transduction Algebra**, described in: -## πŸ‘₯ Authors +- Alfio Gliozzo, Naweed Khan, Christodoulos Constantinides, Nandana Mihindukulasooriya, Nahuel Defosse, Junkyu Lee. + *Transduction is All You Need for Structured Data Workflows* (August 2025). + arXiv:2508.15610 β€” https://arxiv.org/abs/2508.15610 -Developed by Alfio Gliozzo and contributors. -Contributions welcome! +--- -Core team Alfio Gliozzo, Junkyu Lee, Naweed Aghmad, Nahuel Defosse, Christodoulos Constantinides, Mustafa Eyceoz and contributors. +## 🀝 Contributing -## Contributing +Contributions are welcome! +[CONTRIBUTING.md](CONTRIBUTING.md) -Your commit messages should include the line: + Please ensure your commit messages include: -```shell +```text Signed-off-by: Author Name ``` From 3fd26642cbd964d2fce10ac9cf76509b1b2988e3 Mon Sep 17 00:00:00 2001 From: Alfio Gliozzo Date: Thu, 12 Feb 2026 15:52:56 -0500 Subject: [PATCH 19/27] Semantic Operators Documentation --- docs/index.md | 5 +- docs/semantic_operators.md | 390 +++++++++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 3 files changed, 395 insertions(+), 1 deletion(-) create mode 100644 docs/semantic_operators.md diff --git a/docs/index.md b/docs/index.md index df8e8166a..5217bc416 100644 --- a/docs/index.md +++ b/docs/index.md @@ -21,6 +21,9 @@ Agentics is a lightweight, Python-native framework for building **structured and - **[Map-Reduce Operations](map_reduce.md)** πŸ” Scaling transducible functions with map and reduce operations, batch processing patterns, and best practices. +- **[Semantic Operators](semantic_operators.md)** πŸ” + High-level declarative API for data transformations using natural language. Includes `sem_map`, `sem_filter`, `sem_agg`, and more LOTUS-style operations. + - **[Agentics (AG)](agentics.md)** 🧬 Working with `AG` typed state containers, loading data from JSON/CSV/DataFrames, and preserving type information across the pipeline. @@ -97,4 +100,4 @@ You can start the server to visualize this interactively. ```bash mkdocs serve ``` -After started, documentation will be available here [http://127.0.0.1:8000/](http://127.0.0.1:8000/) \ No newline at end of file +After started, documentation will be available here [http://127.0.0.1:8000/](http://127.0.0.1:8000/) diff --git a/docs/semantic_operators.md b/docs/semantic_operators.md new file mode 100644 index 000000000..654b0be66 --- /dev/null +++ b/docs/semantic_operators.md @@ -0,0 +1,390 @@ +# πŸ” Semantic Operators + +Semantic operators provide a high-level, declarative API for performing common data transformation tasks using natural language instructions. Inspired by LOTUS-style semantic operations, these operators enable you to work with structured and unstructured data using LLM-powered transformations. + +--- + +## Overview + +Agentics semantic operators bridge the gap between traditional data manipulation (like pandas operations) and LLM-powered semantic understanding. Each operator accepts either an `AG` (Agentics) or a pandas `DataFrame` as input and returns the same type, making them easy to integrate into existing data pipelines. + +### Available Operators + +| Operator | Description | +|----------|-------------| +| `sem_map` | Map each record using a natural language projection | +| `sem_filter` | Keep records that match a natural language predicate | +| `sem_extract` | Extract one or more attributes from each row | +| `sem_agg` | Aggregate across all records (e.g., for summarization) | +| `sem_topk` | Order records by natural language sorting criteria | +| `sem_join` | Join two datasets based on a natural language predicate | + +--- + +## `sem_map` + +Transform each record in your dataset according to natural language instructions, mapping source data to a target schema. + +### Signature + +```python +async def sem_map( + source: AG | pd.DataFrame, + target_type: Type[BaseModel] | str, + instructions: str, + merge_output: bool = True, + **kwargs, +) -> AG | pd.DataFrame +``` + +### Parameters + +- **`source`** (`AG | pd.DataFrame`): Input data to be mapped +- **`target_type`** (`Type[BaseModel] | str`): Target schema for the output + - If a Pydantic `BaseModel` subclass: used directly as the target type + - If a `str`: a Pydantic model is created dynamically with a single string field +- **`instructions`** (`str`): Natural language description of how to transform the data +- **`merge_output`** (`bool`, default=`True`): + - `True`: Merge mapped fields back into original source records + - `False`: Return only the mapped output +- **`**kwargs`**: Additional arguments forwarded to `AG()` constructor (e.g., model configuration, batching) + +### Returns + +- **`AG | pd.DataFrame`**: Transformed data in the same format as input + +### Example: Basic Mapping + +```python +import pandas as pd +from agentics.core.semantic_operators import sem_map +from pydantic import BaseModel + +# Sample data +df = pd.DataFrame({ + 'review': [ + 'This product is amazing! Best purchase ever.', + 'Terrible quality, broke after one day.', + 'It works okay, nothing special.' + ] +}) + +# Define target schema +class Sentiment(BaseModel): + sentiment: str + confidence: float + +# Map reviews to sentiment +result = await sem_map( + source=df, + target_type=Sentiment, + instructions="Analyze the sentiment of the review and provide a confidence score (0-1)" +) + +print(result) +# Output includes original 'review' column plus 'sentiment' and 'confidence' columns +``` + +### Example: String-based Target Type + +```python +# Using string target type for simpler cases +result = await sem_map( + source=df, + target_type="category", + instructions="Classify the review into one of: positive, negative, neutral" +) + +print(result) +# Output includes original columns plus a 'category' column +``` + +### Example: Extract Without Merge + +```python +# Get only the mapped output without original data +result = await sem_map( + source=df, + target_type=Sentiment, + instructions="Analyze sentiment", + merge_output=False +) + +print(result) +# Output contains only 'sentiment' and 'confidence' columns +``` + +--- + +## `sem_filter` + +Filter records based on a natural language predicate, keeping only those that satisfy the condition. + +### Signature + +```python +async def sem_filter( + source: AG | pd.DataFrame, + predicate_template: str, + **kwargs +) -> AG | pd.DataFrame +``` + +### Parameters + +- **`source`** (`AG | pd.DataFrame`): Input data to be filtered +- **`predicate_template`** (`str`): Natural language condition or LangChain-style template + - Can use `{field}` placeholders to reference source fields + - Or provide a plain text predicate +- **`**kwargs`**: Additional arguments forwarded to `AG()` constructor + +### Returns + +- **`AG | pd.DataFrame`**: Filtered data containing only records that satisfy the predicate + +### Example: Simple Predicate + +```python +from agentics.core.semantic_operators import sem_filter + +df = pd.DataFrame({ + 'product': ['Laptop', 'Phone', 'Tablet', 'Monitor'], + 'description': [ + 'High-performance gaming laptop with RGB keyboard', + 'Budget smartphone with basic features', + 'Premium tablet with stylus support', + '4K monitor for professional work' + ] +}) + +# Filter for premium/high-end products +result = await sem_filter( + source=df, + predicate_template="The product is premium or high-end" +) + +print(result) +# Returns only Laptop, Tablet, and Monitor +``` + +### Example: Template-based Filtering + +```python +# Use field placeholders in the predicate +result = await sem_filter( + source=df, + predicate_template="The {product} described as '{description}' is suitable for gaming" +) + +print(result) +# Returns only the gaming laptop +``` + +--- + +## `sem_agg` + +Aggregate data across all records to produce a summary or consolidated output. + +### Signature + +```python +async def sem_agg( + source: AG | pd.DataFrame, + target_type: Type[BaseModel] | str, + instructions: str = None, + **kwargs, +) -> AG | pd.DataFrame +``` + +### Parameters + +- **`source`** (`AG | pd.DataFrame`): Input data to be aggregated +- **`target_type`** (`Type[BaseModel] | str`): Schema for the aggregated output +- **`instructions`** (`str`, optional): Natural language description of the aggregation +- **`**kwargs`**: Additional arguments forwarded to `AG()` constructor + +### Returns + +- **`AG | pd.DataFrame`**: Aggregated result (typically a single record or summary) + +### Example: Summarization + +```python +from agentics.core.semantic_operators import sem_agg +from pydantic import BaseModel + +df = pd.DataFrame({ + 'review': [ + 'Great product, very satisfied!', + 'Good quality but expensive', + 'Not worth the price', + 'Excellent, would buy again', + 'Decent but has some issues' + ] +}) + +class ReviewSummary(BaseModel): + overall_sentiment: str + key_themes: list[str] + recommendation: str + +# Aggregate all reviews into a summary +result = await sem_agg( + source=df, + target_type=ReviewSummary, + instructions="Summarize all reviews, identify key themes, and provide an overall recommendation" +) + +print(result) +# Returns a single record with aggregated insights +``` + +### Example: Statistical Summary + +```python +class Statistics(BaseModel): + total_count: int + positive_count: int + negative_count: int + average_sentiment: str + +result = await sem_agg( + source=df, + target_type=Statistics, + instructions="Count total reviews, positive reviews, negative reviews, and determine average sentiment" +) +``` + +--- + +## Best Practices + +### 1. Choose the Right Operator + +- **`sem_map`**: Use for 1:1 transformations (each input β†’ one output) +- **`sem_filter`**: Use for selecting subsets based on conditions +- **`sem_agg`**: Use for many:1 transformations (all inputs β†’ one summary) + +### 2. Write Clear Instructions + +```python +# ❌ Vague +instructions = "Process the data" + +# βœ… Clear and specific +instructions = """ +Extract the product name, price, and category from each description. +Normalize prices to USD. Categorize products as: Electronics, Clothing, or Home Goods. +""" +``` + +### 3. Use Appropriate Target Types + +```python +# For simple extractions, use string types +sem_map( + ... + target_type= "category_name" + ... +) + + +# For structured outputs, use Pydantic models +class Product(BaseModel): + name: str + price: float + category: str + +sem_map( + ... + target_type= Product + ... +) +``` + +### 4. Batch Processing + +```python +# Configure batch size for large datasets +result = await sem_map( + source=large_df, + target_type=MyType, + instructions="...", + amap_batch_size=50 # Process 50 records at a time +) +``` + +### 5. Handle Both AG and DataFrame + +```python +# Operators work with both types +df_result = await sem_filter(df, "condition") # Returns DataFrame +ag_result = await sem_filter(ag, "condition") # Returns AG +``` + +--- + +## Performance Considerations + +### Batching + +Semantic operators support batching for efficient processing of large datasets: + +```python +result = await sem_map( + source=df, + target_type=MyType, + instructions="...", + amap_batch_size=20 # Default for sem_filter +) +``` + + +## Integration with Agentics Workflows + +Semantic operators integrate seamlessly with other Agentics features: + +### Chaining Operations + +```python +# Filter β†’ Map β†’ Aggregate pipeline +filtered = await sem_filter(df, "High-value customers") +mapped = await sem_map(filtered, CustomerProfile, "Extract profile details") +summary = await sem_agg(mapped, Summary, "Summarize customer segments") +``` + +### Using with AG + +```python +from agentics import AG + +# Load data into AG +ag = AG.from_dataframe(df) + +# Apply semantic operators +filtered_ag = await sem_filter(ag, "Important records") +mapped_ag = await sem_map(filtered_ag, OutputType, "Transform data") + +# Continue with AG operations +result = await (target_ag << mapped_ag) +``` + +--- + + +## Coming Soon + +The following operators are planned for future releases: + +- **`sem_topk`**: Order and select top-k records by semantic criteria +- **`sem_join`**: Join datasets based on semantic similarity or conditions + +--- + +## See Also + +- [Core Concepts](core_concepts.md) - Understanding Agentics fundamentals +- [Transducible Functions](transducible_functions.md) - Lower-level transformation API +- [Map-Reduce Operations](map_reduce.md) - Scaling transformations +- [Agentics (AG)](agentics.md) - Working with typed state containers diff --git a/mkdocs.yml b/mkdocs.yml index 76386733b..e59dc6992 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -14,6 +14,7 @@ nav: - 🧠 Core Concepts: core_concepts.md - βš™οΈ Transducible Functions: transducible_functions.md - πŸ” Map-Reduce Operations: map_reduce.md + - πŸ” Semantic Operators: semantic_operators.md - 🧬 Agentics (AG): agentics.md - ⚑ Performance Optimization: optimization.md - πŸ”Œ Tool Integration: tool_integration.md From 18007d1a509bd5f638ee68463ccaa006973352a6 Mon Sep 17 00:00:00 2001 From: Junkyu Lee Date: Thu, 12 Feb 2026 19:22:47 -0500 Subject: [PATCH 20/27] reviewed documentation and added references Signed-off-by: Junkyu Lee --- docs/core_concepts.md | 3 + docs/getting_started.md | 3 + docs/index.md | 5 +- docs/map_reduce.md | 4 +- docs/references.md | 45 ++++ docs/semantic_operators.md | 82 ++------ docs/transducible_functions.md | 3 + src/agentics/core/semantic_operators.py | 13 +- tutorials/semantic_operators.ipynb | 259 ++++++++++++++++++++++++ 9 files changed, 346 insertions(+), 71 deletions(-) create mode 100644 docs/references.md create mode 100644 tutorials/semantic_operators.ipynb diff --git a/docs/core_concepts.md b/docs/core_concepts.md index a8d51c555..0aafd2960 100644 --- a/docs/core_concepts.md +++ b/docs/core_concepts.md @@ -256,3 +256,6 @@ A typical workflow looks like this: ## Next - πŸ‘‰ [Transducible Functions](transducible_functions.md) for concrete examples of defining and using transducible functions + +## Go to Index +- πŸ‘‰ [Index](index.md) \ No newline at end of file diff --git a/docs/getting_started.md b/docs/getting_started.md index 1eaac5dc7..fa0cb5709 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -297,3 +297,6 @@ The same transducible function works seamlessly for both single items and batche ## Next - πŸ‘‰ [Core Concepts](core_concepts.md) - Understanding the theoretical foundation + +## Go to Index +- πŸ‘‰ [Index](index.md) \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index 5217bc416..54568920a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -47,7 +47,9 @@ Agentics is a lightweight, Python-native framework for building **structured and End-to-end examples: text-to-SQL, data extraction and enrichment, classification, document workflows, evaluation pipelines, and more. - +## How to Cite Agentics +- **[References](references.md)** πŸ“š + Academic papers and research that form the foundation of Agentics, including transduction algebra, agentic AI, and applications. --- @@ -92,6 +94,7 @@ The subset of input fields that contributed to generating a specific output fiel **Slot** A field in a Pydantic model. "Slot-level provenance" means tracking which input slots contributed to each output slot. +--- ## Documentation diff --git a/docs/map_reduce.md b/docs/map_reduce.md index e660aff39..f66cf5c78 100644 --- a/docs/map_reduce.md +++ b/docs/map_reduce.md @@ -374,6 +374,8 @@ Map operations are automatically parallelized based on `batch_size`. For more co --- ## Next -- πŸ‘‰ [Agentics (AG)](agentics.md) for data modeling patterns and typed state containers - πŸ‘‰ [Map-Reduce Tutorial](../tutorials/map_reduce.ipynb) to see how large-scale execution works in practice +- πŸ‘‰ [Semantic Operators](semantic_operators.md) for performing data transformation tasks using natural language. +## Go to Index +- πŸ‘‰ [Index](index.md) \ No newline at end of file diff --git a/docs/references.md b/docs/references.md new file mode 100644 index 000000000..e8726737c --- /dev/null +++ b/docs/references.md @@ -0,0 +1,45 @@ +# References + +This page contains academic papers and research that form the foundation of Agentics. +If you use Agentics in your research or project, please cite the relevant papers listed below. Each entry includes BibTeX citations for easy integration into your bibliography. + +> **Note:** This list is actively maintained and updated as new papers are published. + +--- + +1. **Transduction is All You Need for Structured Data Workflows** (2025) + - Authors: Alfio Gliozzo, Naweed Khan, Christodoulos Constantinides, Nandana Mihindukulasooriya, Nahuel Defosse, Gaetano Rossiello, Junkyu Lee + - URL: https://arxiv.org/abs/2508.15610 + ```bibtex + @article{gliozzo2025transduction, + title={Transduction is All You Need for Structured Data Workflows}, + author={Gliozzo, Alfio and Khan, Naweed and Constantinides, Christodoulos and Mihindukulasooriya, Nandana and Defosse, Nahuel and Rossiello, Gaetano and Lee, Junkyu}, + journal={arXiv preprint arXiv:2508.15610}, + year={2025} + } + ``` + +2. **Semantic Trading: Agentic AI for Clustering and Relationship Discovery in Prediction Markets** (2025) + - Authors: Agostino Capponi, Alfio Gliozzo, Brian Zhu + - URL: https://arxiv.org/abs/2512.02436 + ```bibtex + @article{capponi2025semantic, + title={Semantic Trading: Agentic AI for Clustering and Relationship Discovery in Prediction Markets}, + author={Capponi, Agostino and Gliozzo, Alfio and Zhu, Brian}, + journal={arXiv preprint arXiv:2512.02436}, + year={2025} + } + ``` + +3. **DAO-AI: Evaluating Collective Decision-Making through Agentic AI in Decentralized Governance** (2026) + - Authors: Chunghyun Han, Alfio Gliozzo, Junkyu Lee, Agostino Capponi + - URL: https://arxiv.org/abs/2510.21117 + ```bibtex + @inproceedings{han2026daoai, + title={{DAO}-{AI}: Evaluating Collective Decision-Making through Agentic {AI} in Decentralized Governance}, + author={Chunghyun Han and Alfio Gliozzo and Junkyu Lee and Agostino Capponi}, + booktitle={AAAI'26 Workshop on Agentic AI in Financial Services}, + year={2026}, + url={https://arxiv.org/abs/2510.21117} + } + ``` \ No newline at end of file diff --git a/docs/semantic_operators.md b/docs/semantic_operators.md index 654b0be66..4b6b78478 100644 --- a/docs/semantic_operators.md +++ b/docs/semantic_operators.md @@ -1,6 +1,6 @@ # πŸ” Semantic Operators -Semantic operators provide a high-level, declarative API for performing common data transformation tasks using natural language instructions. Inspired by LOTUS-style semantic operations, these operators enable you to work with structured and unstructured data using LLM-powered transformations. +Semantic operators provide a high-level, declarative API for performing common data transformation tasks using natural language. Inspired by [LOTUS](https://lotus-data.github.io/)-style semantic operations, these operators enable you to work with structured and unstructured data using LLM-powered transformations. --- @@ -12,12 +12,9 @@ Agentics semantic operators bridge the gap between traditional data manipulation | Operator | Description | |----------|-------------| -| `sem_map` | Map each record using a natural language projection | +| `sem_map` | Map each record using a natural language instruction | | `sem_filter` | Keep records that match a natural language predicate | -| `sem_extract` | Extract one or more attributes from each row | | `sem_agg` | Aggregate across all records (e.g., for summarization) | -| `sem_topk` | Order records by natural language sorting criteria | -| `sem_join` | Join two datasets based on a natural language predicate | --- @@ -51,7 +48,7 @@ async def sem_map( ### Returns -- **`AG | pd.DataFrame`**: Transformed data in the same format as input +- **`AG | pd.DataFrame`**: `AG` or `DataFrame` that contains the transformed data following `target_type` ### Example: Basic Mapping @@ -71,18 +68,21 @@ df = pd.DataFrame({ # Define target schema class Sentiment(BaseModel): - sentiment: str - confidence: float + sentiment: Optional[str] = Field(None, description="The sentiment of the review (e.g., positive, negative, neutral)") + confidence: Optional[float] = Field(None, description="Confidence score of the sentiment analysis btw 0 and 1") # Map reviews to sentiment result = await sem_map( source=df, target_type=Sentiment, - instructions="Analyze the sentiment of the review and provide a confidence score (0-1)" + instructions="Analyze the sentiment of the review and provide a confidence score between 0 and 1." ) -print(result) # Output includes original 'review' column plus 'sentiment' and 'confidence' columns + review sentiment confidence + 0 This product is amazing! Best purchase ever. positive 0.85 + 1 Terrible quality, broke after one day. negative 0.99 + 2 It works okay, nothing special. neutral 0.85 ``` ### Example: String-based Target Type @@ -94,24 +94,6 @@ result = await sem_map( target_type="category", instructions="Classify the review into one of: positive, negative, neutral" ) - -print(result) -# Output includes original columns plus a 'category' column -``` - -### Example: Extract Without Merge - -```python -# Get only the mapped output without original data -result = await sem_map( - source=df, - target_type=Sentiment, - instructions="Analyze sentiment", - merge_output=False -) - -print(result) -# Output contains only 'sentiment' and 'confidence' columns ``` --- @@ -164,7 +146,10 @@ result = await sem_filter( ) print(result) -# Returns only Laptop, Tablet, and Monitor + product description +0 Laptop High-performance gaming laptop with RGB keyboard +1 Tablet Premium tablet with stylus support +2 Monitor 4K monitor for professional work ``` ### Example: Template-based Filtering @@ -175,9 +160,6 @@ result = await sem_filter( source=df, predicate_template="The {product} described as '{description}' is suitable for gaming" ) - -print(result) -# Returns only the gaming laptop ``` --- @@ -354,37 +336,11 @@ mapped = await sem_map(filtered, CustomerProfile, "Extract profile details") summary = await sem_agg(mapped, Summary, "Summarize customer segments") ``` -### Using with AG - -```python -from agentics import AG - -# Load data into AG -ag = AG.from_dataframe(df) - -# Apply semantic operators -filtered_ag = await sem_filter(ag, "Important records") -mapped_ag = await sem_map(filtered_ag, OutputType, "Transform data") - -# Continue with AG operations -result = await (target_ag << mapped_ag) -``` - ---- - - -## Coming Soon - -The following operators are planned for future releases: - -- **`sem_topk`**: Order and select top-k records by semantic criteria -- **`sem_join`**: Join datasets based on semantic similarity or conditions - --- -## See Also +## Next +- πŸ‘‰ [Semantic Operators Tutorial](../tutorials/semantic_operators.ipynb) - Code examples +- πŸ‘‰ [Agentics (AG)](agentics.md) for data modeling patterns and typed state containers -- [Core Concepts](core_concepts.md) - Understanding Agentics fundamentals -- [Transducible Functions](transducible_functions.md) - Lower-level transformation API -- [Map-Reduce Operations](map_reduce.md) - Scaling transformations -- [Agentics (AG)](agentics.md) - Working with typed state containers +## Go to Index +- πŸ‘‰ [Index](index.md) \ No newline at end of file diff --git a/docs/transducible_functions.md b/docs/transducible_functions.md index e6dfcb6f4..faff3ffd5 100644 --- a/docs/transducible_functions.md +++ b/docs/transducible_functions.md @@ -618,3 +618,6 @@ Good reasons to define a separate transducible function: ## Next - πŸ‘‰ **[Map-Reduce Operations](map_reduce.md)** - Scaling with map and reduce, batch processing patterns + +## Go to Index +- πŸ‘‰ [Index](index.md) \ No newline at end of file diff --git a/src/agentics/core/semantic_operators.py b/src/agentics/core/semantic_operators.py index a59a3284a..562726be8 100644 --- a/src/agentics/core/semantic_operators.py +++ b/src/agentics/core/semantic_operators.py @@ -86,11 +86,10 @@ async def sem_map( if isinstance(target_type, str) else target_type ), + instructions=instructions, **kwargs, ) - ag_source.prompt_template = instructions - map_out = await (target_ag << ag_source) output_ag = None if merge_output: @@ -103,7 +102,9 @@ async def sem_map( async def sem_filter( - source: AG | pd.DataFrame, predicate_template: str, **kwargs + source: AG | pd.DataFrame, + predicate_template: str, + **kwargs ) -> AG | pd.DataFrame: """ Agentics-native semantic filter over an `AG` using a LangChain-style condition template. @@ -160,6 +161,7 @@ async def sem_filter( ag_source.prompt_template = predicate_template else: target_ag.instructions += f"\n\nPredicate: {predicate_template}" + map_out = await (target_ag << ag_source) target = ag_source.clone() target.states = [] @@ -190,12 +192,11 @@ async def sem_agg( if isinstance(target_type, str) else target_type ), + instructions=instructions, + transduction_type="areduce", **kwargs, ) - ag_source.prompt_template = instructions - ag_source.transduction_type = "areduce" - output_ag = await (target_ag << ag_source) if type(source) is pd.DataFrame: return output_ag.to_dataframe() diff --git a/tutorials/semantic_operators.ipynb b/tutorials/semantic_operators.ipynb new file mode 100644 index 000000000..caeb0d590 --- /dev/null +++ b/tutorials/semantic_operators.ipynb @@ -0,0 +1,259 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "0", + "metadata": {}, + "source": [ + "# Semantic Operators Tutorial\n", + "\n", + "**Semantic Operators** provide declarative API for performing common data transformation tasks using natural language.\n", + "Here, we show examples of `sem_map`, `sem_filter`, and `sem_agg` that are implmented in `Agentics`." + ] + }, + { + "cell_type": "markdown", + "id": "1", + "metadata": {}, + "source": [ + "### Semantic Map\n", + "\n", + "Transform each record in your dataset according to natural language instructions, mapping source data to a target schema." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c70db513", + "metadata": {}, + "outputs": [], + "source": [ + "from agentics import AG\n", + "from agentics.core.semantic_operators import sem_map, sem_filter, sem_agg\n", + "from typing import Optional\n", + "from pprint import pprint\n", + "\n", + "import pandas as pd\n", + "from pydantic import BaseModel, Field\n", + "\n", + "my_llm = AG.get_llm_provider(\"litellm_proxy\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3", + "metadata": {}, + "outputs": [], + "source": [ + "# Sample data\n", + "df = pd.DataFrame({\n", + " 'review': [\n", + " 'This product is amazing! Best purchase ever.',\n", + " 'Terrible quality, broke after one day.',\n", + " 'It works okay, nothing special.'\n", + " ]\n", + "})\n", + "\n", + "# Define target schema\n", + "class Sentiment(BaseModel):\n", + " sentiment: Optional[str] = Field(None, description=\"The sentiment of the review (e.g., positive, negative, neutral)\")\n", + " confidence: Optional[float] = Field(None, description=\"Confidence score of the sentiment analysis between 0 and 1\")\n", + "\n", + "result = await sem_map(\n", + " source=df,\n", + " target_type=Sentiment,\n", + " instructions=\"Analyze the sentiment of the review and provide a confidence score between 0 and 1.\",\n", + " llm=my_llm)\n", + "print(result)\n", + "\n", + "result = await sem_map(\n", + " source=df,\n", + " target_type=\"category\",\n", + " instructions=\"Classify the review into one of: positive, negative, neutral\",\n", + " llm=my_llm)\n", + "print(result)" + ] + }, + { + "cell_type": "markdown", + "id": "af661f99", + "metadata": {}, + "source": [ + "### Semantic Filter\n", + "\n", + "Filter records based on a natural language predicate, keeping only those that satisfy the condition." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "34bab288", + "metadata": {}, + "outputs": [], + "source": [ + "df = pd.DataFrame({\n", + " 'product': ['Laptop', 'Phone', 'Tablet', 'Monitor'],\n", + " 'description': [\n", + " 'High-performance gaming laptop with RGB keyboard',\n", + " 'Budget smartphone with basic features',\n", + " 'Premium tablet with stylus support',\n", + " '4K monitor for professional work'\n", + " ]\n", + "})\n", + "\n", + "# Filter for premium/high-end products\n", + "result = await sem_filter(\n", + " source=df,\n", + " predicate_template=\"The product is premium or high-end\",\n", + " llm=AG.get_llm_provider(\"litellm_proxy\"),\n", + " verbose_agent=False,\n", + " verbose_transduction=False\n", + ")\n", + "\n", + "print(result)\n", + "\n", + "# Use field placeholders in the predicate\n", + "result = await sem_filter(\n", + " source=df,\n", + " predicate_template=\"The {product} described as '{description}' is suitable for gaming\",\n", + " llm=my_llm)\n", + "\n", + "print(result)" + ] + }, + { + "cell_type": "markdown", + "id": "18401986", + "metadata": {}, + "source": [ + "### Semantic Aggregator\n", + "\n", + "Aggregate data across all records to produce a summary or consolidated output." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "01175466", + "metadata": {}, + "outputs": [], + "source": [ + "df = pd.DataFrame({\n", + " 'review': [\n", + " 'Great product, very satisfied!',\n", + " 'Good quality but expensive',\n", + " 'Not worth the price',\n", + " 'Excellent, would buy again',\n", + " 'Decent but has some issues'\n", + " ]\n", + "})\n", + "\n", + "class ReviewSummary(BaseModel):\n", + " overall_sentiment: str\n", + " key_themes: list[str]\n", + " recommendation: str\n", + "\n", + "# Aggregate all reviews into a summary\n", + "result = await sem_agg(\n", + " source=df,\n", + " target_type=ReviewSummary,\n", + " instructions=\"Summarize all reviews, identify key themes, and provide an overall recommendation\",\n", + " llm=my_llm)\n", + "\n", + "pprint(result)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "461a2acb", + "metadata": {}, + "outputs": [], + "source": [ + "class Statistics(BaseModel):\n", + " total_count: int\n", + " positive_count: int\n", + " negative_count: int\n", + " average_sentiment: str\n", + "\n", + "result = await sem_agg(\n", + " source=df,\n", + " target_type=Statistics,\n", + " instructions=\"Count total reviews, positive reviews, negative reviews, and determine average sentiment\",\n", + " llm=my_llm)\n", + "pprint(result)" + ] + }, + { + "cell_type": "markdown", + "id": "2fe9d161", + "metadata": {}, + "source": [ + "### Integration with Agentics Workflows\n", + "\n", + "Chaining Semantic Operators." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9f40bfa5", + "metadata": {}, + "outputs": [], + "source": [ + "df = pd.DataFrame({\n", + " 'product': ['Laptop', 'Phone', 'Tablet', 'Monitor'],\n", + " 'description': [\n", + " 'High-performance gaming laptop with RGB keyboard',\n", + " 'Budget smartphone with basic features',\n", + " 'Premium tablet with stylus support',\n", + " '4K monitor for professional work'\n", + " ]\n", + "})\n", + "\n", + "filtered = await sem_filter(\n", + " source=df,\n", + " predicate_template=\"The product is premium or high-end\",\n", + " llm=AG.get_llm_provider(\"litellm_proxy\"),\n", + " verbose_agent=False,\n", + " verbose_transduction=False\n", + ")\n", + "pprint(filtered)\n", + "mapped = await sem_map(\n", + " source=filtered,\n", + " target_type=\"price_category\",\n", + " instructions=\"Classify the price of the product into one of: high, medium, low\", \n", + " llm=my_llm)\n", + "pprint(mapped)\n", + "result = await sem_agg(\n", + " source=mapped,\n", + " target_type=\"summary\",\n", + " instructions=\"Summarize all descriptions\",\n", + " llm=my_llm)\n", + "pprint(result)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From ac8c498cfc6e4ddabf2bcd15dcf9a133ea38e0c0 Mon Sep 17 00:00:00 2001 From: Junkyu Lee Date: Thu, 12 Feb 2026 19:25:38 -0500 Subject: [PATCH 21/27] remove specific llm provider name Signed-off-by: Junkyu Lee --- tutorials/semantic_operators.ipynb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tutorials/semantic_operators.ipynb b/tutorials/semantic_operators.ipynb index caeb0d590..823e562cb 100644 --- a/tutorials/semantic_operators.ipynb +++ b/tutorials/semantic_operators.ipynb @@ -36,7 +36,9 @@ "import pandas as pd\n", "from pydantic import BaseModel, Field\n", "\n", - "my_llm = AG.get_llm_provider(\"litellm_proxy\")" + "# Select a model \"watsonx\", \"gemini\", \"openai\", etc. \n", + "# Make sure to set up credentials for the model you choose in your environment variables.\n", + "my_llm = AG.get_llm_provider()" ] }, { From d05ae89c9e1adb2d264f5075ba47d7817c69ab0e Mon Sep 17 00:00:00 2001 From: Junkyu Lee Date: Fri, 13 Feb 2026 14:46:32 -0500 Subject: [PATCH 22/27] review and edit documentations --- docs/core_concepts.md | 9 ++-- docs/getting_started.md | 86 ---------------------------------- docs/transducible_functions.md | 1 + 3 files changed, 6 insertions(+), 90 deletions(-) diff --git a/docs/core_concepts.md b/docs/core_concepts.md index 0aafd2960..6e03052b2 100644 --- a/docs/core_concepts.md +++ b/docs/core_concepts.md @@ -20,12 +20,13 @@ You describe your data using **Pydantic models**: ```python from pydantic import BaseModel +fromp typing import Optional class Product(BaseModel): - id: str | None = None - title: str | None = None - description: str | None = None - price: float | None = None + id: Optional[str] = None + title: Optional[str] = None + description: Optional[str] = None + price: Optional[float] = None ``` These models serve three roles: diff --git a/docs/getting_started.md b/docs/getting_started.md index fa0cb5709..b234296d1 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -208,92 +208,6 @@ for product, tweet in zip(products, tweets): The same transducible function works seamlessly for both single items and batchesβ€”no code changes needed! -### Installation details - -#### Poetry - - Install poetry (skip if available) - - ```bash - curl -sSL https://install.python-poetry.org | python3 - - ``` - - Clone and install agentics - - ```bash - - poetry install - source $(poetry env info --path)/bin/activate - ``` - -#### Python - - > Ensure you have Python 3.11+ 🚨. - > - > ```shell - > python --version - > ``` - - * Create a virtual environment with Python's built in `venv` module. In linux, this - package may be required to be installed with the Operating System package manager. - ```shell - python -m venv .venv - ``` - - * Activate the virtual environment - - `source .venv/bin/activate` # Bash/Zsh - `source .venv/bin/activate.fish` # Fish - - * Install the package - ```bash - python -m pip install ./agentics - ``` - - -#### uv - - * Ensure `uv` is installed. - ```bash - command -v uv >/dev/null && curl -LsSf https://astral.sh/uv/install.sh | sh - # It's recommended to restart the shell afterwards - exec $SHELL - ``` - * `uv venv --python 3.11` - * `uv pip install ./agentics` or `uv add ./agentics` (recommended) - - -#### uvx πŸƒπŸ½ - - > This is a way to run agentics temporarily or quick tests - - * Ensure `uv` is installed. - ```bash - command -v uv >/dev/null && curl -LsSf https://astral.sh/uv/install.sh | sh - # It's recommended to restart the shell afterwards - exec $SHELL - ``` - * uvx --verbose --from ./agentics ipython - - -#### Conda - - 1. Create a conda environment: - ```bash - conda create -n agentics python=3.11 - ``` - In this example the name of the environment is `agetnics` but you can change - it to your personal preference. - - - 2. Activate the environment - ```bash - conda activate agentics - ``` - 3. Install `agentics` from a folder or git reference - ```bash - pip install ./agentics - ``` ## Next - πŸ‘‰ [Core Concepts](core_concepts.md) - Understanding the theoretical foundation diff --git a/docs/transducible_functions.md b/docs/transducible_functions.md index faff3ffd5..b58127f54 100644 --- a/docs/transducible_functions.md +++ b/docs/transducible_functions.md @@ -617,6 +617,7 @@ Good reasons to define a separate transducible function: --- ## Next +- πŸ‘‰ **[Transducible Functions Tutorial](../tutorials/transducible_functions.ipynb)** to see how transducible functions works in practice - πŸ‘‰ **[Map-Reduce Operations](map_reduce.md)** - Scaling with map and reduce, batch processing patterns ## Go to Index From e0c06b069c147cd244638e69f6280bc66f82f655 Mon Sep 17 00:00:00 2001 From: Junkyu Lee Date: Fri, 13 Feb 2026 14:48:10 -0500 Subject: [PATCH 23/27] multiple issues that need to be fixed; default llm provider setting, error outputs are stored in each notebook Signed-off-by: Junkyu Lee --- tutorials/agentics.ipynb | 2 +- tutorials/atypes.ipynb | 202 +++++++++- tutorials/logical_transduction_algebra.ipynb | 175 +++++++- tutorials/map_reduce.ipynb | 401 ++++++++++++++++++- tutorials/synthetic_data_generation.ipynb | 15 +- tutorials/transducible_functions.ipynb | 21 +- 6 files changed, 748 insertions(+), 68 deletions(-) diff --git a/tutorials/agentics.ipynb b/tutorials/agentics.ipynb index 45e47c2f9..9b0d7c240 100644 --- a/tutorials/agentics.ipynb +++ b/tutorials/agentics.ipynb @@ -221,7 +221,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.14" + "version": "3.12.12" } }, "nbformat": 4, diff --git a/tutorials/atypes.ipynb b/tutorials/atypes.ipynb index ab18b724f..20d8128d6 100644 --- a/tutorials/atypes.ipynb +++ b/tutorials/atypes.ipynb @@ -31,7 +31,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 6, "id": "1", "metadata": {}, "outputs": [], @@ -84,13 +84,22 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "id": "3", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'genre': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'movie_name': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'description': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'year': FieldInfo(annotation=Union[int, NoneType], required=False, default=None)}\n" + ] + } + ], "source": [ "from typing import Optional, List\n", "from pydantic import BaseModel, Field\n", + "import agentics\n", "\n", "class Movie(BaseModel):\n", " movie_name: Optional[str] = Field(None, description=\"Movie title.\")\n", @@ -130,10 +139,18 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "id": "5", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'target': FieldInfo(annotation=Union[Summary, NoneType], required=False, default=None), 'source': FieldInfo(annotation=Union[Email, NoneType], required=False, default=None)}\n" + ] + } + ], "source": [ "class Email(BaseModel):\n", " to: Optional[str] = Field(None, description=\"Recipient name or email.\")\n", @@ -169,10 +186,18 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 8, "id": "7", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'target': FieldInfo(annotation=Union[Summary, NoneType], required=False, default=None), 'source': FieldInfo(annotation=Union[EmailAndSentiment, NoneType], required=False, default=None)}\n" + ] + } + ], "source": [ "class Sentiment(BaseModel):\n", " sentiment: Optional[str] = Field(\n", @@ -198,10 +223,19 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 9, "id": "9", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "answer='Paris is the capital of France.' confidence=0.95\n", + "name='Alice' age=None\n" + ] + } + ], "source": [ "from agentics.core.atype import create_pydantic_model\n", "Answer = create_pydantic_model([(\"answer\", \"str\", \"the answer for the question\",False), (\"confidence\", \"float\", None,False)])\n", @@ -244,10 +278,62 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 10, "id": "11", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\n", + " \"student_name\": \"Alice Thompson\",\n", + " \"student_age\": 20,\n", + " \"enrolled_courses\": [\n", + " \"Introduction to Psychology\",\n", + " \"Calculus I\",\n", + " \"Modern World History\"\n", + " ]\n", + "}\n", + "{\n", + " \"student_name\": \"Marcus Rodriguez\",\n", + " \"student_age\": 22,\n", + " \"enrolled_courses\": [\n", + " \"Organic Chemistry\",\n", + " \"Cell Biology\",\n", + " \"Physics for Life Sciences\"\n", + " ]\n", + "}\n", + "{\n", + " \"student_name\": \"Sophia Chen\",\n", + " \"student_age\": 19,\n", + " \"enrolled_courses\": [\n", + " \"Macroeconomics\",\n", + " \"Business Ethics\",\n", + " \"Statistics for Data Science\"\n", + " ]\n", + "}\n", + "{\n", + " \"student_name\": \"Julian Banks\",\n", + " \"student_age\": 21,\n", + " \"enrolled_courses\": [\n", + " \"Digital Marketing\",\n", + " \"Consumer Behavior\",\n", + " \"Creative Writing\"\n", + " ]\n", + "}\n", + "{\n", + " \"student_name\": \"Elena Petrova\",\n", + " \"student_age\": 23,\n", + " \"enrolled_courses\": [\n", + " \"Artificial Intelligence\",\n", + " \"Software Engineering\",\n", + " \"Discrete Mathematics\"\n", + " ]\n", + "}\n" + ] + } + ], "source": [ "from agentics.core.transducible_functions import generate_prototypical_instances\n", "\n", @@ -282,10 +368,82 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 11, "id": "13", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "2026-02-13 14:43:11.581 | DEBUG | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 1\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "2026-02-13 14:43:11.682 | DEBUG | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 2\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "2026-02-13 14:43:11.798 | DEBUG | agentics.core.agentics:__lshift__:650 - Error: 1 states have not been transduced\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n" + ] + }, + { + "ename": "ValidationError", + "evalue": "1 validation error for GeneratedAtype\nmethods\n Input should be a valid list [type=list_type, input_value=None, input_type=NoneType]\n For further information visit https://errors.pydantic.dev/2.12/v/list_type", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mValidationError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[11]\u001b[39m\u001b[32m, line 2\u001b[39m\n\u001b[32m 1\u001b[39m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01magentics\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mcore\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mtransducible_functions\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m generate_atype_from_description\n\u001b[32m----> \u001b[39m\u001b[32m2\u001b[39m person = \u001b[38;5;28;01mawait\u001b[39;00m generate_atype_from_description(\u001b[33m\"\u001b[39m\u001b[33mPerson with demographic information including name, age, email and birthplace\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 3\u001b[39m citizenship = \u001b[38;5;28;01mawait\u001b[39;00m generate_atype_from_description(\u001b[33m\"\u001b[39m\u001b[33mCitizenship information including country of citizenship and visa status\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 4\u001b[39m \u001b[38;5;28mprint\u001b[39m(person.model_dump_json(indent=\u001b[32m2\u001b[39m))\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:609\u001b[39m, in \u001b[36mgenerate_atype_from_description\u001b[39m\u001b[34m(description, retry)\u001b[39m\n\u001b[32m 607\u001b[39m i = \u001b[32m0\u001b[39m\n\u001b[32m 608\u001b[39m \u001b[38;5;28;01mwhile\u001b[39;00m i < retry:\n\u001b[32m--> \u001b[39m\u001b[32m609\u001b[39m generated_atype_ag = \u001b[38;5;28;01mawait\u001b[39;00m (\n\u001b[32m 610\u001b[39m AG(\n\u001b[32m 611\u001b[39m atype=GeneratedAtype,\n\u001b[32m 612\u001b[39m instructions=\u001b[33m\"\"\"\u001b[39m\n\u001b[32m 613\u001b[39m \u001b[33mGenerate python code for the input natural-language type specification.\u001b[39m\n\u001b[32m 614\u001b[39m \n\u001b[32m 615\u001b[39m \u001b[33mRequirements:\u001b[39m\n\u001b[32m 616\u001b[39m \u001b[33m- Define exactly ONE Pydantic BaseModel.\u001b[39m\n\u001b[32m 617\u001b[39m \u001b[33m- Make all fields Optional.\u001b[39m\n\u001b[32m 618\u001b[39m \u001b[33m- Use only primitive types for the fields (str, int, float, bool, list[str], etc.).\u001b[39m\n\u001b[32m 619\u001b[39m \u001b[33m- Avoid nested Pydantic models.\u001b[39m\n\u001b[32m 620\u001b[39m \u001b[33m- Provide descriptions for the class and all its fields using:\u001b[39m\n\u001b[32m 621\u001b[39m \u001b[33m Field(None, description=\u001b[39m\u001b[33m\"\u001b[39m\u001b[33m...\u001b[39m\u001b[33m\"\u001b[39m\u001b[33m)\u001b[39m\n\u001b[32m 622\u001b[39m \u001b[33m- If the input is a question, generate a Pydantic type that can represent\u001b[39m\n\u001b[32m 623\u001b[39m \u001b[33m the answer to that question.\u001b[39m\n\u001b[32m 624\u001b[39m \u001b[33mReturn ONLY valid Python V2 code in `python_code`.\u001b[39m\n\u001b[32m 625\u001b[39m \u001b[33m\"\"\"\u001b[39m,\n\u001b[32m 626\u001b[39m )\n\u001b[32m 627\u001b[39m << description\n\u001b[32m 628\u001b[39m )\n\u001b[32m 630\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m generated_atype_ag.states \u001b[38;5;129;01mand\u001b[39;00m generated_atype_ag[\u001b[32m0\u001b[39m]:\n\u001b[32m 631\u001b[39m generated_atype_ag[\u001b[32m0\u001b[39m].atype = import_pydantic_from_code(\n\u001b[32m 632\u001b[39m generated_atype_ag[\u001b[32m0\u001b[39m].python_code\n\u001b[32m 633\u001b[39m )\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/agentics.py:681\u001b[39m, in \u001b[36mAG.__lshift__\u001b[39m\u001b[34m(self, other)\u001b[39m\n\u001b[32m 679\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mrange\u001b[39m(\u001b[38;5;28mlen\u001b[39m(other)):\n\u001b[32m 680\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(output_states[i], \u001b[38;5;28mself\u001b[39m.atype):\n\u001b[32m--> \u001b[39m\u001b[32m681\u001b[39m output.states.append(\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43matype\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43moutput_states\u001b[49m\u001b[43m[\u001b[49m\u001b[43mi\u001b[49m\u001b[43m]\u001b[49m\u001b[43m.\u001b[49m\u001b[43mmodel_dump\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m)\n\u001b[32m 682\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m 683\u001b[39m output.states.append(\u001b[38;5;28mself\u001b[39m.atype())\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/.venv/lib/python3.12/site-packages/pydantic/main.py:250\u001b[39m, in \u001b[36mBaseModel.__init__\u001b[39m\u001b[34m(self, **data)\u001b[39m\n\u001b[32m 248\u001b[39m \u001b[38;5;66;03m# `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks\u001b[39;00m\n\u001b[32m 249\u001b[39m __tracebackhide__ = \u001b[38;5;28;01mTrue\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m250\u001b[39m validated_self = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m__pydantic_validator__\u001b[49m\u001b[43m.\u001b[49m\u001b[43mvalidate_python\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdata\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mself_instance\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[32m 251\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m validated_self:\n\u001b[32m 252\u001b[39m warnings.warn(\n\u001b[32m 253\u001b[39m \u001b[33m'\u001b[39m\u001b[33mA custom validator is returning a value other than `self`.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33m'\u001b[39m\n\u001b[32m 254\u001b[39m \u001b[33m\"\u001b[39m\u001b[33mReturning anything other than `self` from a top level model validator isn\u001b[39m\u001b[33m'\u001b[39m\u001b[33mt supported when validating via `__init__`.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33m\"\u001b[39m\n\u001b[32m 255\u001b[39m \u001b[33m'\u001b[39m\u001b[33mSee the `model_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.\u001b[39m\u001b[33m'\u001b[39m,\n\u001b[32m 256\u001b[39m stacklevel=\u001b[32m2\u001b[39m,\n\u001b[32m 257\u001b[39m )\n", + "\u001b[31mValidationError\u001b[39m: 1 validation error for GeneratedAtype\nmethods\n Input should be a valid list [type=list_type, input_value=None, input_type=NoneType]\n For further information visit https://errors.pydantic.dev/2.12/v/list_type" + ] + } + ], "source": [ "from agentics.core.transducible_functions import generate_atype_from_description\n", "person = await generate_atype_from_description(\"Person with demographic information including name, age, email and birthplace\")\n", @@ -320,10 +478,22 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 12, "id": "15", "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "NameError", + "evalue": "name 'citizenship' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mNameError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[12]\u001b[39m\u001b[32m, line 2\u001b[39m\n\u001b[32m 1\u001b[39m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01magentics\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mcore\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mtransducible_functions\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m estimateLogicalProximity\n\u001b[32m----> \u001b[39m\u001b[32m2\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;28;01mawait\u001b[39;00m estimateLogicalProximity(\u001b[43mcitizenship\u001b[49m.atype << person.atype))\n", + "\u001b[31mNameError\u001b[39m: name 'citizenship' is not defined" + ] + } + ], "source": [ "from agentics.core.transducible_functions import estimateLogicalProximity\n", "print(await estimateLogicalProximity(citizenship.atype << person.atype))\n" @@ -375,7 +545,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.14" + "version": "3.12.12" } }, "nbformat": 4, diff --git a/tutorials/logical_transduction_algebra.ipynb b/tutorials/logical_transduction_algebra.ipynb index f45d6d18e..751f12064 100644 --- a/tutorials/logical_transduction_algebra.ipynb +++ b/tutorials/logical_transduction_algebra.ipynb @@ -19,7 +19,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "id": "1", "metadata": {}, "outputs": [], @@ -58,10 +58,22 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "id": "3", "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "AttributeError", + "evalue": "'tuple' object has no attribute 'model_dump_json'", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mAttributeError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[2]\u001b[39m\u001b[32m, line 6\u001b[39m\n\u001b[32m 4\u001b[39m write_tweet = Email << GenericInput\n\u001b[32m 5\u001b[39m tweet = \u001b[38;5;28;01mawait\u001b[39;00m write_tweet(\u001b[38;5;28minput\u001b[39m)\n\u001b[32m----> \u001b[39m\u001b[32m6\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[43mtweet\u001b[49m\u001b[43m.\u001b[49m\u001b[43mmodel_dump_json\u001b[49m(indent=\u001b[32m2\u001b[39m))\n", + "\u001b[31mAttributeError\u001b[39m: 'tuple' object has no attribute 'model_dump_json'" + ] + } + ], "source": [ "input = GenericInput(content=\n", " \"\"\"Zoran Mamdani become the NYC mayor\"\"\")\n", @@ -104,10 +116,24 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, "id": "5", "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "TypeError", + "evalue": "transducible() got an unexpected keyword argument 'prompyt_template'", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mTypeError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[3]\u001b[39m\u001b[32m, line 1\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m1\u001b[39m write_mail_to_alfio = \u001b[43mEmail\u001b[49m\u001b[43m<<\u001b[49m\u001b[43m \u001b[49m\u001b[43mWith\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 2\u001b[39m \u001b[43m \u001b[49m\u001b[43mGenericInput\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 3\u001b[39m \u001b[43m \u001b[49m\u001b[43minstructions\u001b[49m\u001b[43m=\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mWrite an email to Alfio Gliozzo\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[32m 4\u001b[39m \u001b[43m \u001b[49m\u001b[43mprompyt_template\u001b[49m\u001b[43m=\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[38;5;132;43;01m{content}\u001b[39;49;00m\u001b[33;43m\"\u001b[39;49m\u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 5\u001b[39m news = GenericInput(content=\u001b[33m\"\u001b[39m\u001b[33mZoran Mandani won the Election in NYC\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 6\u001b[39m mail = \u001b[38;5;28;01mawait\u001b[39;00m write_mail_to_alfio(news)\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:464\u001b[39m, in \u001b[36m_model_lshift\u001b[39m\u001b[34m(OutputModel, InputType)\u001b[39m\n\u001b[32m 462\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(InputType, TransductionConfig):\n\u001b[32m 463\u001b[39m M = InputType.model\n\u001b[32m--> \u001b[39m\u001b[32m464\u001b[39m f = \u001b[43mmake_transducible_function\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 465\u001b[39m \u001b[43m \u001b[49m\u001b[43mInputModel\u001b[49m\u001b[43m=\u001b[49m\u001b[43mM\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 466\u001b[39m \u001b[43m \u001b[49m\u001b[43mOutputModel\u001b[49m\u001b[43m=\u001b[49m\u001b[43mOutputModel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 467\u001b[39m \u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mInputType\u001b[49m\u001b[43m.\u001b[49m\u001b[43mconfig\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 468\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 469\u001b[39m f.\u001b[34m__lshift__\u001b[39m = types.MethodType(_function_lshift, f)\n\u001b[32m 470\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m _wrap_composed(f)\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:367\u001b[39m, in \u001b[36mmake_transducible_function\u001b[39m\u001b[34m(InputModel, OutputModel, function_code, instructions, name, **kwargs)\u001b[39m\n\u001b[32m 364\u001b[39m _auto_fn.\u001b[34m__doc__\u001b[39m = instructions\n\u001b[32m 366\u001b[39m \u001b[38;5;66;03m# Delegate everything to your existing decorator\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m367\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m _wrap_composed(\u001b[43mtransducible\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m(_auto_fn))\n", + "\u001b[31mTypeError\u001b[39m: transducible() got an unexpected keyword argument 'prompyt_template'" + ] + } + ], "source": [ "write_mail_to_alfio = Email<< With(\n", " GenericInput,\n", @@ -139,10 +165,22 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "id": "7", "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "NameError", + "evalue": "name 'write_mail_to_alfio' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mNameError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[4]\u001b[39m\u001b[32m, line 4\u001b[39m\n\u001b[32m 1\u001b[39m news = GenericInput(content=\u001b[33m\"\u001b[39m\u001b[33mZoran Mandani won the Election in NYC, make up a story about that.\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m----> \u001b[39m\u001b[32m4\u001b[39m summary_composite_1 = Summary << \u001b[43mwrite_mail_to_alfio\u001b[49m\n\u001b[32m 5\u001b[39m summary = \u001b[38;5;28;01mawait\u001b[39;00m summary_composite_1(news)\n\u001b[32m 6\u001b[39m \u001b[38;5;28mprint\u001b[39m(summary.model_dump_json(indent=\u001b[32m2\u001b[39m))\n", + "\u001b[31mNameError\u001b[39m: name 'write_mail_to_alfio' is not defined" + ] + } + ], "source": [ "news = GenericInput(content=\"Zoran Mandani won the Election in NYC, make up a story about that.\")\n", "\n", @@ -158,10 +196,18 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "id": "8", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2026-02-13 14:45:29.771 | DEBUG | agentics.core.agentics:from_csv:831 - Importing Agentics of type Movie from CSV str\n" + ] + } + ], "source": [ "\n", "from agentics import AG\n", @@ -190,10 +236,22 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 6, "id": "9", "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "TypeError", + "evalue": "unsupported operand type(s) for @: 'tuple' and 'Movie'", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mTypeError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[6]\u001b[39m\u001b[32m, line 3\u001b[39m\n\u001b[32m 1\u001b[39m classify_genre= Genre << Movie\n\u001b[32m 2\u001b[39m genre = \u001b[38;5;28;01mawait\u001b[39;00m classify_genre(movie)\n\u001b[32m----> \u001b[39m\u001b[32m3\u001b[39m \u001b[38;5;28mprint\u001b[39m((\u001b[43mgenre\u001b[49m\u001b[43m \u001b[49m\u001b[43m@\u001b[49m\u001b[43m \u001b[49m\u001b[43mmovie\u001b[49m).model_dump_json(indent=\u001b[32m2\u001b[39m))\n", + "\u001b[31mTypeError\u001b[39m: unsupported operand type(s) for @: 'tuple' and 'Movie'" + ] + } + ], "source": [ "classify_genre= Genre << Movie\n", "genre = await classify_genre(movie)\n", @@ -202,20 +260,105 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "id": "10", "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "(Genre(genre='crime'), None)" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "await (Genre << movie)" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 8, "id": "11", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "2026-02-13 14:45:40.197 | DEBUG | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 1\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "2026-02-13 14:45:40.304 | DEBUG | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 2\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n", + "2026-02-13 14:45:40.402 | DEBUG | agentics.core.agentics:__lshift__:650 - Error: 1 states have not been transduced\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n", + "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n", + "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n" + ] + }, + { + "ename": "ValidationError", + "evalue": "1 validation error for Explanation\nrelevant_source_attributes\n Input should be a valid dictionary [type=dict_type, input_value=[], input_type=list]\n For further information visit https://errors.pydantic.dev/2.12/v/dict_type", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mValidationError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[8]\u001b[39m\u001b[32m, line 3\u001b[39m\n\u001b[32m 1\u001b[39m classify_genre= Genre << With(Movie,\n\u001b[32m 2\u001b[39m provide_explanation=\u001b[38;5;28;01mTrue\u001b[39;00m)\n\u001b[32m----> \u001b[39m\u001b[32m3\u001b[39m genre, explanation = \u001b[38;5;28;01mawait\u001b[39;00m classify_genre(movie)\n\u001b[32m 4\u001b[39m \u001b[38;5;28mprint\u001b[39m(genre.model_dump_json(indent=\u001b[32m2\u001b[39m))\n\u001b[32m 5\u001b[39m \u001b[38;5;28mprint\u001b[39m(explanation.model_dump_json(indent=\u001b[32m2\u001b[39m))\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:291\u001b[39m, in \u001b[36mtransducible.._transducible..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 289\u001b[39m \u001b[38;5;66;03m# AMAP\u001b[39;00m\n\u001b[32m 290\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(\u001b[38;5;28minput\u001b[39m, (SourceModel, Transduce)):\n\u001b[32m--> \u001b[39m\u001b[32m291\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m wrap_single(\u001b[38;5;28minput\u001b[39m)\n\u001b[32m 292\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(\u001b[38;5;28minput\u001b[39m, AG):\n\u001b[32m 293\u001b[39m \u001b[38;5;28minput\u001b[39m = \u001b[38;5;28minput\u001b[39m.states\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:222\u001b[39m, in \u001b[36mtransducible.._transducible..wrap_single\u001b[39m\u001b[34m(input_obj)\u001b[39m\n\u001b[32m 219\u001b[39m source_ag = source_ag_template.clone()\n\u001b[32m 220\u001b[39m source_ag.states = [pre.object]\n\u001b[32m--> \u001b[39m\u001b[32m222\u001b[39m target_ag = \u001b[38;5;28;01mawait\u001b[39;00m (target_ag_template << source_ag)\n\u001b[32m 224\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(target_ag) == \u001b[32m1\u001b[39m:\n\u001b[32m 225\u001b[39m out = target_ag[\u001b[32m0\u001b[39m]\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/agentics.py:699\u001b[39m, in \u001b[36mAG.__lshift__\u001b[39m\u001b[34m(self, other)\u001b[39m\n\u001b[32m 691\u001b[39m output.transduce_fields = \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[32m 692\u001b[39m target_explanation.instructions = \u001b[33mf\u001b[39m\u001b[33m\"\"\"\u001b[39m\n\u001b[32m 693\u001b[39m \u001b[33mYou have previously transduced an object of type \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mother.atype.\u001b[34m__name__\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m (source) from an object of type \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m.atype.\u001b[34m__name__\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m (target).\u001b[39m\n\u001b[32m 694\u001b[39m \u001b[33mNow look back at both objects and provide a detailed explanation on how each field of the target object was logically derived from the source object.\u001b[39m\n\u001b[32m (...)\u001b[39m\u001b[32m 697\u001b[39m \u001b[33mProvide high confidence score (1.0) only if you are certain that the transduction is logically correct and fully supported by the source object.\u001b[39m\n\u001b[32m 698\u001b[39m \u001b[33m\u001b[39m\u001b[33m\"\"\"\u001b[39m\n\u001b[32m--> \u001b[39m\u001b[32m699\u001b[39m explanation = \u001b[38;5;28;01mawait\u001b[39;00m (target_explanation << output.compose_states(other))\n\u001b[32m 701\u001b[39m \u001b[38;5;28mself\u001b[39m.explanations = explanation.states\n\u001b[32m 702\u001b[39m \u001b[38;5;28mself\u001b[39m.states = output.states\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/agentics.py:674\u001b[39m, in \u001b[36mAG.__lshift__\u001b[39m\u001b[34m(self, other)\u001b[39m\n\u001b[32m 672\u001b[39m allowed = \u001b[38;5;28mself\u001b[39m.atype.model_fields.keys() \u001b[38;5;66;03m# pydantic v2\u001b[39;00m\n\u001b[32m 673\u001b[39m filtered = {k: v \u001b[38;5;28;01mfor\u001b[39;00m k, v \u001b[38;5;129;01min\u001b[39;00m data.items() \u001b[38;5;28;01mif\u001b[39;00m k \u001b[38;5;129;01min\u001b[39;00m allowed}\n\u001b[32m--> \u001b[39m\u001b[32m674\u001b[39m merged = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43matype\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mfiltered\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 676\u001b[39m output.states.append(merged)\n\u001b[32m 677\u001b[39m \u001b[38;5;66;03m# elif is_str_or_list_of_str(other):\u001b[39;00m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/.venv/lib/python3.12/site-packages/pydantic/main.py:250\u001b[39m, in \u001b[36mBaseModel.__init__\u001b[39m\u001b[34m(self, **data)\u001b[39m\n\u001b[32m 248\u001b[39m \u001b[38;5;66;03m# `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks\u001b[39;00m\n\u001b[32m 249\u001b[39m __tracebackhide__ = \u001b[38;5;28;01mTrue\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m250\u001b[39m validated_self = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m__pydantic_validator__\u001b[49m\u001b[43m.\u001b[49m\u001b[43mvalidate_python\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdata\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mself_instance\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[32m 251\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m validated_self:\n\u001b[32m 252\u001b[39m warnings.warn(\n\u001b[32m 253\u001b[39m \u001b[33m'\u001b[39m\u001b[33mA custom validator is returning a value other than `self`.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33m'\u001b[39m\n\u001b[32m 254\u001b[39m \u001b[33m\"\u001b[39m\u001b[33mReturning anything other than `self` from a top level model validator isn\u001b[39m\u001b[33m'\u001b[39m\u001b[33mt supported when validating via `__init__`.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33m\"\u001b[39m\n\u001b[32m 255\u001b[39m \u001b[33m'\u001b[39m\u001b[33mSee the `model_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.\u001b[39m\u001b[33m'\u001b[39m,\n\u001b[32m 256\u001b[39m stacklevel=\u001b[32m2\u001b[39m,\n\u001b[32m 257\u001b[39m )\n", + "\u001b[31mValidationError\u001b[39m: 1 validation error for Explanation\nrelevant_source_attributes\n Input should be a valid dictionary [type=dict_type, input_value=[], input_type=list]\n For further information visit https://errors.pydantic.dev/2.12/v/dict_type" + ] + } + ], "source": [ "classify_genre= Genre << With(Movie,\n", " provide_explanation=True)\n", @@ -241,7 +384,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.14" + "version": "3.12.12" } }, "nbformat": 4, diff --git a/tutorials/map_reduce.ipynb b/tutorials/map_reduce.ipynb index 414336b79..71eac179c 100644 --- a/tutorials/map_reduce.ipynb +++ b/tutorials/map_reduce.ipynb @@ -21,18 +21,26 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "id": "1", "metadata": {}, "outputs": [], "source": [ - "from pydantic import BaseModel\n", "from typing import Optional\n", - "import agentics.core.transducible_functions \n", + "from agentics import AG\n", + "from pydantic import BaseModel, Field\n", + "from dotenv import load_dotenv\n", + "\n", + "# Load .env from project root (current working directory)\n", + "load_dotenv()\n", + "\n", + "llm=AG.get_llm_provider(\"litellm_proxy\")\n", + "\n", "class Number(BaseModel):\n", - " number:Optional[int]\n", + " number:Optional[int] = Field(None, description=\"An integer number\")\n", + "\n", "class RomanNumber(BaseModel):\n", - " roman_number:Optional[str] =None" + " roman_number:Optional[str] = Field(None, description=\"The Roman numeral representation of the number\")" ] }, { @@ -55,14 +63,52 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "id": "3", "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "bf4ad4f61c4a42fe9d28e36d6cb78084", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Output()" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n"
+      ],
+      "text/plain": []
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "(RomanNumber(roman_number='I'), None)\n",
+      "(RomanNumber(roman_number='II'), None)\n",
+      "(RomanNumber(roman_number='III'), None)\n",
+      "(RomanNumber(roman_number='IV'), None)\n"
+     ]
+    }
+   ],
    "source": [
+    "from agentics.core.transducible_functions import transducible, Transduce\n",
+    "\n",
     "to_roman_number = RomanNumber << Number\n",
     "roman_numbers = await to_roman_number([Number(number=i) for i in range(1,5)])\n",
-    "for roman in roman_numbers: print(roman)"
+    "for roman in roman_numbers: \n",
+    "    print(roman)"
    ]
   },
   {
@@ -84,25 +130,145 @@
     "\n",
     "Key points:\n",
     "- The call to to_roman_number([...]) still returns a list[RomanNumber].\n",
-    "- Additionally, every result is appended to /tmp/roman_numbers.jsonl as line-delimited JSON.\n",
     "- If the file already exists, new results are appended, not overwritten."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 7,
    "id": "5",
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "7452adfa257d4262b574d89da740ba9f",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Output()"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "
\n"
+      ],
+      "text/plain": []
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "572e450fb0d740059c69b17af2eb6155",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Output()"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "
\n"
+      ],
+      "text/plain": []
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "e9d1a4c4d53044f783b3479730c5226f",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Output()"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "
\n"
+      ],
+      "text/plain": []
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "c9b51140cd3f4f15930fb83422560039",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Output()"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "
\n"
+      ],
+      "text/plain": []
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "(RomanNumber(roman_number='I'), None)\n",
+      "(RomanNumber(roman_number='II'), None)\n",
+      "(RomanNumber(roman_number='III'), None)\n",
+      "(RomanNumber(roman_number='IV'), None)\n",
+      "(RomanNumber(roman_number='V'), None)\n",
+      "(RomanNumber(roman_number='VI'), None)\n",
+      "(RomanNumber(roman_number='VII'), None)\n",
+      "(RomanNumber(roman_number='VIII'), None)\n",
+      "(RomanNumber(roman_number='IX'), None)\n",
+      "(RomanNumber(roman_number='X'), None)\n",
+      "(RomanNumber(roman_number='XI'), None)\n",
+      "(RomanNumber(roman_number='XII'), None)\n",
+      "(RomanNumber(roman_number='XIII'), None)\n",
+      "(RomanNumber(roman_number='XIV'), None)\n",
+      "(RomanNumber(roman_number='XV'), None)\n",
+      "(RomanNumber(roman_number='XVI'), None)\n",
+      "(RomanNumber(roman_number='XVII'), None)\n",
+      "(RomanNumber(roman_number='XVIII'), None)\n",
+      "(RomanNumber(roman_number='XIX'), None)\n"
+     ]
+    }
+   ],
    "source": [
     "from agentics.core.transducible_functions import With\n",
     "to_roman_number = RomanNumber << With(Number,\n",
     "                                      batch_size = 5,\n",
-    "                                      timeout = 20,\n",
-    "                                      persist_output = \"/tmp/roman_numbers.jsonl\")\n",
+    "                                      timeout = 20)\n",
     "roman_numbers = await to_roman_number([Number(number=i) for i in range(1,20)])\n",
-    "print(open(\"/tmp/roman_numbers.jsonl\").read())\n",
-    "pass"
+    "for roman_number in roman_numbers: \n",
+    "    print(roman_number)"
    ]
   },
   {
@@ -130,10 +296,195 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 8,
    "id": "7",
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "20dfa43c7826436da08de3f409cee78c",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Output()"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "
\n"
+      ],
+      "text/plain": []
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "bfc7d63de00f4dcdba7411fe1ac9e684",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Output()"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "
\n"
+      ],
+      "text/plain": []
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2026-02-13 14:39:53.744 | DEBUG    | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 1\n",
+      "2026-02-13 14:39:53.744 | DEBUG    | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 2\n",
+      "2026-02-13 14:39:53.745 | DEBUG    | agentics.core.agentics:amap:446 - ⚠️ AMAP generated an error processing state # 0: 19 validation errors for ReducedOtherAtype\n",
+      "reduced_other_states.0\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"roman_number='I'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.1\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom..._number='II'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.2\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom...number='III'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.3\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom..._number='IV'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.4\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"roman_number='V'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.5\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom..._number='VI'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.6\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom...number='VII'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.7\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom...umber='VIII'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.8\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom..._number='IX'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.9\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"roman_number='X'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.10\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom..._number='XI'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.11\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom...number='XII'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.12\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom...umber='XIII'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.13\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom...number='XIV'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.14\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom..._number='XV'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.15\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom...number='XVI'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.16\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom...umber='XVII'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.17\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom...mber='XVIII'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
+      "reduced_other_states.18\n",
+      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
+      "  \"rom...number='XIX'\",\n",
+      "  null\n",
+      "]), input_type=Transduce]\n",
+      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n"
+     ]
+    },
+    {
+     "ename": "AttributeError",
+     "evalue": "'list' object has no attribute 'model_dump'",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
+      "\u001b[31mAttributeError\u001b[39m                            Traceback (most recent call last)",
+      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[8]\u001b[39m\u001b[32m, line 6\u001b[39m\n\u001b[32m      1\u001b[39m to_number = Number << RomanNumber\n\u001b[32m      2\u001b[39m reduce = Number << With(Number, \n\u001b[32m      3\u001b[39m     areduce=\u001b[38;5;28;01mTrue\u001b[39;00m,\n\u001b[32m      4\u001b[39m     instructions=\u001b[33m\"\u001b[39m\u001b[33mreturn the sum of the input numbers\u001b[39m\u001b[33m'\u001b[39m\u001b[33m number fields\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m----> \u001b[39m\u001b[32m6\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m reduce(\u001b[38;5;28;01mawait\u001b[39;00m to_number(roman_numbers ))\n",
+      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:273\u001b[39m, in \u001b[36mtransducible.._transducible..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m    270\u001b[39m source_ag = source_ag_template.clone()\n\u001b[32m    271\u001b[39m source_ag.states = [\u001b[38;5;28minput\u001b[39m]\n\u001b[32m--> \u001b[39m\u001b[32m273\u001b[39m intermediate = \u001b[38;5;28;01mawait\u001b[39;00m source_ag.amap(wrap_single)\n\u001b[32m    275\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(intermediate) == \u001b[32m1\u001b[39m:\n\u001b[32m    276\u001b[39m     \u001b[38;5;28;01mreturn\u001b[39;00m intermediate[\u001b[32m0\u001b[39m]\n",
+      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/agentics.py:450\u001b[39m, in \u001b[36mAG.amap\u001b[39m\u001b[34m(self, func, timeout)\u001b[39m\n\u001b[32m    446\u001b[39m     logger.debug(\n\u001b[32m    447\u001b[39m         \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33m⚠️ AMAP generated an error processing state # \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mi\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mresult\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m\"\u001b[39m\n\u001b[32m    448\u001b[39m     )\n\u001b[32m    449\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m i < \u001b[38;5;28mlen\u001b[39m(\u001b[38;5;28mself\u001b[39m.states):\n\u001b[32m--> \u001b[39m\u001b[32m450\u001b[39m     _states.append(\u001b[38;5;28mself\u001b[39m.atype(**\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mstates\u001b[49m\u001b[43m[\u001b[49m\u001b[43mi\u001b[49m\u001b[43m]\u001b[49m\u001b[43m.\u001b[49m\u001b[43mmodel_dump\u001b[49m()))\n\u001b[32m    451\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m    452\u001b[39m     _states.append(\u001b[38;5;28mself\u001b[39m.atype())\n",
+      "\u001b[31mAttributeError\u001b[39m: 'list' object has no attribute 'model_dump'"
+     ]
+    }
+   ],
    "source": [
     "to_number = Number << RomanNumber\n",
     "reduce = Number << With(Number, \n",
@@ -159,10 +510,20 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 9,
    "id": "9",
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "number=100\n",
+      "number=100\n",
+      "number=100\n"
+     ]
+    }
+   ],
    "source": [
     "from agentics.core.transducible_functions import transducible, Transduce\n",
     "@transducible(areduce=True)\n",
@@ -203,7 +564,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.14"
+   "version": "3.12.12"
   }
  },
  "nbformat": 4,
diff --git a/tutorials/synthetic_data_generation.ipynb b/tutorials/synthetic_data_generation.ipynb
index 676a253ad..bbb247ccc 100644
--- a/tutorials/synthetic_data_generation.ipynb
+++ b/tutorials/synthetic_data_generation.ipynb
@@ -29,6 +29,13 @@
     "from pydantic import BaseModel, Field\n",
     "from typing import Optional\n",
     "from agentics.core.transducible_functions import generate_prototypical_instances\n",
+    "from agentics import AG\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "# Load .env from project root (current working directory)\n",
+    "load_dotenv()\n",
+    "\n",
+    "llm=AG.get_llm_provider()\n",
     "\n",
     "class ProductCatalogEntry(BaseModel):\n",
     "    product_id: str = Field(None, description=\"Unique ID for the product\")\n",
@@ -37,9 +44,7 @@
     "    category: str = Field(None, description=\"Product category\")\n",
     "    price: float = Field(None, description=\"Retail price in USD\")\n",
     "\n",
-    "products = await generate_prototypical_instances(ProductCatalogEntry,\n",
-    "                                n_instances=10, \n",
-    "                                llm=\"watsonx/openai/gpt-oss-120b\")\n",
+    "products = await generate_prototypical_instances(ProductCatalogEntry, n_instances=10, llm=llm)\n",
     "\n",
     "for product in products:\n",
     "    print(product.model_dump_json(indent=2))"
@@ -54,7 +59,7 @@
    "source": [
     "from agentics.core.transducible_functions import generate_atype_from_description\n",
     "generated_type = await generate_atype_from_description(\"Person with demographic information including name, age, and email address\")\n",
-    "print(generated_type.model_dump_json(indent=2))\n"
+    "print(generated_type.model_dump_json(indent=2))"
    ]
   },
   {
@@ -80,7 +85,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.14"
+   "version": "3.12.12"
   }
  },
  "nbformat": 4,
diff --git a/tutorials/transducible_functions.ipynb b/tutorials/transducible_functions.ipynb
index 314879069..f417d22ea 100644
--- a/tutorials/transducible_functions.ipynb
+++ b/tutorials/transducible_functions.ipynb
@@ -51,6 +51,10 @@
     "from agentics import AG\n",
     "from pydantic import BaseModel, Field\n",
     "from agentics.core.transducible_functions import transducible, Transduce\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "# Load .env from project root (current working directory)\n",
+    "load_dotenv()\n",
     "\n",
     "llm=AG.get_llm_provider()\n",
     "\n",
@@ -86,11 +90,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "\n",
-    "@transducible(provide_explanation=True, llm=llm)\n",
+    "@transducible(provide_explanation=False, llm=llm)\n",
     "async def write_an_email(state: GenericInput) -> Email:\n",
-    "    \"\"\"Write an email about the provided content. \n",
-    "    Elaborate on that and make up content as needed\"\"\"\n",
+    "    \"\"\"Write an email about the provided content. Elaborate on that and make up content as needed\"\"\"\n",
     "    return Transduce(state)\n",
     "\n",
     "## Transducible functions can be introspected to easily get their input , output , description and original function\n",
@@ -100,9 +102,8 @@
     "print(write_an_email.__original_fn__)\n",
     "\n",
     "\n",
-    "single_mail, explanation = await write_an_email(GenericInput(content=f\"Hi Lisa, I made great progress with the new release of Agentics 2.0\"))\n",
+    "single_mail, explanation = await write_an_email(GenericInput(content=\"Hi Lisa, I made great progress with the new release of Agentics 2.0\"))\n",
     "print(single_mail.model_dump_json(indent=2))\n",
-    "print(explanation)\n",
     "\n"
    ]
   },
@@ -134,7 +135,7 @@
    "source": [
     "import re\n",
     "\n",
-    "@transducible()\n",
+    "@transducible(provide_explanation=False, llm=llm)\n",
     "async def write_an_email_code_only(state: GenericInput) -> Email:  \n",
     "    match = re.match(r\"^(Hi|Dear|Hello|Hey)\\s+([^,]+),\\s*(.+)$\", state.content)\n",
     "    if match:\n",
@@ -142,7 +143,7 @@
     "        return Email(body= body, to=name, subject=\"\")\n",
     "    else: return Email()\n",
     "\n",
-    "@transducible(provide_explanation=True, llm=llm)\n",
+    "@transducible(provide_explanation=False, llm=llm)\n",
     "async def write_an_email_to_lisa(state: GenericInput) -> Email:\n",
     "    \"\"\"Write an email about the provided content. Elaborate on that and make up content as needed\"\"\"\n",
     "    # example code to modify states before transduction\n",
@@ -216,7 +217,7 @@
     "    relevant_sources:Optional[list[str]]=None\n",
     "\n",
     "\n",
-    "@transducible(tools=[web_search], llm=llm, reasoning=True, max_iter=20, provide_explanation=True)\n",
+    "@transducible(tools=[web_search], llm=llm, reasoning=True, max_iter=20, provide_explanation=False)\n",
     "async def answer_question_after_lookup(query: GenericInput) -> WebSearchResult:\n",
     "    \"perform an extensive web search to provide an answer to the input question with supporting evidence. Use your tool to look it up\" \n",
     "    return Transduce(query)\n",
@@ -286,7 +287,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.14"
+   "version": "3.12.12"
   }
  },
  "nbformat": 4,

From 84fd2dbbd2eee488016bb7d802400b4b4eaae434 Mon Sep 17 00:00:00 2001
From: Alfio Gliozzo 
Date: Fri, 13 Feb 2026 16:02:01 -0500
Subject: [PATCH 24/27] sem_filter_fix

---
 src/agentics/core/semantic_operators.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/src/agentics/core/semantic_operators.py b/src/agentics/core/semantic_operators.py
index a59a3284a..29bb92702 100644
--- a/src/agentics/core/semantic_operators.py
+++ b/src/agentics/core/semantic_operators.py
@@ -113,7 +113,7 @@ async def sem_filter(
     predicate. It is an agentic analogue of LOTUS-style semantic filtering.
 
     The `predicate_template` is a **LangChain-style template** (e.g., using `{field}`
-    placeholders) that is rendered against each source state’s fields. The rendered
+    placeholders) that is rendered against each source state's fields. The rendered
     text is then passed to an LLM-based logical classifier which produces a boolean
     decision (`condition_true`) for that state.
 
@@ -156,10 +156,12 @@ async def sem_filter(
         amap_batch_size=20,
         **kwargs,
     )
+    # Keep the more sophisticated version that handles both cases
     if "{" in predicate_template:
         ag_source.prompt_template = predicate_template
     else:
         target_ag.instructions += f"\n\nPredicate: {predicate_template}"
+
     map_out = await (target_ag << ag_source)
     target = ag_source.clone()
     target.states = []
@@ -200,3 +202,6 @@ async def sem_agg(
     if type(source) is pd.DataFrame:
         return output_ag.to_dataframe()
     return output_ag
+
+
+# Made with Bob

From 91e4f270eabd46ab0368c91ffb58cdeb3873137a Mon Sep 17 00:00:00 2001
From: Alfio Gliozzo 
Date: Fri, 13 Feb 2026 17:17:09 -0500
Subject: [PATCH 25/27] sem_filter_fix

---
 src/agentics/core/semantic_operators.py     |  2 +-
 src/agentics/core/transducible_functions.py | 18 ++++++++++++------
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/src/agentics/core/semantic_operators.py b/src/agentics/core/semantic_operators.py
index 971f14dbc..1041c7047 100644
--- a/src/agentics/core/semantic_operators.py
+++ b/src/agentics/core/semantic_operators.py
@@ -177,7 +177,7 @@ async def sem_filter(
 async def sem_agg(
     source: AG | pd.DataFrame,
     target_type: Type[BaseModel] | str,
-    instructions: str = None,
+    instructions: str | None = None,
     # merge_output: bool = True,  ## Target, Merged
     **kwargs,
 ) -> AG | pd.DataFrame:
diff --git a/src/agentics/core/transducible_functions.py b/src/agentics/core/transducible_functions.py
index d156ada24..42b02e879 100644
--- a/src/agentics/core/transducible_functions.py
+++ b/src/agentics/core/transducible_functions.py
@@ -545,21 +545,27 @@ async def semantic_merge(instance1: BaseModel, instance2: BaseModel) -> BaseMode
 
 
 async def generate_prototypical_instances(
-    type: Type[BaseModel], n_instances: int = 10, llm: Any = AG.get_llm_provider()
+    type: Type[BaseModel],
+    n_instances: int = 10,
+    llm: Any = AG.get_llm_provider(),
+    instructions: str = None,
 ) -> list[BaseModel]:
 
     DynamicModel = create_model(
         "ListOfObjectsOfGivenType",
         instances=(list[type] | None, None),  # REQUIRED field
     )
-
-    target = AG(
-        atype=DynamicModel,
-        instructions=f"""
+    full_instructions = f"""
               Generate list of {n_instances} random instances of the following type
               {type.model_json_schema()}.
               fill all attributed for each generated instance
-              """,
+              """
+    if instructions:
+        full_instructions += "Adhere to the following instructions \n" + instructions
+
+    target = AG(
+        atype=DynamicModel,
+        instructions=full_instructions,
         llm=llm,
     )
     generated = await (target << "   ")

From 2d78c285114a990494d0af979ed608d7caacebb0 Mon Sep 17 00:00:00 2001
From: Alfio Gliozzo 
Date: Sun, 15 Feb 2026 14:00:09 -0500
Subject: [PATCH 26/27] Semantic Operators optimized for langflow

---
 src/agentics/core/agentics.py           | 11 ++++-----
 src/agentics/core/semantic_operators.py | 32 +++++++++++++++++++------
 2 files changed, 30 insertions(+), 13 deletions(-)

diff --git a/src/agentics/core/agentics.py b/src/agentics/core/agentics.py
index c19570547..7d1538e9a 100644
--- a/src/agentics/core/agentics.py
+++ b/src/agentics/core/agentics.py
@@ -180,22 +180,21 @@ def timeout(self, value: float):
     def create_crewai_llm(**kwargs):
         return LLM(**kwargs)
 
-    async def generate_atype(
-        self, description: str, retry: int = 3
-    ) -> Tuple[str, Type[BaseModel]] | None:
+    async def generate_atype(self, description: str, retry: int = 3):
         class GeneratedAtype(BaseModel):
             python_code: Optional[str] = Field(
                 None, description="Python Code for the described Pydantic type"
             )
-            methods: list[str] = Field(None, description="Methods for the class above")
+            # methods: list[str] = Field(None, description="Methods for the class above")
 
         i = 0
         while i < retry:
             generated_atype_ag = await (
                 AG(
                     atype=GeneratedAtype,
-                    instructions="""Generate python code for the input nl type specs.
-                Make all fields Optional. Use only primitive types for the fields, avoiding nested.
+                    instructions="""Generate Python code for the pydantic type following the input specs.
+                Make all fields Optional.
+                Use only primitive types for the fields, avoiding nested.
                 Provide descriptions for the class and all its fields, using Field(None,description= "...")
                 If the input nl type spec is a question, generate a pydantic type that can be used to
                 represent the answer to that question.
diff --git a/src/agentics/core/semantic_operators.py b/src/agentics/core/semantic_operators.py
index 1041c7047..57dad9872 100644
--- a/src/agentics/core/semantic_operators.py
+++ b/src/agentics/core/semantic_operators.py
@@ -102,7 +102,10 @@ async def sem_map(
 
 
 async def sem_filter(
-    source: AG | pd.DataFrame, predicate_template: str, **kwargs
+    source: AG | pd.DataFrame,
+    predicate_template: str,
+    sensitivity: float = 0.8,
+    **kwargs,
 ) -> AG | pd.DataFrame:
     """
     Agentics-native semantic filter over an `AG` using a LangChain-style condition template.
@@ -148,10 +151,24 @@ async def sem_filter(
 
     target_ag = AG(
         atype=create_pydantic_model(
-            [("condition_true", "bool", """Condition is True""", False)], name="filter"
+            [
+                (
+                    "sentence_is_true",
+                    "bool",
+                    """Provide True if you think the input sentence is True, False otherwise""",
+                    False,
+                ),
+                (
+                    "truth_score",
+                    "float",
+                    """Provide a number from 0 to 1 to assess the degree to which the input Sentence is True. If you are uncertain, provide a number in the range (0,1)""",
+                    False,
+                ),
+            ],
+            name="filter",
         ),
-        instructions="""You are a Logical Classifier. You have been given an input sentence.
-            Read the input text and return true if the predicate is positive, false otherwise""",
+        instructions="""You have been given an input sentence.
+            Read the input text and return True if the sentence is true, False otherwise""",
         amap_batch_size=20,
         **kwargs,
     )
@@ -164,9 +181,10 @@ async def sem_filter(
     map_out = await (target_ag << ag_source)
     target = ag_source.clone()
     target.states = []
-    for i in range(len(map_out.states)):
-        if map_out[i].condition_true:
-            target.append(ag_source[i])
+
+    for map_out_c, source_c in zip(map_out.states, ag_source.states):
+        if map_out_c.truth_score and map_out_c.truth_score >= sensitivity:
+            target.append(source_c)
 
     if type(source) is pd.DataFrame:
         return target.to_dataframe()

From 0798f6e504fd7e7827b3b16a7422e04352869970 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Nahuel=20Defoss=C3=A9?= 
Date: Tue, 17 Feb 2026 11:40:05 +0300
Subject: [PATCH 27/27] Fix pre-commit issues
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Nahuel DefossΓ© 
---
 docs/agentics.md                             |   2 +-
 docs/core_concepts.md                        |   2 +-
 docs/getting_started.md                      |   2 +-
 docs/map_reduce.md                           |   2 +-
 docs/optimization.md                         |   2 +-
 docs/references.md                           |   2 +-
 docs/semantic_operators.md                   |   2 +-
 docs/tool_integration.md                     |   2 +-
 docs/transducible_functions.md               |   2 +-
 tutorials/atypes.ipynb                       | 199 +---------
 tutorials/logical_transduction_algebra.ipynb | 173 +--------
 tutorials/map_reduce.ipynb                   | 370 +------------------
 tutorials/semantic_operators.ipynb           |  16 +-
 13 files changed, 56 insertions(+), 720 deletions(-)

diff --git a/docs/agentics.md b/docs/agentics.md
index 0c519ac51..b122867e3 100644
--- a/docs/agentics.md
+++ b/docs/agentics.md
@@ -164,4 +164,4 @@ See the [examples directory](../examples/) for practical demonstrations of AG us
 - `emotion_extractor.py` - Text analysis
 
 ## Go to Index
-- πŸ‘‰ [Index](index.md)
\ No newline at end of file
+- πŸ‘‰ [Index](index.md)
diff --git a/docs/core_concepts.md b/docs/core_concepts.md
index 6e03052b2..d05c9b45b 100644
--- a/docs/core_concepts.md
+++ b/docs/core_concepts.md
@@ -259,4 +259,4 @@ A typical workflow looks like this:
 - πŸ‘‰ [Transducible Functions](transducible_functions.md) for concrete examples of defining and using transducible functions
 
 ## Go to Index
-- πŸ‘‰ [Index](index.md)
\ No newline at end of file
+- πŸ‘‰ [Index](index.md)
diff --git a/docs/getting_started.md b/docs/getting_started.md
index b234296d1..d08582027 100644
--- a/docs/getting_started.md
+++ b/docs/getting_started.md
@@ -213,4 +213,4 @@ The same transducible function works seamlessly for both single items and batche
 - πŸ‘‰ [Core Concepts](core_concepts.md) - Understanding the theoretical foundation
 
 ## Go to Index
-- πŸ‘‰ [Index](index.md)
\ No newline at end of file
+- πŸ‘‰ [Index](index.md)
diff --git a/docs/map_reduce.md b/docs/map_reduce.md
index f66cf5c78..ddb9ff0be 100644
--- a/docs/map_reduce.md
+++ b/docs/map_reduce.md
@@ -378,4 +378,4 @@ Map operations are automatically parallelized based on `batch_size`. For more co
 - πŸ‘‰ [Semantic Operators](semantic_operators.md) for performing data transformation tasks using natural language.
 
 ## Go to Index
-- πŸ‘‰ [Index](index.md)
\ No newline at end of file
+- πŸ‘‰ [Index](index.md)
diff --git a/docs/optimization.md b/docs/optimization.md
index 1aae2cca7..bd2e085a9 100644
--- a/docs/optimization.md
+++ b/docs/optimization.md
@@ -321,4 +321,4 @@ async def process_with_error_handling(items):
 - πŸ‘‰ [Transducible Functions](transducible_functions.md) - Core concepts and basic usage
 - πŸ‘‰ [Tool Integration](tool_integration.md) - Using external tools
 - πŸ‘‰ [Map-Reduce Tutorial](../tutorials/map_reduce.ipynb) - Large-scale execution patterns
-- πŸ‘‰ [Index](index.md)
\ No newline at end of file
+- πŸ‘‰ [Index](index.md)
diff --git a/docs/references.md b/docs/references.md
index e8726737c..be79a0237 100644
--- a/docs/references.md
+++ b/docs/references.md
@@ -42,4 +42,4 @@ If you use Agentics in your research or project, please cite the relevant papers
      year={2026},
      url={https://arxiv.org/abs/2510.21117}
    }
-   ```
\ No newline at end of file
+   ```
diff --git a/docs/semantic_operators.md b/docs/semantic_operators.md
index 4b6b78478..35cf18ab7 100644
--- a/docs/semantic_operators.md
+++ b/docs/semantic_operators.md
@@ -343,4 +343,4 @@ summary = await sem_agg(mapped, Summary, "Summarize customer segments")
 - πŸ‘‰ [Agentics (AG)](agentics.md) for data modeling patterns and typed state containers
 
 ## Go to Index
-- πŸ‘‰ [Index](index.md)
\ No newline at end of file
+- πŸ‘‰ [Index](index.md)
diff --git a/docs/tool_integration.md b/docs/tool_integration.md
index 89ceffdc5..5a33f05ee 100644
--- a/docs/tool_integration.md
+++ b/docs/tool_integration.md
@@ -400,4 +400,4 @@ except TimeoutError:
 - πŸ‘‰ [Transducible Functions](transducible_functions.md) - Core concepts and basic usage
 - πŸ‘‰ [Optimization](optimization.md) - Performance tuning and batch processing
 - πŸ‘‰ [Examples](../examples/mcp_server_example.py) - Complete MCP server example
-- πŸ‘‰ [Index](index.md)
\ No newline at end of file
+- πŸ‘‰ [Index](index.md)
diff --git a/docs/transducible_functions.md b/docs/transducible_functions.md
index b58127f54..bb48a86ab 100644
--- a/docs/transducible_functions.md
+++ b/docs/transducible_functions.md
@@ -621,4 +621,4 @@ Good reasons to define a separate transducible function:
 - πŸ‘‰ **[Map-Reduce Operations](map_reduce.md)** - Scaling with map and reduce, batch processing patterns
 
 ## Go to Index
-- πŸ‘‰ [Index](index.md)
\ No newline at end of file
+- πŸ‘‰ [Index](index.md)
diff --git a/tutorials/atypes.ipynb b/tutorials/atypes.ipynb
index 20d8128d6..f751393b6 100644
--- a/tutorials/atypes.ipynb
+++ b/tutorials/atypes.ipynb
@@ -31,7 +31,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": null,
    "id": "1",
    "metadata": {},
    "outputs": [],
@@ -84,18 +84,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": null,
    "id": "3",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'genre': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'movie_name': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'description': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'year': FieldInfo(annotation=Union[int, NoneType], required=False, default=None)}\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from typing import Optional, List\n",
     "from pydantic import BaseModel, Field\n",
@@ -139,18 +131,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": null,
    "id": "5",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'target': FieldInfo(annotation=Union[Summary, NoneType], required=False, default=None), 'source': FieldInfo(annotation=Union[Email, NoneType], required=False, default=None)}\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "class Email(BaseModel):\n",
     "    to: Optional[str] = Field(None, description=\"Recipient name or email.\")\n",
@@ -186,18 +170,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": null,
    "id": "7",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{'target': FieldInfo(annotation=Union[Summary, NoneType], required=False, default=None), 'source': FieldInfo(annotation=Union[EmailAndSentiment, NoneType], required=False, default=None)}\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "class Sentiment(BaseModel):\n",
     "    sentiment: Optional[str] = Field(\n",
@@ -223,19 +199,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": null,
    "id": "9",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "answer='Paris is the capital of France.' confidence=0.95\n",
-      "name='Alice' age=None\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from agentics.core.atype import create_pydantic_model\n",
     "Answer = create_pydantic_model([(\"answer\", \"str\", \"the answer for the question\",False), (\"confidence\", \"float\", None,False)])\n",
@@ -278,62 +245,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": null,
    "id": "11",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "{\n",
-      "  \"student_name\": \"Alice Thompson\",\n",
-      "  \"student_age\": 20,\n",
-      "  \"enrolled_courses\": [\n",
-      "    \"Introduction to Psychology\",\n",
-      "    \"Calculus I\",\n",
-      "    \"Modern World History\"\n",
-      "  ]\n",
-      "}\n",
-      "{\n",
-      "  \"student_name\": \"Marcus Rodriguez\",\n",
-      "  \"student_age\": 22,\n",
-      "  \"enrolled_courses\": [\n",
-      "    \"Organic Chemistry\",\n",
-      "    \"Cell Biology\",\n",
-      "    \"Physics for Life Sciences\"\n",
-      "  ]\n",
-      "}\n",
-      "{\n",
-      "  \"student_name\": \"Sophia Chen\",\n",
-      "  \"student_age\": 19,\n",
-      "  \"enrolled_courses\": [\n",
-      "    \"Macroeconomics\",\n",
-      "    \"Business Ethics\",\n",
-      "    \"Statistics for Data Science\"\n",
-      "  ]\n",
-      "}\n",
-      "{\n",
-      "  \"student_name\": \"Julian Banks\",\n",
-      "  \"student_age\": 21,\n",
-      "  \"enrolled_courses\": [\n",
-      "    \"Digital Marketing\",\n",
-      "    \"Consumer Behavior\",\n",
-      "    \"Creative Writing\"\n",
-      "  ]\n",
-      "}\n",
-      "{\n",
-      "  \"student_name\": \"Elena Petrova\",\n",
-      "  \"student_age\": 23,\n",
-      "  \"enrolled_courses\": [\n",
-      "    \"Artificial Intelligence\",\n",
-      "    \"Software Engineering\",\n",
-      "    \"Discrete Mathematics\"\n",
-      "  ]\n",
-      "}\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from agentics.core.transducible_functions import generate_prototypical_instances\n",
     "\n",
@@ -368,82 +283,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": null,
    "id": "13",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "2026-02-13 14:43:11.581 | DEBUG    | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 1\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n"
-     ]
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "2026-02-13 14:43:11.682 | DEBUG    | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 2\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "2026-02-13 14:43:11.798 | DEBUG    | agentics.core.agentics:__lshift__:650 - Error: 1 states have not been transduced\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n"
-     ]
-    },
-    {
-     "ename": "ValidationError",
-     "evalue": "1 validation error for GeneratedAtype\nmethods\n  Input should be a valid list [type=list_type, input_value=None, input_type=NoneType]\n    For further information visit https://errors.pydantic.dev/2.12/v/list_type",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
-      "\u001b[31mValidationError\u001b[39m                           Traceback (most recent call last)",
-      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[11]\u001b[39m\u001b[32m, line 2\u001b[39m\n\u001b[32m      1\u001b[39m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01magentics\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mcore\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mtransducible_functions\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m generate_atype_from_description\n\u001b[32m----> \u001b[39m\u001b[32m2\u001b[39m person = \u001b[38;5;28;01mawait\u001b[39;00m generate_atype_from_description(\u001b[33m\"\u001b[39m\u001b[33mPerson with demographic information including name, age, email and birthplace\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m      3\u001b[39m citizenship = \u001b[38;5;28;01mawait\u001b[39;00m generate_atype_from_description(\u001b[33m\"\u001b[39m\u001b[33mCitizenship information including country of citizenship and visa status\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m      4\u001b[39m \u001b[38;5;28mprint\u001b[39m(person.model_dump_json(indent=\u001b[32m2\u001b[39m))\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:609\u001b[39m, in \u001b[36mgenerate_atype_from_description\u001b[39m\u001b[34m(description, retry)\u001b[39m\n\u001b[32m    607\u001b[39m     i = \u001b[32m0\u001b[39m\n\u001b[32m    608\u001b[39m     \u001b[38;5;28;01mwhile\u001b[39;00m i < retry:\n\u001b[32m--> \u001b[39m\u001b[32m609\u001b[39m         generated_atype_ag = \u001b[38;5;28;01mawait\u001b[39;00m (\n\u001b[32m    610\u001b[39m             AG(\n\u001b[32m    611\u001b[39m                 atype=GeneratedAtype,\n\u001b[32m    612\u001b[39m                 instructions=\u001b[33m\"\"\"\u001b[39m\n\u001b[32m    613\u001b[39m \u001b[33mGenerate python code for the input natural-language type specification.\u001b[39m\n\u001b[32m    614\u001b[39m \n\u001b[32m    615\u001b[39m \u001b[33mRequirements:\u001b[39m\n\u001b[32m    616\u001b[39m \u001b[33m- Define exactly ONE Pydantic BaseModel.\u001b[39m\n\u001b[32m    617\u001b[39m \u001b[33m- Make all fields Optional.\u001b[39m\n\u001b[32m    618\u001b[39m \u001b[33m- Use only primitive types for the fields (str, int, float, bool, list[str], etc.).\u001b[39m\n\u001b[32m    619\u001b[39m \u001b[33m- Avoid nested Pydantic models.\u001b[39m\n\u001b[32m    620\u001b[39m \u001b[33m- Provide descriptions for the class and all its fields using:\u001b[39m\n\u001b[32m    621\u001b[39m \u001b[33m    Field(None, description=\u001b[39m\u001b[33m\"\u001b[39m\u001b[33m...\u001b[39m\u001b[33m\"\u001b[39m\u001b[33m)\u001b[39m\n\u001b[32m    622\u001b[39m \u001b[33m- If the input is a question, generate a Pydantic type that can represent\u001b[39m\n\u001b[32m    623\u001b[39m \u001b[33m  the answer to that question.\u001b[39m\n\u001b[32m    624\u001b[39m \u001b[33mReturn ONLY valid Python V2 code in `python_code`.\u001b[39m\n\u001b[32m    625\u001b[39m \u001b[33m\"\"\"\u001b[39m,\n\u001b[32m    626\u001b[39m             )\n\u001b[32m    627\u001b[39m             << description\n\u001b[32m    628\u001b[39m         )\n\u001b[32m    630\u001b[39m         \u001b[38;5;28;01mif\u001b[39;00m generated_atype_ag.states \u001b[38;5;129;01mand\u001b[39;00m generated_atype_ag[\u001b[32m0\u001b[39m]:\n\u001b[32m    631\u001b[39m             generated_atype_ag[\u001b[32m0\u001b[39m].atype = import_pydantic_from_code(\n\u001b[32m    632\u001b[39m                 generated_atype_ag[\u001b[32m0\u001b[39m].python_code\n\u001b[32m    633\u001b[39m             )\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/agentics.py:681\u001b[39m, in \u001b[36mAG.__lshift__\u001b[39m\u001b[34m(self, other)\u001b[39m\n\u001b[32m    679\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mrange\u001b[39m(\u001b[38;5;28mlen\u001b[39m(other)):\n\u001b[32m    680\u001b[39m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(output_states[i], \u001b[38;5;28mself\u001b[39m.atype):\n\u001b[32m--> \u001b[39m\u001b[32m681\u001b[39m         output.states.append(\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43matype\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43moutput_states\u001b[49m\u001b[43m[\u001b[49m\u001b[43mi\u001b[49m\u001b[43m]\u001b[49m\u001b[43m.\u001b[49m\u001b[43mmodel_dump\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m)\n\u001b[32m    682\u001b[39m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m    683\u001b[39m         output.states.append(\u001b[38;5;28mself\u001b[39m.atype())\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/.venv/lib/python3.12/site-packages/pydantic/main.py:250\u001b[39m, in \u001b[36mBaseModel.__init__\u001b[39m\u001b[34m(self, **data)\u001b[39m\n\u001b[32m    248\u001b[39m \u001b[38;5;66;03m# `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks\u001b[39;00m\n\u001b[32m    249\u001b[39m __tracebackhide__ = \u001b[38;5;28;01mTrue\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m250\u001b[39m validated_self = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m__pydantic_validator__\u001b[49m\u001b[43m.\u001b[49m\u001b[43mvalidate_python\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdata\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mself_instance\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[32m    251\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m validated_self:\n\u001b[32m    252\u001b[39m     warnings.warn(\n\u001b[32m    253\u001b[39m         \u001b[33m'\u001b[39m\u001b[33mA custom validator is returning a value other than `self`.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33m'\u001b[39m\n\u001b[32m    254\u001b[39m         \u001b[33m\"\u001b[39m\u001b[33mReturning anything other than `self` from a top level model validator isn\u001b[39m\u001b[33m'\u001b[39m\u001b[33mt supported when validating via `__init__`.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33m\"\u001b[39m\n\u001b[32m    255\u001b[39m         \u001b[33m'\u001b[39m\u001b[33mSee the `model_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.\u001b[39m\u001b[33m'\u001b[39m,\n\u001b[32m    256\u001b[39m         stacklevel=\u001b[32m2\u001b[39m,\n\u001b[32m    257\u001b[39m     )\n",
-      "\u001b[31mValidationError\u001b[39m: 1 validation error for GeneratedAtype\nmethods\n  Input should be a valid list [type=list_type, input_value=None, input_type=NoneType]\n    For further information visit https://errors.pydantic.dev/2.12/v/list_type"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from agentics.core.transducible_functions import generate_atype_from_description\n",
     "person = await generate_atype_from_description(\"Person with demographic information including name, age, email and birthplace\")\n",
@@ -478,22 +321,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": null,
    "id": "15",
    "metadata": {},
-   "outputs": [
-    {
-     "ename": "NameError",
-     "evalue": "name 'citizenship' is not defined",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
-      "\u001b[31mNameError\u001b[39m                                 Traceback (most recent call last)",
-      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[12]\u001b[39m\u001b[32m, line 2\u001b[39m\n\u001b[32m      1\u001b[39m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01magentics\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mcore\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mtransducible_functions\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m estimateLogicalProximity\n\u001b[32m----> \u001b[39m\u001b[32m2\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;28;01mawait\u001b[39;00m estimateLogicalProximity(\u001b[43mcitizenship\u001b[49m.atype << person.atype))\n",
-      "\u001b[31mNameError\u001b[39m: name 'citizenship' is not defined"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from agentics.core.transducible_functions import estimateLogicalProximity\n",
     "print(await estimateLogicalProximity(citizenship.atype << person.atype))\n"
diff --git a/tutorials/logical_transduction_algebra.ipynb b/tutorials/logical_transduction_algebra.ipynb
index 751f12064..50022e5bd 100644
--- a/tutorials/logical_transduction_algebra.ipynb
+++ b/tutorials/logical_transduction_algebra.ipynb
@@ -19,7 +19,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "id": "1",
    "metadata": {},
    "outputs": [],
@@ -58,22 +58,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": null,
    "id": "3",
    "metadata": {},
-   "outputs": [
-    {
-     "ename": "AttributeError",
-     "evalue": "'tuple' object has no attribute 'model_dump_json'",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
-      "\u001b[31mAttributeError\u001b[39m                            Traceback (most recent call last)",
-      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[2]\u001b[39m\u001b[32m, line 6\u001b[39m\n\u001b[32m      4\u001b[39m write_tweet = Email << GenericInput\n\u001b[32m      5\u001b[39m tweet = \u001b[38;5;28;01mawait\u001b[39;00m write_tweet(\u001b[38;5;28minput\u001b[39m)\n\u001b[32m----> \u001b[39m\u001b[32m6\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[43mtweet\u001b[49m\u001b[43m.\u001b[49m\u001b[43mmodel_dump_json\u001b[49m(indent=\u001b[32m2\u001b[39m))\n",
-      "\u001b[31mAttributeError\u001b[39m: 'tuple' object has no attribute 'model_dump_json'"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "input = GenericInput(content=\n",
     "    \"\"\"Zoran Mamdani become the NYC mayor\"\"\")\n",
@@ -116,24 +104,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": null,
    "id": "5",
    "metadata": {},
-   "outputs": [
-    {
-     "ename": "TypeError",
-     "evalue": "transducible() got an unexpected keyword argument 'prompyt_template'",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
-      "\u001b[31mTypeError\u001b[39m                                 Traceback (most recent call last)",
-      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[3]\u001b[39m\u001b[32m, line 1\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m1\u001b[39m write_mail_to_alfio = \u001b[43mEmail\u001b[49m\u001b[43m<<\u001b[49m\u001b[43m \u001b[49m\u001b[43mWith\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m      2\u001b[39m \u001b[43m    \u001b[49m\u001b[43mGenericInput\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m      3\u001b[39m \u001b[43m    \u001b[49m\u001b[43minstructions\u001b[49m\u001b[43m=\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mWrite an email to Alfio Gliozzo\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[32m      4\u001b[39m \u001b[43m    \u001b[49m\u001b[43mprompyt_template\u001b[49m\u001b[43m=\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[38;5;132;43;01m{content}\u001b[39;49;00m\u001b[33;43m\"\u001b[39;49m\u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m      5\u001b[39m news = GenericInput(content=\u001b[33m\"\u001b[39m\u001b[33mZoran Mandani won the Election in NYC\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m      6\u001b[39m mail = \u001b[38;5;28;01mawait\u001b[39;00m write_mail_to_alfio(news)\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:464\u001b[39m, in \u001b[36m_model_lshift\u001b[39m\u001b[34m(OutputModel, InputType)\u001b[39m\n\u001b[32m    462\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(InputType, TransductionConfig):\n\u001b[32m    463\u001b[39m     M = InputType.model\n\u001b[32m--> \u001b[39m\u001b[32m464\u001b[39m     f = \u001b[43mmake_transducible_function\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m    465\u001b[39m \u001b[43m        \u001b[49m\u001b[43mInputModel\u001b[49m\u001b[43m=\u001b[49m\u001b[43mM\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m    466\u001b[39m \u001b[43m        \u001b[49m\u001b[43mOutputModel\u001b[49m\u001b[43m=\u001b[49m\u001b[43mOutputModel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m    467\u001b[39m \u001b[43m        \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mInputType\u001b[49m\u001b[43m.\u001b[49m\u001b[43mconfig\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m    468\u001b[39m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m    469\u001b[39m     f.\u001b[34m__lshift__\u001b[39m = types.MethodType(_function_lshift, f)\n\u001b[32m    470\u001b[39m     \u001b[38;5;28;01mreturn\u001b[39;00m _wrap_composed(f)\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:367\u001b[39m, in \u001b[36mmake_transducible_function\u001b[39m\u001b[34m(InputModel, OutputModel, function_code, instructions, name, **kwargs)\u001b[39m\n\u001b[32m    364\u001b[39m     _auto_fn.\u001b[34m__doc__\u001b[39m = instructions\n\u001b[32m    366\u001b[39m     \u001b[38;5;66;03m# Delegate everything to your existing decorator\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m367\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m _wrap_composed(\u001b[43mtransducible\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m(_auto_fn))\n",
-      "\u001b[31mTypeError\u001b[39m: transducible() got an unexpected keyword argument 'prompyt_template'"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "write_mail_to_alfio = Email<< With(\n",
     "    GenericInput,\n",
@@ -165,22 +139,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": null,
    "id": "7",
    "metadata": {},
-   "outputs": [
-    {
-     "ename": "NameError",
-     "evalue": "name 'write_mail_to_alfio' is not defined",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
-      "\u001b[31mNameError\u001b[39m                                 Traceback (most recent call last)",
-      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[4]\u001b[39m\u001b[32m, line 4\u001b[39m\n\u001b[32m      1\u001b[39m news = GenericInput(content=\u001b[33m\"\u001b[39m\u001b[33mZoran Mandani won the Election in NYC, make up a story about that.\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m----> \u001b[39m\u001b[32m4\u001b[39m summary_composite_1 = Summary << \u001b[43mwrite_mail_to_alfio\u001b[49m\n\u001b[32m      5\u001b[39m summary = \u001b[38;5;28;01mawait\u001b[39;00m summary_composite_1(news)\n\u001b[32m      6\u001b[39m \u001b[38;5;28mprint\u001b[39m(summary.model_dump_json(indent=\u001b[32m2\u001b[39m))\n",
-      "\u001b[31mNameError\u001b[39m: name 'write_mail_to_alfio' is not defined"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "news = GenericInput(content=\"Zoran Mandani won the Election in NYC, make up a story about that.\")\n",
     "\n",
@@ -196,18 +158,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": null,
    "id": "8",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "2026-02-13 14:45:29.771 | DEBUG    | agentics.core.agentics:from_csv:831 - Importing Agentics of type Movie from CSV str\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "\n",
     "from agentics import AG\n",
@@ -236,22 +190,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": null,
    "id": "9",
    "metadata": {},
-   "outputs": [
-    {
-     "ename": "TypeError",
-     "evalue": "unsupported operand type(s) for @: 'tuple' and 'Movie'",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
-      "\u001b[31mTypeError\u001b[39m                                 Traceback (most recent call last)",
-      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[6]\u001b[39m\u001b[32m, line 3\u001b[39m\n\u001b[32m      1\u001b[39m classify_genre= Genre << Movie\n\u001b[32m      2\u001b[39m genre = \u001b[38;5;28;01mawait\u001b[39;00m classify_genre(movie)\n\u001b[32m----> \u001b[39m\u001b[32m3\u001b[39m \u001b[38;5;28mprint\u001b[39m((\u001b[43mgenre\u001b[49m\u001b[43m \u001b[49m\u001b[43m@\u001b[49m\u001b[43m \u001b[49m\u001b[43mmovie\u001b[49m).model_dump_json(indent=\u001b[32m2\u001b[39m))\n",
-      "\u001b[31mTypeError\u001b[39m: unsupported operand type(s) for @: 'tuple' and 'Movie'"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "classify_genre= Genre << Movie\n",
     "genre = await classify_genre(movie)\n",
@@ -260,105 +202,20 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": null,
    "id": "10",
    "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "(Genre(genre='crime'), None)"
-      ]
-     },
-     "execution_count": 7,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
     "await (Genre << movie)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": null,
    "id": "11",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "2026-02-13 14:45:40.197 | DEBUG    | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 1\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n"
-     ]
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "2026-02-13 14:45:40.304 | DEBUG    | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 2\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "ERROR:root:OpenAI API call failed: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\n",
-      "2026-02-13 14:45:40.402 | DEBUG    | agentics.core.agentics:__lshift__:650 - Error: 1 states have not been transduced\n"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n",
-      "\u001b[91mAn unknown error occurred. Please check the details below.\u001b[0m\n",
-      "\u001b[91mError details: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-proj-********************************************************************************************************************************************************3h4A. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'code': 'invalid_api_key', 'param': None}, 'status': 401}\u001b[0m\n"
-     ]
-    },
-    {
-     "ename": "ValidationError",
-     "evalue": "1 validation error for Explanation\nrelevant_source_attributes\n  Input should be a valid dictionary [type=dict_type, input_value=[], input_type=list]\n    For further information visit https://errors.pydantic.dev/2.12/v/dict_type",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
-      "\u001b[31mValidationError\u001b[39m                           Traceback (most recent call last)",
-      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[8]\u001b[39m\u001b[32m, line 3\u001b[39m\n\u001b[32m      1\u001b[39m classify_genre= Genre << With(Movie,\n\u001b[32m      2\u001b[39m                             provide_explanation=\u001b[38;5;28;01mTrue\u001b[39;00m)\n\u001b[32m----> \u001b[39m\u001b[32m3\u001b[39m genre, explanation = \u001b[38;5;28;01mawait\u001b[39;00m classify_genre(movie)\n\u001b[32m      4\u001b[39m \u001b[38;5;28mprint\u001b[39m(genre.model_dump_json(indent=\u001b[32m2\u001b[39m))\n\u001b[32m      5\u001b[39m \u001b[38;5;28mprint\u001b[39m(explanation.model_dump_json(indent=\u001b[32m2\u001b[39m))\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:291\u001b[39m, in \u001b[36mtransducible.._transducible..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m    289\u001b[39m \u001b[38;5;66;03m# AMAP\u001b[39;00m\n\u001b[32m    290\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(\u001b[38;5;28minput\u001b[39m, (SourceModel, Transduce)):\n\u001b[32m--> \u001b[39m\u001b[32m291\u001b[39m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mawait\u001b[39;00m wrap_single(\u001b[38;5;28minput\u001b[39m)\n\u001b[32m    292\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(\u001b[38;5;28minput\u001b[39m, AG):\n\u001b[32m    293\u001b[39m     \u001b[38;5;28minput\u001b[39m = \u001b[38;5;28minput\u001b[39m.states\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:222\u001b[39m, in \u001b[36mtransducible.._transducible..wrap_single\u001b[39m\u001b[34m(input_obj)\u001b[39m\n\u001b[32m    219\u001b[39m source_ag = source_ag_template.clone()\n\u001b[32m    220\u001b[39m source_ag.states = [pre.object]\n\u001b[32m--> \u001b[39m\u001b[32m222\u001b[39m target_ag = \u001b[38;5;28;01mawait\u001b[39;00m (target_ag_template << source_ag)\n\u001b[32m    224\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(target_ag) == \u001b[32m1\u001b[39m:\n\u001b[32m    225\u001b[39m     out = target_ag[\u001b[32m0\u001b[39m]\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/agentics.py:699\u001b[39m, in \u001b[36mAG.__lshift__\u001b[39m\u001b[34m(self, other)\u001b[39m\n\u001b[32m    691\u001b[39m output.transduce_fields = \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[32m    692\u001b[39m target_explanation.instructions = \u001b[33mf\u001b[39m\u001b[33m\"\"\"\u001b[39m\n\u001b[32m    693\u001b[39m \u001b[33mYou have previously transduced an object of type \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mother.atype.\u001b[34m__name__\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m (source) from an object of type \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m.atype.\u001b[34m__name__\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m (target).\u001b[39m\n\u001b[32m    694\u001b[39m \u001b[33mNow look back at both objects and provide a detailed explanation on how each field of the target object was logically derived from the source object.\u001b[39m\n\u001b[32m   (...)\u001b[39m\u001b[32m    697\u001b[39m \u001b[33mProvide high confidence score (1.0) only if you are certain that the transduction is logically correct and fully supported by the source object.\u001b[39m\n\u001b[32m    698\u001b[39m \u001b[33m\u001b[39m\u001b[33m\"\"\"\u001b[39m\n\u001b[32m--> \u001b[39m\u001b[32m699\u001b[39m explanation = \u001b[38;5;28;01mawait\u001b[39;00m (target_explanation << output.compose_states(other))\n\u001b[32m    701\u001b[39m \u001b[38;5;28mself\u001b[39m.explanations = explanation.states\n\u001b[32m    702\u001b[39m \u001b[38;5;28mself\u001b[39m.states = output.states\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/agentics.py:674\u001b[39m, in \u001b[36mAG.__lshift__\u001b[39m\u001b[34m(self, other)\u001b[39m\n\u001b[32m    672\u001b[39m         allowed = \u001b[38;5;28mself\u001b[39m.atype.model_fields.keys()  \u001b[38;5;66;03m# pydantic v2\u001b[39;00m\n\u001b[32m    673\u001b[39m         filtered = {k: v \u001b[38;5;28;01mfor\u001b[39;00m k, v \u001b[38;5;129;01min\u001b[39;00m data.items() \u001b[38;5;28;01mif\u001b[39;00m k \u001b[38;5;129;01min\u001b[39;00m allowed}\n\u001b[32m--> \u001b[39m\u001b[32m674\u001b[39m         merged = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43matype\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mfiltered\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m    676\u001b[39m         output.states.append(merged)\n\u001b[32m    677\u001b[39m \u001b[38;5;66;03m# elif is_str_or_list_of_str(other):\u001b[39;00m\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/.venv/lib/python3.12/site-packages/pydantic/main.py:250\u001b[39m, in \u001b[36mBaseModel.__init__\u001b[39m\u001b[34m(self, **data)\u001b[39m\n\u001b[32m    248\u001b[39m \u001b[38;5;66;03m# `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks\u001b[39;00m\n\u001b[32m    249\u001b[39m __tracebackhide__ = \u001b[38;5;28;01mTrue\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m250\u001b[39m validated_self = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m__pydantic_validator__\u001b[49m\u001b[43m.\u001b[49m\u001b[43mvalidate_python\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdata\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mself_instance\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[32m    251\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m validated_self:\n\u001b[32m    252\u001b[39m     warnings.warn(\n\u001b[32m    253\u001b[39m         \u001b[33m'\u001b[39m\u001b[33mA custom validator is returning a value other than `self`.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33m'\u001b[39m\n\u001b[32m    254\u001b[39m         \u001b[33m\"\u001b[39m\u001b[33mReturning anything other than `self` from a top level model validator isn\u001b[39m\u001b[33m'\u001b[39m\u001b[33mt supported when validating via `__init__`.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33m\"\u001b[39m\n\u001b[32m    255\u001b[39m         \u001b[33m'\u001b[39m\u001b[33mSee the `model_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.\u001b[39m\u001b[33m'\u001b[39m,\n\u001b[32m    256\u001b[39m         stacklevel=\u001b[32m2\u001b[39m,\n\u001b[32m    257\u001b[39m     )\n",
-      "\u001b[31mValidationError\u001b[39m: 1 validation error for Explanation\nrelevant_source_attributes\n  Input should be a valid dictionary [type=dict_type, input_value=[], input_type=list]\n    For further information visit https://errors.pydantic.dev/2.12/v/dict_type"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "classify_genre= Genre << With(Movie,\n",
     "                            provide_explanation=True)\n",
diff --git a/tutorials/map_reduce.ipynb b/tutorials/map_reduce.ipynb
index 71eac179c..8fea9b224 100644
--- a/tutorials/map_reduce.ipynb
+++ b/tutorials/map_reduce.ipynb
@@ -21,7 +21,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "id": "1",
    "metadata": {},
    "outputs": [],
@@ -63,45 +63,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": null,
    "id": "3",
    "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "bf4ad4f61c4a42fe9d28e36d6cb78084",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Output()"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "text/html": [
-       "
\n"
-      ],
-      "text/plain": []
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "(RomanNumber(roman_number='I'), None)\n",
-      "(RomanNumber(roman_number='II'), None)\n",
-      "(RomanNumber(roman_number='III'), None)\n",
-      "(RomanNumber(roman_number='IV'), None)\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from agentics.core.transducible_functions import transducible, Transduce\n",
     "\n",
@@ -135,132 +100,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": null,
    "id": "5",
    "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "7452adfa257d4262b574d89da740ba9f",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Output()"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "text/html": [
-       "
\n"
-      ],
-      "text/plain": []
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "572e450fb0d740059c69b17af2eb6155",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Output()"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "text/html": [
-       "
\n"
-      ],
-      "text/plain": []
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "e9d1a4c4d53044f783b3479730c5226f",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Output()"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "text/html": [
-       "
\n"
-      ],
-      "text/plain": []
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "c9b51140cd3f4f15930fb83422560039",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Output()"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "text/html": [
-       "
\n"
-      ],
-      "text/plain": []
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "(RomanNumber(roman_number='I'), None)\n",
-      "(RomanNumber(roman_number='II'), None)\n",
-      "(RomanNumber(roman_number='III'), None)\n",
-      "(RomanNumber(roman_number='IV'), None)\n",
-      "(RomanNumber(roman_number='V'), None)\n",
-      "(RomanNumber(roman_number='VI'), None)\n",
-      "(RomanNumber(roman_number='VII'), None)\n",
-      "(RomanNumber(roman_number='VIII'), None)\n",
-      "(RomanNumber(roman_number='IX'), None)\n",
-      "(RomanNumber(roman_number='X'), None)\n",
-      "(RomanNumber(roman_number='XI'), None)\n",
-      "(RomanNumber(roman_number='XII'), None)\n",
-      "(RomanNumber(roman_number='XIII'), None)\n",
-      "(RomanNumber(roman_number='XIV'), None)\n",
-      "(RomanNumber(roman_number='XV'), None)\n",
-      "(RomanNumber(roman_number='XVI'), None)\n",
-      "(RomanNumber(roman_number='XVII'), None)\n",
-      "(RomanNumber(roman_number='XVIII'), None)\n",
-      "(RomanNumber(roman_number='XIX'), None)\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from agentics.core.transducible_functions import With\n",
     "to_roman_number = RomanNumber << With(Number,\n",
@@ -296,195 +139,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": null,
    "id": "7",
    "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "20dfa43c7826436da08de3f409cee78c",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Output()"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "text/html": [
-       "
\n"
-      ],
-      "text/plain": []
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "bfc7d63de00f4dcdba7411fe1ac9e684",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/plain": [
-       "Output()"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "data": {
-      "text/html": [
-       "
\n"
-      ],
-      "text/plain": []
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "2026-02-13 14:39:53.744 | DEBUG    | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 1\n",
-      "2026-02-13 14:39:53.744 | DEBUG    | agentics.core.async_executor:execute:67 - retrying 1 state(s), attempt 2\n",
-      "2026-02-13 14:39:53.745 | DEBUG    | agentics.core.agentics:amap:446 - ⚠️ AMAP generated an error processing state # 0: 19 validation errors for ReducedOtherAtype\n",
-      "reduced_other_states.0\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"roman_number='I'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.1\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom..._number='II'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.2\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom...number='III'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.3\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom..._number='IV'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.4\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"roman_number='V'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.5\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom..._number='VI'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.6\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom...number='VII'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.7\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom...umber='VIII'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.8\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom..._number='IX'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.9\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"roman_number='X'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.10\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom..._number='XI'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.11\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom...number='XII'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.12\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom...umber='XIII'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.13\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom...number='XIV'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.14\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom..._number='XV'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.15\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom...number='XVI'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.16\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom...umber='XVII'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.17\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom...mber='XVIII'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n",
-      "reduced_other_states.18\n",
-      "  Input should be a valid dictionary or instance of Number [type=model_type, input_value=Transduce(object=[\n",
-      "  \"rom...number='XIX'\",\n",
-      "  null\n",
-      "]), input_type=Transduce]\n",
-      "    For further information visit https://errors.pydantic.dev/2.12/v/model_type\n"
-     ]
-    },
-    {
-     "ename": "AttributeError",
-     "evalue": "'list' object has no attribute 'model_dump'",
-     "output_type": "error",
-     "traceback": [
-      "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
-      "\u001b[31mAttributeError\u001b[39m                            Traceback (most recent call last)",
-      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[8]\u001b[39m\u001b[32m, line 6\u001b[39m\n\u001b[32m      1\u001b[39m to_number = Number << RomanNumber\n\u001b[32m      2\u001b[39m reduce = Number << With(Number, \n\u001b[32m      3\u001b[39m     areduce=\u001b[38;5;28;01mTrue\u001b[39;00m,\n\u001b[32m      4\u001b[39m     instructions=\u001b[33m\"\u001b[39m\u001b[33mreturn the sum of the input numbers\u001b[39m\u001b[33m'\u001b[39m\u001b[33m number fields\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m----> \u001b[39m\u001b[32m6\u001b[39m \u001b[38;5;28;01mawait\u001b[39;00m reduce(\u001b[38;5;28;01mawait\u001b[39;00m to_number(roman_numbers ))\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/transducible_functions.py:273\u001b[39m, in \u001b[36mtransducible.._transducible..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m    270\u001b[39m source_ag = source_ag_template.clone()\n\u001b[32m    271\u001b[39m source_ag.states = [\u001b[38;5;28minput\u001b[39m]\n\u001b[32m--> \u001b[39m\u001b[32m273\u001b[39m intermediate = \u001b[38;5;28;01mawait\u001b[39;00m source_ag.amap(wrap_single)\n\u001b[32m    275\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(intermediate) == \u001b[32m1\u001b[39m:\n\u001b[32m    276\u001b[39m     \u001b[38;5;28;01mreturn\u001b[39;00m intermediate[\u001b[32m0\u001b[39m]\n",
-      "\u001b[36mFile \u001b[39m\u001b[32m~/gitAgentics/documentation/Agentics/src/agentics/core/agentics.py:450\u001b[39m, in \u001b[36mAG.amap\u001b[39m\u001b[34m(self, func, timeout)\u001b[39m\n\u001b[32m    446\u001b[39m     logger.debug(\n\u001b[32m    447\u001b[39m         \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33m⚠️ AMAP generated an error processing state # \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mi\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mresult\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m\"\u001b[39m\n\u001b[32m    448\u001b[39m     )\n\u001b[32m    449\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m i < \u001b[38;5;28mlen\u001b[39m(\u001b[38;5;28mself\u001b[39m.states):\n\u001b[32m--> \u001b[39m\u001b[32m450\u001b[39m     _states.append(\u001b[38;5;28mself\u001b[39m.atype(**\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mstates\u001b[49m\u001b[43m[\u001b[49m\u001b[43mi\u001b[49m\u001b[43m]\u001b[49m\u001b[43m.\u001b[49m\u001b[43mmodel_dump\u001b[49m()))\n\u001b[32m    451\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m    452\u001b[39m     _states.append(\u001b[38;5;28mself\u001b[39m.atype())\n",
-      "\u001b[31mAttributeError\u001b[39m: 'list' object has no attribute 'model_dump'"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "to_number = Number << RomanNumber\n",
     "reduce = Number << With(Number, \n",
@@ -510,20 +168,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": null,
    "id": "9",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "number=100\n",
-      "number=100\n",
-      "number=100\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "from agentics.core.transducible_functions import transducible, Transduce\n",
     "@transducible(areduce=True)\n",
diff --git a/tutorials/semantic_operators.ipynb b/tutorials/semantic_operators.ipynb
index 823e562cb..7a648dcfd 100644
--- a/tutorials/semantic_operators.ipynb
+++ b/tutorials/semantic_operators.ipynb
@@ -24,7 +24,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "c70db513",
+   "id": "2",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -79,7 +79,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "af661f99",
+   "id": "4",
    "metadata": {},
    "source": [
     "### Semantic Filter\n",
@@ -90,7 +90,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "34bab288",
+   "id": "5",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -126,7 +126,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "18401986",
+   "id": "6",
    "metadata": {},
    "source": [
     "### Semantic Aggregator\n",
@@ -137,7 +137,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "01175466",
+   "id": "7",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -169,7 +169,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "461a2acb",
+   "id": "8",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -189,7 +189,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2fe9d161",
+   "id": "9",
    "metadata": {},
    "source": [
     "### Integration with Agentics Workflows\n",
@@ -200,7 +200,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "9f40bfa5",
+   "id": "10",
    "metadata": {},
    "outputs": [],
    "source": [