Skip to content

Move AsyncMinHashLSH from experimental to datasketch.aio #302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ekzhu merged 3 commits into from
Jan 20, 2026
Merged

Move AsyncMinHashLSH from experimental to datasketch.aio #302

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1e8e89a
Initial plan
Copilot Jan 20, 2026
869fcc9
Update documentation and fix RedisStorage import issue
Copilot Jan 20, 2026
3f60b3a
Add clarifying comments for optional Redis dependency handling
Copilot Jan 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions datasketch/aio/storage.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,14 @@
from abc import ABCMeta
from itertools import chain

from datasketch.storage import OrderedStorage, RedisStorage, Storage, UnorderedStorage, _random_name
from datasketch.storage import OrderedStorage, Storage, UnorderedStorage, _random_name

# RedisStorage is only available when redis package is installed (optional dependency)
# Import it conditionally to avoid ImportError when redis is not installed
try:
from datasketch.storage import RedisStorage
except ImportError:
RedisStorage = None

ABC = ABCMeta("ABC", (object,), {})

Expand Down Expand Up @@ -301,7 +308,9 @@ async def remove_val(self, key, val, **kwargs):
await self._collection.find_one_and_delete({"key": key, "vals": val})


if redis is not None:
# Redis-based async storage classes are only defined when both redis package
# and RedisStorage are available (optional dependencies)
if redis is not None and RedisStorage is not None:

class AsyncRedisBuffer(redis.client.Pipeline):
def __init__(self, connection_pool, response_callbacks, transaction, buffer_size, shard_hint=None):
Expand Down
2 changes: 1 addition & 1 deletion docs/documentation.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ MinHash LSH
Asynchronous MinHash LSH
------------------------

.. autoclass:: datasketch.experimental.aio.lsh.AsyncMinHashLSH
.. autoclass:: datasketch.aio.AsyncMinHashLSH
:members:
:special-members:

Expand Down
11 changes: 5 additions & 6 deletions docs/lsh.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,8 +237,7 @@ Asynchronous MinHash LSH at scale
---------------------------------

.. note::
The module supports Python version >=3.6, and is currently experimental.
So the interface may change slightly in the future.
The module supports Python version >=3.6.

This module may be useful if you want to process millions of text documents
in streaming/batch mode using asynchronous RESTful API (for example, aiohttp)
Expand All @@ -256,7 +255,7 @@ The Asynchronous MongoDB storage option can be configured using:

.. code:: python

from datasketch.experimental.aio.lsh import AsyncMinHashLSH
from datasketch.aio import AsyncMinHashLSH
from datasketch import MinHash

_storage = {'type': 'aiomongo', 'mongo': {'host': 'localhost', 'port': 27017, 'db': 'lsh_test'}}
Expand All @@ -277,7 +276,7 @@ The Asynchronous MongoDB storage option can be configured using:

.. code:: python

from datasketch.experimental.aio.lsh import AsyncMinHashLSH
from datasketch.aio import AsyncMinHashLSH
from datasketch import MinHash

_storage = {'type': 'aiomongo', 'mongo': {'host': 'localhost', 'port': 27017, 'db': 'lsh_test'}}
Expand Down Expand Up @@ -329,7 +328,7 @@ To create index for a large number of MinHashes using asynchronous MinHash LSH.

.. code:: python

from datasketch.experimental.aio.lsh import AsyncMinHashLSH
from datasketch.aio import AsyncMinHashLSH
from datasketch import MinHash

def chunk(it, size):
Expand All @@ -355,7 +354,7 @@ To bulk remove keys from LSH index using asynchronous MinHash LSH.

.. code:: python

from datasketch.experimental.aio.lsh import AsyncMinHashLSH
from datasketch.aio import AsyncMinHashLSH
from datasketch import MinHash

def chunk(it, size):
Expand Down