Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/explanation/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ Object-Relational Mappers treat large objects as opaque binary blobs or leave fi

- Relational semantics apply: referential integrity, cascading deletes, query filtering
- Multiple access patterns: lazy `ObjectRef`, streaming via fsspec, explicit download
- Two addressing modes: path-addressed (by primary key) and hash-addressed (deduplicated)
- Two addressing modes: schema-addressed (by primary key) and hash-addressed (deduplicated)

The object store is part of the relational model — queryable and integrity-protected like any other attribute.

Expand Down
2 changes: 1 addition & 1 deletion src/explanation/type-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,7 @@ class GraphCodec(dj.Codec):
"""Store NetworkX graphs."""
name = "graph"

def get_dtype(self, is_external):
def get_dtype(self, is_store):
return "<blob>"

def encode(self, graph, *, key=None, store_name=None):
Expand Down
2 changes: 1 addition & 1 deletion src/how-to/define-tables.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ Codecs serialize Python objects to database storage. Use angle brackets for code
| `<blob@store>` | Serialized objects in object storage |
| `<attach>` | File attachments in database |
| `<attach@store>` | File attachments in object storage |
| `<object@store>` | Files/folders via ObjectRef (path-addressed, supports Zarr/HDF5) |
| `<object@store>` | Files/folders via ObjectRef (schema-addressed, supports Zarr/HDF5) |

Example:

Expand Down
2 changes: 1 addition & 1 deletion src/how-to/use-object-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ class Dataset(dj.Manual):
"""
```

Use path-addressed storage for:
Use schema-addressed storage for:

- Zarr arrays (chunked, appendable)
- HDF5 files
Expand Down
14 changes: 7 additions & 7 deletions src/reference/specs/type-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -410,8 +410,8 @@ class FilepathCodec(dj.Codec):
"""Store-relative file references. External only."""
name = "filepath"

def get_dtype(self, is_external: bool) -> str:
if not is_external:
def get_dtype(self, is_store: bool) -> str:
if not is_store:
raise DataJointError("<filepath> requires @store")
return "json"

Expand Down Expand Up @@ -512,8 +512,8 @@ class BlobCodec(dj.Codec):
"""Serialized Python objects. Supports internal and external."""
name = "blob"

def get_dtype(self, is_external: bool) -> str:
return "<hash>" if is_external else "bytes"
def get_dtype(self, is_store: bool) -> str:
return "<hash>" if is_store else "bytes"

def encode(self, value, *, key=None, store_name=None) -> bytes:
from . import blob
Expand Down Expand Up @@ -551,8 +551,8 @@ class AttachCodec(dj.Codec):
"""File attachment with filename. Supports in-table and in-store."""
name = "attach"

def get_dtype(self, is_external: bool) -> str:
return "<hash>" if is_external else "bytes"
def get_dtype(self, is_store: bool) -> str:
return "<hash>" if is_store else "bytes"

def encode(self, filepath, *, key=None, store_name=None) -> bytes:
path = Path(filepath)
Expand Down Expand Up @@ -662,7 +662,7 @@ def garbage_collect(store_name):
2. **Core types are scientist-friendly**: `float32`, `int8`, `bool`, `bytes` instead of `FLOAT`, `TINYINT`, `LONGBLOB`
3. **Codecs use angle brackets**: `<blob>`, `<object@store>`, `<filepath@main>` - distinguishes from core types
4. **`@` indicates in-store storage**: No `@` = database, `@` present = object store
5. **`get_dtype(is_external)` method**: Codecs resolve dtype at declaration time based on storage mode
5. **`get_dtype(is_store)` method**: Codecs resolve dtype at declaration time based on storage mode
6. **Codecs are composable**: `<blob@>` uses `<hash@>`, which uses `json`
7. **Built-in in-store codecs use JSON dtype**: Stores metadata (path, hash, store name, etc.)
8. **Two OAS regions**: object (PK-addressed) and hash (hash-addressed) within managed stores
Expand Down