Skip to content

[FEAT] Incremental Dataset Import (#140)#144

Merged
JonnyTran merged 1 commit into
developfrom
releases/v0.6.1
Aug 29, 2025
Merged

[FEAT] Incremental Dataset Import (#140)#144
JonnyTran merged 1 commit into
developfrom
releases/v0.6.1

Conversation

@JonnyTran
Copy link
Copy Markdown
Member

  • updates

  • added mapping to DatasetGetterDict

  • added mapping to DatasetModel

  • __fetch_dataset_related_attributes

  • latest

  • updated DatasetMapping

  • latest

  • latest

  • [FEAT] Incremental Dataset Import ([FEAT] Incremental Dataset Import #141)

  • added GET "/datasets/compatible"

  • Add GetImportCompatibleDatasets use case and integrate into dataset configuration

  • Enhance dataset creation workflow with update functionality and new dialog.

  • Added DatasetUpdateDialog component for updating datasets, integrated data source selection, and improved dataset configuration forms. - Updated translations for button labels and added validation for compatible datasets.
  • latest

  • Implement dataset update functionality and improve error handling.

  • Introduced UpdateDatasetUseCase for handling dataset updates.
  • Enhanced DatasetConfigurationForm and DatasetUpdateDialog to support source and target dataset selection.
  • Added error handling and validation for dataset updates in the relevant components.
  • Updated useDatasetConfigurationForm to include the new update method.
  • refactor

  • Refactor dataset creation components and introduce DatasetCreateDialog.

  • Renamed DatasetConfigurationDialog to DatasetCreateDialog for clarity.
  • Added new DatasetCreateDialog component to handle dataset creation with improved UI and validation.
  • Updated useDatasetConfigurationNameAndWorkspace to remove unused imports.
  • refactoring

  • Enhance error handling in AxiosErrorHandler and DocumentRepository.

  • Prioritize specific error messages in AxiosErrorHandler based on business logic, detailed messages, and generic HTTP status messages.
  • Update DocumentRepository to include a new error constant for listing documents and adjust error handling accordingly.
  • Modify error detail in documents.py to provide more specific feedback when no documents are found.
  • Refactor dataset configuration components to support TypeScript.
  • Updated DatasetConfigurationForm, DatasetConfigurationMetadataSelector, and DatasetCreateDialog to use TypeScript for improved type safety.
  • Enhanced validator functions in DatasetConfigurationForm and DatasetCreateDialog to specify parameter types.
  • fix extralit/unit tests

  • fix tests

  • fix tests

  • latest

  • fix tests

  • fix DatasetMapping

  • Refactor DatasetCreation and ImportHistoryDatasetBuilder to replace external_id with source_id and target_id.

  • Updated DatasetCreation to use source_id and target_id for improved clarity in dataset mappings.
  • Modified ImportHistoryDatasetBuilder to align with the new DatasetCreation structure, ensuring proper mapping of source_id and target_id.
  • test fixes

  • fix tests

  • fix tests

  • Revert "fix tests"

This reverts commit 3ecb423.

  • fix tests

  • fix tests

  • Refactor document handling in WorkspacesAPI and update related tests

  • Removed deprecated document methods from WorkspacesAPI.
  • Updated document creation logic to directly use the Document class.
  • Simplified the add_document method in the Workspace class.
  • Cleaned up test cases related to document operations in WorkspacesAPI.

Description

Related Tickets & Documents

Closes #

What type of PR is this? (check all applicable)

  • Refactor
  • Feature
  • Bug Fix
  • Optimization
  • Documentation Update

Steps to QA

Added/updated tests?

  • Yes
  • No, and this is why: please replace this line with details on why tests
    have not been included
  • I need help with writing tests

Added/updated documentations?

  • Yes
  • No, and this is why: please replace this line with details on why tests
    have not been included
  • I need help with writing docs

Checklist

@JonnyTran JonnyTran requested a review from a team as a code owner August 29, 2025 07:16
@JonnyTran JonnyTran merged commit e915eb6 into develop Aug 29, 2025
13 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant