[FEAT] Incremental Dataset Import#141
Merged
JonnyTran merged 24 commits intoAug 29, 2025
Merged
Conversation
…ialog. - Added DatasetUpdateDialog component for updating datasets, integrated data source selection, and improved dataset configuration forms. - Updated translations for button labels and added validation for compatible datasets.
- Introduced UpdateDatasetUseCase for handling dataset updates. - Enhanced DatasetConfigurationForm and DatasetUpdateDialog to support source and target dataset selection. - Added error handling and validation for dataset updates in the relevant components. - Updated useDatasetConfigurationForm to include the new update method.
- Renamed DatasetConfigurationDialog to DatasetCreateDialog for clarity. - Added new DatasetCreateDialog component to handle dataset creation with improved UI and validation. - Updated useDatasetConfigurationNameAndWorkspace to remove unused imports.
- Prioritize specific error messages in AxiosErrorHandler based on business logic, detailed messages, and generic HTTP status messages. - Update DocumentRepository to include a new error constant for listing documents and adjust error handling accordingly. - Modify error detail in documents.py to provide more specific feedback when no documents are found.
- Updated DatasetConfigurationForm, DatasetConfigurationMetadataSelector, and DatasetCreateDialog to use TypeScript for improved type safety. - Enhanced validator functions in DatasetConfigurationForm and DatasetCreateDialog to specify parameter types.
…xternal_id with source_id and target_id. - Updated DatasetCreation to use source_id and target_id for improved clarity in dataset mappings. - Modified ImportHistoryDatasetBuilder to align with the new DatasetCreation structure, ensuring proper mapping of source_id and target_id.
This reverts commit 3ecb423.
- Removed deprecated document methods from WorkspacesAPI. - Updated document creation logic to directly use the Document class. - Simplified the add_document method in the Workspace class. - Cleaned up test cases related to document operations in WorkspacesAPI.
JonnyTran
added a commit
that referenced
this pull request
Aug 29, 2025
* updates * added mapping to DatasetGetterDict * added mapping to DatasetModel * __fetch_dataset_related_attributes * latest * updated DatasetMapping * latest * latest * [FEAT] Incremental Dataset Import (#141) * added GET "/datasets/compatible" * Add GetImportCompatibleDatasets use case and integrate into dataset configuration * Enhance dataset creation workflow with update functionality and new dialog. - Added DatasetUpdateDialog component for updating datasets, integrated data source selection, and improved dataset configuration forms. - Updated translations for button labels and added validation for compatible datasets. * latest * Implement dataset update functionality and improve error handling. - Introduced UpdateDatasetUseCase for handling dataset updates. - Enhanced DatasetConfigurationForm and DatasetUpdateDialog to support source and target dataset selection. - Added error handling and validation for dataset updates in the relevant components. - Updated useDatasetConfigurationForm to include the new update method. * refactor * Refactor dataset creation components and introduce DatasetCreateDialog. - Renamed DatasetConfigurationDialog to DatasetCreateDialog for clarity. - Added new DatasetCreateDialog component to handle dataset creation with improved UI and validation. - Updated useDatasetConfigurationNameAndWorkspace to remove unused imports. * refactoring * Enhance error handling in AxiosErrorHandler and DocumentRepository. - Prioritize specific error messages in AxiosErrorHandler based on business logic, detailed messages, and generic HTTP status messages. - Update DocumentRepository to include a new error constant for listing documents and adjust error handling accordingly. - Modify error detail in documents.py to provide more specific feedback when no documents are found. * Refactor dataset configuration components to support TypeScript. - Updated DatasetConfigurationForm, DatasetConfigurationMetadataSelector, and DatasetCreateDialog to use TypeScript for improved type safety. - Enhanced validator functions in DatasetConfigurationForm and DatasetCreateDialog to specify parameter types. * fix extralit/unit tests * fix tests * fix tests * latest * fix tests * fix DatasetMapping * Refactor DatasetCreation and ImportHistoryDatasetBuilder to replace external_id with source_id and target_id. - Updated DatasetCreation to use source_id and target_id for improved clarity in dataset mappings. - Modified ImportHistoryDatasetBuilder to align with the new DatasetCreation structure, ensuring proper mapping of source_id and target_id. * test fixes * fix tests * fix tests * Revert "fix tests" This reverts commit 3ecb423. * fix tests * fix tests * Refactor document handling in WorkspacesAPI and update related tests - Removed deprecated document methods from WorkspacesAPI. - Updated document creation logic to directly use the Document class. - Simplified the add_document method in the Workspace class. - Cleaned up test cases related to document operations in WorkspacesAPI.
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant enhancements to the dataset creation workflow in the frontend, focusing on adding support for updating existing datasets with imported data, improving dialog management, and refining type safety. The changes include the addition of a new update dialog component, refactoring and renaming of the creation dialog, and updates to form logic and UI to support both creation and update workflows. TypeScript usage is improved across relevant files, and minor UX improvements are made.
Dataset Update Workflow Enhancements:
DatasetUpdateDialog.vuecomponent to support updating existing datasets with imported data, including workspace selection, compatible dataset selection, field mapping preview, and import summary.DatasetConfigurationForm.vueto support both dataset creation and update workflows, including new methods and state for managing dialog visibility and workflow mode. [1] [2] [3]Dialog Refactoring and Type Safety:
DatasetConfigurationDialog.vuetoDatasetCreateDialog.vueand updated all references for clarity and separation of concerns.UI/UX Improvements:
DatasetConfigurationForm.vuefor better spacing and clarity.Minor Fixes and Cleanups:
useDocumentViewModel.tsto streamline error handling.These changes collectively improve the flexibility, usability, and maintainability of the dataset management UI.