Skip to content

view mapped fields only plugin#591

Merged
bangarang merged 1 commit intomainfrom
feat/view-mapped-plugin
Aug 8, 2024
Merged

view mapped fields only plugin#591
bangarang merged 1 commit intomainfrom
feat/view-mapped-plugin

Conversation

@dboskovic
Copy link
Contributor

No description provided.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Aug 7, 2024

Walkthrough

The changes introduce foundational documentation and configuration for the @flatfile/plugin-job-handler and plugins/view-mapped directories. A new changelog and README file enhance project organization, while a Rollup configuration simplifies the build process. The main feature added is the viewMappedPlugin, which filters workbook data to display only user-mapped fields upon completing a mapping job, improving user experience by managing data visibility efficiently.

Changes

File Path Summary
CHANGELOG.md Introduces a changelog for version 1.0.1 of @flatfile/plugin-job-handler.
README.md New placeholder file indicating upcoming documentation for plugins/view-mapped.
rollup.config.mjs New Rollup configuration file for building the project using default settings.
src/index.ts New entry point re-exporting functionality from view-mapped.
src/view-mapped.ts Introduces viewMappedPlugin function to filter displayed fields in the workbook.

Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 3f5a979 and e10ee8c.

Files ignored due to path filters (2)
  • package-lock.json is excluded by !**/package-lock.json, !**/*.json
  • plugins/view-mapped/package.json is excluded by !**/*.json
Files selected for processing (5)
  • plugins/view-mapped/CHANGELOG.md (1 hunks)
  • plugins/view-mapped/README.md (1 hunks)
  • plugins/view-mapped/rollup.config.mjs (1 hunks)
  • plugins/view-mapped/src/index.ts (1 hunks)
  • plugins/view-mapped/src/view-mapped.ts (1 hunks)
Files skipped from review due to trivial changes (4)
  • plugins/view-mapped/CHANGELOG.md
  • plugins/view-mapped/README.md
  • plugins/view-mapped/rollup.config.mjs
  • plugins/view-mapped/src/index.ts
Additional comments not posted (2)
plugins/view-mapped/src/view-mapped.ts (2)

7-8: Good structure for a plugin function.

The viewMappedPlugin function is well-structured and clearly defines its purpose of filtering workbook data to show only mapped fields. The use of event listeners is appropriate for handling job events.


10-26: Ensure robust error handling for API calls.

While the logic for creating a custom job is clear, consider adding error handling for the api.jobs.create call to manage potential failures in job creation.


Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range, codebase verification and nitpick comments (1)
plugins/view-mapped/src/view-mapped.ts (1)

112-119: Improve error message clarity.

The error message in the catch block could be more descriptive to help users understand the issue and possible next steps. Consider including more context or potential resolutions.

-  'An error occured while updating the workbook. See Event Logs.'
+  'An error occurred while updating the workbook. Please check the Event Logs for more details and ensure all configurations are correct.'

Comment on lines +30 to +121
listener
.filter({ job: 'workbook:viewMappedFieldsOnly' })
.on('job:ready', async ({ context: { jobId, workbookId } }) => {
try {
// First, we acknowledge the job
await api.jobs.ack(jobId, {
info: 'Updating the table to only view mapped fields',
progress: 10,
})

// Retrieving the info on the custom job we created in the listener above, and storing that info in its own "customJobInfo" variable
const customJobInfo = await api.jobs.get(jobId)

// From "customJobInfo" variable, retrieving the jobId specifically of the mapping job that completed, and storing it in its own "mappingJobId" variable
const mappingJobId = customJobInfo.data.input.mappingJobId

// Obtaining the mapping job's execution plan to later extract "fieldMapping" out of it, which tells us which fields were mapped in the Matching step
const jobPlan = await api.jobs.getExecutionPlan(mappingJobId)

// Initializing an empty array to store the keys of the mapped fields
const mappedFields = []

// Iterating through all destination fields that are mapped and extracting their field keys. Then, pushing keys of mapped fields to the "mappedFields" variable
for (let i = 0; i < jobPlan.data.plan.fieldMapping.length; i++) {
const destinationFieldKey =
jobPlan.data.plan.fieldMapping[i].destinationField.key

mappedFields.push(destinationFieldKey)
}
// Making an API call to only get the "data" property out of the response, and saving it as its own "fetchedWorkbook" variable
// We need to make this API call and cannot just use what's inside of "workbookOne" because we need data in a specific format
const { data: workbook } = await api.workbooks.get(workbookId)

// Looping through all sheets of the Workbook One. For all fields that are mapped, updating those fields' metadata to "{mapped: true}"
workbook.sheets.forEach((sheet) => {
sheet.config.fields.forEach((field) => {
if (mappedFields.includes(field.key)) {
field.metadata = { mapped: true }
}
})
})

// Looping over each sheet in "workbook" and filtering for fields with metadata "mapped: true". Saving mapped fields per each sheet inside of "filteredWorkbookFields" varibable
const filteredWorkbookFields = workbook.sheets.map((sheet) => {
const fields = sheet.config.fields.filter(
(field) => field.metadata && field.metadata.mapped === true
)
return fields.length > 0 ? fields : null
})

// Updating each sheet in a workbook to only contain fields that a user mapped. This ensures that when the table with data loads, only mapped fields will be displayed
await api.workbooks.update(workbookId, {
// Keeping other non-sheet elements of the workbook untouched (Workbook name, its Submit action, etc)
...workbook,

// Mapping over each sheet to update each to only contain fields that are inside of "filteredWorkbookFields" variable (that have metadata "{mapped: true})"
sheets: workbook.sheets.map((sheet, index) => {
const mappedWorkbookFields = filteredWorkbookFields[index]

// If there are no mapped fields, returning the original sheet structure
if (!mappedWorkbookFields) {
return sheet
}

// If there are mapped fields, returning all properties of the original sheet but updating the "fields" property to the mapped fields
return {
...sheet,
config: {
...sheet.config,
fields: mappedWorkbookFields,
},
}
}),
})

// Completing the job with an appropriate message to the user
await api.jobs.complete(jobId, {
outcome: {
message: 'Table update complete. Please audit the data',
acknowledge: true,
},
})
} catch (error) {
// If something goes wrong while executing the custom job, we fail the job with a message on what next steps to take
await api.jobs.fail(jobId, {
outcome: {
message:
'An error occured while updating the workbook. See Event Logs.',
},
})
}
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optimize field mapping extraction.

The current implementation uses a loop to extract mapped fields. Consider using array methods like map and reduce for a more functional approach, which can improve readability and performance.

-  for (let i = 0; i < jobPlan.data.plan.fieldMapping.length; i++) {
-    const destinationFieldKey =
-      jobPlan.data.plan.fieldMapping[i].destinationField.key
-    mappedFields.push(destinationFieldKey)
-  }
+  const mappedFields = jobPlan.data.plan.fieldMapping.map(
+    (mapping) => mapping.destinationField.key
+  )
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
listener
.filter({ job: 'workbook:viewMappedFieldsOnly' })
.on('job:ready', async ({ context: { jobId, workbookId } }) => {
try {
// First, we acknowledge the job
await api.jobs.ack(jobId, {
info: 'Updating the table to only view mapped fields',
progress: 10,
})
// Retrieving the info on the custom job we created in the listener above, and storing that info in its own "customJobInfo" variable
const customJobInfo = await api.jobs.get(jobId)
// From "customJobInfo" variable, retrieving the jobId specifically of the mapping job that completed, and storing it in its own "mappingJobId" variable
const mappingJobId = customJobInfo.data.input.mappingJobId
// Obtaining the mapping job's execution plan to later extract "fieldMapping" out of it, which tells us which fields were mapped in the Matching step
const jobPlan = await api.jobs.getExecutionPlan(mappingJobId)
// Initializing an empty array to store the keys of the mapped fields
const mappedFields = []
// Iterating through all destination fields that are mapped and extracting their field keys. Then, pushing keys of mapped fields to the "mappedFields" variable
for (let i = 0; i < jobPlan.data.plan.fieldMapping.length; i++) {
const destinationFieldKey =
jobPlan.data.plan.fieldMapping[i].destinationField.key
mappedFields.push(destinationFieldKey)
}
// Making an API call to only get the "data" property out of the response, and saving it as its own "fetchedWorkbook" variable
// We need to make this API call and cannot just use what's inside of "workbookOne" because we need data in a specific format
const { data: workbook } = await api.workbooks.get(workbookId)
// Looping through all sheets of the Workbook One. For all fields that are mapped, updating those fields' metadata to "{mapped: true}"
workbook.sheets.forEach((sheet) => {
sheet.config.fields.forEach((field) => {
if (mappedFields.includes(field.key)) {
field.metadata = { mapped: true }
}
})
})
// Looping over each sheet in "workbook" and filtering for fields with metadata "mapped: true". Saving mapped fields per each sheet inside of "filteredWorkbookFields" varibable
const filteredWorkbookFields = workbook.sheets.map((sheet) => {
const fields = sheet.config.fields.filter(
(field) => field.metadata && field.metadata.mapped === true
)
return fields.length > 0 ? fields : null
})
// Updating each sheet in a workbook to only contain fields that a user mapped. This ensures that when the table with data loads, only mapped fields will be displayed
await api.workbooks.update(workbookId, {
// Keeping other non-sheet elements of the workbook untouched (Workbook name, its Submit action, etc)
...workbook,
// Mapping over each sheet to update each to only contain fields that are inside of "filteredWorkbookFields" variable (that have metadata "{mapped: true})"
sheets: workbook.sheets.map((sheet, index) => {
const mappedWorkbookFields = filteredWorkbookFields[index]
// If there are no mapped fields, returning the original sheet structure
if (!mappedWorkbookFields) {
return sheet
}
// If there are mapped fields, returning all properties of the original sheet but updating the "fields" property to the mapped fields
return {
...sheet,
config: {
...sheet.config,
fields: mappedWorkbookFields,
},
}
}),
})
// Completing the job with an appropriate message to the user
await api.jobs.complete(jobId, {
outcome: {
message: 'Table update complete. Please audit the data',
acknowledge: true,
},
})
} catch (error) {
// If something goes wrong while executing the custom job, we fail the job with a message on what next steps to take
await api.jobs.fail(jobId, {
outcome: {
message:
'An error occured while updating the workbook. See Event Logs.',
},
})
}
})
listener
.filter({ job: 'workbook:viewMappedFieldsOnly' })
.on('job:ready', async ({ context: { jobId, workbookId } }) => {
try {
// First, we acknowledge the job
await api.jobs.ack(jobId, {
info: 'Updating the table to only view mapped fields',
progress: 10,
})
// Retrieving the info on the custom job we created in the listener above, and storing that info in its own "customJobInfo" variable
const customJobInfo = await api.jobs.get(jobId)
// From "customJobInfo" variable, retrieving the jobId specifically of the mapping job that completed, and storing it in its own "mappingJobId" variable
const mappingJobId = customJobInfo.data.input.mappingJobId
// Obtaining the mapping job's execution plan to later extract "fieldMapping" out of it, which tells us which fields were mapped in the Matching step
const jobPlan = await api.jobs.getExecutionPlan(mappingJobId)
// Using map to extract the keys of the mapped fields
const mappedFields = jobPlan.data.plan.fieldMapping.map(
(mapping) => mapping.destinationField.key
)
// Making an API call to only get the "data" property out of the response, and saving it as its own "fetchedWorkbook" variable
// We need to make this API call and cannot just use what's inside of "workbookOne" because we need data in a specific format
const { data: workbook } = await api.workbooks.get(workbookId)
// Looping through all sheets of the Workbook One. For all fields that are mapped, updating those fields' metadata to "{mapped: true}"
workbook.sheets.forEach((sheet) => {
sheet.config.fields.forEach((field) => {
if (mappedFields.includes(field.key)) {
field.metadata = { mapped: true }
}
})
})
// Looping over each sheet in "workbook" and filtering for fields with metadata "mapped: true". Saving mapped fields per each sheet inside of "filteredWorkbookFields" varibable
const filteredWorkbookFields = workbook.sheets.map((sheet) => {
const fields = sheet.config.fields.filter(
(field) => field.metadata && field.metadata.mapped === true
)
return fields.length > 0 ? fields : null
})
// Updating each sheet in a workbook to only contain fields that a user mapped. This ensures that when the table with data loads, only mapped fields will be displayed
await api.workbooks.update(workbookId, {
// Keeping other non-sheet elements of the workbook untouched (Workbook name, its Submit action, etc)
...workbook,
// Mapping over each sheet to update each to only contain fields that are inside of "filteredWorkbookFields" variable (that have metadata "{mapped: true})"
sheets: workbook.sheets.map((sheet, index) => {
const mappedWorkbookFields = filteredWorkbookFields[index]
// If there are no mapped fields, returning the original sheet structure
if (!mappedWorkbookFields) {
return sheet
}
// If there are mapped fields, returning all properties of the original sheet but updating the "fields" property to the mapped fields
return {
...sheet,
config: {
...sheet.config,
fields: mappedWorkbookFields,
},
}
}),
})
// Completing the job with an appropriate message to the user
await api.jobs.complete(jobId, {
outcome: {
message: 'Table update complete. Please audit the data',
acknowledge: true,
},
})
} catch (error) {
// If something goes wrong while executing the custom job, we fail the job with a message on what next steps to take
await api.jobs.fail(jobId, {
outcome: {
message:
'An error occured while updating the workbook. See Event Logs.',
},
})
}
})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants