[2/3] Add AI Assistant Service - add the “Add Type Annotation" Python UDF action #2811

IamtherealBrian · 2024-08-29T21:51:25Z

This PR introduces a new AI-based feature called "Add Type Annotation" that enhances the user experience of the Python UDF on top of the Pyright language server. The following are related PR's:
-PR for the "Add the pyright language server" : #2797.
-PR for the "[1/3] Add AI Assistant Service - Ai Flag" : #2808
-PR for the "[3/3] Add AI Assistant Service - add the "Add All Type Annotation" Python UDF action" : #2812

Key changes:
Added a Monaco action in the frontend Python UDF editor to allow users to automatically add type annotations for an argument each time they select a single argument in their code.

Add Type Annotation:
This action requires the user to select a single argument in their Python UDF code. A UI with a suggestion for the selected argument will pop up. The user can choose to accept or decline the suggestion from OpenAI. If they accept the suggestion, it will be added to their code; otherwise, the code will remain unchanged. The suggestion is provided by OpenAI. After the backend receives the response from OpenAI, it will be forwarded to the frontend via a RESTful API.

Example:

7879e394439dc30d2e09baaf36062bf1.mp4

This is the first PR of the "Add AI Assistant Service" series, which will consist of 3 PRs in total. -PR for the "[2/3] Add AI Assistant Service - add the "Add Type Annotation" Python UDF action" : #2811 -PR for the "[3/3] Add AI Assistant Service - add the "Add All Type Annotation" Python UDF action" : #2812 This PR introduces an AI Assistant flag that allows users to switch between "none" (disabling the AI assistant) and "openai" (using OpenAI as the AI assistant). If the 'none' flag is selected, all AI features will be hidden and unavailable to the user. If the 'openai' flag is selected, AI features will be enabled and accessible in the Python UDF editor. (The implementation of these AI features will be covered in PR2 of 3 and PR3 of 3). **Key idea of each new file:** (1) AiAssistantManager.scala: Handling results corresponding to the two cases obtained from the flag. (2) AiAssistantResource.scala: Implement three RESTful API endpoints to manage AI flag. **Notice:** Since the AI features are in the second and third PRs, the RESTful API functionality in this PR cannot be tested yet, as it will only be used in the second and third PRs. However, if you just want to test the result, you can print the result of the boolean for the testing purpose by calling the frontend method `checkAiAssistantEnabled()`. --------- Co-authored-by: Xudong Wu <2373025856w@gmail.com>

yunyad

LGTM

chenlica

I revised the description of the PR.

Yicong-Huang

A major comment is that we should push prompt generation to backend. Also frontend logic please use Observables and Subscription to implement as Angular's recommendation. more to read https://stackoverflow.com/questions/37364973/what-is-the-difference-between-promises-and-observables

core/amber/src/main/scala/edu/uci/ics/texera/web/resource/aiassistant/AiAssistantResource.scala

core/gui/src/app/dashboard/service/user/ai-assistant/ai-assistant.service.ts

core/gui/src/app/workspace/component/code-editor-dialog/code-editor.component.ts

…#2823) Use `main.ts` which done by `Typefox` to start pyright instead of `startPyright.mjs`. The work done by `Typefox` can be referred to: https://github.com/TypeFox/monaco-languageclient/tree/main/packages/examples/src/python --------- Co-authored-by: Yicong Huang <17627829+Yicong-Huang@users.noreply.github.com> Co-authored-by: Yicong Huang <yicong.huang@observeinc.com>

Yicong-Huang

I left some comments. The frontend would require major changes.

core/amber/src/main/scala/edu/uci/ics/texera/web/resource/aiassistant/AiAssistantResource.scala

core/gui/src/app/dashboard/service/user/ai-assistant/ai-assistant.service.ts

core/gui/src/app/workspace/component/code-editor-dialog/code-editor.component.ts

core/gui/src/app/dashboard/service/user/ai-assistant/ai-assistant.service.ts

Yicong-Huang

LGTM. Left minor comments.

core/gui/src/app/workspace/component/code-editor-dialog/annotation-suggestion.component.ts

core/gui/src/app/workspace/component/code-editor-dialog/code-editor.component.ts

core/gui/src/app/workspace/service/ai-assistant/ai-assistant.service.ts

…thon UDF action (#2857) This PR introduces another AI feature call "Add All Type Annotation" that enhance the user experience of the Python UDF after integrating with the Pyright language server. You can refer to the related PR here: -PR for the "Add the pyright language server" : #2797. -PR for the "[1/3] Add AI Assistant Service - Ai Flag" : #2808 -PR for the "[2/3] Add AI Assistant Service - add the "Add Type Annotation" Python UDF action" : #2811 **Key change:** Added another Monaco action in the frontend Python UDF editor to allow users to automatically add all type annotations for the arguments within their selected code block. **Add All Type Annotation:** This action requires the user to select a code block in their Python UDF code. The action will automatically identify all arguments without type annotations and provide type annotation suggestions for each argument sequentially. The selected code block will be sent to the backend for Python Abstract Syntax Tree (AST) analysis to locate arguments without type annotations, and the result will be sent back to the frontend via a RESTful API. If the user selects a code block that has no arguments or where all the arguments already have type annotations, the code will remain unchanged. Example: https://github.com/user-attachments/assets/d70c1b02-b309-4359-a267-0a5eef831316

This is the first PR of the "Add AI Assistant Service" series, which will consist of 3 PRs in total. -PR for the "[2/3] Add AI Assistant Service - add the "Add Type Annotation" Python UDF action" : #2811 -PR for the "[3/3] Add AI Assistant Service - add the "Add All Type Annotation" Python UDF action" : #2812 This PR introduces an AI Assistant flag that allows users to switch between "none" (disabling the AI assistant) and "openai" (using OpenAI as the AI assistant). If the 'none' flag is selected, all AI features will be hidden and unavailable to the user. If the 'openai' flag is selected, AI features will be enabled and accessible in the Python UDF editor. (The implementation of these AI features will be covered in PR2 of 3 and PR3 of 3). **Key idea of each new file:** (1) AiAssistantManager.scala: Handling results corresponding to the two cases obtained from the flag. (2) AiAssistantResource.scala: Implement three RESTful API endpoints to manage AI flag. **Notice:** Since the AI features are in the second and third PRs, the RESTful API functionality in this PR cannot be tested yet, as it will only be used in the second and third PRs. However, if you just want to test the result, you can print the result of the boolean for the testing purpose by calling the frontend method `checkAiAssistantEnabled()`. --------- Co-authored-by: Xudong Wu <2373025856w@gmail.com>

… UDF action (#2811) This PR introduces a new AI-based feature called "Add Type Annotation" that enhances the user experience of the Python UDF on top of the Pyright language server. The following are related PR's: -PR for the "Add the pyright language server" : #2797. -PR for the "[1/3] Add AI Assistant Service - Ai Flag" : #2808 -PR for the "[3/3] Add AI Assistant Service - add the "Add All Type Annotation" Python UDF action" : #2812 **Key changes:** Added a Monaco action in the frontend Python UDF editor to allow users to automatically add type annotations for an argument each time they select a single argument in their code. **Add Type Annotation:** This action requires the user to select a single argument in their Python UDF code. A UI with a suggestion for the selected argument will pop up. The user can choose to accept or decline the suggestion from OpenAI. If they accept the suggestion, it will be added to their code; otherwise, the code will remain unchanged. The suggestion is provided by OpenAI. After the backend receives the response from OpenAI, it will be forwarded to the frontend via a RESTful API. Example: https://github.com/user-attachments/assets/29cee6c8-c793-4d83-8b27-db64bcc636ae --------- Co-authored-by: Yicong Huang <17627829+Yicong-Huang@users.noreply.github.com> Co-authored-by: Yicong Huang <yicong.huang@observeinc.com>

…thon UDF action (#2857) This PR introduces another AI feature call "Add All Type Annotation" that enhance the user experience of the Python UDF after integrating with the Pyright language server. You can refer to the related PR here: -PR for the "Add the pyright language server" : #2797. -PR for the "[1/3] Add AI Assistant Service - Ai Flag" : #2808 -PR for the "[2/3] Add AI Assistant Service - add the "Add Type Annotation" Python UDF action" : #2811 **Key change:** Added another Monaco action in the frontend Python UDF editor to allow users to automatically add all type annotations for the arguments within their selected code block. **Add All Type Annotation:** This action requires the user to select a code block in their Python UDF code. The action will automatically identify all arguments without type annotations and provide type annotation suggestions for each argument sequentially. The selected code block will be sent to the backend for Python Abstract Syntax Tree (AST) analysis to locate arguments without type annotations, and the result will be sent back to the frontend via a RESTful API. If the user selects a code block that has no arguments or where all the arguments already have type annotations, the code will remain unchanged. Example: https://github.com/user-attachments/assets/d70c1b02-b309-4359-a267-0a5eef831316

all

34f6878

IamtherealBrian requested a review from yunyad August 29, 2024 21:51

IamtherealBrian self-assigned this Aug 29, 2024

IamtherealBrian mentioned this pull request Aug 29, 2024

[3/3] Add AI Assistant Service - add the "Add All Type Annotation" Python UDF action #2812

Closed

IamtherealBrian added the gui label Aug 29, 2024

IamtherealBrian changed the title ~~[2/3] Add AI Assistant Service - add the first Python UDF action~~ [2/3] Add AI Assistant Service - add the “Add Type Annotation" Python UDF action Aug 31, 2024

IamtherealBrian mentioned this pull request Aug 31, 2024

[1/3] Add AI Assistant Service - AI Flag #2808

Merged

IamtherealBrian added 5 commits August 31, 2024 23:15

amber

e0f51d3

fmt

86b3663

frontend logger

bbcc2a3

logger to frontend

980c6b7

yarn fmt

d5a3ded

IamtherealBrian and others added 6 commits September 2, 2024 15:17

Merge branch 'master' into minchongWu-aiFeatures1-pr

dca5a22

fmt

9c316b9

add action

c04d498

debug

081113f

debug

37f9573

fmt

388325a

yunyad approved these changes Sep 3, 2024

View reviewed changes

IamtherealBrian requested a review from chenlica September 3, 2024 04:46

chenlica approved these changes Sep 3, 2024

View reviewed changes

IamtherealBrian requested a review from Yicong-Huang September 3, 2024 05:03

Merge branch 'master' into minchongWu-aiFeatures1-pr

b788554

Yicong-Huang requested changes Sep 3, 2024

View reviewed changes

IamtherealBrian and others added 5 commits September 3, 2024 13:22

change "Ai" to "AI"

acf6a05

move prompt from frontend to backend

4e0b697

resolve conflict?

7bf2483

Merge branch 'master' into minchongWu-aiFeatures1-pr

ed156d7

IamtherealBrian added 6 commits September 9, 2024 22:10

resolve conflict

844e942

fix comments

109d173

fmt

b51bc81

fix comments.

dfa885c

fmt

f5e7c8f

remove import typing modules

c39860a

Yicong-Huang force-pushed the master branch from 240b068 to 2b3e779 Compare September 10, 2024 07:16

Merge branch 'master' into minchongWu-aiFeatures1-pr

6200909

IamtherealBrian requested a review from Yicong-Huang September 19, 2024 00:39

Merge branch 'master' into minchongWu-aiFeatures1-pr

9c997f2

Yicong-Huang requested changes Sep 19, 2024

View reviewed changes

IamtherealBrian and others added 6 commits September 19, 2024 15:54

fix part of the comments

1219868

fix part of the comments

eefc603

add annotation-suggestion.component

2cde080

remove async and change promise to observable

ee50fd7

Merge branch 'master' into minchongWu-aiFeatures1-pr

a6e9d1d

fmt gui

0b108c5

Yicong-Huang approved these changes Sep 20, 2024

View reviewed changes

IamtherealBrian and others added 2 commits September 20, 2024 14:07

fix gui comments

c5759ec

Merge branch 'master' into minchongWu-aiFeatures1-pr

73c4fcc

IamtherealBrian merged commit a892766 into master Sep 20, 2024

IamtherealBrian deleted the minchongWu-aiFeatures1-pr branch September 20, 2024 18:26

IamtherealBrian mentioned this pull request Sep 21, 2024

[3/3] Add AI Assistant Service - add the "Add All Type Annotation" Python UDF action #2857

Merged

Yicong-Huang added frontend Changes related to the frontend GUI and removed gui labels Jul 6, 2025

[2/3] Add AI Assistant Service - add the “Add Type Annotation" Python UDF action #2811

[2/3] Add AI Assistant Service - add the “Add Type Annotation" Python UDF action #2811

Uh oh!

Conversation

IamtherealBrian commented Aug 29, 2024 • edited by chenlica Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yunyad left a comment

Choose a reason for hiding this comment

Uh oh!

chenlica left a comment

Choose a reason for hiding this comment

Uh oh!

Yicong-Huang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Yicong-Huang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Yicong-Huang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

IamtherealBrian commented Aug 29, 2024 •

edited by chenlica

Loading