Skip to content

feat(Engine): initial commit for generate engine metadata#2022

Open
shubhammahure wants to merge 11 commits intodevfrom
1968-generate-engine-meta-data
Open

feat(Engine): initial commit for generate engine metadata#2022
shubhammahure wants to merge 11 commits intodevfrom
1968-generate-engine-meta-data

Conversation

@shubhammahure
Copy link
Contributor

@shubhammahure shubhammahure commented Jan 30, 2026

Description

Using LLM, generate meta data for empty fields within an engine.
Should not overwrite fields that contain data
Generate meta data for empty fields

User should have some way to select what info is being sent to LLM for generation. Ex: give option for engine schema, sample rows, typing in additional details etc. User should always know what is being sent so data is not being leaked unknowingly.

How to Test

You can test in run pixel(Notebook cell) call
just replace your engineid and model id and provide options listed below(engine wise)

GenerateEngineMetadata(engine=["engineid"], model=["modelid"], metaKeys=[["description","tag"]], storagePath=["storagepath"], filePath=['filepath'], options=[{"includeSchema":true,"includeVectorFileNames":false,"includeVectorChunks":true,"includeStorageFileNames":true,"includeModelSmssInfo":true, "includeFunctionSmssInfo":2, "includeStorageFileContent":true }]);

Example for StorageEngine
GenerateEngineMetadata(engine=["ed23348f-842c-417d-...."], model=["4801422a-5c62-421e-a00c-05c6...."], metaKeys=[["description","tag"]], storagePath=["/abcd3"], filePath=['/shubh'], options=[{"includeStorageFileNames":true,"includeStorageFileContent":true }]);

Note: The storagePath and filePath keys are only supported for StorageEngine

Replace options according to your engine:

Supported options:

1) Database

  • includeSchema (boolean):
    Include database schema details such as tables and columns.
  • tableSchemaLimit (number):
    Max tables to include (default: 5)
  • columnSchemaLimit (number):
    Max columns per table (default: 10)

2) Vectors

  • includeVectorFileNames (boolean):
    Include vector document file names.
  • vectorFileLimit (number):
    Maximum number of vector file names to include (default: 5).
  • includeVectorChunks (boolean):
    Include sample text chunks from vector documents.
  • vectorChunkLimit (number):
    Maximum number of vector text chunks to include (default: 3).

3) Storages

  • includeStorageFileNames (boolean):
    Include storage file names.
  • storageFileNameLimit (number):
    Maximum number of storage file names to include (default: 5).
  • includeStorageFileContent (boolean):
    Copy files locally and read content snippets for context.
  • storageFileLimit (number):
    Max number of files to read content from (default: 3).
  • storageCharLimit (number):
    Max characters to read per file (default: 500).

4) Models

  • includeModelSmssInfo (boolean):
    Include model smss file content.

5) Functions

  • includeFunctionSmssInfo (boolean):
    Include function smss file content.

6) User-specific options

  • useExistingDescription (boolean):
    Refine and enhance an existing description instead of generating a new one.
  • additionalContext (string):
    Extra user-provided context to guide metadata generation.
  • tone (string):
    Writing tone for the generated description.
    Supported values: neutral | business | scientific (default: neutral).****

@shubhammahure shubhammahure self-assigned this Jan 30, 2026
@shubhammahure shubhammahure linked an issue Jan 30, 2026 that may be closed by this pull request
3 tasks
@shubhammahure shubhammahure marked this pull request as ready for review February 5, 2026 14:10
@shubhammahure shubhammahure requested a review from a team as a code owner February 5, 2026 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generate Engine Meta Data

2 participants