Skip to content

Conversation

@dartanghan
Copy link
Contributor

Added a feature to scan .ipynb (jupyter notebook) files, able to read SQL and PYTHON cells.
This will be very useful for data engineers that want their data products validated.

  • Added .ipynb in the README.doc
  • Added ipynb in pyproject.toml
  • Created ipynb_plugin.py
  • Added rules in plugins.yaml

Sample code:

%sql
SET password := "secret"
SELECT :password
%python

@dlt.table
def filtered_taxis():
    password="SQLSERVER"
    username="SQLUSER"
    print("username: ", username)
    print("password: ", password)

Sample result:

File: notebook.ipynb
 Identified issue 1: Hardcoded SQL password in notebook
    Snippet: %sql
SET password := "secret"
SELECT :password
    Line number: 62
    CWE: CWE-798
    Severity: High
...

File: dlt_pipeline.ipynb
 Identified issue 1: Hardcoded credentials in code
    Snippet: password="SQLSERVER"
    username="SQLUSER"
    Line number: 64
    CWE: CWE-798
    Severity: High
...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant