Skip to content

API for general program improvement without slicing #463

@marov

Description

@marov

Is your feature request related to a problem? Please describe.
Currently Linea can extract ("slice") source code required to produce some variable.
i.e. linea.slice("c") in

a = 2
b = 2
c = a + b
d = c / a

would produce

a = 2
b = 2
c = a + b

as a does not depend ond
However, one may want to output the entire code, including d

Describe the solution you'd like
"Slicing" the entire code would produce the code identical to the input, so what is the point?
In the trivial example above - none, however, leveraging additional capabilities of Linea we can produce useful outcomes:

  • Airflow (Perfect. Argo, Ray ...) DAG encompassing the entire code
  • Modularization for effectiveness and efficiency - we may output 2 separate functions producing a and d, which can be run parallel, restarted/cache independently etc.
  • Replacing in-mem pandas object by distributed datasets (Dask, Spar, Ray ...) for scalability.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions