Skip to content

Unlimited backtrace is impractical #9

@bertsky

Description

@bertsky

When globally aligning sequences that deviate much, combinatory explosion can quickly leed to excessive runtime memory consumption in the current implementation. And it is not always easy to detect those cases by score heuristics in a prior backtrace=False pass.

I believe these should be added:

  1. a package-exposed variable with a default limit (perhaps relative to the sequences' length)
  2. an optional parameter with an override limit to be able to control the quality-performance trade-off.

(The limit could be based on stack depth or number of alternatives, for example.)

Example: I am trying to align OCRed images of German Fraktur script with their corresponding ground truth text. Sometimes the OCR fails miserably like so:
Mitreden andrer 274. Günſtiger Eindruck der Staatsrathsſitzungen 274. (original line)
*0obe-ondrer '? '-änſiger Eindrue der Torerotheflgg,, (OCR result)
In this case, using StrictGlobalSequenceAligner tries to take more than 20 GB RSS (at which point I quit).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions