Add Fast Segformer Processor by capnmav77 · Pull Request #37024 · huggingface/transformers

capnmav77 · 2025-03-27T01:28:54Z

What does this PR do?

Adding fast processor for Segformer.

github-actions · 2025-03-27T01:29:08Z

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the Ready for review button (at the bottom of the PR page). This will assign reviewers and trigger CI.

- removed redundant params like do_center_crop,center_crop which aren't present in the original segformer class

yonigozlan

Hi @capnmav77 , Thanks for working on this! Looks like the segmentation_maps processing logic from the slow image processor is missing here. You can take a look at how it's being handled for BeitImageProcessorFast as it is very similar to Segformer image processor! Here is the PR BeitImageProcessorFast #37005

…essing module with references from beitimageprocessing fast

capnmav77 · 2025-03-28T04:19:14Z

Hi @capnmav77 , Thanks for working on this! Looks like the segmentation_maps processing logic from the slow image processor is missing here. You can take a look at how it's being handled for BeitImageProcessorFast as it is very similar to Segformer image processor! Here is the PR BeitImageProcessorFast #37005

hey @yonigozlan i've added the segmentation_maps processing logic from the slow image processor also i've taken references from the beitimageprocessorFast it really helped a lot , do let me know what to do next

yonigozlan

Hey @capnmav77 thanks a lot for iterating, looks much better! Still a few things to change, and now the important part is to check that all tests pass with

RUN_SLOW=1 python -m pytest tests/models/segformer/test_image_processing_segformer.py

You should also override test_slow_fast_equivalence to also test segmentation maps processing. You can check how it was done in the Beit PR.

Also all the formatting/repo consistency tests should pass. You can use the utilities make fixup, make fix-copies, make style, make quality to help.

…smoothly

capnmav77 · 2025-04-02T04:39:25Z

Hey @yonigozlan , i have modified the code accordingly but there's a issue i'm not able to figure out , one local test fails :

FAILED tests/models/segformer/test_image_processing_segformer.py::SegformerImageProcessingTest::test_slow_fast_equivalence - AssertionError: False is not true

do you have a fix for this i'm not able to understand why this is happening , the dims of the fast processor are 1 , C , H , W , which shouldn't be the case compared to the slow processor which has 1, H , W , i presume the labels are messed up somewhere . Please do lemme know the fix for that and do suggest code changes .

here's the log output of the labels :

tensor([[[ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  3,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  3,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  3,  3,  2,  2,
           2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  3,  2,  2,  2,
           2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  3,  3,  3,  3,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  3,  2,  2,  2,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2, 18, 18,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2, 18,  2,  2,  2,  2,  2],
         [ 5,  5,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2, 18,  2,  2,  2,  2,  2],
         [ 5,  5,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2, 18,  2,  2,  2,  2,  2],
         [ 5,  5,  5,  5,  5,  5,  5,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2, 18, 18,  2,  2,  2,  2],
         [ 5,  5,  5,  5,  5,  5,  5,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2, 18, 18, 18,  2,  2,  2,  2],
         [ 5,  5,  5,  5,  5,  5,  5,  1,  1,  0,  0,  0,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2, 18, 18, 18,  2,  2,  2,  2],
         [ 1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  2,  2,
           2,  2,  2,  2,  2,  2, 18, 18, 18,  2,  2,  2,  2],
         [ 1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  2,  2,
          10, 10,  2,  2,  2, 18, 18, 18, 18, 18, 18, 18,  2],
         [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
          10, 10, 10, 10, 10, 10, 10, 18, 18, 18, 18, 18, 18],
         [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
          10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10],
         [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
          10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10],
         [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
          10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10],
         [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
          10, 10, 10, 10, 10, 10, 10, 10,  7,  7,  7,  7,  7]]])
tensor([[[[ 86,  87,  86,  ...,  82,  81,  80],
          [ 89,  90,  90,  ...,  83,  82,  80],
          [ 90,  92,  93,  ...,  85,  84,  83],
          ...,
          [ 99, 100, 101,  ..., 105, 101,  99],
          [102, 107, 107,  ..., 110, 108, 104],
          [ 97,  97, 103,  ..., 121, 119, 117]],

         [[138, 138, 138,  ..., 132, 131, 131],
          [140, 141, 141,  ..., 133, 132, 131],
          [142, 143, 143,  ..., 135, 134, 132],
          ...,
          [109, 108, 108,  ..., 108, 103, 102],
          [107, 108, 105,  ..., 111, 108, 106],
          [102,  99, 102,  ..., 114, 114, 114]],

         [[191, 193, 194,  ..., 191, 189, 188],
          [194, 196, 197,  ..., 194, 191, 189],
          [196, 199, 200,  ..., 196, 193, 191],
          ...,
          [ 62,  63,  63,  ...,  66,  64,  63],
          [ 62,  66,  66,  ...,  71,  70,  71],
          [ 58,  58,  61,  ..., 100, 102, 102]]]])
          ```

yonigozlan

Hi @capnmav77 thanks for iterating! Sorry for the delayed review. Almost ready to go, although some logic is missing for segmentation maps

capnmav77 · 2025-04-16T08:54:35Z

Hi, @yonigozlan , all the tests pass apart from the following :
FAILED tests/models/segformer/test_image_processing_segformer.py::SegformerImageProcessingTest::test_slow_fast_equivalence - AssertionError: False is not true

the tensors from the fast processor does not match the tensor from the slow one ? i've been trying to crack this issue but i'm not getting to the root cause , i assume it's some mishap in the reduce_label function ? or something related to labelling can you suggest fixes ?

slow: 
tensor([[[ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  3,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  3,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           3,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
           2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  3,  3,  2,  2,
           2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  3,  2,  2,  2,
           2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  3,  3,  3,  3,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  3,  2,  2,  2,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2, 18, 18,  2,  2,  2,  2],
         [ 3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2, 18,  2,  2,  2,  2,  2],
         [ 5,  5,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2, 18,  2,  2,  2,  2,  2],
         [ 5,  5,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2, 18,  2,  2,  2,  2,  2],
         [ 5,  5,  5,  5,  5,  5,  5,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2,  2, 18, 18,  2,  2,  2,  2],
         [ 5,  5,  5,  5,  5,  5,  5,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2, 18, 18, 18,  2,  2,  2,  2],
         [ 5,  5,  5,  5,  5,  5,  5,  1,  1,  0,  0,  0,  2,  2,  2,  2,  2,
           2,  2,  2,  2,  2,  2, 18, 18, 18,  2,  2,  2,  2],
         [ 1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  2,  2,
           2,  2,  2,  2,  2,  2, 18, 18, 18,  2,  2,  2,  2],
         [ 1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  2,  2,
          10, 10,  2,  2,  2, 18, 18, 18, 18, 18, 18, 18,  2],
         [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
          10, 10, 10, 10, 10, 10, 10, 18, 18, 18, 18, 18, 18],
         [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
          10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10],
         [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
          10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10],
         [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
          10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10],
         [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
          10, 10, 10, 10, 10, 10, 10, 10,  7,  7,  7,  7,  7 ]]]) - here is the change
 Fast: 
tensor([[[[ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
            3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
            3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
            3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
            3,  3,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
            3,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,
            3,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,
            2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  3,  3,  2,  2,
            2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  3,  2,  2,  2,
            2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  3,  3,  3],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  3,  3,  3,  3,  3,  3,  2,  2],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  3,  3,  3,  3,  2,  2,  2,  2],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2],
          [ 3,  3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  2,  2,  2,  4,  4,  2,  2,  2],
          [ 3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  2,  2,  9,  9,  2,  2,  2,  2],
          [ 3,  3,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  2,  3, 14,  8,  2,  2,  2,  2],
          [ 4,  4,  3,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  2,  4, 15,  9,  2,  2,  2,  2],
          [ 5,  5,  4,  3,  3,  3,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  2,  4, 16,  9,  2,  2,  2,  2],
          [ 5,  5,  5,  4,  5,  5,  4,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  2,  5, 17, 10,  2,  2,  2,  2],
          [ 5,  5,  5,  5,  5,  5,  5,  3,  2,  2,  2,  2,  2,  2,  2,  2,  2,
            2,  2,  2,  2,  2,  2, 10, 18, 10,  2,  2,  2,  2],
          [ 4,  4,  4,  4,  4,  3,  4,  2,  1,  1,  1,  1,  1,  1,  2,  2,  2,
            2,  2,  2,  2,  2,  4, 14, 18, 10,  2,  2,  2,  2],
          [ 1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  2,  2,
            2,  2,  2,  2,  2,  8, 18, 18, 12,  3,  2,  2,  2],
          [ 1,  1,  1,  1,  1,  1,  2,  2,  2,  2,  2,  2,  3,  3,  3,  4,  5,
            6,  6,  5,  5,  4, 12, 18, 18, 17, 13, 11, 10,  4],
          [ 6,  7,  8,  8,  8,  8,  8,  9,  9,  8,  8,  9,  9,  9, 10, 10, 10,
           10, 10, 10, 10, 10, 11, 13, 14, 16, 17, 17, 17, 14],
          [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
           10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 12, 12],
          [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
           10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10],
          [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
           10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,  9,  9],
          [ 9,  9,  9,  9,  9,  9,  9,  9,  9,  9,  9,  9,  9,  9,  9, 10, 10,
           10, 10, 10, 10, 10, 10,  9,  9,  8,  7,  7,  7,  6 ]]]])
           
           ```

- modified the preprocessing of segmentation maps to use tensors - added batch support

ariG23498 · 2025-05-07T05:40:06Z

Hi @capnmav77 now that the #37005 is merged, you could take a look at the segmentation maps logic. I think that could be of use to you.

capnmav77 · 2025-05-21T01:45:07Z

hey @yonigozlan , the image processing fast segformer batched works perfectly but the labels of a single image are not matched , this discrepancy is likely due to inherent differences in the NEAREST interpolation between PIL (used by the slow processor) and TorchVision (used by the fast processor), which can be very difficult to perfectly align. what do you suggest i do ?

out of all the tests , The batched tests pass , but the test_slow_fast_equivalence for a single image which matches labels fails , we can modify the testcase to check for dtype and labels size which pass .

yonigozlan · 2025-05-28T15:54:06Z

Hey @capnmav77 ! Sorry for the delay. Have you tried with NEAREST_EXACT? It could reduce the differences. If you still have issues we might just increase the tolerance for equivalence tests for this model

capnmav77 · 2025-06-04T14:31:43Z

Hey @yonigozlan , i've done the required changes and apparantly the maximum difference is 2 hence i've updated the tolerance value to 2.5 in the testcode and all the testcases pass . Let me know if i need to change anything else now ?

simonreise · 2025-06-10T10:39:59Z

Hi @capnmav77! You should probably implement the same fix as in #38042 , because as far as I can see your current code applies reduce_labels not only to the labels, but also to the images

yonigozlan

Hey @capnmav77 ! Sorry for the delay. Updated the PR to follow the last fast image processor standards and use modular, will merge once the CI passes!

HuggingFaceDocBuilderDev · 2025-07-25T20:54:15Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

github-actions · 2025-07-28T19:10:56Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto, beit, segformer

* Add Fast Segformer Processor * Modified the params according to segformer model * modified test_image_processing_Segformer_fast args - removed redundant params like do_center_crop,center_crop which aren't present in the original segformer class * added segmentation_maps processing logic form the slow segformer processing module with references from beitimageprocessing fast * fixed code_quality * added recommended fixes and tests to make sure everything processess smoothly * Fixed SegmentationMapsLogic - modified the preprocessing of segmentation maps to use tensors - added batch support * fixed some mismatched files * modified the tolerance for tests * use modular * fix ci --------- Co-authored-by: yonigozlan <yoni.gozlan@huggingface.co>

Original PR #37024 by capnmav77 Original: huggingface/transformers#37024

Merged from original PR #37024 Original: huggingface/transformers#37024

Original PR #37024 by capnmav77 Original: huggingface/transformers#37024

Merged from original PR #37024 Original: huggingface/transformers#37024

Add Fast Segformer Processor

5d7f02e

github-actions Bot marked this pull request as draft March 27, 2025 01:29

capnmav77 mentioned this pull request Mar 27, 2025

[Contributions Welcome] Add Fast Image Processors #36978

Closed

81 tasks

capnmav77 marked this pull request as ready for review March 27, 2025 02:18

github-actions Bot requested review from ydshieh and yonigozlan March 27, 2025 02:18

capnmav77 added 2 commits March 27, 2025 08:15

Modified the params according to segformer model

ef33597

modified test_image_processing_Segformer_fast args

421f902

- removed redundant params like do_center_crop,center_crop which aren't present in the original segformer class

yonigozlan reviewed Mar 27, 2025

View reviewed changes

added segmentation_maps processing logic form the slow segformer proc…

d26e961

…essing module with references from beitimageprocessing fast

Merge branch 'main' into main

c3c0f7d

capnmav77 requested a review from yonigozlan March 28, 2025 04:26

fixed code_quality

192cef8

yonigozlan reviewed Mar 31, 2025

View reviewed changes

capnmav77 force-pushed the main branch from dda9d01 to 192cef8 Compare April 2, 2025 04:31

added recommended fixes and tests to make sure everything processess …

133f163

…smoothly

capnmav77 requested a review from yonigozlan April 2, 2025 04:39

yonigozlan reviewed Apr 15, 2025

View reviewed changes

Comment thread src/transformers/models/segformer/image_processing_segformer_fast.py Outdated

Comment thread src/transformers/models/segformer/image_processing_segformer_fast.py Outdated

Comment thread src/transformers/models/segformer/image_processing_segformer_fast.py Outdated

Fixed SegmentationMapsLogic

a69d71d

- modified the preprocessing of segmentation maps to use tensors - added batch support

capnmav77 force-pushed the main branch from 03f735d to a69d71d Compare April 16, 2025 09:01

capnmav77 requested a review from yonigozlan April 16, 2025 09:34

Merge remote-tracking branch 'upstream/main'

86358a8

capnmav77 force-pushed the main branch from c21f35d to 86358a8 Compare May 22, 2025 22:37

fixed some mismatched files

ae2ae30

modified the tolerance for tests

c0734e7

yonigozlan added 2 commits July 25, 2025 20:14

Merge remote-tracking branch 'upstream/main' into segformer-fast

63210d0

use modular

71d17b9

yonigozlan approved these changes Jul 25, 2025

View reviewed changes

fix ci

5274dd9

yonigozlan enabled auto-merge (squash) July 25, 2025 20:41

fixup

4a594e1

yonigozlan added 2 commits July 28, 2025 18:00

Merge remote-tracking branch 'upstream/main' into segformer-fast

22723da

fix modular

723f577

yonigozlan merged commit 4f8f51b into huggingface:main Jul 28, 2025
25 checks passed

snorkelopstesting2-coder pushed a commit to snorkel-marlin-repos/huggingface_transformers_pr_37024_faf590bd-3a4b-4b4c-a2d1-f75157d959c5 that referenced this pull request Oct 11, 2025

Add Fast Segformer Processor

317d142

Original PR #37024 by capnmav77 Original: huggingface/transformers#37024

snorkelopstesting2-coder mentioned this pull request Oct 11, 2025

Add Fast Segformer Processor snorkel-marlin-repos/huggingface_transformers_pr_37024_faf590bd-3a4b-4b4c-a2d1-f75157d959c5#1

Merged

snorkelopstesting2-coder pushed a commit to snorkel-marlin-repos/huggingface_transformers_pr_37024_d323a227-32a6-40b4-b248-6e4bf86c8c42 that referenced this pull request Oct 11, 2025

Add Fast Segformer Processor

74df38f

Original PR #37024 by capnmav77 Original: huggingface/transformers#37024

snorkelopstesting2-coder mentioned this pull request Oct 11, 2025

Add Fast Segformer Processor snorkel-marlin-repos/huggingface_transformers_pr_37024_d323a227-32a6-40b4-b248-6e4bf86c8c42#1

Merged

Conversation

capnmav77 commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Uh oh!

github-actions Bot commented Mar 27, 2025

Uh oh!

yonigozlan left a comment

Choose a reason for hiding this comment

Uh oh!

capnmav77 commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yonigozlan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

capnmav77 commented Apr 2, 2025

Uh oh!

yonigozlan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

capnmav77 commented Apr 16, 2025

Uh oh!

ariG23498 commented May 7, 2025

Uh oh!

capnmav77 commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yonigozlan commented May 28, 2025

Uh oh!

capnmav77 commented Jun 4, 2025

Uh oh!

simonreise commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yonigozlan left a comment

Choose a reason for hiding this comment

Uh oh!

HuggingFaceDocBuilderDev commented Jul 25, 2025

Uh oh!

github-actions Bot commented Jul 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

capnmav77 commented Mar 27, 2025 •

edited

Loading

capnmav77 commented Mar 28, 2025 •

edited

Loading

capnmav77 commented May 21, 2025 •

edited

Loading

simonreise commented Jun 10, 2025 •

edited

Loading