Add rtdetr-v2 version of code

[SangbumChoi]

What does this PR do?

This is the code of compatible for rtdetr-v2. https://github.com/lyuwenyu/RT-DETR/blob/main/rtdetrv2_pytorch/configs/rtdetrv2/rtdetrv2_r18vd_120e_coco.yml

At this moment I just uploaded rtdetrv2_r18vd for the test, but while in the reviewing code I will also upload other model weight also. https://huggingface.co/danelcsb/rtdetr_v2_r18vd/tree/main

@qubvel @amyeroberts

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

[SangbumChoi]

CI error seems unrelated (jax, albumentation uninstalled error)

[amyeroberts]

Thanks for adding @SangbumChoi!

As v2 is released with a new paper, it should be added as it's own, separate model in the repo.

[SangbumChoi]

@amyeroberts There is no problem making with v2 independent repo, however should I make it compatible to import v1 configuration in v2?

[amyeroberts]

@SangbumChoi As there appears to be v2 specific checkpoints, I'd say no.

[SangbumChoi]

Files to be transferred https://huggingface.co/danelcsb

[SangbumChoi]

Need to be changed after approval

[SangbumChoi]

Need to be changed after approval

[SangbumChoi]

Need to be changed after approval

[SangbumChoi]

Need to be changed after approval

[SangbumChoi]

@amyeroberts RTDetrV2 is ready 👍🏼

[amyeroberts]

Looks great - thanks for adding!

Just a few small comments. Final step after addressing these is running the slow tests for the model before merge. Could you push an empty commit with the message [run_slow] rt_detr_v2?

[amyeroberts]

We should use # Copied from for the tests too

[SangbumChoi]

@amyeroberts Well actually since the configuration of RTDetr and RTDetrV2 is different. However, I will add the part that I can do e.g. RTDetrV2ResNetModelTester

[SangbumChoi]

Wait how can we use # Copied from since it starts from transformers ?

# Copied from transformers.models.rt_detr.modeling_rt_detr.RTDetrPreTrainedModel with RTDetr->RTDetrV2,rt_detr->rt_detr_v2

[amyeroberts]

As above - this makes it more explicit which dimensions are being used wrt the scope

Suggested change

	num_backbone_outs = len(config.decoder_in_channels)
	decoder_input_proj_list = []
	for _ in range(num_backbone_outs):
	in_channels = config.decoder_in_channels[_]
	decoder_input_proj_list.append(
	nn.Sequential(
	nn.Conv2d(in_channels, config.d_model, kernel_size=1, bias=False),
	nn.BatchNorm2d(config.d_model, config.batch_norm_eps),
	)
	)
	for _ in range(config.num_feature_levels - num_backbone_outs):
	decoder_input_proj_list.append(
	nn.Sequential(
	nn.Conv2d(in_channels, config.d_model, kernel_size=3, stride=2, padding=1, bias=False),
	nn.BatchNorm2d(config.d_model, config.batch_norm_eps),
	)
	)
	in_channels = config.d_model
	decoder_input_proj_list = []
	for in_channels in config.decoder_in_channels:
	decoder_input_proj_list.append(
	nn.Sequential(
	nn.Conv2d(in_channels, config.d_model, kernel_size=1, bias=False),
	nn.BatchNorm2d(config.d_model, config.batch_norm_eps),
	)
	)
	decoder_input_proj_list.append(
	nn.Sequential(
	nn.Conv2d(config.decoder_in_channels[-1], config.d_model, kernel_size=3, stride=2, padding=1, bias=False),
	nn.BatchNorm2d(config.d_model, config.batch_norm_eps),
	)
	)
	for _ in range(config.num_feature_levels - num_backbone_outs - 1):
	decoder_input_proj_list.append(
	nn.Sequential(
	nn.Conv2d(config.d_model, config.d_model, kernel_size=3, stride=2, padding=1, bias=False),
	nn.BatchNorm2d(config.d_model, config.batch_norm_eps),
	)
	)

[SangbumChoi]

@amyeroberts Unlike above case I think this is not always true when config.num_feature_levels = num_backbone_outs. Let me think of it and try to fix it

[yonigozlan]

Hi @SangbumChoi! Excited to see RT-DETR-V2 in Transformers thanks for working on this!
As the implementation is so similar to RT-DETR and contains a lot of copied from, I think it could really benefit from using the new Modular system: more info and examples here and here.

As most of the work is already done in this PR, using Modular should be straightforward: you could put every module that does not include a copied from inside the modular file and discard the ones that do, and you should also be able to simplify the part that don't use copied from if they are similar/only add logic to parts in RT-DETR, using inheritance.

Happy to help if you have any questions!

[SangbumChoi]

close since there is another PR for modular function