bug: Sometimes "reference_img_f" is ignored.

[akurmustafa]

When I run execute following code snippet,

registered_slide_dst_dir = dst_dir + "registered_slides"
align_to_reference = True
image_type = "brightfield"
do_rigid = True
imgs_ordered = True
crop="overlap"
img_list = [
    immune_path,
    neuronal_path
]
reference_img_f = he_path
registrar = registration.Valis(src_dir=src_dir, img_list=img_list, dst_dir=dst_dir, reference_img_f=reference_img_f, align_to_reference=align_to_reference, do_rigid=do_rigid, image_type=image_type, imgs_ordered=imgs_ordered)

In the registration result, I see the registered versions of the immune_path and neuronal_path. However, they are not registered according to image at reference_img_f. They are registered, as if I did not pass any reference_img_f where image at neuronal_path is registered into image at immune_path.
I verified this behaviour with following, when I print using after initialization

print(registrar.reference_img_f)

result is

<PATH>HandE_annotations/HandE/Batch1/001_2_A_(3)_5.tif-2.tif

after

rigid_registrar, non_rigid_registrar, error_df = registrar.register()

same print print(registrar.reference_img_f)
generates

<PATH>Registered-R0Q_Q0c2.Q0c3.Q0c4.Q0c5_01-2-A-1_scene002_c2_ORG.tif

which is immune_path.
I am a bit confused. Do I missing something?

[cdgatenbee]

Hi @akurmustafa,
Thanks for bringing this to my attention. It seems like this should work, but I also think I know why it's not. I believe it may have something to do with the the reference image not being in the img_list. I'll need to put together a test case to verify, but in the meantime, can you try changing the parameters to the following and let me know if it works as expected?

imgs_ordered = True
crop="overlap"
img_list = [
    immune_path,
    he_path,
    neuronal_path
]
reference_img_f = he_path

Having he_path in the middle should ensure that the other images are aligned to it. Please try that out and let me know how it goes.

Best,
-Chandler

[akurmustafa]

Thanks @cdgatenbee for the quick response.

I tried your advice for this case, Here are my new findings:

Before the registration print(registrar.reference_img_f) emits he_path which is reference_img_f argument. However, after registration, it emits neuronal_path. In short, I thinkreference_img_f is overwritten at some point during processing.

Additionally, In the deformation_fields out directory, I saw that he_path is prependended with 0, neuronal_path is prepended with 1 and immune_path is prepended with 2. Assuming middle image (the image prepended with 1) is the reference version. This is consistent with my findings. Also deformation field image corresponding to the neuronal_path (image that is prepended with 1) doesn't contain any warping. This further suggests that this is indeed the image used as reference.

Maybe I should give some additional details, regarding the inputs. If these are invalid, or wrong observed behaviour may be stemming from these inconsistencies: immune_path is a gray-scale image with size: (16768, 33280), neuronal_path is a gray-scale image with size: (18624, 33281), whereas he_path image is an RGB image with size: (3, 10024, 19828). Also, I use image_type as brightfield.

For immune_path and neuronal_path registration should be done at the same pyramid level, However, for he_path registration should be done at another pyramid level. Different pyramid levels and mixing gray scale and RGB images might be problematic. And at some point, maybe algorithm decides to overwrite reference to obtain better result (just hypothesizing).

By the way, the registration results seem good to me. I am only inquiring because the behavior is a bit strange, and I want to understand the reason behind it to make sure I am not messing up somethings. Thanks again @cdgatenbee for the follow-up and suggestions.

Best,

[cdgatenbee]

Hi @akurmustafa,
Thanks for reporting back, this is certainly unexpected output. I'll see if I can figure out what is going on by putting together a dataset with the same filenames and parameters. I'll let you know how it goes.

Best,
-Chandler

[YiyuHong]

valis/valis/warp_tools.py

Line 67 in 6610b06

name_matches = [re.search(ref_img_name.lower(), n) for n in img_names]

I think the bug that ignore reference_img_f is caused by this line. It can't find matched name....

[cdgatenbee]

@YiyuHong, you are exactly correct, that line is the source of the issue here. Thanks for pinpointing it! In particular, the trouble is the file names have several special characters that need to be escaped before matching using regex. It can be resolved by changing those lines from

img_names = [valtils.get_name(f).lower() for f in img_f_list]
name_matches = [re.search(ref_img_name.lower(), n) for n in img_names]
ref_img_idx = [i for i in range(n_imgs) if name_matches[i] is not None]

to

img_names = [valtils.get_name(f).lower() for f in img_f_list]
ref_escape = re.escape(ref_img_name.lower()) # Escape special characters
name_matches = [re.search(ref_escape, n) for n in img_names]
ref_img_idx = [i for i in range(n_imgs) if name_matches[i] is not None]

Now it it will correctly find the reference image, even if the filenames have special characters.

Thanks again for your insights @YiyuHong! And thank you for @akurmustafa for sharing the detailed info about your images, it wouldn't have been possible to debug this without it. This fix will be included in the next update, the last part of which involves debugging the Docker image. As soon as that's figured out I'll push the update and close this issue.

Best,
-Chandler

[minajamshidi]

Hi @cdgatenbee,

I also encountered this issue. However, your suggested correction in warp_tools doesn't work for me because my slide names include multiple dots (for example X.X.X.X..X_12345.svs with X being a placeholder). Therefore, if ref_img_name=X.X.X.X..X_12345, valtils.get_name returns X.X.X.X.. This would not be an issue if the slide names do not include any dots.

As I understood, when get_ref_img_idx is called, valtils.get_name is already applied on slide names, Therefore, this step could be skipped in warp_tools. However, in order to keep get_ref_img_idx general, I modified the above-mentioned lines in your last reply as the following:

if os.path.exists(ref_img_name):  
    ref_img_name = valtils.get_name(os.path.split(ref_img_name)[1])

img_names = []
for f in img_f_list:
    if os.path.exists(f):
        img_names.append(valtils.get_name(f).lower())
    else:
        img_names.append(f.lower())

Best,
Mina