Toronto Metropolitan University
Browse

File(s) not publicly available

Improving Interactive Segmentation using a Novel Weighted Loss Function with an Adaptive Click Size and Two-Stream Fusion

journal contribution
posted on 2023-05-03, 16:01 authored by Ragavie Pirabaharan, Naimul KhanNaimul Khan

Interactive segmentation has recently attracted at-tention for specialized segmentation tasks where expert input is required to further enhance the segmentation performance. In this work, we propose a novel interactive segmentation framework, where user clicks are dynamically adapted in size based on the current segmentation mask. The clicked regions form a weight map and are fed to a deep neural network together with the image, where the network learns to discriminate important regions through a novel weighted loss function. To evaluate our loss function, a state-of-the-art interactive V-Net (IV-Net) model which utilizes both foreground and background user clicks as the main method of interaction is employed. To further improve on the IV-Net, we propose the use of a two-stream fusion interactive If-Net (TSFIV-Net) which applies multimodal fusion to allow for the propagation of image feature information throughout the architecture. We train and validate both the models on the BCV dataset, while testing on both seen and unseen structures from the MSD dataset to determine the models generalization and segmentation abilities in comparison to the standard IV-Net. Applying adaptive user click sizes increases the overall dice score by 4.86 % and 8.59 % for seen and unseen structures respectively by utilizing only a single user interaction on the IV-Net compared to the original version, and 9.88% and 10.35% on the TSFIV-Net.

 

History

Language

English

Usage metrics

    Computer Engineering

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC