Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement functions to determine ITSP bounds and compute quantification ambiguity range (QAR) #5

Open
AjeyPaiK opened this issue Oct 30, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@AjeyPaiK
Copy link
Member

AjeyPaiK commented Oct 30, 2024

@JorenB and I spent some time to understand errors that occur when a segmentation model makes predictions. By using the dice metric, we can come up with upper and lower bounds expected for the errors when a deep learning model generates segmentations. These bounds can be used to derive something called the "quantification ambiguity range" that represents the range of ambiguity we can expect when the model estimates ITSP. It's much like the perceptual ambiguity in humans.

@AjeyPaiK AjeyPaiK self-assigned this Oct 30, 2024
@AjeyPaiK AjeyPaiK changed the title Implement functions to determine ITSP bounds and compute quantitative ambiguity range (QAR) Implement functions to determine ITSP bounds and compute quantification ambiguity range (QAR) Oct 30, 2024
@AjeyPaiK AjeyPaiK added the enhancement New feature or request label Oct 30, 2024
@AjeyPaiK
Copy link
Member Author

AjeyPaiK commented Oct 30, 2024

Sensitivity of segmentation errors on volume and dice score

Note: \mathcal isn't supported on github markdown.

Today, I looked at how significantly do the volume and the dice score effect the segmentation errros.
For Eq 5 and Eq 6 from my lab journal, we can write the partial derivative of errors with respect to volume and dice score as follows:

Sensitivity of Segmentation Errors to Volume:

$$ \frac{\partial E_{\text{over}}}{\partial V} = \frac{2}{d} - 2 $$

And

$$ \frac{\partial E_{\text{under}}}{\partial V} = \frac{2 (1 - d)}{2 - d}. $$

These partial derivatives show that sensitivity to $V$ does not depend on $V$ itself but instead depends on $d$. As $d$ increases, these derivatives approach zero, meaning that for high dice scores, segmentation error becomes almost independent of $V$.

Sensitivity of Segmentation Errors to Dice Score:

$$ \frac{\partial \mathcal{E}_{\text{over}}}{\partial d} = -\frac{2V}{d^2} $$

And

$$ \frac{\partial \mathcal{E}_{\text{under}}}{\partial d} = -\frac{2V}{(2 - d)^2}. $$

These partial derivatives also indicate a larger sensitivity to $d$ due to the inverse square proportionality. Consequently, we can also conclude that QAR is more sensitive to the dice score compared to the volumes.


Below is an output from a simulation that illustrates the points made above. The simulation is now part of itsper.
vd_vs_error

@AjeyPaiK
Copy link
Member Author

Along the same lines, I concluded that the dependence of QAR on the proportions of tumor and stroma volumes behaves similarly as the perceptual ambiguity seen in humans conducting the same task. In that, the QAR is maximum when the tumor volume and stroma volume are equal. Whereas, QAR decreases when the volumes are unequal. Humans also have most ambiguity when the tumor and stroma compartments are equal in a given image.

qar_variation

@AjeyPaiK
Copy link
Member Author

AjeyPaiK commented Nov 6, 2024

qar_variation

Implemented better ways to simulate tumor and stroma compartments. Now it closely mimics real situations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant