Browsing by Author "Albrecht, Jake"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Best practices to evaluate the impact of biomedical research software-metric collection beyond citations
    (Oxford University Press, 2024) Afiaz, Awan; Ivanov, Andrey A.; Chamberlin, John; Hanauer, David; Savonen, Candace L.; Goldman, Mary J.; Morgan, Martin; Reich, Michael; Getka, Alexander; Holmes, Aaron; Pati, Sarthak; Knight, Dan; Boutros, Paul C.; Bakas, Spyridon; Caporaso, J. Gregory; Del Fiol, Guilherme; Hochheiser, Harry; Haas, Brian; Schloss, Patrick D.; Eddy, James A.; Albrecht, Jake; Fedorov, Andrey; Waldron, Levi; Hoffman, Ava M.; Bradshaw, Richard L.; Leek, Jeffrey T.; Wright, Carrie; Pathology and Laboratory Medicine, School of Medicine
    Motivation: Software is vital for the advancement of biology and medicine. Impact evaluations of scientific software have primarily emphasized traditional citation metrics of associated papers, despite these metrics inadequately capturing the dynamic picture of impact and despite challenges with improper citation. Results: To understand how software developers evaluate their tools, we conducted a survey of participants in the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We found that although developers realize the value of more extensive metric collection, they find a lack of funding and time hindering. We also investigated software among this community for how often infrastructure that supports more nontraditional metrics were implemented and how this impacted rates of papers describing usage of the software. We found that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seemed to be associated with increased mention rates. Analysing more diverse metrics can enable developers to better understand user engagement, justify continued funding, identify novel use cases, pinpoint improvement areas, and ultimately amplify their software's impact. Challenges are associated, including distorted or misleading metrics, as well as ethical and security concerns. More attention to nuances involved in capturing impact across the spectrum of biomedical software is needed. For funders and developers, we outline guidance based on experience from our community. By considering how we evaluate software, we can empower developers to create tools that more effectively accelerate biological and medical research progress. Availability and implementation: More information about the analysis, as well as access to data and code is available at https://github.com/fhdsl/ITCR_Metrics_manuscript_website.
  • Thumbnail Image
    Item
    The Brain Tumor Segmentation (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI
    (ArXiv, 2023-06-01) Moawad, Ahmed W.; Janas, Anastasia; Baid, Ujjwal; Ramakrishnan, Divya; Jekel, Leon; Krantchev, Kiril; Moy, Harrison; Saluja, Rachit; Osenberg, Klara; Wilms, Klara; Kaur, Manpreet; Avesta, Arman; Cassinelli Pedersen, Gabriel; Maleki, Nazanin; Salimi, Mahdi; Merkaj, Sarah; von Reppert, Marc; Tillmans, Niklas; Lost, Jan; Bousabarah, Khaled; Holler, Wolfgang; Lin, MingDe; Westerhoff, Malte; Maresca, Ryan; Link, Katherine E.; Tahon, Nourel Hoda; Marcus, Daniel; Sotiras, Aristeidis; LaMontagne, Pamela; Chakrabarty, Strajit; Teytelboym, Oleg; Youssef, Ayda; Nada, Ayaman; Velichko, Yuri S.; Gennaro, Nicolo; Connectome Students; Group of Annotators; Cramer, Justin; Johnson, Derek R.; Kwan, Benjamin Y. M.; Petrovic, Boyan; Patro, Satya N.; Wu, Lei; So, Tiffany; Thompson, Gerry; Kam, Anthony; Guzman Perez-Carrillo, Gloria; Lall, Neil; Group of Approvers; Albrecht, Jake; Anazodo, Udunna; Lingaru, Marius George; Menze, Bjoern H.; Wiestler, Benedikt; Adewole, Maruf; Anwar, Syed Muhammad; Labella, Dominic; Li, Hongwei Bran; Iglesias, Juan Eugenio; Farahani, Keyvan; Eddy, James; Bergquist, Timothy; Chung, Verena; Shinohara, Russel Takeshi; Dako, Farouk; Wiggins, Walter; Reitman, Zachary; Wang, Chunhao; Liu, Xinyang; Jiang, Zhifan; Van Leemput, Koen; Piraud, Marie; Ezhov, Ivan; Johanson, Elaine; Meier, Zeke; Familiar, Ariana; Kazerooni, Anahita Fathi; Kofler, Florian; Calabrese, Evan; Aneja, Sanjay; Chiang, Veronica; Ikuta, Ichiro; Shafique, Umber; Memon, Fatima; Conte, Gian Marco; Bakas, Spyridon; Rudie, Jeffrey; Aboian, Mariam; Radiology and Imaging Sciences, School of Medicine
    Clinical monitoring of metastatic disease to the brain can be a laborious and timeconsuming process, especially in cases involving multiple metastases when the assessment is performed manually. The Response Assessment in Neuro-Oncology Brain Metastases (RANO-BM) guideline, which utilizes the unidimensional longest diameter, is commonly used in clinical and research settings to evaluate response to therapy in patients with brain metastases. However, accurate volumetric assessment of the lesion and surrounding peri-lesional edema holds significant importance in clinical decision-making and can greatly enhance outcome prediction. The unique challenge in performing segmentations of brain metastases lies in their common occurrence as small lesions. Detection and segmentation of lesions that are smaller than 10 mm in size has not demonstrated high accuracy in prior publications. The brain metastases challenge sets itself apart from previously conducted MICCAI challenges on glioma segmentation due to the significant variability in lesion size. Unlike gliomas, which tend to be larger on presentation scans, brain metastases exhibit a wide range of sizes and tend to include small lesions. We hope that the BraTS-METS dataset and challenge will advance the field of automated brain metastasis detection and segmentation.