Thazin Aung, PhD | Yale School of Medicine
In melanoma diagnostics, accurate and consistent evaluation of immune cell infiltration is critical for guiding treatment decisions. However, traditional visual assessments by pathologists can vary significantly between observers, leading to potential inconsistencies in clinical care. In a recent interview with Dermatology Times, Thazin Aung, PhD, an associate research scientist at the Yale University School of Medicine in New Haven, Connecticut, discusses her team’s efforts to develop an AI-driven tool that standardizes immune cell scoring and lays the groundwork for future clinical implementation.
DT: What motivated you and your team to do a comparison of AI scoring and traditional methods for evaluating in melanoma?
Aung: I have been working at Yale for over 6 years, and my interest is artificial intelligence and machine learning. And I was interested in translational research, so I was always looking for ways to integrate these machine learning and AI in cancer research, translational cancer research. So I started this work about 3 years ago with AI in melanoma. But that time, it was just validating the previous publications and results. But now it’s a step further. After validating the previous results in this work, we try to make it more consistent across users. So if we want to ever replace human pathologists later down the line, we hope to be ready with that work.
Right now, in clinical projects, pathologists actually look at the slides and a microscope and estimate how many immune cells are in the tumor. These immune cells are the body’s defense mechanism, and they can either attack their cancer cells or kill the cancer cells. But the problem is that the scoring of the pathologists is not consistent. So they vary from person to person, because they are eyeballing. It’s not always accurate counting. So on the other hand, the AI model that we developed was trained on 1000s of cells and melanoma tissue lines, so it can automatically find and count those immune cells. Because it looks at the patterns across the slide, it gives much more consistent immune cell counts, and that is our motivation. And if we are getting consistent results with our method, we can replace the human pathologist and actually get much more efficient work. So that’s what we want. That’s why we did this work.
DT: In the study itself, AI significantly outperformed those visual assessments in reproducibility. Can you walk through why consistency is so critical when evaluating these biomarkers?
Aung: When 2 pathologists look at the same slide and patient, for example, and give two different scores…one gives, let’s say, 60% of the immune cells, which is good because if there are more immune cells, then they are going to fight more cancer cells. And then the other pathologist comes in and says, “Okay, no, this patient only has a 45%.” The clinical decision-making is now very difficult, so that is very important. And so in our study, we actually had about 40 pathologists and over 90 operators…more than half of them are scientists and some of them are actually pathologists who used this algorithm to analyze the immune cells. And the consistency between the people who use the AI algorithm is very tight. They are very consistent. But within the pathologies, we call it the coefficient of variation, meaning they vary too much. So the AI has much more reliability [and] that’s what we meant in the article. So the coefficients of variation, it’s very important, and it needs to be as small as possible. And the AI model that we built did that and gave us very tight coefficient of variations.
DT: Given that the study was retrospective in nature, what are the next steps to move the open-source AI tool toward clinical implementation?
Aung: Currently, we build the algorithm on the open-source software. So it’s very convenient – as long as you have a computer, you can use that. You can run our algorithm on the open-source software, and you can just evaluate if you have that tissue slide. But the only disadvantage of the open-source software is that it’s low maintenance. It may be difficult later because the technology is improving, and if the software doesn’t match with the latest technology, it’s going to be hard. So for next steps, we are thinking of developing this interface of our own and then making this algorithm to be able to run on our own interface, instead of the open-source software. That’s the first step. The second step is, because it is retrospective, we need to evaluate our AI model prospectively. To be able to evaluate it, we have to link it to treatment outcomes. So we have a lot of work to do to get to that point, but we are now thinking of working with the oncologist, working with the dermatologist, and running the prospective trial. And we have to also find a cut point for the patients who have high or low immune cells by using this AI algorithm. So that’s the next step we have in mind.
DT: How do you see open-source AI tools such as this one, transforming clinical workflows in dermatopathology, not just for melanoma, but even other skin cancers as well?
Aung: Like I said, open-source software is very good because everybody can use it, but because it is low maintenance, it’s going to be a little bit challenging later down the line. But for research purposes, no doubt this will be very helpful. Not only in melanoma, but also other skin cancers, like mucosal or acromelanoma or other types of skin cancer, or even different types of cancer, like head and neck cancer. So if you want to adopt the AI tool in clinics, I think you need to actually lock down the model as well as the interface that you’re using. Because open-source software to start with, is really good. You could do your research without having to worry about developing your own tools, and without having to spend too much time on that. But using it in the clinic is another question. So you have to actually have it as a fixed one. Unless you know, you can use this open-source software for another 20 years without having to worry about updating all of that. But I think having your own interface to use in clinics is the way to go.
DT: Is there anything else you’d like to share with our audience of dermatologists?
Aung: Although the AI did pretty well, it gave us very consistent results and everything, we’re not able to replace pathologists. We still need the pathologist’s opinions to mark the cells. As a scientist, we know generally what cells look like, what types they are. But pathologists know better, because they’ve been trained for 8 years. We still need to have the ground truth and opinion from the pathologist. But to be able to run in the clinic, once we’ve already developed the algorithm, I think the scientists can do the job.
[Transcript has been edited for clarity]