AI diagnostic support may increase inequity
Biases in AI models worsen skin colour-related differences in diagnostic accuracy among non-specialists (1,500 words, 7 minutes)
Machine learning has long touted promises of supercharging medical expertise through troves of data, helping physicians harness insights at scale for superior outcomes. But an expansive digitized dermatology experiment reveals artificial intelligence (AI) also risks reinforcing human blindspots — specifically around disease recognition in darker skin tones.
Findings from a large-scale digital experiment of dermatology diagnoses by doctors and doctors paired with artificial intelligence (AI) tools have found that support from AI improved the diagnostic accuracy of both specialists and generalists by more than 33%, but exacerbated the gap in the diagnostic accuracy of generalists across skin tones.
The study was published in Nature Medicine.
In a press release, the paper’s authors note research suggests that decision support from deep learning systems (DLS) can help improve the accuracy of a physician’s diagnosis—both primary care physicians and dermatology specialists—but the gap in the accuracy of primary care physicians across skin tones widens.
“While research has shown there is less representation of dark skin in textbooks and dermatology resident programs, there's minimal research around how accurate doctors are on light or dark skin with respect to diagnosing diseases,” said Matthew Groh, PhD. “Our study reveals that there are disparities in the accuracy of physicians on light versus dark skin. And in this case, it’s not the AI that is biased, it’s how physicians use it.”
Dr. Groh, assistant professor of management and organizations at Northwestern University’s Kellogg School of Management, led the research team.
The team found, without AI assistance, that across all skin tones and skin conditions dermatology specialists were 38% accurate in their diagnosis, and primary care physicians were 19% accurate.
Primary care providers who reported seeing mostly or all White patients were less accurate on dark versus light skin.
When decision support from a DLS was introduced, diagnostic accuracy increased by 33% among dermatologists and 69% among primary care physicians. Among dermatologists, DLS supports increased accuracy relatively evenly across skin tones. However, the same was not true for primary care physicians—their accuracy increased more in light skin tones than in dark ones. AI assistance exacerbated the accuracy disparities among primary care physicians by five percentage points, which is statistically significant.
“We suspected bias, but specialists don't have this AI-exacerbated bias, whereas primary care physicians do,” Dr. Groh said. “When a specialist sees advice from AI, they take their own vast knowledge into account when diagnosing. Primary care physicians might not have that same deep intuition of pattern matching, so they go with the AI suggestion on patterns that they are aware of.”
For the study, the researchers recruited more than 1,100 physicians, including dermatologists and primary care physicians. They were then presented with 364 images spanning 46 skin diseases and asked to submit up to four different diagnoses.
Bottom line: AI support for dermatologic diagnoses can improve accuracy, but skin colour-related biases in the systems can worsen the discrepancy in diagnostic accuracy among non-specialist physicians.
From the literature on dermatologic diagnosis
Understanding skin colour bias in deep learning-based skin lesion segmentation
This study evaluates skin tone bias within prevalent neural networks for skin lesion segmentation. Since no information about skin colour exists in widely used datasets, to quantify the bias the researchers used three distinct skin colour estimation methods: Fitzpatrick skin type estimation, Individual Typology Angle estimation as well as manual grouping of images by skin colour. The authors assess bias across common models by training a variety of U-Net-based models on three widely used datasets with 1,758 different dermoscopic and clinical images. They also evaluate commonly suggested methods to mitigate bias.
The investigators expose a significant and large correlation between segmentation performance and skin colour, which they say reveals consistent challenges in segmenting lesions for darker skin tones across diverse datasets. They find significant bias in skin lesion segmentation against darker-skinned individuals when evaluated both in and out-of-sample. They also found that commonly used methods for bias mitigation do not result in any significant reduction in bias.
Dermascopic profiling of inverted follicular keratosis in differing skin phenotypes
In this paper, researchers describe the prevalent dermascopic features of inverted follicular keratosis—a rare benign tumour of the follicular infundibulum—especially in patients with skin of colour.
To do this they retrospectively analyzed 35 histopathologically verified cases of inverted follicular keratosis from a single university hospital in Turkey. Of those, two (5.7%), 12 (34.3%), 16 (45.7%) and five (14.3%) patients had Fitzpatrick phototypes II, III, IV, and V, respectively. Clinically, the majority of inverted follicular keratoses were hypopigmented or non-pigmented (82.9%). Pink-white structureless areas (54.3%), ulceration (54.3%), central keratin mass (42.9%), and blood spots on keratin mass (42.9%) were the most frequent dermascopic findings. Pigmented structures were observed as blue-grey structureless areas in 12 lesions and as blue-grey clods in five lesions, primarily in Fitzpatrick's phototypes IV and V. The incidence of a pink, structureless area and blood spots on ulceration was found to be statistically significantly higher in individuals with fair skin types, while a greater prevalence of blue-grey coloration was observed in those with skin of colour (p<0.05).
Using artificial intelligence on dermatology conditions in Uganda: A case for diversity in training data sets for machine learning
Researchers assessed the diagnostic performance of an AI-powered dermatological algorithm called Skin Image Search on dermatologic conditions in Fitzpatrick VI skin.
The investigators retrospectively extracted 123 dermatologic images from a total of 173 images from the electronic database of a Ugandan telehealth company, The Medical Concierge Group (TMCG), after getting their consent. They then analyzed details of age, gender, and dermatologic clinical diagnosis using R on R studio software to assess the diagnostic accuracy of the AI app along with disease diagnosis and body part. Predictability levels of the AI app were graded on a scale of 0 to 5, where 0 meant no prediction was made and 1-5 demonstrated a reduction in the incorrect diagnosis prediction rate of the AI. A total of 76 (62%) of the images were from females and 47 (38%) were from males.
Researchers found the overall diagnostic accuracy of the AI app on dermatologic conditions in Black skin was low at 17% (21 out of 123 predictable images) compared to 69.9% performance on Caucasian skin type as reported from the training results. Among the different diagnoses, the highest correctness performance was with dermatitis (80%).
Dermoscopic evaluation of actinic changes in the lips of Indigenous children living at high altitude in Ecuador: A descriptive study
This study aimed to describe the clinical and dermoscopic actinic changes in the lips of 25 Indigenous children living at high altitudes in Ecuador.
The observational study was conducted in a public school in the Andes region of Ecuador (Aug.-Nov. 2019). Researchers assessed males and females aged five to 15 years by complete physical examination, digital dermoscopic photographs, and punch biopsies.
Clinical lips findings included desquamation [52% Upper Lip (UL); 40% Lower Lip (LL)], fissuring (8% UL; 8% LL), scabs (8% UL; 8% LL), and discoloration (40% UL; 20% LL). Dermoscopic features included a white-yellow lip colour (24% UL; p=0.02). The main morphologic pattern of blood vessels was monomorphic (88% UL; p<0.001), polymorphous (60% LL; p<0.001), dotted pattern (64% UL; 28% LL; p=0.02), and linear-irregular (32% UL; 72% LL; p=0.01). Female children had radiating white structures on UL (p=0.025), while boys presented white structureless areas (UL 63.6%; LL 77.8%; p=0.032). No differences in dermoscopic findings were observed according to the Fitzpatrick scale score. Punch biopsies showed no indications of actinic cheilitis.
VIDEO: Vulvar lichen sclerosus in darker skin types
Dr. Akinshemoyin Vaughn, Assistant Professor of Dermatology at the Medical College of Wisconsin in Milwaukee, discusses vulvar lichen sclerosus in darker skin types. The discussion covers misdiagnoses and how the condition presents in darker skin types. The talk also touches on the presentation of lichen simplex chronicus and erosive lichen planus.
At the intersection of skin and society
A corporation co-owned by 13 Mi’kmaw communities is investing in new off-peak electricity storage facilities with Nova Scotia Power in what both parties are calling a step toward reconciliation, reports CityNews.
The project is intended to draw and store electricity during off-peak periods and release it back to the grid when needed.
An equity loan of up to $18 million is being provided by the Canada Infrastructure Bank to help facilitate the partnership.
Crystal Nicholas, president of the corporation, Wskijinu’k Mtmo’taqnuow Agency Ltd. (WMA), said creating a greener future is a priority for the Mi’kmaw Nation, and the investment in the storage facility marks “true economic reconciliation.”
“I’m very optimistic that this will continue to open doors for the WMA to partner with a lot of other companies,” said Nicholas.
Construction of what will be the largest energy storage project in Atlantic Canada is set to begin this year in White Rock, Bridgewater, and Waverly and continue through 2026.
The first site is expected to be operational next year.
Peter Gregg, the president and CEO of Nova Scotia Power, said the partnership will help mitigate project costs and allow the utility to conduct meaningful work with Mi’kmaw communities.
The equity loans are part of the Canada Infrastructure Bank’s goal to invest at least $1 billion in Indigenous infrastructure by accelerating projects and providing access to capital.
This week
Feb. 29 is Rare Disease Day
March is Endometriosis Awareness Month
March 1 is Zero Discrimination Day
Something to think about in the week ahead. . .
— Kofi Annan, Ghanaian statesman and former Secretary-General of the United Nations (1938 to 2018)
Next week
Toronto-based dermatologist Dr. Marissa Joseph describes photoaging in darker skin types and provides her recommendations on sun protection for both visible and ultraviolet light.
If you like Skin Spectrum Weekly, why not check out Chronicle’s other publications, podcasts, and portal?
Established in 1995, The Chronicle of Skin & Allergy is a scientific newspaper providing news and information on practical therapeutics and clinical progress in dermatologic medicine. The latest issue features:
Drs. Lauren Lam (Calgary), Sam Hanna (Toronto), Benjamin Barankin (Toronto), and Renée A. Beach, (Toronto) discuss many of the new therapeutic options that arrived in 2023.
Dr. Andrew F. Alexis (New York) details pigmentary disorders in patients with skin of colour.
An essay from Dr. Sheila Wang (Montreal) submitted to the 2022 Dermatology Industry Taskforce on Inclusion, Diversity and Equity (DiTiDE) short essay contest. Dr. Wang wrote about the development of new imaging technology to detect and assess ‘beneath-the-skin’ inflammation, perfusion, and tissue oxygenation in any skin phototype.
Plus regular features, including the popular column “Vender on Psoriasis” by dermatologist Dr. Ron Vender
Read a recent digital edition of The Chronicle of Skin & Allergy here. To apply for a complimentary* subscription or to receive a sample copy, please email health@chronicle.org with your contact information.
The Women in Dermatology e-newsletter updates new findings concerning dermatologic issues that affect women and the female dermatologists who care for them. Read the current issue here.
Season three of the Vender on Psoriasis podcast with Dr. Ron Vender has begun. Listen to the new season here. In episode five, Dr. Vender discusses whether or not vitamin D has any impact on psoriasis severity, sex differences in psoriatic inflammation itch, and the risk of psychiatric disorders associated with acitretin.
And if you’re looking for a web destination for all things derm, visit derm.city, “Where Dermatology Lives.” Please like, rate, review, and share it with your colleagues.