Comparison of three short video platforms in China in 2023
Video content and publisher distribution
In this cross-sectional study, we examined videos pertaining to pancreatic cancer available on three prominent short video sharing platforms in China: TikTok, Bilibili, and Kwai. Examining the content categories of the videos, TikTok (34%) and Kwai (45%) predominantly featured videos related to disease knowledge, while Bilibili (45%) primarily published news and reports. This distribution aligns partially with our expectations. Bilibili, known for its young user base and entertainment-oriented content [20, 21], may prioritize news updates to attract viewership, whereas TikTok and Kwai’s focus on disease knowledge likely reflects their broader user demographics and medical practitioners’ active participation. However, all platforms posted the fewest videos on preventive knowledge, followed by treatment-related knowledge and invalid information. The scarcity of prevention-related content (5.3% overall) was unexpected, given the modifiable risk factors (e.g., smoking, diet) emphasized in clinical guidelines [9, 37]. This gap suggests that we may have missed opportunities to leverage these modifiable risk factors for delivering targeted public health information. Furthermore, as we expected, since medical practitioners have a professional medical knowledge background, the video content they produce focuses more on the field of medical expertise, such as the disease knowledge, prevention, and treatment. The high degree of fit between professional qualities and content output reflects the unique advantages of medical practitioners in health information dissemination. In contrast, science communicators tended to publish disease knowledge and news/reports, potentially balancing educational goals with audience engagement strategies. Patients published a lot of invalid information, likely due to personal biases or lack of medical training [38], highlighting the risks of non-professional health communication. It is noteworthy that no video content across these platforms provided information about postoperative follow-up and nursing care specific to pancreatic cancer. This indicates the necessity to incorporate such content in future health education videos, including postoperative follow-up schedules (e.g., imaging surveillance intervals), management of chemotherapy/radiotherapy side effects (e.g., nausea, fatigue), nutritional support (e.g., dietary plans for weight maintenance), pain control strategies, and psychological counseling for patients. These topics are critical for improving long-term quality of life but were notably absent in the analyzed videos, mirroring findings from Zhao et al.’s 2024 study on lung cancer [39].
Quality and reliability of short videos on pancreatic cancer
Although user engagement metrics across various platforms were relatively high, objective quality assessments using GQS, mDISCERN, and MQ-VET revealed generally mediocre video content quality (median GQS score: 3 [IQR, 2–3]), low reliability (median mDISCERN score: 2 [IQR, 1.75–3]), and an overall unsatisfactory performance (median MQ-VET score: 45 [IQR, 38–50]). These scores fell below established thresholds for high-quality medical information in previous studies (typically ≥ 4 points for GQS and mDISCERN scales, and ≥ 64 points for MQ-VET scale) [23,24,25,26]. Additionally, no substantial inter-platform differences were detected in terms of video quality and reliability.
The quality and reliability of short videos about pancreatic cancer are contingent upon the inherent characteristics of these videos (such as duration, content depth, visual/auditory clarity, use of visual aids like medical illustrations or animations, and citation of authoritative sources), the constraints of video sharing platforms, and the publishers themselves [23]. Primarily, due to their abbreviated duration, simplistic composition, and entertainment-based design, these short videos inherently suffer from content constraints. In addition, some videos may sacrifice the depth of their content to cater to the audience’s preference for fragmented information. Moreover, the platform lacks the ability to review medical professional content and cannot limit the upload of videos with extremely poor reliability. In this study, only two videos were discovered cited both reference bases and other information sources, achieving a score of 5 on the mDISCERN scale, while most videos failed to provide any reference or links to other information channels, rendering the video content unreliable. This aligns with previous research findings where merely 4% of short videos in liver cancer analyses [23] and 2.33% in gastric cancer research [26] offered verifiable references. Such low citation rates indicate poor traceability of health information disseminated through current short video platforms, where the brevity and entertainment value of health-related content often take precedence over academic rigor. The absence of references undermines content credibility, as viewers are unable to verify the reliability of the health information.
We found that the quality and reliability of videos posted by medical practitioners were significantly higher than those of non-medical practitioners, which is consistent with public expectation and similar to previous relevant research results [23, 25, 39]. Medical practitioners usually have professional medical knowledge and clinical experience, so they have a more accurate and in-depth understanding of pancreatic cancer related knowledge. When making videos, they rely on clinical guidelines and medical research on pancreatic cancer to ensure the accuracy and reliability of the information conveyed. This accuracy plays a crucial role in maintaining and enhancing public trust, because viewers, especially Chinese viewers, tend to trust qualified publishers more when obtaining health information [24]. Therefore, we suggest that medical professionals engage in proper content planning during the creation of health—related videos. This includes ensuring that the videos are clear, logical and comprehensible, and citing authoritative medical literature or data to enhance credibility. These efforts can help build a good trust relationship with the public and further promote the public’s understanding and management of pancreatic cancer. In addition, there is no significant difference in the quality and reliability of videos posted by pancreatic disease related experts and non-pancreatic disease related experts. This may be attributed to the fact that while the latter group does not specialize in pancreatic diseases, their medical background provides them with solid foundational scientific literacy. For instance, endocrinologists are highly skilled in diabetes diagnosis and treatment. Given the close link between type 2 diabetes and pancreatic cancer onset, and that blood sugar management after pancreatic cancer resection is a common issue in their field, they also have a relatively clear understanding of pancreatic cancer [40, 41]. This indirectly reflects the necessity of implementing a multidisciplinary team (MDT) diagnostic and treatment model for pancreatic cancer [12, 42]. Moreover, their content creation also followed authoritative standards of treatment such as the NCCN guidelines, which ensured the accuracy of the information. This finding suggests that healthcare practitioners outside pancreatic specialties can also serve as credible sources of health information, and that their interdisciplinary perspectives may even enhance public understanding in a more layman’s way, expanding the reach of pancreatic disease education. However, videos posted by traditional Chinese medicine experts exhibit significantly less quality and reliability, feasibly owing to our evaluation criteria. This is similar to the findings of Zheng et al. 2023 on short videos related to liver cancer [23]. When evaluating expertise in pancreatic cancer, we adhered to NCCN guidelines [37], which do not incorporate traditional Chinese medicine. Importantly, videos from TikTok medical practitioners showed slightly higher quality and reliability than those from other platforms, perhaps due to the professional certification most TikTok practitioners possess and the minimal proportion of traditional Chinese medicine experts. This contrasts with the study by Yang et al. in 2022, in which Chinese medicine videos accounted for a quarter of the top 100 videos for thyroid cancer search results on TikTok in their study [43]. However, in a 2024 study on laryngeal cancer [44], only one Chinese medicine video appeared on TikTok. This marked change may be linked to TikTok’s recent strict certification rules. Many traditional Chinese medical doctors working in relatively lower-level hospitals are not allowed to post Chinese medicine videos on TikTok as they fail to obtain certification. In addition, the videos posted by patients or their families may lack professional depth and can be influenced by emotional experiences and mental state changes related to pancreatic cancer [38], potentially resulting in compromised informational accuracy and diminished content quality. However, these narratives also offer unique insights into the emotional and psychological challenges of living with the disease, addressing themes such as grief, identity shifts, and coping strategies—aspects rarely covered in clinical content. Although their educational utility may be limited, the emotional resonance triggered by these real-life experiences has significant value in building psychological support networks between patient populations and families.
Meanwhile, we revealed significant distinctions in video quality across diverse content types. Videos related to the categories of “disease”, “prevention”, and “treatment” show high quality and reliability, but there is no significant difference among the three. In contrast, news/report videos and those with invalid information showed poor quality and reliability. The videos about invalid information generally emanated from patients or relatives sharing patient life circumstances while the videos of the news/report were mainly the doctor’s outpatient clinic, which lacked the introduction of patient’s specific condition, examination information and knowledge popularization. Moreover, this may lead to some patients receiving misleading information about diagnosis and treatment that is not consistent with their own condition.
From the video examples chosen across different platforms, we have observed some characteristics of these platforms. On TikTok, we have found high-quality videos posted by medical practitioners, such as a popular education video published by a gastroenterologist of a tertiary hospital about the risk factors, screening, and diagnosis of pancreatic cancer. With professional medical illustrations and clear treatment procedures, these videos help the public to better understand the causes, symptoms, and diagnostic means of pancreatic cancer. On Bilibili, some popular science communicators create videos with humorous and interesting illustrations. They comprehensively explain aspects like pancreatic cancer’s epidemiology, etiology, clinical manifestations, screening methods, and treatments, thus attracting many young audiences. In contrast, some videos on Kwai, posted by non-medical practitioners like patients sharing personal treatment experiences, have certain reference value but lack accuracy and comprehensiveness.
The relationship between video quality and ranking and audience recognition
Interestingly, our correlation analysis revealed only a weak connection between video quality and video ranking, and GQS, mDISCERN, and MQ-VET scores are not independent influencing factors for ranking. This result indicated that the current video ranking recommended by default algorithm of short video platforms did not adequately represent video quality and reliability pertaining to pancreatic cancer, thereby limiting viewers’access to top-quality information on “pancreatic cancer” on short video platforms. In addition, previous study suggests that algorithms employed by platforms such as TikTok may prioritize content with high entertainment value or emotional appeal, which often lacks high-quality and reliable health information [24, 45]. For example, videos with sensationalized titles or dramatic stories may attract more user interaction, even if their content is incomplete or misleading. This discrepancy between the prioritization of platform recommendation algorithms and the educational quality of videos explains why high-quality videos (such as those with evidence-based treatment guidelines) do not consistently receive higher rankings. Furthermore, video quality exhibited no significant correlation with likes and comments, having only a weak positive relationship with “save”, a finding consistent with previous research by Zheng et al. [23]. Past video analyses of cholelithiasis [34] and atopic eczema [31] showed a negative relationship between video quality and audience approval. The relationship between video quality and both ranking and viewer approval may also reflect viewers’dual motivations when accessing health-related content. While some audiences seek evidence-based medical information, others may prioritize emotional resonance or coping strategies derived from shared experiences of patients and families, particularly in contexts where psychological support constitutes the primary informational need. For instance, videos featuring personal narratives or simplified advice from experts might provide psychological comfort, even if their scientific rigor is limited. This reveals the complex interplay between informational credibility assessment and affective participation in public health information-seeking behaviors. Overall, the discrepancy in these results suggests that video rankings and engagement metrics may tend to reflect the entertainment value of videos rather than the quality of their educational content. In addition, audience assessment of video quality in health education varies among diseases, warranting more research to establish the relationship between video quality and audience recognition.
The analysis of word cloud map
The word cloud map reflected both the principal message conveyed and the viewer and publisher’s topical concerns [46, 47]. The frequent keywords indicated that people were most concerned about the treatment of pancreatic cancer, followed by early symptoms. Although surgery is the only known radical pancreatic cancer treatment, only 15–20% qualify for early radical surgery [11]. Furthermore, the prognosis for some patients receiving surgery and adjuvant chemotherapy remains unfavorable [48]. However, the frequent mention of”Traditional Chinese Medicine”(n = 41, ranking fifth among Chinese keywords) in keyword analysis suggests this topic may reflect emerging public or patient interest in alternative therapeutic approaches. Previous studies have reported that some patients with advanced cancers, including pancreatic cancer, explore complementary therapies such as traditional Chinese medicine due to limited efficacy of conventional treatments [49, 50]. Several traditional Chinese medicines contain biologically active components, and some of them have anticancer activity against pancreatic cancer, which may serve as candidates for pancreatic cancer treatment [49]. However, because traditional Chinese medicine belongs to empirical medicine and most treatment plans are lacking in evidence-based medicine, making them potentially unsuitable for widespread use [50].
Conversely, we observed more attention should be accorded to imaging examination—an essential tool for early pancreatic cancer screening and diagnosis, especially abdominal enhanced thin-slice computed tomography (CT) with pancreatic cancer detection rates ranging between 76%−96% [51]. Additionally, diet captured considerable attention, with studies demonstrating a correlation between high intakes of saturated fat and candy, red meat, and processed meat, and a higher risk of pancreatic cancer [52], and Yang et al. found that a ketogenic diet can increase chemotherapy sensitivity in pancreatic cancer patients, suggesting dietary adjuvant therapy’s potential [53]. Finally, emerging treatments—including neoadjuvant therapy, radiofrequency ablation, and HIFU—are attracting interest. Despite negligible improvements in pancreatic cancer survival rates over the past decades, neoadjuvant therapy offers potential surgical opportunities for patients with borderline resectable pancreatic cancer (BRPC) and locally advanced pancreatic cancer (LAPC) and the related concept deserves to be publicized and promoted on a wider scale [12, 42]. At present, the commonly used local treatments for LAPC mainly include radiotherapy, microwave or radiofrequency ablation, irreversible electroporation and high-intensity focused ultrasound [54]. These local treatments can not only relieve pain symptoms, but also improve the local control rate, surgical conversion rate and overall survival [54, 55]. In addition, local treatment exerts an effect on tumor tissue, changes its composition and tumor microenvironment, may enhance the effect of chemotherapy, increase antigen release and presentation through T cell activation, and reduce tumor induced immunosuppression [56].
In summary, word cloud analysis visually delineates the current key topics of public concern in pancreatic cancer video content while identifying critical knowledge gaps requiring prioritized attention in health education. In the future, video creators should prioritize integrating educational content with established public needs and develop targeted educational materials to address critical knowledge gaps, thereby enhancing public comprehension and management of diseases.
Comparison of Chinese TikTok platforms in 2024 and 2023
Compared to 2023, the proportion of medical practitioners who publish pancreatic cancer-related videos on Chinese TikTok platform further increases in 2024, especially the participation of experts in the field of pancreas significantly increased to 56%. This reflects that the medical community is increasingly aware of the important role of social media in health education and is willing to disseminate health knowledge to the public through this channel [57]. In addition, the number of “news and reports” videos surged to 58%, surpassing “disease” videos. This change may be related to the adjustment of the platform’s recommendation mechanism or the public’s heightened attention to current affairs news [58]. Therefore, medical practitioners were more inclined to publish news and reports videos rather than systematic videos introducing pancreatic cancer-related knowledge, resulting in a decrease in the GQS score of their videos. Videos published by non-medical practitioners had lower GQS and mDISCERN scores than the previous year, once again exposing the public’s lack of ability to identify and disseminate high-quality health information [34].
Comparison of TikTok platforms in the USA and China in 2024
Pancreatic cancer-related videos on the TikTok platforms in the USA and China showed significant differences in terms of publisher identity and video quality. The USA platform was dominated by non-medical practitioners, especially patients and family members (47% of the total), while China was dominated by medical practitioners. This difference may stem from the different healthcare systems, cultural backgrounds and public health awareness in the two countries [59, 60]. In the USA, medical information dissemination is strictly regulated, but social media content creation is relatively free, promoting diverse voices and personal expression. This cultural environment may encourage more patients and families to share their experiences, creating a patient-centered content ecosystem [45]. In contrast, Chinese social media platforms have stricter content review and professional certification requirements, making medical professionals the dominant force in health information dissemination [24, 44]. The USA healthcare system emphasizes patient autonomy and personalized treatment, which may be reflected in the videos posted by patients and their family members, who tend to share personal treatment experiences and viewpoints. In contrast, the Chinese healthcare system focuses on professional medical team guidance and systematic treatment plans, with video content often related to outpatient records, disease knowledge, and professional treatment recommendations [24]. In terms of video quality, the overall quality of videos posted by both medical and non-medical practitioners on the USA TikTok platform was higher than that in China, suggesting that USA publishers might have more professional medical knowledge or higher production literacy related to new media compared to similar populations in China. However, in Wu et al.’s cross-national study on non-alcoholic fatty liver disease (NAFLD)-related short videos [24], there was no significant difference in the quality of videos published by non-medical professionals between the USA and China. This suggests varying levels of public understanding of different diseases across countries. It is worth noting that compared to medical practitioners, the quality of videos published by USA non-medical practitioners was lower, while their larger number on the USA platform thus cancelling out the advantage of the USA medical practitioners to a certain extent, resulting in no significant difference in the overall quality of the videos between the two countries. The word cloud maps showed that treatment, symptoms and diet were the focus of attention in both countries, reflecting the importance of early identification of pancreatic cancer, choice of treatment options and dietary management [61, 62]. In addition, the word cloud map of the USA highlighted “nursing”, which showed that the American society attached great importance to the nursing care of pancreatic cancer patients, including pain management, nutrition, and psychological support [63], which was a reference for the video publishers in China.
Evaluation and comparison of scientific scoring tools
In this cross-sectional study, we evaluated the quality and reliability of short videos using two methods: a combination of (1) GQS and mDISCERN, and (2) MQ-VET. The use of the MQ-VET scale aims to cover a broader range of evaluation criteria (such as use of visual aids, use of medical terminology, video recording, and update dates) to conduct a more comprehensive assessment of video quality and reliability. The strong correlation observed between GQS, mDISCERN, and MQ-VET scores underscores a significant alignment among these tools in assessing video content [24, 64]. This congruence not only validates the robustness of MQ-VET as a newly developed instrument but also reinforces the reliability of our findings, as multiple independent metrics converge on similar evaluation results. Such consistency is pivotal for future research, as it suggests that these tools can be used complementarily to provide a multifaceted assessment of health-related videos. However, while our findings support MQ-VET’s applicability here, its generalizability necessitates validation across other medical topics. For instance, conditions with varying public awareness levels or differing clinical complexities may require tailored evaluation criteria, highlighting the need for further cross-disease and interdisciplinary studies. From a practical perspective, the high correlation among these tools suggests that future platform algorithms could integrate these scoring systems to prioritize videos that excel across multiple quality dimensions, thereby enhancing public access to reliable health information.
Practical significance and future suggestions
Pancreatic cancer is a highly malignant disease with relatively limited public awareness, the dissemination of erroneous information through low-quality videos can have”devastating”consequences for patients [65, 66]. However, at the same time, high-quality videos have the potential to improve the dietary habits of high-risk individuals, raise awareness about early diagnosis of pancreatic cancer, and expose patients to more advanced treatment concepts, thus offering hope for reducing the incidence of pancreatic cancer and extending the survival of patients [15, 67]. Therefore, we put forward the following suggestions to contribute to the standardized and comprehensive development of field of short videos about pancreatic cancer. Short video platforms should collaborate with governments, professional organizations, and medical practitioners to review video content from an evidence-based medicine perspective and actively combat misinformation. In addition, platforms should enhance their video recommendation algorithms, differentiating health education videos from ordinary ones, and prioritize promoting higher quality and more reliable content. Moreover, platforms can draw on successful cases of other health information interventions to design pancreatic cancer related health education programs, improving public awareness of pancreatic cancer. For instance, in Galiano Castillo et al.’s study, an internet-based exercise intervention enhanced the quality of life for breast cancer patients [68]. Also, in Maria Scheerman et al.’s research, using a theory-based health intervention plan on the social media platform Telegram successfully improved oral hygiene behaviors and oral health outcomes among Iranian adolescents [69]. To enhance the visibility and influence of medical practitioners’videos, the platform can provide dedicated display sections or channels for medical practitioners. Additionally, on search result pages related to health information, the platform should prioritize the display of medical practitioners’videos. This strategy can increase the exposure of their videos, thereby boosting their visibility and influence, and promoting the widespread dissemination of high quality health information. From the publisher’s perspective, reliable reference materials should be provided in videos and medical practitioners should employ a simpler and more easily understandable approach in video production to enhance audience receptiveness. In terms of video content, pancreatic cancer is prone to recurrence after surgery [70], we found that there are no videos in China that mentioned content related to follow-up review, treatment, and care after radical pancreatic cancer surgery, so future videos on pancreatic cancer science should supplement for this content. Finally, based on the aforementioned findings and informed by existing intervention approaches in prior research, we propose a potential model utilizing short video interventions to modify pancreatic health-related behaviors. This integrated framework comprises four pivotal components: First, healthcare professionals should collaborate with video-sharing platforms to develop and disseminate evidence-based content addressing pancreatic cancer prevention, early detection, and therapeutic management. Second, platforms should employ targeted algorithms to prioritize distribution of these educational materials to high-risk demographics, particularly individuals with familial histories of pancreatic malignancy or those exhibiting modifiable risk factors (e.g., tobacco use, obesity). Third, video content should incorporate interactive components such as embedded calls-to-action and knowledge-assessment quizzes to enhance viewer engagement and facilitate practical application of health information. Finally, implementation of reinforcement mechanisms including automated reminders and online community support groups to sustain behavioral modifications longitudinally. This comprehensive model not only addresses the dissemination of accurate health information but also actively promotes its application in real-world health decisions and behaviors.
Limitations
This study has some limitations. First, we only evaluated the top 100 pancreatic cancer videos in the three platforms of TikTok, Bilibili, and Kwai, instead of evaluating all videos. Current studies indicate that, due to platform recommendation algorithms, these top 100 videos generally reflect the quality of content that most users view [23, 31, 32, 34]. However, this method may prioritize videos with high engagement or those recently uploaded, potentially excluding less popular but high-quality content (e.g., videos with lower visibility despite scientific rigor might be excluded). Future study could mitigate this limitation by expanding the sample size to include all videos in the field to obtain a broader spectrum of content. Second, this was a cross-sectional study, Videos related to pancreatic cancer on the TikTok, Bilibili, and Kwai platforms may change over time and as platform content is updated. Third, there may be some selection bias in our study. By relying only on publicly published and accessible videos, some high-quality videos may not be included in the study due to inaccessibility. Moreover, similar to many previous short video-related studies, our search strategy solely depended on the single Chinese keyword “胰腺癌” (pancreatic cancer). Although it is the most commonly used keyword by the public, it may have missed videos using other synonyms like “胰腺肿瘤” (pancreatic tumor), or compound terms such as “胰腺癌预防” (pancreatic cancer prevention), “胰腺癌治疗” (pancreatic cancer treatment). Future studies could incorporate a broader range of search terms to enhance the comprehensiveness of video sampling and mitigate terminology-related biases. In addition, the content review policies of different platforms can have an impact on the publication and visibility of videos. While strict review policies may filter out some low-quality or misleading videos, they may also result in some high-quality videos not being presented. Lastly, our analysis only includes Chinese videos on Chinese video sharing platforms and English videos on USA video sharing platforms, so the findings may not be generalizable to other language platforms. In the future, more cross-language studies are needed to fill this gap and confirm our findings.