Introduction
A medicine package leaflet contains information about a drug’s safe and effective use, such as its active ingredients, indications, dosage, administration, precautions, and storage conditions. A patient-centered package leaflet can enhance the readability and usability of the information provided. Readability refers to how easily and clearly the text can be read and understood based on linguistic complexity, while usability focuses on the user’s experience in finding, comprehending, and applying the information.1 It plays a crucial role in promoting appropriate medication use by facilitating informed decision-making and helping healthcare professionals counsel patients, which can enhance medication adherence.2 Given its importance, the leaflet is a key document reviewed during the drug approval process to ensure it meets legal requirements and is well prepared. In many countries, including South Korea, medicine package leaflets are printed and distributed on paper, enclosed in the product container or packaging, and provided to patients or healthcare professionals, such as doctors and pharmacists.
Recent technological advancements have led to several efforts to provide information about medicines online. The COVID-19 pandemic significantly accelerated the digital transformation in the healthcare sector,3 making e-labeling increasingly popular. Although there is no universally accepted definition of e-labeling, it typically refers to delivering medicine package leaflets electronically.4 E-labeling is also known as electronic Product Information (ePI), e-label or e-leaflet, depending on the country or region. Despite the variations in terminology, recent initiatives from regulatory authorities and the pharmaceutical industry have converged on a common understanding of e-labeling as a tool to complement or replace traditional paper leaflets. Research on e-labeling has highlighted several benefits,4–6 including providing patients and healthcare professionals with instant access to up-to-date drug information, personalizing content for user-friendliness, enhancing information sharing through improved compatibility, and offering environmental benefits by reducing the need for paper leaflets.
Adoption of e-labeling at the national level is occurring worldwide. In Japan, the Act on “Securing Quality, Efficacy, and Safety of Products Including Pharmaceuticals and Medical Devices” was amended in December 2019 and enforced in August 2021, allowing prescription drug leaflets to be provided exclusively in electronic format.7 Singapore has been running an e-label pilot since 2019, allowing pharmaceutical companies to adopt e-labeling voluntarily.8 Initially, it was only applied to prescription drugs, but as of April 2024, e-labeling has been extended to non-prescription drugs. The European Union (EU) established key principles for the development of ePIs for medicinal products for human use in January 2020.9 Some EU member states, including Denmark, Netherlands, Spain, and Sweden, have run a pilot project from July 2023 to August 2024 to examine the utility of ePIs.10 In South Korea, legal amendments took effect in January 2024, allowing certain prescription drugs to be distributed without paper leaflets, relying solely on electronic versions.11
However, the benefits of e-labeling may not be uniformly applicable to different countries and populations because of discrepancies in the management of drug leaflets and e-labeling. Various studies on patient perceptions of e-labeling12–14 have shown that some patients appreciate e-labeling, while others—particularly older adults—express concerns about accessibility, privacy, and difficulty in finding specific information.12,13 Surveys indicate that many patients would accept e-labeling if pharmacies continued providing paper copies upon request.14 If e-labeling proves to be less effective than paper-based package leaflets in terms of readability and usability, it may increase the risk of inappropriate medicine use and reduce medication adherence. These limitations are particularly concerning for individuals with limited digital and/or medication literacy, such as older adults, and may ultimately undermine public health outcomes while impeding efforts to promote patient empowerment. While some studies only address patient perceptions and expectations of e-labeling, very few empirical studies have demonstrated whether e-labeling can actually increase the readability and usability of drug leaflets and meet patient expectations. Notably, no such studies have been conducted in the Korean context.
This study investigates whether e-labeling improves readability and usability compared to traditional paper leaflets in the South Korean context. Using a user testing method, we explored factors that enhance or hinder the effectiveness of e-labeling. User testing is a mixed-methods approach that incorporates both quantitative and qualitative findings to evaluate the readability and usability of package leaflets based on participants’ ability to use them.15 The guideline published by the European Commission recommends user testing for consulting with patient populations to ensure readability and usability of package leaflets under Article 59(3) of Directive 2001/83/EC.16 Unlike traditional content-based readability assessments—such as the Flesch–Kincaid formula, SMOG formula, or FOG readability test—user testing provides insight into actual health literacy skills, rather than relying solely on word or sentence length.17 User testing is not currently mandatory for developing drug leaflets in South Korea, and no study has evaluated the utilization of e-labeling and compared it to the readability of paper leaflets. The results of this study can provide empirical evidence for countries considering introducing e-labeling and developing relevant policies.
Materials and Methods
Study Design
We used user testing, a mixed-methods approach to evaluate the readability and usability of package leaflets in both paper and electronic formats. Quantitative data were collected through a user testing questionnaire to evaluate participants’ ability to locate and understand information in the package leaflets, specifically assessing traceability, tracking speed, and comprehension. Additionally, qualitative data were gathered through semi-structured interviews to explore participants’ perceptions of the leaflet they used, the challenges they faced in finding and understanding information, and their views on the potential adoption of e-labeling.
User testing procedures followed the recommendations outlined in the European Commission guideline,16 which includes standards for participant selection, testing protocols, and evaluation criteria. According to the guideline,16 if deficiencies are identified during testing, the leaflet should be revised and retested with new participants until the criteria are met. However, this study did not aim to optimize or validate a specific leaflet version for regulatory approval. Instead, it focused on evaluating the current readability and usability of package leaflets in both printed and electronic formats. Therefore, the testing procedure was conducted once for each format without repetition.
Tested Materials
We conducted a comparative assessment of the paper and electronic versions of package leaflets (e-labeling) for two therapeutic drug classes: antihypertensive (prescription drug) and non-steroidal anti-inflammatory drug (NSAID) (non-prescription drug). Based on publicly available national supply data from the Korean Statistical Information Service (KOSIS), these classes were identified as top-supply scale products. Antihypertensives require high medication adherence for chronic disease management,18 making clear and accessible information particularly critical. NSAIDs were selected as the most frequently supplied non-prescription drugs and the class associated with the highest number of adverse event reports in South Korea as of 2020.19 Final products within each class were selected based on the following criteria: (1) dispensing and administration practices, particularly whether package inserts are provided to patients in South Korean healthcare settings; (2) formulation type, with a focus on tablets due to their widespread use; and (3) the complexity of package leaflets, including document length, structure, and the amount of safety-related information.
Paper leaflets were evaluated using the actual inserts provided in each product’s packaging. The electronic versions of package leaflets were retrieved as PDF versions from the official website of the Ministry of Food and Drug Safety (MFDS), the national regulatory authority in South Korea. These PDF files maintained the original layout and structure of the paper format and were not adapted for web-based presentation.
Participants
Participants were recruited through convenience sampling via public notices posted in Goyang-si, where the interviews were conducted. Following the European Commission guideline,16 which recommends 10 to 20 participants per user testing condition, a total of 77 individuals were enrolled and allocated across four study groups (antihypertensive vs NSAID × paper leaflet vs e-labeling). Group assignment considered participants’ sex, age, and education level to promote demographic balance. Healthcare professionals (e.g., physicians, nurses, pharmacists) and individuals unable to read the materials, even with visual aids, were excluded. Additionally, participants who had used either drug within the past six months were excluded to minimize potential bias from prior familiarity.
Questionnaires
The questions were developed specifically for this study based on the principles outlined in the guideline,16 which recommends that user testing questions address important and complex issues, assess participants’ ability to find and understand the information, and evaluate their capacity to act appropriately based on it. The questions covered all necessary aspects of medicine use, including safety-related issues such as side effects, interactions, dosage, and indications. The questions were distributed in accordance with the amount of information provided in the leaflets. The guideline suggested using 12–15 questions; therefore, we developed 15 questions for each drug. The proportion of questions for both medicines was designed to be similar to minimize differences in difficulty. When developing the questions, we tried to avoid emphasizing either the positive or negative aspects of the medicines. The questions were organized randomly so that the flow did not follow the order presented in the leaflets. A pilot test was conducted with three participants for each study condition to ensure that patients understood the questions well. Minor revisions were made to improve clarity and flow based on their feedback. The questions for each drug are listed in Boxes 1 and 2.
|
Box 1 User Testing Questions for the Antihypertensive Leaflet
|
 |
Box 2 User Testing Questions for the NSAID Leaflet
|
Data Collection
User-testing interviews were conducted for one month, from September 14, 2020, to October 14, 2020. These interviews took place individually in a quiet, private room on the Dongguk University campus, with the author (JW) conducting each face-to-face interview one-on-one with the participants. Participants were asked to read the package leaflets thoroughly before answering the interviewer’s questions. They could take as much time as they needed to view the content of the paper leaflets and e-labeling. Each user testing interview was designed to last no more than 45 min to avoid tiring the participants.16 Participants were allowed to use a pen to underline or mark information while reading the paper leaflets. When using the e-labeling, participants scanned the QR code created for this study which was placed on the outer packaging of the drug, using their own mobile phones. This approach enabled them to interact with the e-labeling content in a natural and familiar manner, utilizing features such as zoom or search functions with comfort. After reading the leaflets, the interviewer asked the participants to answer questions, which they could answer while looking at the leaflets.
Quantitative data were collected by three criteria. The first criterion was traceability.16 The participants were asked to indicate where they could find the information in the leaflet. Participants were rated as “unsuccessful” if they failed to find the correct information, gave up trying to find it, or took more than 3 min per question. The second criterion was tracking speed, which is not specified in the European Commission guideline16 but was adopted from a previous study20 conducted in South Korea. This measure allowed for a detailed comparison between paper leaflets and e-labeling. It refers to how quickly participants found the desired information in the leaflets. The interviewer evaluated it on a 5-point scale (5: found the information immediately or within 30 seconds, 4: found the information within 1 min, 3: found the information within 1–2 min, 2: found the information after more than 2 min, 1: failed to find the information, including cases of incorrect answers or delays exceeding 3 min).20 The third criterion was comprehension.16 Since we did not measure the participants’ pre-existing medical and pharmaceutical knowledge or memory skills, they were asked to rephrase the content of the documents in their own words. All three criteria were applied consistently to evaluate both leaflet formats (paper and e-labeling).
Once participants had answered all the questions, qualitative data were collected through short, semi-structured interviews. Participants rated their satisfaction with the paper leaflets and e-labeling. They were also encouraged to elaborate on factors that assisted or hindered their ability to find and understand the information. We also asked about their perceptions of implementing e-labeling and how they would feel if paper leaflets were no longer available and could only be viewed electronically, such as on a mobile phone or computer.
Analysis
All interviews were audio-recorded and transcribed verbatim. The first author (JW) checked the accuracy of the transcripts by reviewing the audio recordings. The qualitative data were managed and analyzed using MAXQDA 24 (VERBI Software GmbH, Berlin, Germany). A content analysis approach was used to identify and categorize themes and patterns.21 Both authors (JW and KH) read each transcript independently, identified codes and grouped them into themes. Any discrepancies were resolved through group discussions until a consensus was reached. Quantitative data were analyzed using IBM SPSS Statistics 29.0 (IBM Corp., Armonk, NY, USA). To assess potential baseline imbalances between participants assigned to the paper leaflet and e-labeling groups within each therapeutic drug class, chi-squared tests (or Fisher’s exact tests, where appropriate) were used for categorical variables and independent t-tests for continuous variables. User testing outcomes (traceability, tracking speed, and comprehension) were presented descriptively.
Ethics
The study was approved by the Dongguk University Institutional Review Board on May 15, 2020 (Approval Number: DUIRB-202005-00) and conducted in accordance with the Declaration of Helsinki for research on human subjects. All participants provided written informed consent prior to participation, including consent for audio recording and the use of anonymized, direct quotes in publication. Before the interviews, participants were fully informed of the study’s purpose, procedures, and interview duration. They were also informed that participation was voluntary, and that they could withdraw at any time. To ensure participants’ anonymity and data confidentiality, all data were de-identified and stored securely in accordance with relevant guidelines and regulations.
Results
Participants’ Characteristics
The demographic characteristics of the participants are presented in Table 1. To assess baseline imbalance, we compared sex, age, education level, average monthly household income, occupational status (clerical vs non-clerical), and marital status across study groups. The analysis revealed no statistically significant differences, indicating that participants were broadly similar in their ability to find and understand information in the package leaflet.
 |
Table 1 Participants’ Characteristics
|
Quantitative Findings
Table 2 summarizes the results of the readability assessment for paper leaflets and e-labeling. For the antihypertensive, information traceability in the paper leaflet was 68.8%, reduced by 4.7% when provided electronically. The average tracking speed was also 0.08 points lower for electronic delivery than for paper, and comprehension decreased by 3.3%.
 |
Table 2 Summary of User Test Results
|
The NSAID showed a more pronounced drop in all metrics than the antihypertensive when the information was provided electronically rather than on paper. Its traceability dropped by 7.0%, tracking speed decreased by 0.19 points, and comprehension declined by 5.7%. Although the decline was more substantial for the NSAID, the antihypertensive had lower values for all metrics, making it more challenging to track and understand the desired information.
Tables 3 and 4 show the differences in traceability, tracking speed, and comprehension by question. According to the guidelines,16 a leaflet is considered readable if 90% of participants can locate the correct information and 90% of them understand it. However, none of the drugs and delivery methods evaluated in this study met these criteria. For the antihypertensive, no question in the paper leaflet was successfully tracked by all participants. Four questions (2, 3, 14, and 15) had 100% traceability for the e-labeling group. However, question 6 had 0% traceability, as no one could track the information. The paper leaflet’s lowest tracking rates were for questions 4 and 6. When comparing the number of questions answered correctly by more than half of the participants, 11 out of 15 (73.3%) were answered correctly using the paper leaflet, compared to 9 out of 15 (60.0%) with e-labeling, showing a slight decrease in accuracy with electronic delivery. For some questions, participants struggled to understand the content even after finding the correct information, which indicated decreased comprehension.
 |
Table 3 Traceability, Tracking Speed, and Comprehension for the Antihypertensive by Question
|
 |
Table 4 Traceability, Tracking Speed, and Comprehension for the NSAID by Question
|
For the NSAID, no question except for question 9 achieved 100% traceability in either format. For questions with a tracking rate of 50% or higher, 13 out of 15 (86.7%) were identified in the paper leaflet, with 12 out of 15 (80.0%) in the e-labeling, which is a relatively high rate compared with the antihypertensive. The lowest-tracked questions by the delivery method were question 10 in the paper leaflet and questions 6 and 10 in the e-labeling. These included questions regarding interactions with other medicines (question 6) and availability by prescription (question 10). Some questions were misunderstood, even when the correct information was tracked, as was also observed with the antihypertensive. Figures 1 and 2 visualize these outcomes by question.
 |
Figure 1 Comparison of traceability, tracking speed, and comprehension for the antihypertensive by question. Abbreviations: EL, Electronic Leaflet; PL, Paper Leaflet.
|
 |
Figure 2 Comparison of traceability, tracking speed, and comprehension for the NSAID by question. Abbreviations: EL, Electronic Leaflet; PL, Paper Leaflet.
|
Qualitative Findings
Barriers to Readability and Usability in Paper Leaflets
Participants complained about the paper leaflets, with negative evaluations being predominant in terms of readability and usability. Three main issues were identified: small font size, complicated terminology, and document structure obscuring important information. The font size was the most straightforward complaint among users using paper leaflets. Many suggested that increasing the text size would encourage more people to read more leaflets, even if it required a larger paper size or more sheets.
I have always felt manufacturers should make the font size of the leaflets bigger. I ignore the leaflets because it (font size) is too small. I just put them (leaflets) away whenever I read. (P19, 40s, NSAID paper leaflet group)
Complicated terminology substantially contributed to poor readability. Terms only understood by experts made it challenging for users to grasp the content of the leaflets. It requires additional effort from the user, who must repeatedly read the leaflet to understand it, reducing its overall utilization.
Professionals might know all the terms, but most people still don’t. I don’t understand what the leaflets are saying; it’s the same information, but I can’t make sense of it. (P18, 40s, NSAID paper leaflet group)
Most participants agreed that leaflet formatting was a major problem, noting that it complicated the process of distinguishing important information and understanding the document’s flow and structure. Several alternatives were suggested to improve the problem, such as highlighting essential information using bold text and prominent colors or placing it at the front of the leaflets.
Everything is written in the same font size, which is too small. The manufacturer should use different colors or make the important information larger. Alternatively, just put the important stuff in the front. (P17, 50s, antihypertensive paper leaflet group)
Mixed Perceptions of E-Labeling: Advantages and Usability Issues
Participants’ perceptions of e-labeling, particularly in terms of readability and usability, were mixed, encompassing both positive and negative aspects. Since the paper leaflets were merely converted into PDF format for use as e-labeling, issues such as the use of complicated terminology and poorly structured content that obscured key information remained evident in the e-labeling format. However, participants positively evaluated e-labeling for two reasons: it freed them from the constraints of text size and allowed them to find the information through the search function. Viewing e-labeling on a mobile phone allowed a larger display, which many participants utilized during interviews. Even though some participants did not use the search function during the interviews, they were aware of it and expressed plans to use it in the future when accessing e-labeling.
It is more convenient to see the text because I cannot adjust the size of the text on paper, but I can adjust the size of the text on the smartphone. (P19, 40s, NSAID e-labeling group)
If I use e-labeling on the Internet, I can search and type in the word for the information I want and get to that part quickly, rather than reading it in lines. It is more convenient. (P10, 20s, antihypertensive e-labeling group)
However, there were limitations in navigating information on mobile phones when using e-labeling. Typically, users refer to the table of contents or skim headings and subheadings to grasp the content and find the specific information they need. However, viewing e-labeling on a mobile phone does not allow one to grasp the contents of the entire document at a glance but only allows zooming in on a specific location, which may further limit the utilization of drug leaflets. Many participants expressed frustration with navigating back and forth within the document to find the information they needed. They also found it challenging to determine which of the search results or browsed content contained the exact information they were looking for. Overall, paper leaflets were considered more convenient for understanding the structure and finding specific details.
I can increase the font size on my phone, but it takes me a long time to zoom in to find what I am looking for because I do not know where the information is. With paper leaflets, I can see the document’s structure at a glance and find the information faster. (P5, 20s, antihypertensive e-labeling group)
I wish only the exact information I was looking for would show up, but there’s so much similar content scattered throughout the leaflet. To make sure what I’m looking for, I have to go through everything, so I end up reading it all from the beginning again. (P16, 40s, antihypertensive e-labeling group)
Participants also shared their views on the accessibility of e-labeling, which was often compared to that of paper leaflets. These responses revealed a mix of both positive and negative aspects. For example, strong resistance to and negative perceptions of e-labeling were primarily driven by concerns over reduced accessibility and usability of drug information among older adults. Older people are less likely to use electronic devices such as mobile phones, which raised concerns that this could prevent them from accessing information. The additional step of scanning a QR code to view medicine information could be cumbersome and reduce people’s willingness to use the e-labeling. Paper leaflets were valued for their immediate accessibility, as they were available immediately after opening the drug container.
Paper leaflets are convenient and available right out of the box. Why would I go through the trouble of searching on my phone? I wouldn’t use my phone to look up information more than I already do with paper leaflets. (P12, 50s, antihypertensive e-labeling group)
On the other hand, some participants expected e-labeling would reduce the need for storing printed copies of drug leaflets. For example, when using non-prescription medicines, patients often discard the paper package inserts; however, if a QR code is on the container or packaging, they could retrieve the drug information without relying on paper leaflets. Another reported advantage was the ability to download and save e-labeling on mobile phones, which enabled access to drug information anytime and anywhere. The summary of qualitative findings on e-labeling are presented in Table 5.
When I open the medicine packaging, I sometimes throw away the paper leaflets right away. Sometimes, the leaflets are wrinkled, dirty, or torn, making them difficult to read. I might even lose them. In that case, I think it would be nice to be able to look up the information I want on my phone. (P7, 20s, NSAID e-labeling group)
 |
Table 5 Summary of Qualitative Findings on E-Labeling
|
Patient Preferences and Conditions for Successful E-Labeling Adoption
When considering the advantages and disadvantages of paper leaflets and e-labeling, most participants expressed a clear preference for a hybrid approach that incorporates both formats. They emphasized that offering both options would maximize usability and ensure that patients can choose the format that best meets their needs. However, if the national policy mandates a single delivery method, participants strongly advocated the retention of paper package leaflets to ensure availability of drug information through an existing and familiar format.
If there is a paper leaflet that is also available online, I’d prefer that. However, if the government gets rid of all the paper leaflets, the elderly won’t understand it, no matter how much you explain it. Many people still have difficulty using QR codes. Also, the elderly take more medication. I think e-labeling is too hard for them. (P20, 20s, NSAID e-labeling group)
Even participants who were optimistic about using e-labeling, either in combination with paper leaflets or alone, emphasized the need for improvements to enhance the usefulness of e-labeling. Several improvements were suggested to maximize the benefits of e-labeling. The improvements can be summarized into four: (1) formatting important information in a way that makes it easier to find and understand; (2) creating a table of contents and executive summary; (3) enhancing readability through visual elements such as color and images, which are less feasible in print due to cost, and (4) providing additional information, such as via hyperlinks and videos, to help people understand the content. Participants agreed that simply transferring the current paper leaflet content and format to the e-labeling would not be beneficial and could reduce their usage. They said e-labeling needs to be improved to make it easier to identify important information quickly. Solutions include highlighting important information with color, underlining or bolding, increasing the font size of certain information, or putting important information at the front of the leaflet. While the definition of “important information” may vary, typical examples include warnings, dosage instructions, and contraindications.
Why don’t they just make the information that they are legally required to provide and that people are asking for bigger and readily available? It would be less bulky, and they could create a separate online link for people who want to go into more detail. (P15, 40s, NSAID e-labeling group)
Participants suggested the creation of a table of contents and executive summary for ease of understanding the structure of the entire leaflet. Since e-labeling makes it harder to view the document’s overall layout at a glance, improving the format is essential. Suggestions included placing the table of contents at the beginning of the leaflet and incorporating digital features that would allow users to jump directly to the information they need, making the search process faster and more efficient.
I would like to utilize links to get the information I am looking for quickly. For example, suppose I want to find the precautions for specific situations while taking medicines. In that case, I can search and click the words I am looking for and gather the relevant information faster. (P17, 40s, NSAID paper leaflet group)
Finally, some commented that it would be helpful to have the ability to view additional information that cannot be provided in the leaflets via hyperlinks or videos. For example, they expressed a desire to click on a complicated term in the leaflet to see a further explanation or a direct link to a video to watch an expert explain a drug or disease in detail. In addition, e-labeling can be made more patient-friendly by using various techniques to improve readability—such as the use of color and images—without concerns about the cost limitations associated with printed materials. By actively utilizing these technological advantages enabled by digital platforms, it becomes possible to overcome the inherent limitations of paper leaflets and offer more user-centered and accessible drug information.
Discussion
In this study, we used user testing, as recommended by the European Commission,16 to assess the readability and usability of package leaflets in both paper and electronic formats. According to the European guideline,16 a package leaflet is considered readable when 90% of the participants can locate the information, and 90% of them understand it. However, the paper leaflets for both drugs failed to meet this standard, and e-labeling demonstrated even lower performance in terms of traceability, tracking speed, and comprehension. Qualitative feedback from participants revealed several shortcomings. Many found it difficult to locate key information in paper leaflets due to technical jargon, poor visual layout, and an unclear information hierarchy. In the case of e-labeling, the need to zoom in to read small text often disrupted their ability to grasp the overall structure of the leaflet. These findings indicate that current package leaflets in South Korea are not adequately tailored to patients’ needs and comprehension levels, and thus require improvements in both content and format before fully transitioning to e-labeling.
The poor performance of e-labeling observed in the user testing is likely due to the inherent limitations of its current structure and use, as revealed in our qualitative findings. Participants reported difficulties in grasping the overall structure of the document at a glance and expressed uncertainty about whether the information they found was relevant to their needs. When users relied on zoom-in and zoom-out functions to navigate the document, they often lost the contextual understanding necessary for accurate interpretation of the information. Even when using the search function, participants found it challenging to determine which search result was most relevant, especially when similar information appeared in multiple sections of the leaflet, such as when adverse effects or precautions were repeated. Interview data further revealed that participants frequently revisited the surrounding context of the information they found to confirm its relevance, resulting in slower tracking speeds and, in many cases, either failing to find the answer or failing to fully comprehend it once found. Although users could enlarge the text, the small font size on mobile phones, combined with complex terminology throughout the leaflet, significantly hindered readability and further limited the practical usability of e-labeling. Being “dropped into” a specific point in the e-labeling via keyword search without sufficient orientation or guidance often left them confused and unable to use the information effectively. Previous studies have found that even younger adults report concerns about e-labeling, often related to difficulties in navigating digital formats.12,13 Our findings support these previous studies, emphasizing that concerns about e-labeling are not primarily due to resistance to digital formats themselves, but rather arise from practical barriers, such as insufficient guidance on how to access and use the information, a lack of intuitive navigation, and poorly organized layouts.12,22
While previous studies have quantitatively demonstrated mixed public attitudes toward e-labeling,12,13 our study also highlights valuable qualitative insights, particularly regarding accessibility. Participants valued the potential convenience of e-labeling, particularly the ability to access information anytime and anywhere without relying on paper leaflets. However, others expressed concern that removing paper leaflets, which are considered the most intuitive and familiar format, could make it more challenging to obtain the necessary drug information. Notably, participants considered not only themselves but also expressed concern for others, particularly older adults and individuals with limited digital skills, who might struggle to access information in digital formats. This broader perspective contributed to their reluctance to support a nationwide transition to e-labeling. These findings highlight that for e-labeling to be successfully implemented, it must ensure universal accessibility and demonstrate clear advantages over existing paper leaflets in terms of readability and usability.
It is essential to avoid simply converting paper package leaflets into digital formats such as PDFs to maximize the benefits of e-labeling, as seen in the case of South Korea. Instead, the focus should shift toward optimizing digital delivery formats by utilizing features such as intuitive navigation, structured content, interactive functions, and customization options.23 In particular, introducing a “key information” section, which can help patients quickly locate key information, has been discussed as a strategy to enhance user comprehension, though implementation challenges such as standardization and regulatory burden remain.24 Furthermore, our study identified several desired features suggested by participants, such as headline sections, color, pictograms, hyperlinks, videos, and customized leaflets for patients. These features are supported by previous literature for their effectiveness in improving information accessibility and understanding,25–28 underscoring the importance of integrating them into future e-labeling formats. Overall, patient-centered approaches are crucial to ensure that e-labeling effectively serves its intended purpose of enhancing access and understanding across diverse user populations.
Implications for Policy and Practice
As of 2023, South Korea has one of the most advanced digital infrastructures globally, with household internet access reaching 99.9%,29 a smartphone penetration rate of 94.8%,30 and high rates of device ownership even among older adults (e.g., 96.2% in the 60s).30 The household internet penetration rate stands at 82.5%, and computer ownership at 78.1%, indicating widespread digital access across the population.31 These conditions help explain the government’s recent decision to implement a national e-labeling system using QR codes. However, while there is substantial policy interest in expanding digital accessibility, comparatively less attention has been given to improving e-health literacy and users’ ability to engage with health-related information.32 National surveys reveal substantial disparities in digital health literacy across different subpopulations,33 particularly among older adults and other information-vulnerable groups.34–36 Without targeted interventions to address these gaps, the shift to e-labeling may inadvertently limit access to essential medicine information and exacerbate health inequities.
Unlike the European Union, where user testing is an established component of the regulatory process, South Korea currently lacks such a requirement.37 Moreover, while countries that have adopted e-labeling—such as Japan and Singapore—do not mandate user testing, they have made efforts to develop patient-targeted package leaflets. In contrast, South Korea has yet to establish practices for designing package leaflets tailored to patient needs, indicating a clear area for improvement. As e-labeling advances, there is a growing need not only to evaluate the readability and usability of digital materials through user testing, but also to assess user acceptance—whether patients from diverse backgrounds can and are willing to engage with digital formats. In this context, developing patient-centered content and verifying its acceptability and usability through real-world testing will be critical. Reflecting these considerations, several countries have initially limited e-labeling to hospital-only medicines,38 which are dispensed and administered in controlled settings under the supervision of healthcare professionals. Expansion to non-prescription products is still considered premature by many experts,39 highlighting the need for careful regulatory planning and staged implementation. Therefore, the transition from paper leaflets to e-labeling should proceed gradually, guided by empirical evidence and continuous feedback from various stakeholders. Several policy considerations must be addressed to support the effective implementation of e-labeling. First, e-labeling should be introduced gradually, starting with medicines that are administered and monitored by healthcare professionals, such as injectable drugs, while maintaining printed leaflets for self-administered medicines. Second, to ensure equitable access, healthcare systems must establish mechanisms that allow patients with limited digital literacy to obtain printed leaflets at no cost upon request. Third, e-labeling must be grounded in patient-centered principles, providing intuitive interfaces and leveraging digital tools to maximize its benefits. Its effectiveness should be validated through real-world pilot studies. Lastly, comprehensive education and awareness campaigns targeting both patients and healthcare professionals are crucial for promoting an understanding of the purpose and benefits of e-labeling. These efforts should also include practical guidance on how to use e-labeling and aim to promote its acceptance in every healthcare practice. Such an approach is essential to ensure that e-labeling enhances—not hinders—access, comprehension, and the safe use of medicines across all patient groups.
Strengths and Limitations
To the best of our knowledge, this is the first to assess the actual use of e-labeling by patients through user testing, a method widely recognized as valuable in the European regulatory review process. While several studies have evaluated the acceptance and expectations of e-labeling, our study provides in-depth insights into the barriers that may hinder its use and accessibility. By identifying key factors necessary for the effective implementation of e-labeling, we aim to support future initiatives. Additionally, there are few studies that apply user testing in countries where English is not the native language, making our findings even more significant. The results of this study will be beneficial for countries with diverse regulatory frameworks that plan to introduce e-labeling policies in the future.
This study has some limitations. First, the user testing method employed in this study to evaluate the leaflets was initially designed to assess the readability of paper leaflets and was not specifically designed for e-labeling. However, the strength of this approach lies in its ability to gather both quantitative and qualitative data, providing a comprehensive evaluation of how patients actually use the leaflets, which is the primary objective of the study.40–42 Second, participants’ education levels differed slightly from those of the general population. In South Korea, approximately 50.7% of individuals graduated from university in 2020.43 In contrast, the percentage of participants who graduated from university ranged from 33.3% to 80.0%, which may have influenced the study’s outcomes. Third, e-labeling was evaluated by scanning a QR code on the container and packaging with participants’ mobile phones to view PDF files of the leaflets. However, the readability of e-labeling might vary depending on the electronic devices or platforms used for information delivery. Despite this variability, the approach used in this study is consistent with the current practices in South Korea’s e-labeling pilot studies and may represent the most practical way to implement e-labeling. Lastly, the generalizability of the findings may be limited, as the study was conducted in a single, conveniently selected region (Goyang-si), focused solely on the antihypertensive and NSAID, and involved a relatively small sample size.
Implications for Future Research
Future studies should recruit larger and more diverse populations, include a broader range of therapeutic categories, and be conducted across multiple geographic regions to enhance the generalizability of findings. Specifically, quantitative research that identifies factors influencing the use and acceptance of e-labeling, along with subgroup analysis based on participant characteristics, would provide deeper insights into information-seeking behaviors related to e-labeling. Future studies should also explore the perspectives of other stakeholders, including healthcare professionals, the pharmaceutical industry, and regulators. For instance, physicians, pharmacists, and nurses may have different views on the value, feasibility, and clinical utility of e-labeling. The economic implications of e-labeling also deserve further investigation. While pharmaceutical companies may save costs by eliminating paper leaflets, these savings may be negated by increased expenses in other areas of the healthcare system, such as pharmacies, where printed leaflets might still be necessary to ensure equitable access upon patient requests.24 Therefore, future research should examine the broader systemic impacts of e-labeling adoption.
Conclusion
This study presents patient-centered evidence indicating that current medicine package leaflets in South Korea, whether in paper or electronic format, do not adequately ensure readability and usability for patients. While e-labeling may offer some potential advantages, it does not inherently enhance readability and usability. To facilitate the successful adoption of e-labeling, improvements in content formatting, navigation structure, equitable access mechanisms, and the use of digital features are essential prerequisites. Stakeholders must understand that e-labeling is not just a technological transition; it is also a patient communication strategy that requires careful design and validation. Pilot implementation, supported by real-world testing and feedback from multiple stakeholders, will be crucial for the effective adoption of e-labeling and for promoting safe medicine use in the digital era. Patients also need practical guidance on how to use e-labeling, along with educational and awareness campaigns led by healthcare professionals and pharmaceutical companies. Regulators should ensure that no patients are left behind in the era of e-labeling and maintain their commitment to safeguarding public health.
Data Sharing Statement
The study data are not available as the participants have not provided consent for the data to be made available beyond the research team. Furthermore, the research team only has the participants’ consent to publish de-identified group data.
Acknowledgments
The authors would like to thank all study participants for their time and contributions.
Author Contributions
All authors made substantial contributions to conceptualization, study design, data acquisition, analysis, and interpretation of data; drafted or revised the article; agreed on the journal to which the article was submitted; gave final approval of the version to be published; and committed to being accountable for all aspects of the work.
Funding
The study reported in this publication was supported by grants 22183MFDS368 and RS-2024-00332213 from the Ministry of Food and Drug Safety in 2025. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Ministry of Food and Drug Safety.
Disclosure
The authors report no conflicts of interest in this work.
References
1. Pires C, Vigário M, Cavaco A. Readability of medicinal package leaflets: a systematic review. Rev Saude Publica. 2015;49:4. doi:10.1590/S0034-8910.2015049005559
2. Raynor DK, Blenkinsopp A, Knapp P, et al. A systematic review of quantitative and qualitative research on the role and effectiveness of written information available to patients about individual medicines. Health Technol Assess. 2007;11(5):1–160. doi:10.3310/hta11050
3. Bolislis WR, de Lucia ML, Dolz F, et al. Regulatory agilities in the time of COVID-19: overview, trends, and opportunities. Clin Ther. 2021;43(1):124–139. doi:10.1016/j.clinthera.2020.11.015
4. Matsui R, Yamaguchi K, Lee JJV, et al. Survey result for E-labeling initiatives in Asia. Ther Innov Regul Sci. 2023;57(2):251–260. doi:10.1007/s43441-022-00462-5
5. Bolislis WRR, Mortazavi C, Riccioni R, et al. From print to screen: regulatory considerations to adopting innovative approaches for patient information and safety. Ther Innov Regul Sci. 2020;54(4):831–838. doi:10.1007/s43441-019-00018-0
6. Roberts K, Thakkar R, Autor D, et al. Creating E-Labeling platforms: an industry vision. Clin Pharmacol Ther. 2020;108(4):716–718. doi:10.1002/cpt.1865
7. Pharmaceuticals and Medical Devices Agency (PMDA). Information regarding newly introduced electronic package inserts. Available from: https://www.pmda.go.jp/english/safety/info-services/e-pack-ins/0001.html.
8. Health Sciences Authority (HSA). Guidance on electronic labelling for therapeutic products. Available from: https://www.hsa.gov.sg/docs/default-source/hprg-tpb/guidances/tpb-gn-021-000_appendix-7a-guidance-on-electronic-labelling-for-therapeutic-products.pdf?sfvrsn=1f3ad3d0_6.
9. European Medicines Agency (EMA). Electronic product information for human medicines in the EU: key principles. Available from: https://www.ema.europa.eu/en/news/key-principles-use-electronic-product-information-eu-medicines.
10. European Medicines Agency (EMA). Electronic product information (ePI). Available from: https://www.ema.europa.eu/en/human-regulatory-overview/marketing-authorisation/product-information-requirements/electronic-product-information-epi.
11. Pharmaceutical Affairs Act (Act No. 19897). Law.go.kr. Available from: https://www.law.go.kr/lsInfoP.do?lsiSeq=257861&ancYd=20240102&ancNo=19897&efYd=20240102&nwJoYnInfo=N&efGubun%20=Y&chrClsCd=010202&ancYnChk=0.
12. Hammar T, Nilsson AL, Hovstadius B. Patients’ views on electronic patient information leaflets. Pharm Pract. 2016;14(2):702–710. doi:10.18549/PharmPract.2016.02.702
13. Bamberger M, De Loof H, Marstboom C, et al. Replacing vaccine paper package inserts: a multi-country questionnaire study on the acceptability of an electronic replacement in different target groups. BMC Public Health. 2022;22(1):156. doi:10.1186/s12889-022-12510-8
14. Bechini A, Chiesi F, Giammarco B, et al. Electronic package leaflets for vaccines: what are people’s perceptions in Italy? Vaccines. 2022;10(7):1075. doi:10.3390/vaccines10071075
15. Sless D, Shrensky R. Writing About Medicines for People. Sydney: Australian Self-Medication Industry; 2006.
16. European Commission. Guideline on the readability of the labelling and package leaflet of medicinal products for human use. Available from: https://health.ec.europa.eu/system/files/2016-11/2009_01_12_readability_guideline_final_en_0.pdf.
17. Raynor DK. User testing in developing patient medication information in Europe. Res Social Adm Pharm. 2013;9(5):640–645. doi:10.1016/j.sapharm.2013.02.007
18. Kim SJ, Kwon OD, Cho B, et al. Effects of combination drugs on antihypertensive medication adherence in a real-world setting: a Korean Nationwide Study. BMJ Open. 2019;9(6):e029862. doi:10.1136/bmjopen-2019-029862
19. Ministry of Food and Drug Safety (MFDS). Trends in reporting of safety information on medicines in 2020. Available from: https://nedrug.mfds.go.kr/bbs/2/57/.
20. Kim DJ. The user readability test of package inserts on oral contraceptives in Korea [master’s thesis]. Seoul: Chung-Ang University; 2009.
21. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–1288. doi:10.1177/1049732305276687
22. Loh XY, Woo AL, Haris A, et al. Acceptance of electronic labeling for medicinal product information among Malaysian hospital patients: cross-sectional study. J Med Internet Res. 2024;26:e56591. doi:10.2196/56591
23. Sirkas K, Juppo A, Miettinen M, et al. Could paper package leaflets be left out from hospital products? Explor Res Clin Soc Pharm. 2022;26(7):100176. doi:10.1016/j.rcsop.2022.100176
24. European Federation of Pharmaceutical Industries and Associations (EFPIA). Position papers on electronic product information (ePI). Available from: https://www.efpia.eu/media/2sgl2zyh/iatf-position-papers-on-epi.pdf.
25. Dickinson R, Raynor DK, Knapp P, et al. Do patients use a headline section in a leaflet to find key information about their medicines? Findings from a user-test study. Ther Innov Regul Sci. 2016;50(5):581–591. doi:10.1177/2168479016639080
26. Shrestha A, Rajesh V, Dessai SS, et al. Preparation, validation and user-testing of pictogram-based patient information leaflets for tuberculosis. Pulm Pharmacol Ther. 2018;51:26–31. doi:10.1016/j.pupt.2018.05.002
27. van den Berg LN, Chavannes NH, Aardoom JJ. Using animated videos to promote the accessibility and understandability of package leaflets: retrospective observational study evaluating the first year of implementation. J Med Internet Res. 2023;25:e40914. doi:10.2196/40914
28. Munsour EE, Awaisu A, Ahmad Hassali MA, et al. Impact of customized-consumer medication information on health-related quality of life among patients with type 2 diabetes mellitus. Res Social Adm Pharm. 2020;16(6):793–799. doi:10.1016/j.sapharm.2019.08.038
29. OECD. ICT access and usage by households. Available from: https://data-explorer.oecd.org/vis?tm=DF_HH&pg=0&snb=1&vw=tb&df[ds]=dsDisseminateFinalDMZ&df[id]=DSD_ICT_HH_IND%40DF_HH&df[ag]=OECD.STI.DEP&df[vs]=&pd=2012%2C&dq=.A.B1_HH._T._T.&to[TIME_PERIOD]=false.
30. Korea Communications Commission (KCC). 2023 Survey on media usage behavior. Available from: https://kcc.go.kr/user.do?mode=view&page=A02060100&dc=K02060100&boardId=1027&cp=1&boardSeq=59157.
31. Ministry of Science and ICT (MSIT). MSIT announces 2022 survey result on digital divide. Available from: https://www.msit.go.kr/eng/bbs/view.do?sCode=eng&mId=4&mPid=2&pageIndex=&bbsSeqNo=42&nttSeqNo=782&searchOpt=ALL&searchTxt=.
32. Choi SK, Chun H, Choi EJ. Review of recent digital health literacy programs in Europe and the United States. Korean J Health Educ Promot. 2022;39(4):15–28. doi:10.14367/kjhep.2022.39.4.15
33. Ministry of Science and ICT (MSIT). The report on the digital divide in 2023. Available from: https://www.msit.go.kr/bbs/view.do?sCode=user&mId=99&mPid=74&bbsSeqNo=79&nttSeqNo=3173616.
34. Kim M, Lee J, Doo EY. Factors influencing healthcare provider-patient communication of patients with chronic diseases. J Korean Acad Nurs Adm. 2020;26(2):73–83. doi:10.11111/jkana.2020.26.2.73
35. Kim HJ, Kim M. Comparison study of e-health literacy and health promoting behaviors of cancer patients and nurses. Asian Oncol Nurs. 2020;20(2):100–109. doi:10.5388/aon.2020.20.2.100
36. Song JH, Shin SJ. The effects of e-health literacy and subjective health status on health-seeking behaviors of elderly using the Internet in the community. J Digit Converg. 2020;18(1):321–332. doi:10.14400/JDC.2020.18.1.321
37. Yang JW, Kwon KH. A comparative analysis on common technical document module 1 submission form of drug labeling in South Korea, United States and Europe. Yakhak Hoeji. 2023;67(6):404–414. doi:10.17480/psk.2023.67.6.404
38. Skogman-Lindqvist C, Lapatto-Reiniluoto O, Sirviö M, et al. Benefits and challenges of electronic package leaflet (ePL) – review of ePL pilots in hospital settings in Europe. Eur J Pharm Sci. 2023;191:106605. doi:10.1016/j.ejps.2023.106605
39. Lambot N, Vande Ginste M. Electronic leaflet pilot in Belgium and Luxembourg hospitals. Regulatory Rapporteur. 2022;19(4):3–5.
40. Jay E, Aslani P, Raynor DK. User testing of consumer medicine information in Australia. Health Educ J. 2011;70(4):420–427. doi:10.1177/0017896910376131
41. Tong V, Raynor DK, Aslani P. User testing as a method for identifying how consumers say they would act on information related to over-the-counter medicines. Res Social Adm Pharm. 2017;13(3):476–484. doi:10.1016/j.sapharm.2016.06.001
42. Tong V, Aslani P, Raynor DK, et al. Developing and user testing new pharmacy label formats – a study to inform labelling standards. Health Expect. 2021;24(4):1125–1136. doi:10.1111/hex.13203
43. OECD. Education at a glance 2024. Available from: https://www.oecd.org/en/publications/education-at-a-glance-2024_c00cad36-en.html.