Eintrag weiter verarbeiten
A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment
Gespeichert in:
| Zeitschriftentitel: | JMIR Medical Informatics |
|---|---|
| Personen und Körperschaften: | , , , , , , , , |
| In: | JMIR Medical Informatics, 9, 2021, 7, S. e24022 |
| Format: | E-Article |
| Sprache: | Englisch |
| veröffentlicht: |
JMIR Publications Inc.
|
| Schlagwörter: |
| author_facet |
Zheng, Wu Yi Van Dort, Bethany Marcilly, Romaric Day, Richard Burke, Rosemary Shakib, Sepehr Ku, Young Reid-Anderson, Hannah Baysari, Melissa Zheng, Wu Yi Van Dort, Bethany Marcilly, Romaric Day, Richard Burke, Rosemary Shakib, Sepehr Ku, Young Reid-Anderson, Hannah Baysari, Melissa |
|---|---|
| author |
Zheng, Wu Yi Van Dort, Bethany Marcilly, Romaric Day, Richard Burke, Rosemary Shakib, Sepehr Ku, Young Reid-Anderson, Hannah Baysari, Melissa |
| spellingShingle |
Zheng, Wu Yi Van Dort, Bethany Marcilly, Romaric Day, Richard Burke, Rosemary Shakib, Sepehr Ku, Young Reid-Anderson, Hannah Baysari, Melissa JMIR Medical Informatics A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment Health Information Management Health Informatics |
| author_sort |
zheng, wu yi |
| spelling |
Zheng, Wu Yi Van Dort, Bethany Marcilly, Romaric Day, Richard Burke, Rosemary Shakib, Sepehr Ku, Young Reid-Anderson, Hannah Baysari, Melissa 2291-9694 JMIR Publications Inc. Health Information Management Health Informatics http://dx.doi.org/10.2196/24022 <jats:sec> <jats:title>Background</jats:title> <jats:p>It is well known that recommendations from electronic medication alerts are seldom accepted or acted on by users. Key factors affecting the effectiveness of medication alerts include system usability and alert design. Thus, human factors principles that apply knowledge of human capabilities and limitations are increasingly used in the design of health technology to improve the usability of systems.</jats:p> </jats:sec> <jats:sec> <jats:title>Objective</jats:title> <jats:p>This study aims to evaluate a newly developed evidence-based self-assessment tool that allows the valid and reliable evaluation of computerized medication alerting systems. This tool was developed to be used by hospital staff with detailed knowledge of their hospital’s computerized provider order entry system and alerts to identify and address potential system deficiencies. In this initial assessment, we aim to determine whether the items in the tool can measure compliance of medication alerting systems with human factors principles of design, the tool can be consistently used by multiple users to assess the same system, and the items are easy to understand and perceived to be useful for assessing medication alerting systems.</jats:p> </jats:sec> <jats:sec> <jats:title>Methods</jats:title> <jats:p>The Tool for Evaluating Medication Alerting Systems (TEMAS) was developed based on human factors design principles and consisted of 66 items. In total, 18 staff members recruited across 6 hospitals used the TEMAS to assess their medication alerting systems. Data collected from participant assessments were used to evaluate the validity, reliability, and usability of the TEMAS. Validity was assessed by comparing the results of the TEMAS with those of prior in-house evaluations. Reliability was measured using Krippendorff α to determine agreement among assessors. A 7-item survey was used to determine usability.</jats:p> </jats:sec> <jats:sec> <jats:title>Results</jats:title> <jats:p>The participants reported mostly negative (n=8) and neutral (n=7) perceptions of alerts in their medication alerting system. However, the validity of the TEMAS could not be directly tested, as participants were unaware of any results from prior in-house evaluations. The reliability of the TEMAS, as measured by Krippendorff α, was low to moderate (range 0.26-0.46); however, participant feedback suggests that individuals’ knowledge of the system varied according to their professional background. In terms of usability, 61% (11/18) of participants reported that the TEMAS items were generally easy to understand; however, participants suggested the revision of 22 items to improve clarity.</jats:p> </jats:sec> <jats:sec> <jats:title>Conclusions</jats:title> <jats:p>This initial assessment of the TEMAS allowed the identification of its components that required modification to improve usability and usefulness. It also revealed that for the TEMAS to be effective in facilitating a comprehensive assessment of a medication alerting system, it should be completed by a multidisciplinary team of hospital staff from both clinical and technical backgrounds to maximize their knowledge of systems.</jats:p> </jats:sec> A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment JMIR Medical Informatics |
| doi_str_mv |
10.2196/24022 |
| facet_avail |
Online Free |
| finc_class_facet |
Informatik Medizin |
| format |
ElectronicArticle |
| fullrecord |
blob:ai-49-aHR0cDovL2R4LmRvaS5vcmcvMTAuMjE5Ni8yNDAyMg |
| id |
ai-49-aHR0cDovL2R4LmRvaS5vcmcvMTAuMjE5Ni8yNDAyMg |
| institution |
DE-Gla1 DE-Zi4 DE-15 DE-Pl11 DE-Rs1 DE-105 DE-14 DE-Ch1 DE-L229 DE-D275 DE-Bn3 DE-Brt1 DE-Zwi2 DE-D161 |
| imprint |
JMIR Publications Inc., 2021 |
| imprint_str_mv |
JMIR Publications Inc., 2021 |
| issn |
2291-9694 |
| issn_str_mv |
2291-9694 |
| language |
English |
| mega_collection |
JMIR Publications Inc. (CrossRef) |
| match_str |
zheng2021atoolforevaluatingmedicationalertingsystemsdevelopmentandinitialassessment |
| publishDateSort |
2021 |
| publisher |
JMIR Publications Inc. |
| recordtype |
ai |
| record_format |
ai |
| series |
JMIR Medical Informatics |
| source_id |
49 |
| title |
A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| title_unstemmed |
A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| title_full |
A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| title_fullStr |
A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| title_full_unstemmed |
A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| title_short |
A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| title_sort |
a tool for evaluating medication alerting systems: development and initial assessment |
| topic |
Health Information Management Health Informatics |
| url |
http://dx.doi.org/10.2196/24022 |
| publishDate |
2021 |
| physical |
e24022 |
| description |
<jats:sec>
<jats:title>Background</jats:title>
<jats:p>It is well known that recommendations from electronic medication alerts are seldom accepted or acted on by users. Key factors affecting the effectiveness of medication alerts include system usability and alert design. Thus, human factors principles that apply knowledge of human capabilities and limitations are increasingly used in the design of health technology to improve the usability of systems.</jats:p>
</jats:sec>
<jats:sec>
<jats:title>Objective</jats:title>
<jats:p>This study aims to evaluate a newly developed evidence-based self-assessment tool that allows the valid and reliable evaluation of computerized medication alerting systems. This tool was developed to be used by hospital staff with detailed knowledge of their hospital’s computerized provider order entry system and alerts to identify and address potential system deficiencies. In this initial assessment, we aim to determine whether the items in the tool can measure compliance of medication alerting systems with human factors principles of design, the tool can be consistently used by multiple users to assess the same system, and the items are easy to understand and perceived to be useful for assessing medication alerting systems.</jats:p>
</jats:sec>
<jats:sec>
<jats:title>Methods</jats:title>
<jats:p>The Tool for Evaluating Medication Alerting Systems (TEMAS) was developed based on human factors design principles and consisted of 66 items. In total, 18 staff members recruited across 6 hospitals used the TEMAS to assess their medication alerting systems. Data collected from participant assessments were used to evaluate the validity, reliability, and usability of the TEMAS. Validity was assessed by comparing the results of the TEMAS with those of prior in-house evaluations. Reliability was measured using Krippendorff α to determine agreement among assessors. A 7-item survey was used to determine usability.</jats:p>
</jats:sec>
<jats:sec>
<jats:title>Results</jats:title>
<jats:p>The participants reported mostly negative (n=8) and neutral (n=7) perceptions of alerts in their medication alerting system. However, the validity of the TEMAS could not be directly tested, as participants were unaware of any results from prior in-house evaluations. The reliability of the TEMAS, as measured by Krippendorff α, was low to moderate (range 0.26-0.46); however, participant feedback suggests that individuals’ knowledge of the system varied according to their professional background. In terms of usability, 61% (11/18) of participants reported that the TEMAS items were generally easy to understand; however, participants suggested the revision of 22 items to improve clarity.</jats:p>
</jats:sec>
<jats:sec>
<jats:title>Conclusions</jats:title>
<jats:p>This initial assessment of the TEMAS allowed the identification of its components that required modification to improve usability and usefulness. It also revealed that for the TEMAS to be effective in facilitating a comprehensive assessment of a medication alerting system, it should be completed by a multidisciplinary team of hospital staff from both clinical and technical backgrounds to maximize their knowledge of systems.</jats:p>
</jats:sec> |
| container_issue |
7 |
| container_start_page |
0 |
| container_title |
JMIR Medical Informatics |
| container_volume |
9 |
| format_de105 |
Article, E-Article |
| format_de14 |
Article, E-Article |
| format_de15 |
Article, E-Article |
| format_de520 |
Article, E-Article |
| format_de540 |
Article, E-Article |
| format_dech1 |
Article, E-Article |
| format_ded117 |
Article, E-Article |
| format_degla1 |
E-Article |
| format_del152 |
Buch |
| format_del189 |
Article, E-Article |
| format_dezi4 |
Article |
| format_dezwi2 |
Article, E-Article |
| format_finc |
Article, E-Article |
| format_nrw |
Article, E-Article |
| _version_ |
1792347439403892738 |
| geogr_code |
not assigned |
| last_indexed |
2024-03-01T17:55:19.022Z |
| geogr_code_person |
not assigned |
| openURL |
url_ver=Z39.88-2004&ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fvufind.svn.sourceforge.net%3Agenerator&rft.title=A+Tool+for+Evaluating+Medication+Alerting+Systems%3A+Development+and+Initial+Assessment&rft.date=2021-07-16&genre=article&issn=2291-9694&volume=9&issue=7&pages=e24022&jtitle=JMIR+Medical+Informatics&atitle=A+Tool+for+Evaluating+Medication+Alerting+Systems%3A+Development+and+Initial+Assessment&aulast=Baysari&aufirst=Melissa&rft_id=info%3Adoi%2F10.2196%2F24022&rft.language%5B0%5D=eng |
| SOLR | |
| _version_ | 1792347439403892738 |
| author | Zheng, Wu Yi, Van Dort, Bethany, Marcilly, Romaric, Day, Richard, Burke, Rosemary, Shakib, Sepehr, Ku, Young, Reid-Anderson, Hannah, Baysari, Melissa |
| author_facet | Zheng, Wu Yi, Van Dort, Bethany, Marcilly, Romaric, Day, Richard, Burke, Rosemary, Shakib, Sepehr, Ku, Young, Reid-Anderson, Hannah, Baysari, Melissa, Zheng, Wu Yi, Van Dort, Bethany, Marcilly, Romaric, Day, Richard, Burke, Rosemary, Shakib, Sepehr, Ku, Young, Reid-Anderson, Hannah, Baysari, Melissa |
| author_sort | zheng, wu yi |
| container_issue | 7 |
| container_start_page | 0 |
| container_title | JMIR Medical Informatics |
| container_volume | 9 |
| description | <jats:sec> <jats:title>Background</jats:title> <jats:p>It is well known that recommendations from electronic medication alerts are seldom accepted or acted on by users. Key factors affecting the effectiveness of medication alerts include system usability and alert design. Thus, human factors principles that apply knowledge of human capabilities and limitations are increasingly used in the design of health technology to improve the usability of systems.</jats:p> </jats:sec> <jats:sec> <jats:title>Objective</jats:title> <jats:p>This study aims to evaluate a newly developed evidence-based self-assessment tool that allows the valid and reliable evaluation of computerized medication alerting systems. This tool was developed to be used by hospital staff with detailed knowledge of their hospital’s computerized provider order entry system and alerts to identify and address potential system deficiencies. In this initial assessment, we aim to determine whether the items in the tool can measure compliance of medication alerting systems with human factors principles of design, the tool can be consistently used by multiple users to assess the same system, and the items are easy to understand and perceived to be useful for assessing medication alerting systems.</jats:p> </jats:sec> <jats:sec> <jats:title>Methods</jats:title> <jats:p>The Tool for Evaluating Medication Alerting Systems (TEMAS) was developed based on human factors design principles and consisted of 66 items. In total, 18 staff members recruited across 6 hospitals used the TEMAS to assess their medication alerting systems. Data collected from participant assessments were used to evaluate the validity, reliability, and usability of the TEMAS. Validity was assessed by comparing the results of the TEMAS with those of prior in-house evaluations. Reliability was measured using Krippendorff α to determine agreement among assessors. A 7-item survey was used to determine usability.</jats:p> </jats:sec> <jats:sec> <jats:title>Results</jats:title> <jats:p>The participants reported mostly negative (n=8) and neutral (n=7) perceptions of alerts in their medication alerting system. However, the validity of the TEMAS could not be directly tested, as participants were unaware of any results from prior in-house evaluations. The reliability of the TEMAS, as measured by Krippendorff α, was low to moderate (range 0.26-0.46); however, participant feedback suggests that individuals’ knowledge of the system varied according to their professional background. In terms of usability, 61% (11/18) of participants reported that the TEMAS items were generally easy to understand; however, participants suggested the revision of 22 items to improve clarity.</jats:p> </jats:sec> <jats:sec> <jats:title>Conclusions</jats:title> <jats:p>This initial assessment of the TEMAS allowed the identification of its components that required modification to improve usability and usefulness. It also revealed that for the TEMAS to be effective in facilitating a comprehensive assessment of a medication alerting system, it should be completed by a multidisciplinary team of hospital staff from both clinical and technical backgrounds to maximize their knowledge of systems.</jats:p> </jats:sec> |
| doi_str_mv | 10.2196/24022 |
| facet_avail | Online, Free |
| finc_class_facet | Informatik, Medizin |
| format | ElectronicArticle |
| format_de105 | Article, E-Article |
| format_de14 | Article, E-Article |
| format_de15 | Article, E-Article |
| format_de520 | Article, E-Article |
| format_de540 | Article, E-Article |
| format_dech1 | Article, E-Article |
| format_ded117 | Article, E-Article |
| format_degla1 | E-Article |
| format_del152 | Buch |
| format_del189 | Article, E-Article |
| format_dezi4 | Article |
| format_dezwi2 | Article, E-Article |
| format_finc | Article, E-Article |
| format_nrw | Article, E-Article |
| geogr_code | not assigned |
| geogr_code_person | not assigned |
| id | ai-49-aHR0cDovL2R4LmRvaS5vcmcvMTAuMjE5Ni8yNDAyMg |
| imprint | JMIR Publications Inc., 2021 |
| imprint_str_mv | JMIR Publications Inc., 2021 |
| institution | DE-Gla1, DE-Zi4, DE-15, DE-Pl11, DE-Rs1, DE-105, DE-14, DE-Ch1, DE-L229, DE-D275, DE-Bn3, DE-Brt1, DE-Zwi2, DE-D161 |
| issn | 2291-9694 |
| issn_str_mv | 2291-9694 |
| language | English |
| last_indexed | 2024-03-01T17:55:19.022Z |
| match_str | zheng2021atoolforevaluatingmedicationalertingsystemsdevelopmentandinitialassessment |
| mega_collection | JMIR Publications Inc. (CrossRef) |
| physical | e24022 |
| publishDate | 2021 |
| publishDateSort | 2021 |
| publisher | JMIR Publications Inc. |
| record_format | ai |
| recordtype | ai |
| series | JMIR Medical Informatics |
| source_id | 49 |
| spelling | Zheng, Wu Yi Van Dort, Bethany Marcilly, Romaric Day, Richard Burke, Rosemary Shakib, Sepehr Ku, Young Reid-Anderson, Hannah Baysari, Melissa 2291-9694 JMIR Publications Inc. Health Information Management Health Informatics http://dx.doi.org/10.2196/24022 <jats:sec> <jats:title>Background</jats:title> <jats:p>It is well known that recommendations from electronic medication alerts are seldom accepted or acted on by users. Key factors affecting the effectiveness of medication alerts include system usability and alert design. Thus, human factors principles that apply knowledge of human capabilities and limitations are increasingly used in the design of health technology to improve the usability of systems.</jats:p> </jats:sec> <jats:sec> <jats:title>Objective</jats:title> <jats:p>This study aims to evaluate a newly developed evidence-based self-assessment tool that allows the valid and reliable evaluation of computerized medication alerting systems. This tool was developed to be used by hospital staff with detailed knowledge of their hospital’s computerized provider order entry system and alerts to identify and address potential system deficiencies. In this initial assessment, we aim to determine whether the items in the tool can measure compliance of medication alerting systems with human factors principles of design, the tool can be consistently used by multiple users to assess the same system, and the items are easy to understand and perceived to be useful for assessing medication alerting systems.</jats:p> </jats:sec> <jats:sec> <jats:title>Methods</jats:title> <jats:p>The Tool for Evaluating Medication Alerting Systems (TEMAS) was developed based on human factors design principles and consisted of 66 items. In total, 18 staff members recruited across 6 hospitals used the TEMAS to assess their medication alerting systems. Data collected from participant assessments were used to evaluate the validity, reliability, and usability of the TEMAS. Validity was assessed by comparing the results of the TEMAS with those of prior in-house evaluations. Reliability was measured using Krippendorff α to determine agreement among assessors. A 7-item survey was used to determine usability.</jats:p> </jats:sec> <jats:sec> <jats:title>Results</jats:title> <jats:p>The participants reported mostly negative (n=8) and neutral (n=7) perceptions of alerts in their medication alerting system. However, the validity of the TEMAS could not be directly tested, as participants were unaware of any results from prior in-house evaluations. The reliability of the TEMAS, as measured by Krippendorff α, was low to moderate (range 0.26-0.46); however, participant feedback suggests that individuals’ knowledge of the system varied according to their professional background. In terms of usability, 61% (11/18) of participants reported that the TEMAS items were generally easy to understand; however, participants suggested the revision of 22 items to improve clarity.</jats:p> </jats:sec> <jats:sec> <jats:title>Conclusions</jats:title> <jats:p>This initial assessment of the TEMAS allowed the identification of its components that required modification to improve usability and usefulness. It also revealed that for the TEMAS to be effective in facilitating a comprehensive assessment of a medication alerting system, it should be completed by a multidisciplinary team of hospital staff from both clinical and technical backgrounds to maximize their knowledge of systems.</jats:p> </jats:sec> A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment JMIR Medical Informatics |
| spellingShingle | Zheng, Wu Yi, Van Dort, Bethany, Marcilly, Romaric, Day, Richard, Burke, Rosemary, Shakib, Sepehr, Ku, Young, Reid-Anderson, Hannah, Baysari, Melissa, JMIR Medical Informatics, A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment, Health Information Management, Health Informatics |
| title | A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| title_full | A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| title_fullStr | A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| title_full_unstemmed | A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| title_short | A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| title_sort | a tool for evaluating medication alerting systems: development and initial assessment |
| title_unstemmed | A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment |
| topic | Health Information Management, Health Informatics |
| url | http://dx.doi.org/10.2196/24022 |