Publications

Objectives: To compare how clinical researchers generate data-driven hypotheses with a visualinteractive analytic tool (VIADS, a visual interactive analysis tool for filtering and summarizinglarge data sets coded with hierarchical terminologies) or other tools.

Methods: We recruited clinical researchers and separated them into “experienced” and“inexperienced” groups. Participants were randomly assigned to a VIADS or control groupwithin the groups. Each participant conducted a remote 2-hour study session for hypothesisgeneration with the same study facilitator on the same datasets by following a think-aloudprotocol. Screen activities and audio were recorded, transcribed, coded, and analyzed.Hypotheses were evaluated by seven experts on their validity, significance, and feasibility. Weconducted multilevel random effect modeling for statistical tests.

Results: Eighteen participants generated 227 hypotheses, of which 147 (65%) were valid. TheVIADS and control groups generated a similar number of hypotheses. The VIADS group took asignificantly shorter time to generate one hypothesis (e.g., among inexperienced clinicalresearchers, 258 seconds versus 379 seconds, p = 0.046, power = 0.437, ICC = 0.15). TheVIADS group received significantly lower ratings than the control group on feasibility and thecombination rating of validity, significance, and feasibility.

Conclusion: The role of VIADS in hypothesis generation seems inconclusive. The VIADSgroup took a significantly shorter time to generate each hypothesis. However, the combinedvalidity, significance, and feasibility ratings of their hypotheses were significantly lower. Furthercharacterization of hypotheses, including specifics on how they might be improved, could guidefuture tool development.

Keywords: scientific hypothesis generation; clinical research; VIADS; utility study; secondarydata analysis tools
(PDF) Data-driven hypothesis generation among inexperienced clinical researchers: A comparison of secondary data analyses with visualization (VIADS) and other tools. Available from: https://www.researchgate.net/publication/377170726_Data-driven_hypothesis_generation_among_inexperienced_clinical_researchers_A_comparison_of_secondary_data_analyses_with_visualization_VIADS_and_other_tools [accessed Jan 18 2024].

This study detailed the course design principles and implementation of project-based learning (PBL) in a technology-themed graduate-level online course. Students were trained to develop knowledge and skills in instructional leadership, such as the capability to design, deliver, and evaluate educational technology professional development programs. Pre- and post-survey data were collected to examine any change in students' knowledge and skills in instructional leadership by completing this course (N = 18). Quantitative findings revealed positive learning outcomes, and there was statistical significance regarding student improvement in knowledge and skills of instructional leadership, rendering the PBL approach viable.

• Keywords: 
• instructional leadership, 
• project-based learning, 
• online education, 
• educational technology, 
• professional development

Citation: Min Lun Wu, Lan Li, Yuchun Zhou. Enhancing technology leaders' instructional leadership through a project-based learning online course[J]. STEM Education, 2023, 3(2): 89-102. doi: 10.3934/steme.2023007

Related Papers:

Mixed methods research (MMR) has been widely adopted in a plethora of disciplines. Integration is the pressing issue regarding the legitimation, the added value, and the quality of using MMR, though inadequate literature has discussed effective strategies used in the field of education, including school psychology, counseling, and teacher education. This study reviewed 119 recently published MMR articles in education using a four-dimension codebook with the goal to explore generic integration strategies and innovative strategies used by educational researchers in practice. As a result, three most commonly used generic integration strategies were identified, including (1) using a good mixed methods (MM) research question to guide research design, (2) using appropriate MM sampling strategy to obtain good data for achieving integration, and (3) using multiple MM data mixing strategies to facilitate integration. Moreover, five creative integration strategies were found at the method level: (1) using an innovative survey to collect both qualitative and quantitative data, (2) using visual support to collect data, (3) using high-tech methods to facilitate data collection, (4) using data visualization in mixing, and (5) quantitizing categorized QUAL data. This review summarizes and analyzes the effective integration strategies commonly used at the research design level and at the method level. It also provides valuable recommendations for educational researchers to explore creative strategies to achieve efficient integration when they conduct mixed methods research.

Background: Visualization can be a powerful tool to comprehend data sets, especially when they can be represented via hierarchical structures. Enhanced comprehension can facilitate the development of scientific hypotheses. However, the inclusion of excessive data can make visualizations overwhelming.

Objective: We developed a visual interactive analytic tool for filtering and summarizing large health data sets coded with hierarchical terminologies (VIADS). In this study, we evaluated the usability of VIADS for visualizing data sets of patient diagnoses and procedures coded in the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM).

Methods: We used mixed methods in the study. A group of 12 clinical researchers participated in the generation of data-driven hypotheses using the same data sets and time frame (a 1-hour training session and a 2-hour study session) utilizing VIADS via the think-aloud protocol. The audio and screen activities were recorded remotely. A modified version of the System Usability Scale (SUS) survey and a brief survey with open-ended questions were administered after the study to assess the usability of VIADS and verify their intense usage experience with VIADS.

Results: The range of SUS scores was 37.5 to 87.5. The mean SUS score for VIADS was 71.88 (out of a possible 100, SD 14.62), and the median SUS was 75. The participants unanimously agreed that VIADS offers new perspectives on data sets (12/12, 100%), while 75% (8/12) agreed that VIADS facilitates understanding, presentation, and interpretation of underlying data sets. The comments on the utility of VIADS were positive and aligned well with the design objectives of VIADS. The answers to the open-ended questions in the modified SUS provided specific suggestions regarding potential improvements for VIADS, and the identified problems with usability were used to update the tool.

Conclusions: This usability study demonstrates that VIADS is a usable tool for analyzing secondary data sets with good average usability, good SUS score, and favorable utility. Currently, VIADS accepts data sets with hierarchical codes and their corresponding frequencies. Consequently, only specific types of use cases are supported by the analytical results. Participants agreed, however, that VIADS provides new perspectives on data sets and is relatively easy to use. The VIADS functionalities most appreciated by participants were the ability to filter, summarize, compare, and visualize data.

International Registered Report Identifier (IRRID): RR2-10.2196/39414

JMIR Hum Factors 2023;10:e44644

doi:10.2196/44644

Teachers are gatekeepers of technology integration in the classroom. Pre-service teachers’ attitudes, confidence, and competence in exploring emerging technologies play a critical role in teachers’ adoption of technology in teaching. This study examined the effects of a gamified technology course on pre-service teachers’ confidence, intention, and motivation in integrating technology into teaching. A sample of pre-service teachers (N = 84) at a Midwestern university in the United States in the academic year of 2021–22 was surveyed. The regression results revealed that the gamified course significantly and positively influenced pre-service teachers’ confidence in using technology in education, intention to adopt gamification, and motivation to explore more emerging technologies for teaching, after controlling for gender. In contrast, gender did not affect pre-service teachers’ confidence, intention, and motivation in integrating technology into instruction after controlling for the gamified course effects. Suggestions on gamifying course design while leveraging quest-based learning and active learning principles to enhance students' positive attitudes and motivation to explore technology integration are discussed.

There is a paucity of empirical research on teaching mixed methods. To fill this gap in literature, this convergent mixed methods study explores the effectiveness of using active learning approaches in teaching a mixed methods course. The qualitative data, including 10 individual interviews, 29 students’ reflections, and 26 teaching evaluation surveys, were used to examine students’ learning experience and outcomes. Students’ presentations (N = 29) and final papers (N = 29) were transformed into numbers as the quantitative data. The converged results indicated that students were actively engaged in learning and achieved the expected learning outcomes. This study makes valuable contributions to the mixed methods pedagogical culture by providing details and suggestions on how to use active learning approaches in teaching mixed methods.

Background:Scientific hypothesis generation is a critical step in scientific research that determines the direction and impact of any investigation. Despite its vital role, we have limited knowledge of the process itself, thus hindering our ability to address some critical questions.

Objective:This study aims to answer the following questions: To what extent can secondary data analytics tools facilitate the generation of scientific hypotheses during clinical research? Are the processes similar in developing clinical diagnoses during clinical practice and developing scientific hypotheses for clinical research projects? Furthermore, this study explores the process of scientific hypothesis generation in the context of clinical research. It was designed to compare the role of VIADS, a visual interactive analysis tool for filtering and summarizing large data sets coded with hierarchical terminologies, and the experience levels of study participants during the scientific hypothesis generation process.

Methods:This manuscript introduces a study design. Experienced and inexperienced clinical researchers are being recruited since July 2021 to take part in this 2×2 factorial study, in which all participants use the same data sets during scientific hypothesis–generation sessions and follow predetermined scripts. The clinical researchers are separated into experienced or inexperienced groups based on predetermined criteria and are then randomly assigned into groups that use and do not use VIADS via block randomization. The study sessions, screen activities, and audio recordings of participants are captured. Participants use the think-aloud protocol during the study sessions. After each study session, every participant is given a follow-up survey, with participants using VIADS completing an additional modified System Usability Scale survey. A panel of clinical research experts will assess the scientific hypotheses generated by participants based on predeveloped metrics. All data will be anonymized, transcribed, aggregated, and analyzed.

Results:Data collection for this study began in July 2021. Recruitment uses a brief online survey. The preliminary results showed that study participants can generate a few to over a dozen scientific hypotheses during a 2-hour study session, regardless of whether they used VIADS or other analytics tools. A metric to more accurately, comprehensively, and consistently assess scientific hypotheses within a clinical research context has been developed.

Conclusions:The scientific hypothesis–generation process is an advanced cognitive activity and a complex process. Our results so far show that clinical researchers can quickly generate initial scientific hypotheses based on data sets and prior experience. However, refining these scientific hypotheses is a much more time-consuming activity. To uncover the fundamental mechanisms underlying the generation of scientific hypotheses, we need breakthroughs that can capture thinking processes more precisely.

International Registered Report Identifier (IRRID):DERR1-10.2196/39414

JMIR Res Protoc 2022;11(7):e39414

doi:10.2196/39414

Background:Disease status (eg, cancer stage) has been used in routine clinical practice to determine more accurate treatment plans. Health-related indicators, such as mortality, morbidity, and population group life expectancy, have also been used. However, few studies have specifically focused on the comprehensive and objective measures of individual health status.

Objective:The aim of this study was to analyze the perspectives of the public toward 29 health indicators obtained from a literature review to provide evidence for further prioritization of the indicators. The difference between health status and disease status should be considered.

Methods: This study used a cross-sectional design. Online surveys were administered through Ohio University, ResearchMatch, and Clemson University, resulting in three samples. Participants aged 18 years or older rated the importance of the 29 health indicators. The rating results were aggregated and analyzed as follows (in each case, the dependent variables were the individual survey responses): (1) to determine the agreement among the three samples regarding the importance of each indicator, where the independent variables (IVs) were the three samples; (2) to examine the mean differences between the retained indicators with agreement across the three samples, where the IVs were the identified indicators; and (3) to rank the groups of indicators into various levels after grouping the indicators with no mean differences, where the IVs were the groups of indicators.

Results: In total, 1153 valid responses were analyzed. Descriptive statistics revealed that the top five-rated indicators were drug or substance abuse, smoking or tobacco use, alcohol abuse, major depression, and diet and nutrition. Among the 29 health indicators, the three samples agreed upon the importance of 13 indicators. Inferential statistical analysis indicated that some of the 13 indicators held equal importance. Therefore, the 13 indicators were categorized by rank into seven levels: level 1 included blood sugar level and immunization and vaccination; level 2 included LDL cholesterol; level 3 included HDL cholesterol, blood triglycerides, cancer screening detection, and total cholesterol; level 4 included health literacy rate; level 5 included personal care needs and air quality index greater than 100; level 6 included self-rated health status and HIV testing; and level 7 included the supply of dentists. Levels 1 to 3 were rated significantly higher than levels 4 to 7.

Conclusions: This study provides a baseline for prioritizing 29 health indicators, which can be used by electronic health records or personal health record system designers or developers to determine what can be included in the systems to capture an individual’s health status. Currently, self-rated health status is the predominantly used health indicator. Additionally, this study provides a foundation for tracking and measuring preventive healthcare services more accurately and for developing an individual health status index.

J Med Internet Res 2022;24(6):e38099

doi:10.2196/38099

As integration in mixed methods studies is of great interest, we reviewed published articles between 2007 and 2019 in two major journals of mixed methods research to identify the integration challenges mixed methods researchers experienced. This study contributes to mixed methods by summarizing the reported challenges in empirical mixed methods articles based on the concept of integration. We found mixed methods users’ experienced uncertainty during integration due to lack of guidance and confidence. We advocate that researchers should be proactive in expounding methodological challenges that they confronted in practice and report what they did, how they did it, and why they did it in such ways to approach full integration. By doing this, researchers would make valuable contributions to this methodology’s development.

Mixed methods research (MMR) has been adopted in a variety of disciplines, including health science, social studies, psychology, early childhood, and business. More recently, higher education researchers adopted this methodology, but with limited evaluation on the quality of published mixed methods research. To enhance researchers’ understanding of using MMR, we developed a 4-dimension codebook and reviewed 14 recently published mixed methods research articles that were published in 3 top journals in higher education. We found that researchers used 3 effective strategies to approach integration: (a) using a mixed methods research question to guide the mixed methods (MM) design, (b) using effective mixed methods sampling strategies to collect different types of data for the purpose of integration, and (c) using mixing techniques to interpret results for optimal maximum integration. Apart from the 3 effective integration strategies, a few insufficiencies have been singled out: (a) infrequent use of MM terminologies in the article title, (b) limited familiarity with MM data mixing strategies, and (c) non-optimal reporting of MM results. At the end, we synthesized the useful integration strategies and developed a checklist for higher education researchers to design high-quality mixed methods research studies.