Publications

Objectives: To compare how clinical researchers generate data-driven hypotheses with a visualinteractive analytic tool (VIADS, a visual interactive analysis tool for filtering and summarizinglarge data sets coded with hierarchical terminologies) or other tools.

Methods: We recruited clinical researchers and separated them into “experienced” and“inexperienced” groups. Participants were randomly assigned to a VIADS or control groupwithin the groups. Each participant conducted a remote 2-hour study session for hypothesisgeneration with the same study facilitator on the same datasets by following a think-aloudprotocol. Screen activities and audio were recorded, transcribed, coded, and analyzed.Hypotheses were evaluated by seven experts on their validity, significance, and feasibility. Weconducted multilevel random effect modeling for statistical tests.

Results: Eighteen participants generated 227 hypotheses, of which 147 (65%) were valid. TheVIADS and control groups generated a similar number of hypotheses. The VIADS group took asignificantly shorter time to generate one hypothesis (e.g., among inexperienced clinicalresearchers, 258 seconds versus 379 seconds, p = 0.046, power = 0.437, ICC = 0.15). TheVIADS group received significantly lower ratings than the control group on feasibility and thecombination rating of validity, significance, and feasibility.

Conclusion: The role of VIADS in hypothesis generation seems inconclusive. The VIADSgroup took a significantly shorter time to generate each hypothesis. However, the combinedvalidity, significance, and feasibility ratings of their hypotheses were significantly lower. Furthercharacterization of hypotheses, including specifics on how they might be improved, could guidefuture tool development.

Keywords: scientific hypothesis generation; clinical research; VIADS; utility study; secondarydata analysis tools
(PDF) Data-driven hypothesis generation among inexperienced clinical researchers: A comparison of secondary data analyses with visualization (VIADS) and other tools. Available from: https://www.researchgate.net/publication/377170726_Data-driven_hypothesis_generation_among_inexperienced_clinical_researchers_A_comparison_of_secondary_data_analyses_with_visualization_VIADS_and_other_tools [accessed Jan 18 2024].

Hypothesis generation is an early and critical step in any hypothesis-driven clinical research project. Because it is not yet a well-understood cognitive process, the need to improve the process goes unrecognized. Without an impactful hypothesis, the significance of any research project can be questionable, regardless of the rigor or diligence applied in other steps of the study, e.g., study design, data collection, and result analysis. In this perspective article, the authors provide a literature review on the following topics first: scientific thinking, reasoning, medical reasoning, literature-based discovery, and a field study to explore scientific thinking and discovery. Over the years, scientific thinking has shown excellent progress in cognitive science and its applied areas: education, medicine, and biomedical research. However, a review of the literature reveals the lack of original studies on hypothesis generation in clinical research. The authors then summarize their first human participant study exploring data-driven hypothesis generation by clinical researchers in a simulated setting. The results indicate that a secondary data analytical tool, VIADS—a visual interactive analytic tool for filtering, summarizing, and visualizing large health data sets coded with hierarchical terminologies, can shorten the time participants need, on average, to generate a hypothesis and also requires fewer cognitive events to generate each hypothesis. As a counterpoint, this exploration also indicates that the quality ratings of the hypotheses thus generated carry significantly lower ratings for feasibility when applying VIADS. Despite its small scale, the study confirmed the feasibility of conducting a human participant study directly to explore the hypothesis generation process in clinical research. This study provides supporting evidence to conduct a larger-scale study with a specifically designed tool to facilitate the hypothesis-generation process among inexperienced clinical researchers. A larger study could provide generalizable evidence, which in turn can potentially improve clinical research productivity and overall clinical research enterprise.

Collapse

Traditional teaching approaches in research methods courses demotivate
students, hindering their ability to learn and apply research concepts
in practical settings. Instructors who are devoted to employing active
learning pedagogy in teaching research courses have observed favorable

outcomes. However, a dearth of literature has focused on students’ percep-
tions and evaluation of active learning pedagogy. This study explores how

students perceive active learning approaches via in-person and written-
response interviews. Eighteen participants volunteered and self-selected

one of the interview forms. In addition to exploring students’ perspec-
tives on active learning pedagogy, we conducted a comparison of those

interviewing modes. Our analysis revealed that regardless of interview
type, participants provided similar results. In-person interviews generated
more information segments; however, the segments from written-response
interviews provided more informative meaning. Combining all data, we
developed three themes and 29 codes about students’ perceptions of active
learning pedagogy. We found that students valued self-exploration as well

as interactive activities that promoted deep learning and knowledge trans-
fer. Based on the results, we provide practical recommendations for how

instructors can design active learning-based research courses to facilitate
students in active and self-regulated learning.

This study detailed the course design principles and implementation of project-based learning (PBL) in a technology-themed graduate-level online course. Students were trained to develop knowledge and skills in instructional leadership, such as the capability to design, deliver, and evaluate educational technology professional development programs. Pre- and post-survey data were collected to examine any change in students' knowledge and skills in instructional leadership by completing this course (N = 18). Quantitative findings revealed positive learning outcomes, and there was statistical significance regarding student improvement in knowledge and skills of instructional leadership, rendering the PBL approach viable.

• Keywords: 
• instructional leadership, 
• project-based learning, 
• online education, 
• educational technology, 
• professional development

Citation: Min Lun Wu, Lan Li, Yuchun Zhou. Enhancing technology leaders' instructional leadership through a project-based learning online course[J]. STEM Education, 2023, 3(2): 89-102. doi: 10.3934/steme.2023007

Related Papers:

Mixed methods research (MMR) has been widely adopted in a plethora of disciplines. Integration is the pressing issue regarding the legitimation, the added value, and the quality of using MMR, though inadequate literature has discussed effective strategies used in the field of education, including school psychology, counseling, and teacher education. This study reviewed 119 recently published MMR articles in education using a four-dimension codebook with the goal to explore generic integration strategies and innovative strategies used by educational researchers in practice. As a result, three most commonly used generic integration strategies were identified, including (1) using a good mixed methods (MM) research question to guide research design, (2) using appropriate MM sampling strategy to obtain good data for achieving integration, and (3) using multiple MM data mixing strategies to facilitate integration. Moreover, five creative integration strategies were found at the method level: (1) using an innovative survey to collect both qualitative and quantitative data, (2) using visual support to collect data, (3) using high-tech methods to facilitate data collection, (4) using data visualization in mixing, and (5) quantitizing categorized QUAL data. This review summarizes and analyzes the effective integration strategies commonly used at the research design level and at the method level. It also provides valuable recommendations for educational researchers to explore creative strategies to achieve efficient integration when they conduct mixed methods research.

Background: Visualization can be a powerful tool to comprehend data sets, especially when they can be represented via hierarchical structures. Enhanced comprehension can facilitate the development of scientific hypotheses. However, the inclusion of excessive data can make visualizations overwhelming.

Objective: We developed a visual interactive analytic tool for filtering and summarizing large health data sets coded with hierarchical terminologies (VIADS). In this study, we evaluated the usability of VIADS for visualizing data sets of patient diagnoses and procedures coded in the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM).

Methods: We used mixed methods in the study. A group of 12 clinical researchers participated in the generation of data-driven hypotheses using the same data sets and time frame (a 1-hour training session and a 2-hour study session) utilizing VIADS via the think-aloud protocol. The audio and screen activities were recorded remotely. A modified version of the System Usability Scale (SUS) survey and a brief survey with open-ended questions were administered after the study to assess the usability of VIADS and verify their intense usage experience with VIADS.

Results: The range of SUS scores was 37.5 to 87.5. The mean SUS score for VIADS was 71.88 (out of a possible 100, SD 14.62), and the median SUS was 75. The participants unanimously agreed that VIADS offers new perspectives on data sets (12/12, 100%), while 75% (8/12) agreed that VIADS facilitates understanding, presentation, and interpretation of underlying data sets. The comments on the utility of VIADS were positive and aligned well with the design objectives of VIADS. The answers to the open-ended questions in the modified SUS provided specific suggestions regarding potential improvements for VIADS, and the identified problems with usability were used to update the tool.

Conclusions: This usability study demonstrates that VIADS is a usable tool for analyzing secondary data sets with good average usability, good SUS score, and favorable utility. Currently, VIADS accepts data sets with hierarchical codes and their corresponding frequencies. Consequently, only specific types of use cases are supported by the analytical results. Participants agreed, however, that VIADS provides new perspectives on data sets and is relatively easy to use. The VIADS functionalities most appreciated by participants were the ability to filter, summarize, compare, and visualize data.

International Registered Report Identifier (IRRID): RR2-10.2196/39414

JMIR Hum Factors 2023;10:e44644

doi:10.2196/44644

Teachers are gatekeepers of technology integration in the classroom. Pre-service teachers’ attitudes, confidence, and competence in exploring emerging technologies play a critical role in teachers’ adoption of technology in teaching. This study examined the effects of a gamified technology course on pre-service teachers’ confidence, intention, and motivation in integrating technology into teaching. A sample of pre-service teachers (N = 84) at a Midwestern university in the United States in the academic year of 2021–22 was surveyed. The regression results revealed that the gamified course significantly and positively influenced pre-service teachers’ confidence in using technology in education, intention to adopt gamification, and motivation to explore more emerging technologies for teaching, after controlling for gender. In contrast, gender did not affect pre-service teachers’ confidence, intention, and motivation in integrating technology into instruction after controlling for the gamified course effects. Suggestions on gamifying course design while leveraging quest-based learning and active learning principles to enhance students' positive attitudes and motivation to explore technology integration are discussed.

There is a paucity of empirical research on teaching mixed methods. To fill this gap in literature, this convergent mixed methods study explores the effectiveness of using active learning approaches in teaching a mixed methods course. The qualitative data, including 10 individual interviews, 29 students’ reflections, and 26 teaching evaluation surveys, were used to examine students’ learning experience and outcomes. Students’ presentations (N = 29) and final papers (N = 29) were transformed into numbers as the quantitative data. The converged results indicated that students were actively engaged in learning and achieved the expected learning outcomes. This study makes valuable contributions to the mixed methods pedagogical culture by providing details and suggestions on how to use active learning approaches in teaching mixed methods.

Background:Scientific hypothesis generation is a critical step in scientific research that determines the direction and impact of any investigation. Despite its vital role, we have limited knowledge of the process itself, thus hindering our ability to address some critical questions.

Objective:This study aims to answer the following questions: To what extent can secondary data analytics tools facilitate the generation of scientific hypotheses during clinical research? Are the processes similar in developing clinical diagnoses during clinical practice and developing scientific hypotheses for clinical research projects? Furthermore, this study explores the process of scientific hypothesis generation in the context of clinical research. It was designed to compare the role of VIADS, a visual interactive analysis tool for filtering and summarizing large data sets coded with hierarchical terminologies, and the experience levels of study participants during the scientific hypothesis generation process.

Methods:This manuscript introduces a study design. Experienced and inexperienced clinical researchers are being recruited since July 2021 to take part in this 2×2 factorial study, in which all participants use the same data sets during scientific hypothesis–generation sessions and follow predetermined scripts. The clinical researchers are separated into experienced or inexperienced groups based on predetermined criteria and are then randomly assigned into groups that use and do not use VIADS via block randomization. The study sessions, screen activities, and audio recordings of participants are captured. Participants use the think-aloud protocol during the study sessions. After each study session, every participant is given a follow-up survey, with participants using VIADS completing an additional modified System Usability Scale survey. A panel of clinical research experts will assess the scientific hypotheses generated by participants based on predeveloped metrics. All data will be anonymized, transcribed, aggregated, and analyzed.

Results:Data collection for this study began in July 2021. Recruitment uses a brief online survey. The preliminary results showed that study participants can generate a few to over a dozen scientific hypotheses during a 2-hour study session, regardless of whether they used VIADS or other analytics tools. A metric to more accurately, comprehensively, and consistently assess scientific hypotheses within a clinical research context has been developed.

Conclusions:The scientific hypothesis–generation process is an advanced cognitive activity and a complex process. Our results so far show that clinical researchers can quickly generate initial scientific hypotheses based on data sets and prior experience. However, refining these scientific hypotheses is a much more time-consuming activity. To uncover the fundamental mechanisms underlying the generation of scientific hypotheses, we need breakthroughs that can capture thinking processes more precisely.

International Registered Report Identifier (IRRID):DERR1-10.2196/39414

JMIR Res Protoc 2022;11(7):e39414

doi:10.2196/39414

Background:Disease status (eg, cancer stage) has been used in routine clinical practice to determine more accurate treatment plans. Health-related indicators, such as mortality, morbidity, and population group life expectancy, have also been used. However, few studies have specifically focused on the comprehensive and objective measures of individual health status.

Objective:The aim of this study was to analyze the perspectives of the public toward 29 health indicators obtained from a literature review to provide evidence for further prioritization of the indicators. The difference between health status and disease status should be considered.

Methods: This study used a cross-sectional design. Online surveys were administered through Ohio University, ResearchMatch, and Clemson University, resulting in three samples. Participants aged 18 years or older rated the importance of the 29 health indicators. The rating results were aggregated and analyzed as follows (in each case, the dependent variables were the individual survey responses): (1) to determine the agreement among the three samples regarding the importance of each indicator, where the independent variables (IVs) were the three samples; (2) to examine the mean differences between the retained indicators with agreement across the three samples, where the IVs were the identified indicators; and (3) to rank the groups of indicators into various levels after grouping the indicators with no mean differences, where the IVs were the groups of indicators.

Results: In total, 1153 valid responses were analyzed. Descriptive statistics revealed that the top five-rated indicators were drug or substance abuse, smoking or tobacco use, alcohol abuse, major depression, and diet and nutrition. Among the 29 health indicators, the three samples agreed upon the importance of 13 indicators. Inferential statistical analysis indicated that some of the 13 indicators held equal importance. Therefore, the 13 indicators were categorized by rank into seven levels: level 1 included blood sugar level and immunization and vaccination; level 2 included LDL cholesterol; level 3 included HDL cholesterol, blood triglycerides, cancer screening detection, and total cholesterol; level 4 included health literacy rate; level 5 included personal care needs and air quality index greater than 100; level 6 included self-rated health status and HIV testing; and level 7 included the supply of dentists. Levels 1 to 3 were rated significantly higher than levels 4 to 7.

Conclusions: This study provides a baseline for prioritizing 29 health indicators, which can be used by electronic health records or personal health record system designers or developers to determine what can be included in the systems to capture an individual’s health status. Currently, self-rated health status is the predominantly used health indicator. Additionally, this study provides a foundation for tracking and measuring preventive healthcare services more accurately and for developing an individual health status index.

J Med Internet Res 2022;24(6):e38099

doi:10.2196/38099