The LearnLab Summer School offers an immersive, one-week intensive course designed to equip participants with the skills to create cutting-edge technology-enhanced learning experiments and build intelligent tutoring systems. This program provides a robust conceptual foundation alongside significant hands-on experience in all stages of technology-enhanced learning research. From the initial design and setup of experiments to execution and data analysis, participants gain practical expertise in a technology-supported environment. Notably, prior programming experience is not required to benefit fully from this comprehensive summer school experience within the Learn Lab setting.
The summer school’s curriculum is thoughtfully structured over five days, with a balanced approach that dedicates equal time to lectures and practical, hands-on activities. Each day is segmented into lectures, interactive discussion sessions, and laboratory sessions. During these lab sessions, participants collaborate to develop a small-scale prototype experiment. These experiments focus on enhancing learning within core academic areas such as mathematics, science, and language acquisition. Participants utilize state-of-the-art tools throughout the week, gaining familiarity with platforms such as the Open Learning Initiative (OLI) development environment, Cognitive Tutor Authoring Tools, and various other resources crucial for contemporary course development. Furthermore, the summer school introduces tools for authoring natural language dialog, TagHelper tools for the semi-automated coding of verbal data, and DataShop, a powerful resource for storing and analyzing student interaction data and assessing student knowledge and performance. The culmination of the week involves student teams presenting their project accomplishments to their peers, fostering a collaborative and insightful learning environment. Participants are expected to engage in preparatory activities before the summer school commences to maximize their learning experience.
The LearnLab Summer School is organized into six specialized, parallel tracks, allowing participants to focus on areas of particular interest. These tracks include: Building Online Courses with OLI (BOLI), Chemistry Education (CE), Computational Models of Learning (CML), Intelligent Tutor Systems development (ITS), Educational Data Mining (EDM), and, newly added in 2023, Computer Science Education Research (CER). The program particularly encourages applications from individuals focused on addressing inequities within education and those with a strong interest in advancing computer science education.
While these tracks operate in parallel, there is a degree of overlap in content and shared learning opportunities. However, they diverge significantly in the hands-on activities, which constitute approximately half of the summer school’s duration. Participants are assigned to a primary track based on their preferences indicated in their application. Recognizing the diverse interests and learning goals of attendees, the summer school promotes a flexible approach, allowing participants to “shop around” and engage in activities from tracks outside their official assignment. The paramount objective is to ensure that every participant gains a valuable and enriching learn lab experience.
A defining feature of the LearnLab Summer School is the intensive mentoring provided by experienced LearnLab researchers. This mentorship begins with email communication prior to the summer school’s commencement, facilitating the selection of an appropriate subject domain and project task. It continues throughout the week with dedicated one-on-one time during the hands-on laboratory sessions. Mentors are carefully assigned based on the interests expressed by participants in their applications. While all participants have the opportunity to interact with all course instructors, they benefit from more frequent and focused engagement with their designated mentor, ensuring personalized guidance and support within the learn lab environment.
The esteemed researchers expected to serve as mentors and instructors include: Ken Koedinger, Vincent Aleven, Peter Brusilovsky, David Yaron, Mark Blaser, Thomas Price, Norman Bier, Erin Czerwinski, John Stamper, Erik Harpstead, and Carolyn Rose, among other distinguished experts in the field.
Explore Six Specialized Tracks at LearnLab Summer School
Computer Science Education Research (CER) Track: This track provides participants with the essential knowledge and skills to create, refine, and rigorously evaluate interactive learning content, instructional materials, learning technologies, and analytics specifically designed for computer science education. Participants will learn how to ground their design processes in existing Computer Science Education Research (CER) findings. Crucially, they will also learn to leverage their research outcomes to deepen the understanding of how individuals learn computer science and how to effectively improve this learning process. CER is inherently interdisciplinary, drawing upon insights from education research, computer science, psychology, and related fields to address critical questions surrounding computer science learning and teaching within a learn lab context. For those seeking a deeper dive into CER, Dr. Amy Ko’s CER FAQ offers valuable resources.
Within the CER track, participants will investigate how students grasp complex computer science concepts and skills. They will analyze the impact of diverse pedagogical approaches on student learning outcomes and engagement levels. Furthermore, the track will demonstrate how instrumenting a CS course or module for data-driven analysis enables the application of educational data mining techniques as powerful tools for enhancing learning. A key focus will also be on methods to reduce barriers to participation and promote greater diversity within computer science education.
Participants in the CER track might engage in a variety of impactful projects, such as:
- Designing innovative systems or interventions, firmly rooted in educational theory, to improve learning within both formal CS classroom settings and informal learning environments.
- Creating and evaluating data-driven algorithms to automate aspects of CS instruction, such as providing personalized feedback or optimizing problem selection.
- Designing rigorous studies to evaluate the effectiveness of interventions in CS classrooms, utilizing data to gain deeper insights into how students interact with and benefit from these interventions within the learn lab.
- Instrumenting existing CS learning environments and developing analytics to gain a comprehensive understanding of student learning patterns within those environments.
- Applying learning analytic approaches to existing datasets from CS classrooms to address specific research questions about how students learn computer science effectively.
Building Online Courses with OLI – OLI Track: The Open Learning Initiative (OLI) track centers on the fundamental elements of effective online course design, with a particular emphasis on the crucial connection between clearly defined learning objectives and measurable learning outcomes. Participants in this track will begin by identifying a specific course module they aspire to create and articulating the expected learning outcomes for that module. Throughout the week, they will engage in a structured process to: 1) refine their learning outcomes to ensure they are precise and readily measurable, and 2) develop comprehensive content, engaging activities, and robust assessments specifically designed to support these outcomes. Time permitting, participants may also formulate a strategic plan for developing additional course modules, further extending their impact within the learn lab environment. The modules created during the summer school can be directly implemented in live classrooms through the OLI platform and iteratively improved using valuable data gathered from learner interactions over time. Participants will not only create OLI courseware but will also gain lasting proficiency in using OLI tools and techniques beyond the duration of the summer session.
This track adopts a two-tiered pedagogical approach, providing participants with a thorough introduction to both the underlying pedagogical principles and design philosophy that underpin effective OLI learning experiences, while simultaneously guiding them in the practical application of the tools and technologies that constitute the OLI platform. Carnegie Mellon’s Open Learning Initiative (OLI) is dedicated to developing online learning environments that seamlessly integrate research and practice to deliver highly effective learning experiences while simultaneously advancing the fundamental understanding of human learning processes within a learn lab setting. OLI technologies offer a standard suite of learning activities coupled with the flexibility to integrate additional educational technologies. For example, an OLI course could incorporate technologies and approaches from other Summer School tracks, such as integrating an intelligent tutor or collaborative learning experience into a broader learning environment, or leveraging EDM techniques to analyze data generated from their OLI course.
Computational Models of Learning (CML) Track: A central theme of the Computational Models of Learning (CML) track is exploring the synergistic relationship between the study of machine learning and human learning within a learn lab context. Computational Models of Learning (CML) are designed to simulate how students’ knowledge representations evolve and adapt in response to individual learning opportunities. While Educational Data Mining (EDM) employs a data-driven approach to analyze the progression of student performance with practice, CML adopts a theory-first approach to model learning. This is achieved through the development of sophisticated machine-learning algorithms that simulate how specific learning experiences induce changes in students’ internal knowledge representations. By simulating learning as a dynamic developmental process, CML-based student simulations can replicate the patterns of mistakes and correct responses observed in real students as they learn directly from online educational materials, including Intelligent Tutoring Systems (ITSs). Attendees will gain in-depth knowledge of recently developed Computational Models of Learning, understand how their underlying theories are rigorously tested against real student data, and learn how CML distinguishes itself from other forms of student modeling and simulation. Participants will have the opportunity to apply computational models of learning for several practical purposes, including: 1) as simulated learners to generate synthetic student data for research and development, 2) as innovative authoring tools that are taught interactively rather than requiring traditional programming, and 3) as powerful tools for generating explainable knowledge representations to analyze both expert knowledge and the dynamic evolution of students’ knowledge throughout the learning process within the learn lab. To facilitate attendees in simulating or supporting learning within their specific domains of interest, CML track participants often acquire foundational knowledge from the ITS and EDM tracks, encouraging interdisciplinary learning and skill development.
Chemistry Education (CE) Track: The Chemistry Education track delves into the intricacies of how students learn chemistry, utilizing REAL Chem courseware, a comprehensive resource that provides a full year’s worth of instructional materials. REAL Chem has amassed millions of data records collected from a diverse range of two- and four-year higher education institutions. Unlike traditional pre- and post-tests, this rich dataset spans entire semesters, offering a continuous and granular view of student learning progression and enabling the discovery of patterns across various chemistry topics. (For example, see DOI: 10.26434/chemrxiv-2024-bms6x).
This track provides a hands-on introduction to the field of chemistry education research within a learn lab setting. Participants will utilize REAL Chem materials and datasets to design their own research projects, gaining practical experience with LearnSphere’s DataShop, a secure platform for advanced data analysis. DataShop empowers researchers to uncover valuable insights, such as learning curves and performance trends, from complex educational datasets. Additionally, participants will explore Torus, an adaptive learning platform designed for modifying and creating instructional materials that are precisely tailored to specific research objectives.
By the end of the week, participants will have formulated a clear research question and developed a robust study design aligned with their research goals. Potential research topics may include investigating problem-solving strategies in chemistry, exploring conceptual understanding of core chemical principles, or analyzing the effectiveness of molecular visualization tools in chemistry education within the learn lab. Whether participants are experienced chemistry instructors initiating research endeavors or learning scientists venturing into chemistry education research, this track provides the essential tools, expert support, and intellectual inspiration to advance both teaching practices and research initiatives.
EDM (Educational Datamining) Track: Participants selecting the Educational Data Mining (EDM) track will focus on the practical application of data mining tools and methods to analyze real-world educational datasets within a learn lab environment. The dataset for analysis can be chosen from the extensive collection currently available in LearnLab’s DataShop, or participants have the option to bring their own dataset, allowing for personalized research inquiries. (It is recommended to coordinate dataset selection with the assigned mentor prior to the summer school). A typical dataset in this context would comprise a detailed record of student interactions with a computer tutor over a defined period, potentially extending over a semester or academic year. However, the track welcomes and encourages the use of diverse and interesting datasets beyond this typical example. Participants will identify a driving research question to guide their analysis, ideally one that aligns with their existing research interests. Examples of such questions include understanding how students’ strategy use evolves over time or identifying the relationships between specific aspects of students’ meta-cognitive abilities and their learning outcomes within the learn lab. The analytical process involves several key steps: becoming thoroughly familiar with the chosen dataset, potentially by engaging with the learning activities represented in the data as a student; operationalizing the overarching hypothesis into more detailed and testable analysis questions; executing the analysis using a range of tools (e.g., DataShop, TagHelper, R, SPSS, Weka, RapidMiner, or other relevant software packages) and algorithms (e.g., Logistic Regression, exponential-family Principle Component Analysis); interpreting the results of the analysis in a meaningful way; and preparing a concise summary poster presentation to share findings with peers and mentors.
ITS (Intelligent Tutoring System) Track: In the Intelligent Tutoring System (ITS) development track, participants will engage in the hands-on implementation of a prototype computer-based tutor, leveraging authoring tools specifically developed by LearnLab researchers. A primary tool utilized in this track is CTAT (Cognitive Tutor Authoring Tools), which is specifically designed to facilitate the creation of intelligent tutoring systems, even for individuals without prior programming experience, making the learn lab experience accessible to a broad range of participants. Depending on individual interests and research goals, the developed tutor prototype might be related to a planned or potential experiment, perhaps even an in vivo experiment conducted in a real-world classroom setting. Alternatively, it could contribute to an ongoing tutor development project or a course that the participant is currently teaching or planning to launch. CTAT-built tutors are particularly well-suited for tutoring multi-step problem-solving tasks, commonly found in subjects like mathematics, physics, and chemistry. However, they are also being increasingly and successfully applied to language learning, where exercises often have a finer level of granularity. During the week, participants will initiate their tutor development process by conducting cognitive task analysis to thoroughly understand the nature of the problems for which their tutor will provide targeted tutoring. Subsequently, based on their specific interests, they will utilize one or more of the aforementioned tools to implement a functional computer-based tutor prototype. By the conclusion of the week, participants will have a working prototype ready for further development and refinement within their learn lab projects. In fact, participants who choose to concentrate on intelligent tutoring systems development will have already implemented some core intelligent tutor behavior, specifically an Example-Tracing Tutor, by the end of just the second day of the summer school, demonstrating rapid progress and tangible outcomes.
