The HINTS Lab
Help through Intelligent Support
The Help through INTelligent Support (HINTS) Lab, directed by Dr. Thomas Price, works to develop learning environments that automatically support students with AI and data-driven help features. With a focus on computing education, our goal is to reimagine programming environments as adaptive, interactive systems that help students to pursue learning goals that are meaningful to them. We believe that every student should be able to learn computing with the support they need to be successful, working on projects that match their values and interests. Our research emphasizes practical methods that can scale to new classrooms and contexts, without placing additional burden on instructors. Examples of our research include:
- Supporting students to design and implement on open-ended programming projects.
- Generating data-driven programming help, such as hints and worked examples.
- Developing machine learning models to better understand student code and predict learning outcomes.
- Understanding how programmers seek and use help from web-based help, AI systems, and humans.
- Evaluating the impact of programming support in classroom and lab studies.
For more about our research, see this video:
Projects
*
HINTS Lab Members
Principal Investigators
Thomas W. Price
Assistant Professor
Graduate Students
Yang Shi
Ph.D. Candidate
James Skripchuk
Ph.D. Candidate
John Thomas Bacher
Ph.D. Student
Keith Tran
Ph.D. Student
Alumni
Emma Wang
Machine Learning Engineer, Apple
Samiha Marwan
Post Doctoral Researcher (CI Fellow)
Rui Zhi
Software Engineer
HINTS Lab Director
Thomas W. Price is an Assistant Professor in the Computer Science Department at North Carolina State University, where he runs the HINTS Lab. He is also a part of NCSU’s Center for Educational Informatics.
Thomas’ research goal is to re-imagine educational programming environments as adaptive, data-driven systems that support students automatically as they pursue learning goals that are meaningful to them. His work has focused on the domain of computing education, where he has developed techniques for automatically generating programming hints and feedback for students in real-time by leveraging student data. His HINTS lab focuses on supporting students working in creative, open-ended and block-based learning contexts, leading to novel data-driven programming support, including adaptive examples, subgoal feedback, and models to predict student outcomes.
To learn more about me, check out:
Download my CV.
Interests
- Computing Education Research
- Educational Data Mining
- Advanced Learning Technologies
- Human-Computer Interaction
Education
-
Ph.D. in Computer Science, 2018
North Carolina State University
-
M.S. in Computer Science, 2015
North Carolina State University
-
B.S. in Computer Science, 2013
Elon University
Recent Publications
See some of our most recent work or view all publications here.
(2023).
A Case Study on When and How Novices Use Code Examples in Open-Ended Programming.
Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education.
(2023).
An Experience Report on Introducing Explicit Strategies into Testing Checklists for Advanced Beginners.
Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education.
(2023).
Analysis of Novices’ Web-Based Help-Seeking Behavior While Programming.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2023).
Do Intentions to Persist Predict Short-Term Computing Course Enrollments?.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2023).
KC-Finder: Automated Knowledge Component Discovery for Programming Problems.
Proceedings of the International Conference on Educational Data Mining.
(2022).
Case Studies on the Use of Storyboarding by Novice Programmers.
Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1.
(2022).
Exploring Design Choices to Support Novices' Example Use During Creative Open-Ended Programming.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2022).
Automating Personalized Feedback to Improve Students' Persistence in Computing.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2022).
Check It Off: Exploring the Impact of a Checklist Intervention on the Quality of Student-authored Unit Tests.
Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1.
(2022).
Code-DKT: A Code-based Knowledge Tracing Model for Programming Tasks..
In Proceedings of the 15th International Conference on Educational Data Mining (EDM) 2022.
(2022).
Adaptive Immediate Feedback for Block-Based Programming: Design and Evaluation.
IEEE Transactions on Learning Technologies.
(2022).
Cross-Lingual Adversarial Domain Adaptation for Novice Programming.
Proceedings of the Association for the Advancement of Artificial Intelligence Conference.
(2022).
Gender, Self-Assessment, and Persistence in Computing: How gender differences in self-assessed ability reduce women's persistence in computer science.
Proceedings of the International Computing Education Research Conference.
(2022).
How to Catch Novice Programmers’ Struggles: Detecting Moments of Struggle in Open-Ended Block-Based Programming Projects using Trace Log Data.
Proceedings of the 6th Workshop on Educational Data Mining in Computer Science Education @ EDM'22.
(2022).
How, when, and why do novices struggle in programming? Exploring the experiences and perceptions of common programming moments in block-based environments..
Proceedings of the 6th Workshop on Educational Data Mining in Computer Science Education @ EDM'22.
(2022).
Identifying Common Errors in Open-Ended Machine Learning Projects.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2022).
Increasing Students' Persistence in Computer Science through a Lightweight Scalable Intervention.
Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1.
(2022).
iSnap: Evolution and Evaluation of a Data-Driven Hint System for Block-based Programming.
IEEE Transactions on Learning Technologies.
(2022).
Pinpoint: A Record, Replay, and Extract System to Support Code Comprehension and Reuse.
Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).
(2022).
Proceedings of the 6th Workshop on Educational Data Mining in Computer Science Education @ EDM'22.
(2021).
Just a Few Expert Constraints Can Help: Humanizing Data-Driven Subgoal Detection for Novice Programming.
Proceedings of the International Conference on Educational Data Mining 2021.
(2021).
Toward Semi-Automatic Misconception Discovery Using Code Embeddings.
Proceedings of the 11th International Learning Analytics and Knowledge Conference (LAK21).
(2021).
Early Performance Prediction using Interpretable Patterns in Programming Process Data.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2021).
Execution-Trace-Based Feature Engineering To Enable Formative Feedback on Visual, Interactive Programs.
Proceedings of the 5th Workshop on Educational Data Mining in Computer Science Education (CSEDM) at EDM'21.
(2021).
Exploring Design Choices in Data-driven Hints for Python Programming Homework.
Proceedings of the ACM Conference on Learning@ Scale.
(2021).
Automated classification of visual, interactive programs using execution traces.
Proceedings of the International Conference on Educational Data Mining.
(2021).
Knowing both when and where: Temporal-ASTNN for Early Prediction of Student Success in Novice Programming Tasks.
In Proceedings of the 14th International Conference on Educational Data Mining (EDM) 2021.
(2021).
More with less: Exploring how to use deep learning effectively through semi-supervised learning for automatic bug detection in student code..
In Proceedings of the 14th International Conference on Educational Data Mining (EDM) 2021.
(2021).
Novices' Learning Barriers When Using Code Examples in Open-Ended Programming.
Proceedings of the International Conference on Innovation and Technology in Computer Science Education(31% acceptance rate; 84/275 full papers.).
(2021).
Novices' Learning Barriers When Using Code Examples in Open-Ended Programming.
Proceedings of the International Conference on Innovation and Technology in Computer Science Education.
(2021).
PlanIT! A new integrated tool to help novices design for open-ended projects.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2021).
Proceedings of the 5th Workshop on Educational Data Mining in Computer Science Education @ EDM'21.
(2021).
Promoting Students' Progress-Monitoring Behavior during Block-Based Programming.
Proceedings of the Koli Calling International Conference on Computing Education Research.
(2021).
Scaffolding Game Design: Towards Tool Support for Planning Open-Ended Projects in an Introductory Game Design Class.
Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).
(2021).
Snap-Check: Automated Testing for Graphical Interactive Programs.
Proceedings of the International Conference on Innovation and Technology in Computer Science Education(31% acceptance rate; 84/275 full papers.).
(2021).
SnapCheck: Automated Testing for Snap Programs.
Proceedings of the International Conference on Innovation and Technology in Computer Science Education.
(2021).
Identifying Struggling Students in Novice Programming Course with Knowledge Tracing.
Proceedings of the 5th Workshop on Educational Data Mining in Computer Science Education (CSEDM) at EDM'21.
(2021).
Using Student Trace Logs to Determine Meaningful Progress and Struggle during Programming Problem Solving..
Proceedings of the International Conference on Educational Data Mining.
(2020).
Adaptive Immediate Feedback Can Improve Novice Programming Engagement and Intention to Persist in Computer Science.
Proceedings of the International Computing Education Research Conference (22.7% acceptance rate; 27/119 full papers).
(2020).
An Evaluation of Data-driven Programming Hints in a Classroom Setting.
Proceedings of the International Conference on Artificial Intelligence in Education.
(2020).
Comparing Feature Engineering Approaches to Predict Complex Programming Behaviors.
Proceedings of the 4th Workshop on Educational Data Mining in Computer Science Education (CSEDM) at EDM'20.
(2020).
Crescendo: Engaging Students to Self-Paced Programming Practices.
Proceedings of the ACM Technical Symposium on Computer Science Education(31.4% acceptance rate; 171/544 papers.).
(2020).
Engaging Students with Instructor Solutions in Online Programming Homework.
ACM CHI Conference on Human Factors in Computing Systems.
(2020).
Unproductive Help-seeking in Programming: What it is and How to Address it?.
Proceedings of the International Conference on Innovation and Technology in Computer Science Education.
(2020).
Proceedings of the 4th Workshop on Educational Data Mining in Computer Science Education @ EDM'20.
(2020).
What Time is It? Student Modeling Needs to Know.
Proceedings of the International Conference on Educational Data Mining.
(2020).
ProgSnap2: A Flexible Format for Programming Process Data.
Proceedings of the International Conference on Innovation and Technology in Computer Science Education.
(2020).
Step Tutor: Supporting Students through Step-by-Step Example-Based Feedback.
Proceedings of the International Conference on Innovation and Technology in Computer Science Education(27.6% acceptance rate; 72/261 full papers.).
(2020).
The Impact of Data-driven Positive Programming Feedback: When it Helps, What Happens when it Goes Wrong, and How Students Respond.
Proceedings of the 4th Workshop on Educational Data Mining in Computer Science Education (CSEDM) at EDM'20.
(2020).
Immediate Data-Driven Positive Feedback Increases Engagement on Programming Homework for Novices.
Proceedings of the 4th Workshop on Educational Data Mining in Computer Science Education (CSEDM) at EDM'20.
(2019).
Toward Data-Driven Example Feedback for Novice Programming.
Proceedings of the International Conference on Educational Data Mining.
(2019).
A Comparison of Two Designs for Automated Programming Hints.
Proceedings of the Educational Data Mining in Computer Science Workshop in the Companion Proceedings of the International Conference on Learning Analytics and Knowledge.
(2019).
An Evaluation of the Impact of Automated Programming Hints on Performance and Learning.
Proceedings of the International Computing Education Research Conference.
(2019).
Defining Tinkering Behavior in Open-ended Block-based Programming Assignments.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2019).
Evaluating the Effectiveness of Parsons Problems for Block-based Programming.
Proceedings of the International Computing Education Research Conference.
(2019).
Exploring the Impact of Worked Examples in a Novice Programming Environment.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2019).
Lightning Talk: Curating Analyses for Programming Log Data.
Proceedings of SPLICE 2019 Workshop Computing Science Education Infrastructure: From Tools to Data at 15th ACM International Computing Education Research Conference.
(2019).
One minute is enough: Early Prediction of Student Success and Event-level Difficulty during a Novice Programming Task.
Proceedings of the International Conference on Educational Data Mining.
(2019).
Proceedings of the 2nd Workshop on Educational Data Mining in Computer Science Education @ LAK'19.
(2019).
Proceedings of the 3rd Workshop on Educational Data Mining in Computer Science Education @ AIED'19.
(2019).
ProgSnap2: A Flexible Format for Programming Process Data.
Proceedings of the Educational Data Mining in Computer Science Workshop in the Companion Proceedings of the International Conference on Learning Analytics and Knowledge.
(2019).
Resource rush: towards an open-ended programming game.
2019 IEEE Blocks and Beyond Workshop (B&B).
(2019).
The Impact of Adding Textual Explanations to Next-step Hints in a Novice Programming Environment.
Proceedings of the Annual Conference on Innovation and Technology in Computer Science Education (28% acceptance rate; 67/243 full papers).
(2019).
A Comparison of the Quality of Data-driven Programming Hint Generation Algorithms.
International Journal of Artificial Intelligence in Education.
(2018).
Exploring Instructional Support Design in an Educational Game for K-12 Computing Education.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2018).
iSnap: Automatic Hints and Feedback for Block-based Programming.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2018).
Reducing the State Space of Programming Problems through Data-Driven Feature Detection.
Proceedings of the Educational Data Mining in Computer Science Education Workshop at the International Conference on Educational Data Mining.
(2018).
The Continuous Hint Factory - Providing Hints in Vast and Sparsely Populated Edit Distance Spaces.
Journal of Educational Data Mining.
(2018).
The impact of data quantity and source on the quality of data-driven hints for programming.
Proceedings of the International Conference on Artificial Intelligence in Education.
(2017).
Showpiece: iSnap Demonstration.
Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing.
(2017).
Factors Influencing Students' Help-Seeking Behavior while Programming with Human and Computer Tutors.
Proceedings of the International Computing Education Research Conference.
(2017).
Hint Generation Under Uncertainty: The Effect of Hint Quality on Help-Seeking Behavior.
Proceedings of the International Conference on Artificial Intelligence in Education (30% acceptance rate; 36/121 full papers).
(2017).
iSnap: Towards Intelligent Tutoring in Novice Programming Environments.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2017).
Position Paper: Block-based Programming Should Offer Intelligent Support for Learners.
Proceedings of the 2nd Blocks and Beyond Workshop at VL/HCC.
(2017).
Sharing and Using Programming Log Data.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2017).
Evaluation of a Data-driven Feedback Algorithm for Open-ended Programming.
Proceedings of the International Conference on Educational Data Mining.
(2016).
The Impact of Granularity on the Effectiveness of Students ' Pedagogical Decision.
Annual Meeting of the Cognitive Science Society.
(2016).
Evaluation of a Frame-based Programming Editor.
Proceedings of the International Computing Education Research Conference.
(2016).
Generating Data-driven Hints for Open-ended Programming.
Proceedings of the International Conference on Educational Data Mining.
(2016).
Clashroom: A Game to Enhance the Classroom Experience.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2016).
Question answering in the context of stories generated by computers.
Advances in Cognitive Systems.
(2016).
Lessons Learned from \"BJC\" CS Principles Professional Development.
Proceedings of the ACM Technical Symposium on Computer Science Education.
(2015).
An Exploration of Data-Driven Hint Generation in an Open-Ended Programming Problem.
Proceedings of the Workshop on Graph-based Educational Data Mining at the International Conference on Educational Data Mining.
(2015).
BJC in Action: Comparison of Student Perceptions of a Computer Science Principles Course.
Proceedings of the Annual Conference For Research On Equity & Sustained Participation In Computing, Engineering, & Technology.
(2015).
Comparing Textual and Block Interfaces in a Novice Programming Environment.
Proceedings of the International Computing Education Research Conference.
(2015).
Determining the Impact of Teacher Professional Development on Perceived Ability to Teach a Computer Science Principles Course..
Proceedings of the International Computing Education Research Conference.
(2015).
The Impact of Granularity on Worked Examples and Problem Solving.
Proceedings of the Annual Meeting of the Cognitive Science Society.
(2015).
Using the hint factory to compare model-based tutoring systems.
Proceedings of the Workshop on Graph-based Educational Data Mining at the International Conference on Educational Data Mining.
(2015).
An Improved Data-Driven Hint Selection Algorithm for Probability Tutors.
Proceedings of the International Conference on Educational Data Mining.
(2014).
Towards an Extended Declarative Representation for Camera Planning.
Workshop on Intelligent Cinematography and Editing at the AAAI Conference on Artificial Intelligence.