Publications

35publications

In review

LAK 2027·Submitted

Making Interpretations Accountable: Propagating Validation Evidence Through a Human–AI Multimodal Learning Analytics Pipeline

Fonteles, J.H., Cohn, C., Zhou, M., Ayalon, E., Ashwin, T.S., & Biswas, G.

Learning and Individual Differences·Submitted

Expanding Access to Collaborative Computational Thinking through the ReCode Loop and Role-Differentiated AI Agents

Fonteles, J.H., Ayalon, E., Shaer, A., Enyedy, N., Dror, R., & Biswas, G.

arXiv:2505.17238·Preprint

Personalizing Student–Agent Interactions Using Log-Contextualized Retrieval-Augmented Generation (RAG)

Cohn, C., Rayala, S., Snyder, C., Fonteles, J.H., Jain, S., Mohammed, N., et al.

British Journal of Educational Technology·Major revision

A Human–AI Multimodal Analysis of Collaborative Knowledge Integration through Embodiment and Reflection

Fonteles, J.H., Zhou, M., Cohn, C., Ayalon, E., Enyedy, N., & Biswas, G.

ACM Computing Surveys·Under review

Multimodal Methods for Analyzing Learning and Training Environments: A Systematic Literature Review

Cohn, C., Davalos, E., Vatral, C., Fonteles, J.H., Wang, H.D., Coursey, A., Rayala, S., Ma, M., & Biswas, G.

2026

Conference/AIED 2026 – International Conference on Artificial Intelligence in Education
★

A Human–AI Partnership Framework for Multimodal Analysis in Embodied Learning Environments

Fonteles, J.H., & Biswas, G.

Workshop/CV4Edu Workshop @ CVPR 2026

Do Emotion Recognition Models Generalize to Classrooms? Robustness and Fairness Analysis

Sadashiva, A.T., Prasanna, S.M., Fonteles, J.H., & Biswas, G.

Workshop/ACII 2026 Workshops and Demos

When Modalities Disagree: Multimodal Sentiment Conflict as an Affective Signal in Collaborative Learning

Sadashiva, A.T., Prasanna, S.M., Fonteles, J.H., & Biswas, G.

Conference/ISLS 2026

A Multimodal Study of Emotion Regulation in Student Dyads During Computational Model Building

Sadashiva, A.T., Prasanna, S.M., Fonteles, J.H., & Biswas, G.

Conference/EARLI SIG: AI in Learning, Teaching and Education 2026

Multimodal Engagement Analytics for Collaborative Learning: A Human-in-the-Loop Approach

Ashwin, T.S., Gupta, V., Cohn, C., Prasanna, S.M., Zou, X., Fonteles, J.H., Hmelo-Silver, C., Lester, J., & Biswas, G.

Journal/Learning and Instruction
★

Analyzing Embodied Learning in Classroom Settings: A Human-in-the-Loop AI Approach for Multimodal Learning Analytics

Fonteles, J.H., Cohn, C., Ayalon, E., Zhou, M., Ashwin, T.S., Davalos, E., Li, Z., Rayala, S., Mereddy, D., Coursey, A., Jain, S., Zhang, Y., Enyedy, N., Danish, J., & Biswas, G.

Conference/LAK 2026 – International Learning Analytics and Knowledge Conference
★

A Novel Approach to Evaluating the Effectiveness of Large Language Models for Multimodal Analysis of Embodied Learning in Classrooms

Fonteles, J.H., Sivakumaran, N., Cohn, C., Coursey, A., Yu, S., Stengel-Eskin, E., Ashwin, T.S., Bansal, M., & Biswas, G.

Conference/AAAI 2026 – Proceedings of the AAAI Conference on Artificial Intelligence
★

A Theory of Adaptive Scaffolding for LLM-Based Pedagogical Agents

Cohn, C., Rayala, S., Srivastava, N., Fonteles, J.H., Jain, S., Luo, X., Mereddy, D., Mohammed, N., & Biswas, G.

2025

Conference/US-RSE 2025

SyncFlow: A Scalable Platform for Multimodal Learning Analytics

Timalsina, U., Davalos, E., Sanda, N., Zhang, Y., Fonteles, J.H., Ashwin, T.S., & Biswas, G.

Workshop/MMLA Workshop @ EDM 2025
★

Exploring Agentic Multimodal Late Fusion with LLMs for Embodied Learning

Fonteles, J.H., Cohn, C., Mereddy, D., Ashwin, T.S., & Biswas, G.

Conference/CSCL 2025 – International Conference on Computer-Supported Collaborative Learning

Exploring the Design of Pedagogical Agent Roles in Collaborative STEM+C Learning

Cohn, C., Fonteles, J.H., Snyder, C., Srivastava, N., Ashwin, T.S., Campbell, D., Montenegro, J., & Biswas, G.

Conference/ICLS 2025 – International Conference of the Learning Sciences

Interaction Analysis Practice Within Computational Trends: Promises and Challenges

Zhou, M., Danish, J., Fonteles, J.H., Davalos, E., Dim, E., Lane, A.C., Ashwin, T.S., Srivastava, N., Steinberg, S., Anton, G., Enyedy, N., & Biswas, G.

Journal/Learning and Individual Differences

Using Collaborative Interactivity Metrics to Analyze Students' Problem-Solving Behaviors during STEM+C Computational Modeling Tasks

Snyder, C., Cohn, C., Fonteles, J.H., & Biswas, G.

Journal/British Journal of Educational Technology
★

A Multimodal Approach to Support Teacher, Researcher and AI Collaboration in STEM+C Learning Environments

Cohn, C., Snyder, C., Fonteles, J.H., Ashwin, T.S., Montenegro, J., & Biswas, G.

2024

Conference/ICCE 2024 – 32nd International Conference on Computers in Education
★

Designing an AI-Enhanced Timeline for Monitoring Multimodal Interactions in Embodied Learning Environments

Fonteles, J.H., Srivastava, N., Davalos, E., Ashwin, T.S., & Biswas, G.

Conference/AIED 2024 – Artificial Intelligence in Education (LNCS)

A First Step in Using Machine Learning Methods to Enhance Interaction Analysis for Embodied Learning Environments

Fonteles, J.H., Davalos, E., Ashwin, T.S., Zhang, Y., Zhou, M., Ayalon, E., Lane, A.C., Steinberg, S., Anton, G., Danish, J., Enyedy, N., & Biswas, G.

Conference/IDC 2024 – 23rd ACM Interaction Design and Children Conference

Promoting Equitable Learning Outcomes for Underserved Students in Open-Ended Learning Environments

Fonteles, J.H., Akpanoko, C., Wisniewski, P., & Biswas, G.

Conference/ICLS 2024 – International Conference of the Learning Sciences

Exploring Artificial Intelligence Supported Interaction Analysis

Zhou, M., Fonteles, J.H., Danish, J., Davalos, E., Steinberg, S., Biswas, G., & Enyedy, N.

Conference/ICMI Companion '24 – 26th International Conference on Multimodal Interaction

3D Gaze Tracking for Studying Collaborative Interactions in Mixed-Reality Environments

Davalos, E., Zhang, Y., Ashwin, T.S., Fonteles, J.H., Timalsina, U., & Biswas, G.

Conference/LAK 2024 – 14th Learning Analytics and Knowledge Conference

Analyzing Students Collaborative Problem-Solving Behaviors in Synergistic STEM+C Learning

Snyder, C., Hutchins, N.M., Cohn, C., Fonteles, J.H., & Biswas, G.

2023

Conference/IEEE BigData 2023 – IEEE International Conference on Big Data

ChimeraPy: A Scientific Distributed Streaming Framework for Real-time Multimodal Data Retrieval and Processing

Davalos, E., Timalsina, U., Zhang, Y., Wu, J., Fonteles, J.H., & Biswas, G.

Conference/LAK23 – 13th International Learning Analytics and Knowledge Conference

Identifying Gaze Behavior Evolution via Temporal Fully-Weighted Scanpath Graphs

Davalos, E., Vatral, C., Cohn, C., Fonteles, J.H., Biswas, G., Mohammed, N., et al.

2021

Journal/Entertainment Computing

User Experience in a Kinect-Based Conducting System for Visualization of Musical Structures

Fonteles, J.H., & Rodrigues, M.A.F.

2018

Conference/IEEE SeGAH 2018 – 6th International Conference on Serious Games and Applications for Health

Gesture-Controlled Interactive Musical Game to Practice Hand Therapy Exercises and Learn Rhythm and Melodic Structures

Fonteles, J.H., Serpa, Y.R., Barbosa, R.G., Rodrigues, M.A.F., & Alves, M.S.

2017

Conference/SVR 2017 – 19th Symposium on Virtual and Augmented Reality

Interactive Musical Game with a Gesture-Controlled Virtual Puppet

Fonteles, J.H., Fernandes, M.S.S., Queiroz, D.L., & Rodrigues, M.A.F.

2nd Place, Best Full Paper

2015

Journal/Journal on Interactive Systems

Visual and Gesture-Driven Interaction Using the Leap Motion to Conduct a 3D Particle System: Evaluation and Analysis of an Orchestral Performance

Fonteles, J.H., Sousa, E.S., & Rodrigues, M.A.F.

Conference/SVR 2015 – XVII Symposium on Virtual and Augmented Reality

Visual and Interactive Performance of Particles Conducted by the Leap Motion

Fonteles, J.H., Sousa, E.S., & Rodrigues, M.A.F.

Workshop/XV Workshop de Informática Médica – XXXV Congresso da Sociedade Brasileira de Computação

Apresentando o Tratbem: Suporte Social, Informacional e Psicológico ao Tabagista

Oliveira, F., Sales, M.P., Fonteles, J.H., Silva, N., Pereira, F., Melo, J., Cavalcante, I., & Moraes, L.

2014

Conference/SVR 2014 – XVI Symposium on Virtual and Augmented Reality

Real-Time Animations of Virtual Fountains Based on a Particle System for Visualizing Musical Structures

Fonteles, J.H., Rodrigues, M.A.F., & Basso, V.E.

2013

Journal/Journal of Visual Languages & Computing

Creating and Evaluating a Particle System for Music Visualization

Fonteles, J.H., Rodrigues, M.A.F., & Basso, V.E.