Benson Fernandes

Final Project

AI-assisted language learning APP

In today’s interconnected world, proficient English pronunciation is crucial for international students pursuing education abroad. My project explores the potential of using Artificial Intelligence (AI) within mobile language learning apps to improve English acquisition.

By harnessing AI avatars and AI feedback systems, I aimed to improve the English pronunciation learning experience for international students, making it more engaging, personalised, and effective. This showcase will unveil the exciting developments in my journey to redefine language learning through AI integration.

Design Objectives & Hypothesis

Research Objective: 

To investigate the impact of advanced technology (AI) on the perceived usability and engagement of international students using mobile language learning apps for English pronunciation in multicultural settings.

Hypothesis: 

The integration of AI into mobile language learning apps will likely have a positive influence on international students’ perceived usability and engagement. Leading to more effective English pronunciation within multicultural contexts. I further anticipated that the combined use of AI avatars and feedback systems would yield the most favourable results for users’ perceived usability and engagement.

Research Development

Literature Review: 

The literature review was instrumental in evaluating the current state of the AI market and its potential implications for mobile language learning apps. Carefully selected and analysed relevant academic papers and industry reports to lay a solid foundation for the project.

Thematic Analysis: 

To better understand the needs and concerns of the target user groups, I conducted semi-structured interviews to empathise with the users, gaining insights into their interests and pain points.

Language Learning Model

To design an optimal user flow for the app prototype, I developed a language learning model rooted in an existing study on English pronunciation theory (Theory, Exposure, and Pronunciation Mode).

Theory Mode: In this phase, users receive instruction on the phonetic sounds of words. Breaking down keywords within the context of a body of text for better comprehension.

Exposure Mode: This stage employs repetition to enhance users’ recognition and recall abilities, both visually and acoustically.

Pronunciation Mode: To assess the effectiveness of the Theory and Exposure modes, users undergo pronunciation tests against a given text and score. Users can receive feedback on their pronunciation.

Design Development:

My journey in developing the prototype commenced with a comprehensive analysis of existing language learning products, including a thorough examination of design, layout, features, and character traits in popular apps such as Duolingo.

Drawing inspiration from this analysis, I used the insights to create a Figma prototype for a mobile language learning app. (ClearSpeak)

Key features, including Voice Calibration, Mouth Warmup Exercises, and Pronunciation Mode Theories, were strategically integrated to enhance the user experience. The aim was to create a more effective and engaging UI interface for mobile language learning apps.

Prototype Development

AI Avatars: The user-friendly AI teacher avatar is a supportive guide throughout the learning journey. This feature was developed using a range of AI-powered tools, including image generation, text-to-speech technology, and image animation, ensuring a dynamic and engaging learning experience. 

AI Feedback: The feedback feature evaluates the user’s pronunciation against a predetermined text. It assesses various aspects, including phonetic accuracy, fluency, and completeness, providing users with valuable insights to enhance their language learning.

4 final video prototypes were created: 

Prototype A: AI Avatar (No) & AI feedback (No)

Prototype B: AI Avatar (NO) & AI feedback (Yes)

Prototype C: AI Avatar (Yes) & AI feedback (No)

Prototype D: AI Avatar (Yes) & AI feedback (Yes)

User Testing

The user testing phase involved 26 international students who participated in online or in-person sessions. A sample population of Indian, Chinese, and European international students provided valuable feedback for all four prototypes.

Testing Process: Each participant experienced a controlled randomised sequence of all four prototypes and had their pronunciation assessed against the test body of text. Subsequently, they completed questionnaires to evaluate the perceived usability (utilizing SUS scores and the NASA Task-Load Index) and user engagement (through a Post-Task Survey).

Project Hypothoisi and Data Analysis (Key Findings)

SUS Score: Notably, Prototype (B) emerged as the leader in system usability, with participants highlighting its practicality for language learning.

NASA (TLX): The data indicated that all four prototypes placed a medium to high mental task load on participants, with no statistically significant differences.

Post-Task Survey: Prototype (D) garnered the most praise from participants, being perceived as engaging and useful. Repeated exposure to the AI avatar built familiarity and user preference.

Hypothesis Assessment: The data strongly supports rejecting the null hypothesis, suggesting that the incorporation of AI in language learning positively impacts international students’ English pronunciation acquisition.