Report on a Presentation at the 5th Workshop 2013:
Future Education and Learning Environments/General with Smart Devices, Japanese Society for Information and Systems in Education (JSiSE)
"Impact of a Testing Method Using Smart Phones on Test-Takers’ Motivation and Results"

On January 11, 2014, I participated in a workshop of Japanese Society for Information and Systems in Education (JSiSE) held at the Education and Research Building A of Kochi University of Technology. I gave a presentation titled "Impact of a Testing Method Using Smart Phones on Test-Takers’ Motivation and Results."


The research presented stems from the fact that it is now required that universities improve their information education classes, as information-related skills possessed by freshmen change year after year. The reason for such changes is partly due to changes in high-school education curricula, such that the academic subject of information was reformed and, in 2013, divided into two subjects—Society and Information and Science of Information—which have been gradually implemented. At the same time, faculty development has become compulsory, and universities are required to improve their classes for promoting student learning outside of the class, improving students’ academic ability, and helping them acquire sound knowledge.


To deal with the issues above, blended learning—which blends face-to-face lessons with an e-learning system—is one measure taken to reform college information education. Since most college students today possess a mobile phone and since the ratio of students with smart phones is increasing, more attention has been paid to mobile learning (m-learning) that uses these devices. With this learning, we can expect to promote learning outside of the classroom and facilitate students to gain sound knowledge.


To help students gain sound knowledge, some previous works have proposed conducting tests. Regarding tests using smart phones, it has been asked which method—the one that displays all questions on the screen or the one that exchanges questions and answers—causes less burden for learners and motivates them to try harder.


To examine the foundation subject of information at college, in this research we established an experimental environment that assumes test delivery and independent testing using a smart phones outside of class, in an environment of face-to-face class blended with an m-learning system. The research aimed to investigate differences in different testing methods while considering college students’ approach–avoidance trend toward testing from the standpoints of test-takers’ motivation to take the test, self-efficacy, and rate of correct answers.


In this research, by targeting 60 college students, we conducted a test that mixes multiple-choice and fill-in-the-blank questions (15 questions in total) using a smart phone. We divided students into two groups: one that could see all questions at once on the screen (30 students, hereafter "displaying all questions" group), and the other that could answer questions one at a time (30 students, hereafter "exchanging questions and answers" group). To help students gain the soundness of knowledge through testing, since previous research has suggested focusing on students' viewpoint for tests (i.e., approach–avoidance trend toward testing), we analyzed the trend of students' approaching or avoiding tests. As a result, we could not confirm any bias in the trend for the groups and so concluded that the method used to divide the groups was appropriate.


Next, we conducted a comparative analysis of the rate of correct answers and testing time, as well as students’ burden, motivation for testing, and self-efficacy. As a result, we found that the exchanging questions and answers group expressed feeling burdened and less motivated when all questions been displayed at once. By contrast, regarding self-efficacy, the displaying all questions group showed significantly higher recognition through the questionnaire and generally indicated having a full understanding of the content of the lesson. In regards also to the rate of correct answers, it was shown that the displaying all questions group had significantly higher recognition. These findings suggest us that, though it is easier to work with the questions-and-answers testing method, students can get better learning efficacy by being shown all questions at once.


As our future tasks, we will need to:

1)  Explain why a higher rate of correct answers and efficacy are achieved with the method that displays all questions at once;

2)  Analyze the answering procedure;

3)  Conduct survey analysis using a tablet;

4)  Conduct a long-term survey that targets actual class lessons.


Lastly, let me extend my sincere appreciation to many of you who gave me valuable advice at the workshop.

 (Takeshi Kitazawa, CRET Researcher)

Takeshi Kitazawa

CRET Researcher / Associate Professor of Graduate School of Education, Tokyo Gakugei University

Hobbies: Traveling, swimming, and taking a walk around town

Research topic: I conduct research on the educational effects of using ICT, focusing particularly on learners’ motivation.


<< | 1 | >>

Areas of Reasearch in CRET

This laboratory conducts research on test evaluation and analysis. We also perform joint research and exchange programs with overseas testing research institutes.

>> Click here
for the laboratory

This laboratory conducts research and development into testing approaches that measure communication skills, teamwork skills, and social skills, etc.

Dr. Atsushi Aikawa

Faculty of Human Sciences,
University of Tsukuba
Ph.D. in Psychology

>> Click here
for the laboratory

This laboratory conducts research on the foundation of computer-based testing, and basic research on media and recognition, as well as applied and practical research
that utilize such knowledge.

Dr. Kanji Akahori

Professor Emeritus of
Tokyo Institute of

>> Click here
for the laboratory