Report on CRET Research Exchange with OECD, DIPF, and IEA

 From October 11th to October 17th 2009, CRET researchers, led by Mr. Kenichi Arai, the Chair of CRET, visited OECD , DIPF , and IEA . A useful exchange took place concerning competencies measured by international assessments such as PIAAC2011(Programme for the International Assessment of Adult Competencies), PISA2012(The Programme for International Student Assessment), their measurement methodologies, as well as CRET current research activities. The following is the latest report on PISA2012, PIAAC2011, and Computer Based Test (CBT).

 (1) PISA2012 and PIAAC2011

 OECD has been implementing PISA for students aged 15, reaching the end of compulsory education, every three years from the year 2000. In PISA2012, standard questions of mathematical literacy, reading literacy and scientific literacy will be included, with the emphasis on mathematical literacy. As an option, using CBT to measure three types of literacy and problem-solving abilities is scheduled to be executed. 
 In 2011, PIAAC is scheduled to be implemented for those from 16 to 64 years of age. The predecessors to this survey are IALS(International Adult Literacy Survey, 1994 - 1998) and ALL(Adult Literacy and Life Skills Survey, 2002 - 2006) , measuring competencies in three areas: reading literacy, numeracy and problem-solving abilities. The home-visit survey method will be used. It will be available both as a CBT and a Paper Based Test (PBT). The examiner will choose the appropriate test form depending on the examinee’s ability to operate a computer. 

 Approximately 60% of PIAAC2011 is to resemble the questions used for IALS and ALL. Referring to the questions of ALL publicly disclosed , the questions are oriented to assess basic literacy necessary at work and in daily life, such as the ability to read gasoline meters. In the preliminary PIAAC survey scheduled to take place from March to April 2010, 28 countries including Japan are expected to participate. Past surveys indicated that some adults equipped with basic reading and essay writing skills were unable to use them effectively in real life. We shall see how the 2011 survey will turn out.

 The reports on the two assessments are expected to come out in 2013. The possible perspectives of data analysis include the comparison of the test results of PISA2012 and PIAAC2011 for the group of 16-year olds, and the PIAAC2011 and PISA2000 comparison of the group of 28-year olds who took PISA 2000.

 (2) On CBT

 There are several points to note in developing CBT. Here are two of them.
 One is the effect on the test score of the examinee’s affinity to computers. This was also mentioned in the report on JART’s special lecture in September where Dr. Randy Bennett (ETS) gave a speech . Considering this point, in the CBT for PISA and PIAAC, tests will be based on the premise that computers can be used with ease by test takers, comparable to the use of pencils for PBTs, based on what we heard during our visits. 
 The second point of concern with CBT is the increased burden on test-takers, resulting from the need to scroll pages as sentences get longer. As a possible measure to reduce the need for scrolling, CRET has been exploring the possibility of replacing written questions with animated or other images. Much interest was shown in the CRET report presented at e-Learn2009 in October, entitled ?Use of Animation to Display Math Problems in a Computer-Based Test.

 We have learned that steady preparations are being made for international tests to incorporate the measurement of competencies to effectively use computers for cross-disciplinary problem-solving requirements in modern society. CRET intends to keep up with this global trend in search of the optimum education for Japanese. 

(Chie Hoshi, CRET Researcher)


Chie Hoshi

CRET Researcher

World Watch

<< | 1 | 2 | >>

Areas of Reasearch in CRET

This laboratory conducts research on test evaluation and analysis. We also perform joint research and exchange programs with overseas testing research institutes.

>> Click here
for the laboratory

This laboratory conducts research and development into testing approaches that measure communication skills, teamwork skills, and social skills, etc.

Dr. Atsushi Aikawa

Faculty of Human Sciences,
University of Tsukuba
Ph.D. in Psychology

>> Click here
for the laboratory

This laboratory conducts research on the foundation of computer-based testing, and basic research on media and recognition, as well as applied and practical research
that utilize such knowledge.

Dr. Kanji Akahori

Professor Emeritus of
Tokyo Institute of

>> Click here
for the laboratory