View source for Programme for International Student Assessment (2000 to 2012)

{{short description|Educational assessment and evaluation}}
{{for|the latest PISA test, in 2018|Programme for International Student Assessment}}
{{Use dmy dates|date=May 2021}}

The '''[[Programme for International Student Assessment]]''' has had several runs before [[Programme for International Student Assessment#PISA 2012|the most recent one in 2012]]. The first PISA assessment was carried out in 2000. The results of each period of assessment take about one year and a half to be analysed. First results were published in November 2001. The release of raw data and the publication of technical report and data handbook only took place in spring 2002. The triennial repeats follow a similar schedule; the process of seeing through a single PISA cycle, start-to-finish, always takes over four years. 470,000 15-year-old students representing 65 nations and territories participated in PISA 2009. An additional 50,000 students representing nine nations were tested in 2010.<ref>PISA 2009 Technical Report, 2012, OECD, http://www.oecd.org/dataoecd/60/31/50036771.pdf</ref>

Every period of assessment focuses on one of the three competence fields of reading, math, science; but the two others are tested as well. After nine years, a full cycle is completed: after 2000, reading was again the main domain in 2009.

{| class="wikitable"
|-
! Period
! Focus
! OECD countries
! Partner countries
! Participating students
! Notes
|-
| 2000
| Reading
| 28
| 4 + 11
| 265,000
| The Netherlands disqualified from data analysis. 11 additional non-OECD countries took the test in 2002.
|-
| 2003
| Mathematics
| 30
| 11
| 275,000
| UK disqualified from data analysis. Also included test in [[problem solving]].
|-
| 2006
| Science
| 30
| 27
| 400,000
| Reading scores for US excluded from analysis due to misprint in testing materials.<ref>{{citation|url = http://nces.ed.gov/pubs2008/2008016.pdf|title = Highlights From PISA 2006: Performance of U.S. 15-Year-Old Students in Science and Mathematics Literacy in an International Context|date = 2007-12-10 |first1 = Stéphane |last1 = Baldi |first2 = Ying|last2 = Jin|first3 = Melanie|last3 = Skemer|first4 = Patricia J|last4 = Green|first5 = Deborah|last5 = Herget|first6 = Holly|last6 = Xie|accessdate = 2013-12-14|publisher = [[National Center for Education Statistics|NCES]]|quote = PISA 2006 reading literacy results are not reported for the United States because of an error in printing the test booklets. Furthermore, as a result of the printing error, the mean performance in mathematics and science may be misestimated by approximately 1 score point. The impact is below one standard error.}}</ref> 
|-
| 2009<ref name=PISA2009/>
| Reading
| 34
| 41 + 10
| 470,000
| 10 additional non-OECD countries took the test in 2010.<ref name=PISA2009Plus />
|-
| 2012<ref name=PISA2012>{{citation|title = PISA 2012 Results in Focus|publisher = [[OECD]]|date = 3 December 2013|accessdate = 4 December 2013|url = http://www.oecd.org/pisa/keyfindings/pisa-2012-results-overview.pdf}}</ref>
| Mathematics
| 34
| 31
| 510,000
|
|}

==Results==

===PISA 2012===
{{collapse top|title=PISA 2012}}
[[File:PISA-Maths-2012.svg|350px|right|thumb|The results for the 2012 "Maths" section on a world map.]]
[[File:PISA-Science-2012.svg|350px|right|thumb|The results for the 2012 "Science" section on a world map.]]
[[File:PISA-Reading-2012.svg|350px|right|thumb|The results for the 2012 "Reading" section on a world map.]]
{{PISA 2012}}
PISA 2012 was presented on 3 December 2013, with results for around 510,000 participating students in all 34 OECD member countries and 31 partner countries.<ref name=PISA2012>{{citation|title = PISA 2012 Results in Focus|publisher = [[OECD]]|date = 3 December 2013|accessdate = 4 December 2013|url = http://www.oecd.org/pisa/keyfindings/pisa-2012-results-overview.pdf}}</ref> This testing cycle had a particular focus on mathematics, where the mean score was 494. A sample of 1,688 students from Puerto Rico took the assessment, scoring 379 in math, 404 in reading and 401 in science.<ref>CB Online Staff. [http://www.caribbeanbusinesspr.com/news/pr-scores-low-on-global-report-card-100911.html "PR scores low on global report card"] {{webarchive|url=https://web.archive.org/web/20150103080012/http://www.caribbeanbusinesspr.com/news/pr-scores-low-on-global-report-card-100911.html |date=2015-01-03 }}, ''[[Caribbean Business]]'', 26 September 2014. Retrieved on 3 January 2015.</ref> A subgroup of  44 countries and economies with about 85 000 students also took part in an optional computer-based assessment of problem solving.<ref>OECD (2014): PISA 2012 results: Creative problem solving: Students’ skills in tackling real-life problems (Volume V), http://www.oecd-ilibrary.org/education/pisa-2012-results-skills-for-life-volume-v_9789264208070-en</ref>

[[Shanghai]] had the highest score in all three subjects. It was followed by Singapore, Hong Kong, Chinese Taipei and Korea in mathematics; Hong Kong, Singapore, Japan and Korea in reading and Hong Kong, Singapore, Japan and Finland in science.

They were a sample of about 28 million in the same age group in 65 countries and economies,<ref name=oecd>[http://www.oecd.org/pisa/keyfindings/pisa-2012-results.htm PISA 2012 Results] [[OECD]]. Retrieved 4 December 2013</ref> including the [[OECD#Current members|OECD countries]], several [[China|Chinese]] cities, [[Vietnam]], [[Indonesia]] and several countries in [[South America]].<ref name=PISA2012/>

The test lasted two hours, was paper-based and included both open-ended and [[multiple-choice questions]].<ref name=oecd/>

The students and school staff also answered a [[questionnaire]] to provide background information about the students and the schools.<ref name=PISA2012/><ref name=oecd/>

PISA 2012 was presented on 3 December 2013, with results for around 510,000 participating students in all 34 OECD member countries and 31 partner countries.<ref name=PISA2012/> This testing cycle had a particular focus on mathematics, where the mean score was 494. The mean score in reading was 496 and in science 501.{{fact|date=July 2022}}

The results show distinct groups of high-performers in mathematics: the East Asian countries, with [[Shanghai]], scoring the best result of 613, followed closely by [[Hong Kong]], [[Japan]], [[Chinese Taipei]] and [[South Korea]]. Among the Europeans, [[Liechtenstein]] and [[Switzerland]] performed best, with [[Netherlands]], [[Estonia]], [[Finland]], [[Poland]], [[Belgium]], [[Germany]], [[Austria]] all posting mathematics scores "not significantly statistically different from" one another. The [[United Kingdom]], [[Ireland]], [[Australia]] and [[New Zealand]] were similarly clustered around the OECD average of 494, with the USA trailing this group at 481.<ref name=PISA2012/>

[[Qatar]], [[Kazakhstan]] and [[Malaysia]] were the countries which showed the greatest improvement in mathematics. The USA and the United Kingdom showed no significant change.<ref name=Guardian>{{citation|first1 = Ami|last1 = Sedghi|first2 = George|last2 = Arnett|first3 = Mona|last3 = Chalabi|date = 2013-12-03|title = Pisa 2012 results: which country does best at reading, maths and science?|url = https://www.theguardian.com/news/datablog/2013/dec/03/pisa-results-country-best-reading-maths-science|accessdate = 2013-02-14|work = [[The Guardian]]}}</ref> Sweden had the greatest fall in mathematics performance over the last ten years, with a similar falling trend also in the two other subjects, and leading politicians in Sweden expressed great worry over the results.<ref>{{citation|first = Richard|last = Adams|date = 2013-12-03|url = https://www.theguardian.com/world/2013/dec/03/swedish-results-fall-free-schools-pisa-oecd|title = Swedish results fall abruptly as free school revolution falters|work = [[The Guardian]]|accessdate = 2013-12-03}}</ref><ref>{{citation|first = Jens|last = Kärrman|date = 2013-12-03|url = http://www.dn.se/nyheter/sverige/lofven-om-pisa-nationell-kris/|title = Löfven om Pisa: Nationell kris|newspaper = [[Dagens Nyheter]]|accessdate = 2013-12-03}}</ref>

On average boys scored better than girls in mathematics, girls scored better than boys in reading and the two sexes had quite similar scores in science.<ref name=Guardian />

Indonesia, Albania, Peru, Thailand and Colombia were the countries where most students reported being happy at school, while students in Korea, the Czech Republic, the Slovak Republic, Estonia and Finland reported least happiness.<ref name=oecd/>

{{Clear}}
{{Collapse bottom}}

===PISA 2009===
{{collapse top|title=PISA 2009}}
The PISA 2009 cycle included results in mathematics, science and reading for all 36 OECD member countries and 37 partner countries.<ref name=PISA2009>{{citation|url = http://www.oecd.org/pisa/pisaproducts/46619703.pdf|date = 2010-12-07|publisher = [[OECD]]|title = PISA 2009 Results: Executive Summary}}</ref><ref>{{citation|url = http://pisa2009.acer.edu.au/multidim.php|title = Multi-dimensional Data Request|publisher = [[OECD]]|year = 2010|accessdate = 2012-06-28|url-status = dead|archiveurl = https://web.archive.org/web/20120714072212/http://pisa2009.acer.edu.au/multidim.php|archivedate = 2012-07-14}}</ref><ref>{{citation|url = http://www.oecd.org/dataoecd/54/12/46643496.pdf|title = PISA 2009 Results: Executive Summary (Figure 1 only)|publisher = [[OECD]]|accessdate = 2012-06-28|year = 2010}}</ref>

Of the partner countries, only selected areas of three countries—[[India]], [[Venezuela]] and [[China]]—were assessed. PISA 2009+, released in December 2011, included data from 10 additional partner countries which had testing delayed from 2009 to 2010 because of scheduling constraints.<ref name=PISA2009Plus>{{citation|url = http://www.acer.edu.au/media/acer-releases-results-of-pisa-2009-participant-economies|title = ACER releases results of PISA 2009+ participant economies|publisher = [[Australian Council for Educational Research|ACER]]|date = 2011-12-16|access-date = 2016-04-15|archive-url = https://web.archive.org/web/20141008170044/http://www.acer.edu.au/media/acer-releases-results-of-pisa-2009-participant-economies/|archive-date = 2014-10-08|url-status = dead}}</ref><ref>{{citation|title = PISA 2009 Plus Results|first = Maurice|last = Walker|year = 2011|publisher = [[OECD]]|url = https://mypisa.acer.edu.au/images/mypisadoc/acer_pisa%202009%2B%20international.pdf|archive-url = https://web.archive.org/web/20111222065347/https://mypisa.acer.edu.au/images/mypisadoc/acer_pisa%202009%2B%20international.pdf|url-status = dead|archive-date = 2011-12-22|accessdate = 2012-06-28}}</ref>
{{PISA 2009}}
{{Clear}}
{{Collapse bottom}}

===PISA 2006===
{{collapse top|title=PISA 2006}}
{{PISA 2006}}
{{Clear}}
{{collapse bottom}}

===PISA 2003===
The results for PISA 2003 were released on 14 December 2004. This PISA cycle tested 275,000 15 year-olds on mathematics, science, reading and problem solving and involved schools from 30 OECD member countries and 11 partner countries.<ref>{{citation|url = http://www.oecd.org/edu/school/programmeforinternationalstudentassessmentpisa/34002216.pdf|title = Learning for Tomorrow's World First Results from PISA 2003|publisher = [[OECD]]|date = 2004-12-14|accessdate= 2014-01-06}}</ref> Note that for Science and Reading, the means displayed are for "All Students", but for these two subjects (domains), not all of the students answered questions in these domains.  In the 2003 OECD Technical Report (pages 208, 209), there are different country means (different than those displayed below) available for students who had exposure to these domains.<ref>{{citation|url = http://www.oecd.org/edu/school/programmeforinternationalstudentassessmentpisa/35188570.pdf|title = PISA 2003 Technical Report|publisher = [[OECD]]}}</ref> 
{{collapse top|title=PISA 2003}}
{| cellpadding="3" align="left" cellspacing="0" border="1" style="border: gray solid 1px; border-collapse:collapse; font-size:85%;"
|+OECD members at the time of the study are in '''boldface'''.
|+The United Kingdom was disqualified due to a low response rate.
|-
!bgcolor="#CECEEE" |Mathematics
!bgcolor="#CECECE" |Science
!bgcolor="#FFCCCC" |Reading
!bgcolor="#CEFFCE" |Problem solving
|-
|
{| width=240
|1
|{{flagcountry|Hong Kong}}, [[China]]
|550
|-
|2
|'''{{flagcountry|Finland}}'''
|544
|-
|3
|'''{{flagcountry|Korea}}'''
|542
|-
|4
|'''{{flagcountry|Netherlands}}'''
|538
|-
|5
|{{flagcountry|Liechtenstein}}
|536
|-
|6
|'''{{flagcountry|Japan}}'''
|534
|-
|7
|'''{{flagcountry|Canada}}'''
|532
|-
|8
|'''{{flagcountry|Belgium}}'''
|529
|-
|9
|{{flagcountry|Macao}}, [[China]]
|527
|-
|10
|'''{{flagcountry|Switzerland}}'''
|527
|-
|11
|'''{{flagcountry|Australia}}'''
|524
|-
|12
|'''{{flagcountry|New Zealand}}'''
|523
|-
|13
|'''{{flagcountry|Czech Republic}}'''
|516
|-
|14
|'''{{flagcountry|Iceland}}'''
|515
|-
|15
|'''{{flagcountry|Denmark}}'''
|514
|-
|16
|'''{{flagcountry|France}}'''
|511
|-
|17
|'''{{flagcountry|Sweden}}'''
|509
|-
|18
|'''{{flagcountry|Austria}}'''
|506
|-
|19
|'''{{flagcountry|Germany}}'''
|503
|-
|20
|'''{{flagcountry|Ireland}}'''
|503
|-
|21
|'''{{flagcountry|Slovakia}}'''
|498
|-
|22
|'''{{flagcountry|Norway}}'''
|495
|-
|23
|'''{{flagcountry|Luxembourg}}'''
|493
|-
|24
|'''{{flagcountry|Poland}}'''
|490
|-
|25
|'''{{flagcountry|Hungary}}'''
|490
|-
|26
|'''{{flagcountry|Spain}}'''
|485
|-
|27
|{{flagcountry|Latvia}}
|483
|-
|28
|'''{{flagcountry|United States}}'''
|483
|-
|29
|{{flagcountry|Russia}}
|468
|-
|30
|'''{{flagcountry|Portugal}}'''
|466
|-
|31
|'''{{flagcountry|Italy|2003}}'''
|466
|-
|32
|'''{{flagcountry|Greece}}'''
|445
|-
|33
|{{flagcountry|Serbia and Montenegro}}
|437
|-
|34
|'''{{flagcountry|Turkey}}'''
|423
|-
|35
|{{flagcountry|Uruguay}}
|422
|-
|36
|{{flagcountry|Thailand}}
|417
|-
|37
|'''{{flagcountry|Mexico}}'''
|385
|-
|38
|{{flagcountry|Indonesia}}
|360
|-
|39
|{{flagcountry|Tunisia}}
|359
|-
|40
|{{flagcountry|Brazil}}
|356
|-
|}
|
{| width=240
|1
|'''{{flagcountry|Finland}}'''
|548
|-
|2
|'''{{flagcountry|Japan}}'''
|548
|-
|3
|{{flagcountry|Hong Kong}}, [[China]]
|539
|-
|4
|'''{{flagcountry|Korea}}'''
|538
|-
|5
|{{flagcountry|Liechtenstein}}
|525
|-
|6
|'''{{flagcountry|Australia}}'''
|525
|-
|7
|{{flagcountry|Macao}}, [[China]]
|525
|-
|8
|'''{{flagcountry|Netherlands}}'''
|524
|-
|9
|'''{{flagcountry|Czech Republic}}'''
|523
|-
|10
|'''{{flagcountry|New Zealand}}'''
|521
|-
|11
|'''{{flagcountry|Canada}}'''
|519
|-
|12
|'''{{flagcountry|Switzerland}}'''
|513
|-
|13
|'''{{flagcountry|France}}'''
|511
|-
|14
|'''{{flagcountry|Belgium}}'''
|509
|-
|15
|'''{{flagcountry|Sweden}}'''
|506
|-
|16
|'''{{flagcountry|Ireland}}'''
|505
|-
|17
|'''{{flagcountry|Hungary}}'''
|503
|-
|18
|'''{{flagcountry|Germany}}'''
|502
|-
|19
|'''{{flagcountry|Poland}}'''
|498
|-
|20
|'''{{flagcountry|Slovakia}}'''
|495
|-
|21
|'''{{flagcountry|Iceland}}'''
|495
|-
|22
|'''{{flagcountry|United States}}'''
|491
|-
|23
|'''{{flagcountry|Austria}}'''
|491
|-
|24
|{{flagcountry|Russia}}
|489
|-
|25
|{{flagcountry|Latvia}}
|489
|-
|26
|'''{{flagcountry|Spain}}'''
|487
|-
|27
|'''{{flagcountry|Italy|2003}}'''
|486
|-
|28
|'''{{flagcountry|Norway}}'''
|484
|-
|29
|'''{{flagcountry|Luxembourg}}'''
|483
|-
|30
|'''{{flagcountry|Greece}}'''
|481
|-
|31
|'''{{flagcountry|Denmark}}'''
|475
|-
|32
|'''{{flagcountry|Portugal}}'''
|468
|-
|33
|{{flagcountry|Uruguay}}
|438
|-
|34
|{{flagcountry|Serbia and Montenegro}}
|436
|-
|35
|'''{{flagcountry|Turkey}}'''
|434
|-
|36
|{{flagcountry|Thailand}}
|429
|-
|37
|'''{{flagcountry|Mexico}}'''
|405
|-
|38
|{{flagcountry|Indonesia}}
|395
|-
|39
|{{flagcountry|Brazil}}
|390
|-
|40
|{{flagcountry|Tunisia}}
|385
|-
|}
|
{| width=240
|1
|'''{{flagcountry|Finland}}'''
|543
|-
|2
|'''{{flagcountry|Korea}}'''
|534
|-
|3
|'''{{flagcountry|Canada}}'''
|528
|-
|4
|'''{{flagcountry|Australia}}'''
|525
|-
|5
|{{flagcountry|Liechtenstein}}
|525
|-
|6
|'''{{flagcountry|New Zealand}}'''
|522
|-
|7
|'''{{flagcountry|Ireland}}'''
|515
|-
|8
|'''{{flagcountry|Sweden}}'''
|514
|-
|9
|'''{{flagcountry|Netherlands}}'''
|513
|-
|10
|{{flagcountry|Hong Kong}}, [[China]]
|510
|-
|11
|'''{{flagcountry|Belgium}}'''
|507
|-
|12
|'''{{flagcountry|Norway}}'''
|500
|-
|13
|'''{{flagcountry|Switzerland}}'''
|499
|-
|14
|'''{{flagcountry|Japan}}'''
|498
|-
|15
|{{flagcountry|Macao}}, [[China]]
|498
|-
|16
|'''{{flagcountry|Poland}}'''
|497
|-
|17
|'''{{flagcountry|France}}'''
|496
|-
|18
|'''{{flagcountry|United States}}'''
|495
|-
|19
|'''{{flagcountry|Denmark}}'''
|492
|-
|20
|'''{{flagcountry|Iceland}}'''
|492
|-
|21
|'''{{flagcountry|Germany}}'''
|491
|-
|22
|'''{{flagcountry|Austria}}'''
|491
|-
|23
|{{flagcountry|Latvia}}
|491
|-
|24
|'''{{flagcountry|Czech Republic}}'''
|489
|-
|25
|'''{{flagcountry|Hungary}}'''
|482
|-
|26
|'''{{flagcountry|Spain}}'''
|481
|-
|27
|'''{{flagcountry|Luxembourg}}'''
|479
|-
|28
|'''{{flagcountry|Portugal}}'''
|478
|-
|29
|'''{{flagcountry|Italy|2003}}'''
|476
|-
|30
|'''{{flagcountry|Greece}}'''
|472
|-
|31
|'''{{flagcountry|Slovakia}}'''
|469
|-
|32
|{{flagcountry|Russia}}
|442
|-
|33
|'''{{flagcountry|Turkey}}'''
|441
|-
|34
|{{flagcountry|Uruguay}}
|434
|-
|35
|{{flagcountry|Thailand}}
|420
|-
|36
|{{flagcountry|Serbia and Montenegro}}
|412
|-
|37
|{{flagcountry|Brazil}}
|403
|-
|38
|'''{{flagcountry|Mexico}}'''
|400
|-
|39
|{{flagcountry|Indonesia}}
|382
|-
|40
|{{flagcountry|Tunisia}}
|375
|-
|}
|
{| width=240
|1
|'''{{flagcountry|Korea}}'''
|550
|-
|2
|{{flagcountry|Hong Kong}}, [[China]]
|548
|-
|3
|'''{{flagcountry|Finland}}'''
|548
|-
|4
|'''{{flagcountry|Japan}}'''
|547
|-
|5
|'''{{flagcountry|New Zealand}}'''
|533
|-
|6
|{{flagcountry|Macao}}, [[China]]
|532
|-
|7
|'''{{flagcountry|Australia}}'''
|530
|-
|8
|{{flagcountry|Liechtenstein}}
|529
|-
|9
|'''{{flagcountry|Canada}}'''
|529
|-
|10
|'''{{flagcountry|Belgium}}'''
|525
|-
|11
|'''{{flagcountry|Switzerland}}'''
|521
|-
|12
|'''{{flagcountry|Netherlands}}'''
|520
|-
|13
|'''{{flagcountry|France}}'''
|519
|-
|14
|'''{{flagcountry|Denmark}}'''
|517
|-
|15
|'''{{flagcountry|Czech Republic}}'''
|516
|-
|16
|'''{{flagcountry|Germany}}'''
|513
|-
|17
|'''{{flagcountry|Sweden}}'''
|509
|-
|18
|'''{{flagcountry|Austria}}'''
|506
|-
|19
|'''{{flagcountry|Iceland}}'''
|505
|-
|20
|'''{{flagcountry|Hungary}}'''
|501
|-
|21
|'''{{flagcountry|Ireland}}'''
|498
|-
|22
|'''{{flagcountry|Luxembourg}}'''
|494
|-
|23
|'''{{flagcountry|Slovakia}}'''
|492
|-
|24
|'''{{flagcountry|Norway}}'''
|490
|-
|25
|'''{{flagcountry|Poland}}'''
|487
|-
|26
|{{flagcountry|Latvia}}
|483
|-
|27
|'''{{flagcountry|Spain}}'''
|482
|-
|28
|{{flagcountry|Russia}}
|479
|-
|29
|'''{{flagcountry|United States}}'''
|477
|-
|30
|'''{{flagcountry|Portugal}}'''
|470
|-
|31
|'''{{flagcountry|Italy|2003}}'''
|469
|-
|32
|'''{{flagcountry|Greece}}'''
|448
|-
|33
|{{flagcountry|Thailand}}
|425
|-
|34
|{{flagcountry|Serbia and Montenegro}}
|420
|-
|35
|{{flagcountry|Uruguay}}
|411
|-
|36
|'''{{flagcountry|Turkey}}'''
|408
|-
|37
|'''{{flagcountry|Mexico}}'''
|384
|-
|38
|{{flagcountry|Brazil}}
|371
|-
|39
|{{flagcountry|Indonesia}}
|361
|-
|40
|{{flagcountry|Tunisia}}
|345
|-
|}
|}
{{Clear}}
{{Collapse bottom}}

===PISA 2000===
The results for the first cycle of the PISA survey were released on 14 November 2001. 265,000 15 year-olds were tested in 28 OECD countries and 4 partner countries on mathematics, science and reading. An additional 11 countries were tested later in 2002.<ref>{{citation|url = http://www.oecd.org/edu/school/programmeforinternationalstudentassessmentpisa/33690591.pdf|title = Literacy Skills for the World of Tomorrow: Further Results from PISA 2000|publisher = [[OECD]]|year = 2003|accessdate = 2014-01-06}}</ref>
{{collapse top|title=PISA 2000}}
{{PISA 2000 2003 2006}}
{{Clear}}
{{collapse bottom}}

==Comparison with other studies==
The [[Pearson product-moment correlation coefficient|correlation]] between PISA 2003 and TIMSS 2003 grade 8 country means is 0.84 in mathematics, 0.95 in science. The values go down to 0.66 and 0.79 if the two worst performing developing countries are excluded. Correlations between different scales and studies are around 0.80. The high correlations between different scales and studies indicate common causes of country differences (e.g. educational quality, culture, wealth or genes) or a homogenous underlying factor of cognitive competence. [[European Economic Area]] countries perform slightly better in PISA; the [[Commonwealth of Independent States]] and Asian countries in TIMSS. Content balance and years of schooling explain most of the variation.<ref>M. L. Wu: ''A Comparison of PISA and TIMSS 2003 achievement results in Mathematics.'' Paper presented at the AERA Annual Meeting, New York, March 2008.</ref>

==Reception==
The results from PISA 2003 and PISA 2006 were featured in the 2010 documentary [[Waiting for "Superman"]].<ref>{{cite web|url=https://www.youtube.com/watch?v=ZKTfaro96dg|title=Waiting for "Superman" trailer|website=[[YouTube]] |date=7 May 2010 |accessdate=8 October 2010}}</ref>

===China===
Education professor Yong Zhao has noted that PISA 2009 did not receive much attention in the Chinese media, and that the high scores in China are due to excessive workload and testing, adding that it's "no news that the [[Chinese Education System|Chinese education system]] is excellent in preparing outstanding test takers, just like other education systems within the [[East Asian cultural sphere|Confucian cultural circle]]: Singapore, Korea, Japan, and Hong Kong."<ref name=zhao>{{citation|url = http://zhaolearning.com/2010/12/10/a-true-wake-up-call-for-arne-duncan-the-real-reason-behind-chinese-students-top-pisa-performance|title = A True Wake-up Call for Arne Duncan: The Real Reason Behind Chinese Students Top PISA Performance|author = Yong Zhao|date = 10 December 2010}}</ref>

Students from [[Shanghai]], [[China]], had the top scores of every category ([[Mathematics]], [[Reading (process)|Reading]] and [[Science]]) in PISA 2009. In discussing these results, PISA spokesman [[Andreas Schleicher]], Deputy Director for Education and head of the analysis division at the OECD’s directorate for education, described Shanghai as a pioneer of educational reform in which "there has been a sea change in pedagogy". Schleicher stated that Shanghai abandoned its "focus on educating a small elite, and instead worked to construct a more inclusive system. They also significantly increased teacher pay and training, reducing the emphasis on rote learning and focusing classroom activities on problem solving."<ref>{{citation|url = http://www.time.com/time/world/article/0,8599,2035586,00.html#ixzz17XACd2S2|title = China Beats Out Finland for Top Marks in Education|first = Peter|last = Gumbel|magazine = [[Time (magazine)|Time]]|date = 7 December 2010|archive-url = https://web.archive.org/web/20101210114159/http://www.time.com/time/world/article/0,8599,2035586,00.html#ixzz17XACd2S2|accessdate = 27 June 2012|archive-date = 10 December 2010}}</ref>

Schleicher also states that PISA tests administered in rural China have produced some results approaching the OECD average: Citing further, as-yet-unpublished OECD research, Schleicher said, "We have actually done Pisa in 12 of the provinces in China. Even in some of the very poor areas you get performance close to the OECD average."<ref name="ft-1">{{citation|url = http://www.ft.com/cms/s/0/20770bf6-01e7-11e0-b66c-00144feabdc0.html|work = [[Financial Times]]|date = 7 December 2010|title = Shanghai tops global state school rankings|first = Chris|last = Cook|accessdate = 28 June 2012}}</ref> Schleicher says that for a developing country, China's 99.4% enrollment in primary education is "the envy of many countries". He maintains that junior secondary school participation rates in China are now 99%; and in Shanghai, not only has senior secondary school enrollment attained 98%, but admissions into higher education have achieved 80% of the relevant age group. Schleicher believes that this growth reflects quality, not just quantity, which he contends the top PISA ranking of Shanghai's secondary education confirms.<ref name="ft-1" /> Schleicher believes that China has also expanded school access and has moved away from learning by rote.<ref>{{citation|url = http://blogs.ft.com/beyond-brics/2010/12/07/why-are-chinese-schoolkids-so-good/|title = Why are Chinese schoolkids so good?|first = Henry|last = Mance|date = 7 December 2010|accessdate = 28 June 2012|work = [[Financial Times]]}}</ref>  According to Schleicher, Russia performs well in rote-based assessments, but not in PISA, whereas China does well in both rote-based and broader assessments.<ref name="ft-1" />

===Denmark===
[[University of Copenhagen]] Professor Svend Kreiner, who examined in detail PISA's 2006 reading results, noted that in 2006 only about ten percent of the students who took part in PISA were tested on all 28 reading questions. "This in itself is ridiculous," Kreiner told Stewart. "Most people don't know that half of the students taking part in PISA (2006) do not respond to any reading item at all. Despite that, PISA assigns reading scores to these children."<ref>{{Cite web |url=https://ifsv.sund.ku.dk/biostat/biostat_annualreport/images/c/ca/ResearchReport-2011-1.pdf |title=Archived copy |access-date=15 April 2016 |archive-date=4 March 2016 |archive-url=https://web.archive.org/web/20160304034725/https://ifsv.sund.ku.dk/biostat/biostat_annualreport/images/c/ca/ResearchReport-2011-1.pdf |url-status=dead }}</ref>

=== Finland ===
The stable, high marks of Finnish students have attracted a lot of attention. According to Hannu Simola<ref>{{citation|last = Simola|first = Hannu|year = 2005|url = http://www.mv.helsinki.fi/home/hsimola/Finnish%20PISA%20Miracle%2005.pdf|title = The Finnish miracle of PISA: Historical and sociological remarks on teaching and teacher education|journal = Comparative Education|volume = 41|issue = 4|pages = 455–470|doi = 10.1080/03050060500317810|s2cid = 145325152}}</ref> the results reflect a paradoxical mix of progressive policies implemented through a rather conservative pedagogic setting, where the high levels of teachers' academic preparation, social status, professionalism and motivation for the job are concomitant with the adherence to traditional roles and methods by both teachers and pupils in Finland's changing, but still quite paternalistic culture. Others advance Finland's low poverty rate as a reason for its success.<ref name=NEA /><ref name=NSSP /> Finnish education reformer Pasi Sahlberg attributes Finland's high educational achievements to its emphasis on social and educational equality and stress on cooperation and collaboration, as opposed to the competition among teachers and schools that prevails in other nations.<ref>{{cite web|last=Cleland |first=Elizabeth |url=https://www.theatlantic.com/national/archive/2011/12/what-americans-keep-ignoring-about-finlands-school-success/250564/ |title=What Americans Keep Ignoring About Finland's School Success – Anu Partanen |work=The Atlantic |date=29 December 2011 }}</ref>

=== India ===
Of the 74 countries tested in the PISA 2009 cycle including the "+" nations, the two Indian states came up 72nd and 73rd out of 74 in both reading and mathematics, and 73rd and 74th in science. India's poor performance may not be linguistic as some suggested. 12.87% of US students, for example, indicated that the language of the test differed from the language spoken at home. while 30.77% of Himachal Pradesh students indicated that the language of the test differed from the language spoken at home, a significantly higher percent<ref name="autogenerated1">{{cite web |url=http://pisa2009.acer.edu.au/interactive_results.php |title=Database – PISA 2009 |publisher=Pisa2009.acer.edu.au |access-date=2016-04-15 |archive-url=https://web.archive.org/web/20160322032020/http://pisa2009.acer.edu.au/interactive_results.php |archive-date=2016-03-22 |url-status=dead }}</ref> However, unlike American students, those Indian students with a different language at home did better on the PISA test than those with the same language.<ref name="autogenerated1"/> India's poor performance on the PISA test is consistent with India's poor performance in the only other instance when India's government allowed an international organization to test its students<ref>http://ddp-ext.worldbank.org/EdStats/INDprwp08b.pdf {{Bare URL PDF|date=March 2022}}</ref> and consistent with India's own testing of its elite students in a study titled ''Student Learning in the Metros 2006.'' <ref>{{citation|url = http://www.ei-india.com/wp-content/uploads/2012/07/Student-Learning-in-the-Metros-Issue-2.pdf|title = Student Learning in the Metros|first = Educational|last = Initiatives|date = November 2006|work = Educational Initiatives}}</ref> These studies were conducted using TIMSS questions.  The poor result in PISA was greeted with dismay in the Indian media.<ref>{{citation|url = http://www.indianexpress.com/news/poor-pisa-ranks-hrd-seeks-reason/896777|title = Poor PISA ranks: HRD seeks reason|first = Anubhuti|last = Vishnoi|date = 7 January 2012|work = The Indian Express}}</ref> The BBC reported that as of 2008, only 15% of India's students reach high school.<ref>{{cite news|url = http://news.bbc.co.uk/2/hi/business/7267315.stm|publisher = BBC News|first = Zareer|last = Masani|title = India still Asia's reluctant tiger|date = 27 February 2008}}</ref>

===Italy / South Tyrol===
In 2003 [[South Tyrol]] (Provincia Autonoma di Bolzano / Autonome Provinz Bozen), a predominantly German-speaking province in the north of Italy, took part in the PISA project for the first time in order to have a regional result as an ''adjudicated region''. In the rest of Italy PISA is conducted by INVALSI (Istituto nazionale per la valutazione del sistema educativo di istruzione e di formazione), a formally independent research institution affiliated to the Ministry of Education, whereas in South Tyrol PISA was carried out by the regional Education Authority itself (''Intendenza scolastica'' / ''Schulamt'', since 2018 renamed into ''Bildungsdirektion''),<ref>Cf. http://www.provincia.bz.it/bildung-sprache/deutschsprachige-schule/mitteilungen.asp?publ_action=300&publ_image_id=469121. Retrieved 11 April 2021.</ref> which is part of the South Tyrolean regional government.
At the end of 2004, in the months prior to the announcement of the test results, the regional Education Authority in Bolzano / Bozen downplayed the validity of the PISA assessment and commissioned alternative school evaluations, preparing the public for a mediocre test result. According to the official PISA report 2003, however, South Tyrol seemed to even beat the PISA world champion Finland.

'''Critique'''

Right from the beginning, there was scepticism as to how South Tyrol succeeded in outdoing the neighbouring Italian and Austrian provinces. On the front page of its weekend edition for 29/30 January 2005, the South Tyrolean newspaper ''Neue Südtiroler Tageszeitung'' published a harsh critique and revealed that the South Tyrolean Education Authority had secretly eliminated more than 300 students from the 1500 students officially drawn as South Tyrolean test sample by the PISA Consortium, and soon more inconsistencies were to surface:
* '''Lack of independence of the South Tyrolean PISA board:''' In South Tyrol, PISA was not conducted by a nominally independent body like the local university ''Freie Universität Bozen'' or the European Academy EURAC, both of which have ample expertise in the field of education, but by the so-called Pedagogical Institute (Pädagogisches Institut, director: Rudolf Meraner), which was part of the Education Authority, which in its turn was part of the regional government. A few years later the Pedagogical Institute was nominally absorbed into the Education Authority and renamed into Department for Innovation and Counselling (with the same director: Rudolf Meraner).
* '''Exploitation for political ends''': In the most influential mass media, the regional PISA results were presented as a triumph of the regional government and the SVP (''Südtiroler Volkspartei'' or South Tyrolean People’s Party) ruling party’s policy, although the legal framework for all high schools in Italy is a purely national domain. For all the political autonomy granted to South Tyrol, this region still has the same types of schools and follows the same curricula as do all Italian regions. Mass media like the most-read South Tyrolean newspaper, ''Dolomiten'', whose owner Michel Ebner is a prominent party member of the ruling SVP, did not either try to explain why the secondary schools attended by the Italian speaking students in the same Province did considerably worse, although it is the same regional government run by the SVP which is in charge of the Italian and the German school administration.<ref>Cf. ''INFO,'' December 2004 (i.e. a Circular Letter edited by the regional Education Authority): «In all fields, the South Tyrolean schools achieved a first-rate performance» (p. 2); Mr Höllrigl, then director of the Education Authority, and Mr Meraner, then head of the PISA board, in ''INFO,'' January 2005: «I am surprised that we have already become world leaders» (p. 12); Mr Meraner in the most-read daily ''Dolomiten,'' 18 February 2005: «We are the world champions even in Problem Solving».</ref>
* '''Harassment of Critics:''' People who criticized the official PISA results and pointed out violations of the technical rules were officially threatened by the provincial governor, Luis Durnwalder, with libel action for slandering South Tyrol. On 16 March 2006, Durnwalder announced in a press conference that an Austrian teacher would be prosecuted and sued for damages simply because the teacher, in a letter sent to the Austrian Ministry of Education, had mentioned the fact that the South Tyrolean Education Authority had eliminated 17 percent of the students from the regional sample, thereby rendering invalid the regional PISA result. A year later, however, Mr Durnwalder had to admit that he had never taken legal action against the teacher and that there were no legal proceedings obviously because the critique was correct.<ref>Cf. the South Tyrolean weekly ''FF,'' 17 February 2005, p. 10: "Land klagt Lehrer [Regional Government’s Action against Teacher", and ''FF,'' 16 March 2006, in which Mr Durnwalder admits to the failed legal suit.</ref> It is also noteworthy that the director of the Pedagogical Institute, Rudolf Meraner, and others have constantly deleted the original German Wikipedia article about the South Tyrolean PISA results, replacing it with public government statements.
* '''Regional results deliberately misrepresented as national results:''' By definition, the PISA result of South Tyrol is a ''subregional result,'' which is not fully valid because of the small sample of 1500 students. Such regional results mainly serve documentary purposes and cannot be compared with national results. Nonetheless, the South Tyrolean Education Authority and the regional government repeatedly, and falsely, claimed that the South Tyrolean test results are national, i.e. fully valid results, whereas the neighbouring regions like Tyrol (Austria) and Trentino (Italy), according to South Tyrolean press releases, only had subnational results.<ref>Cf. ''Dolomiten,'' 27 January 2005: Mr Hilpold misinformed the press on behalf of the regional government: «South Tyrol was assessed as a nation [Land]. It is due to the fact that we were assessed as a nation that we may compare our results with other nations.» Mr Meraner, director of the Pedagogical Institute, also wrongly claimed that the South Tyrolean overall result may be compared to that of other «nations» because, as he falsely stated, South Tyrol had a national result of its own.</ref>
* '''Manipulation of the sample: Elimination of 17 percent of students:''' For a subnational result, regions had to test 1500 students. Among all regions with a subnational result, worldwide, South Tyrol was the only one that failed to test 1500 students. For reasons never specified, the Education Authority had eliminated 292 students (i.e. 17 percent) from the 1500 students sample, testing only 1208 of the students selected by the PISA consortium. Failing to test all 1500 students, the South Tyrolean Education Authority violated the technical PISA rules and deprived the PISA result even of its limited technical validity as a subnational result. Later on the Education Authority had to admit that it had actually excluded all vocational students, whose performance is generally considered to be inferior to that of high schools, and all students of the third forms of so-called middle schools (scuole medie / Mittelschulen), who would have been part of the sample only if they repeated a class and were considered underachievers because normally 15 year old students are enrolled at high schools. From the official PISA report this manipulation can be easily deduced as the difference between ''total population of 15-year-olds'' (4.908) and ''total enrolled population of 15-year-olds at grade 7 or above'' (4.087).<ref>Cf. the PISA report: ''Learning for Tomorrow’s World. First Results from PISA 2003.'' Paris, 2004. p. 469; online version: http://www.oecd.org/dataoecd/1/60/34002216.pdf. Retrieved 8 March 2012.</ref> In subsequent PISA assessments, the OECD or rather the PISA consortium did not publish these key figures so that it was no longer possible to ascertain the number of students eliminated from the sample against the rules.
* '''Incorrect figures about ''Target Population'' and actual sample:''' In Italy, according to the figures officially made available by the INVALSI, the number of all 15-year-old students (574.611) paradoxically exceeds the number of all 15-year-old people (561.304).<ref>Cf. the PISA report: ''Learning for Tomorrow’s World. First Results from PISA 2003.'' Paris, 2004. p. 321; online version: http://www.oecd.org/dataoecd/1/60/34002216.pdf.</ref> Due to this mistake, it is impossible to establish how many Italian students were actually identified as Italian ''target population'', and it is also impossible to find out which percentage of South Tyrolean students had actually been presented as ''target population'' to the PISA consortium. In theory, the Education Authority could have excluded certain types of students from the target population even before the sample of 1500 students was drawn by the PISA consortium, from which the Education Authority later on eliminated 17 percent.

'''Comparison with similar assessments'''

The stunning South Tyrolean 2003 PISA results can hardly be reconciled with similar high school evaluations, which were not conducted or influenced by the South Tyrolean Education Authority itself. Three international or national large scale assessment projects painted a gloomy picture of the South Tyrolean students’ performance.
* '''Admission Test at Austrian Universities of Medicine (EMS = Eignungstest für das Medizinstudium):''' Over the last two decades, candidates graduating from South Tyrolean high schools have traditionally scored very badly at the entry examination. The quota system at Austrian Universities of Medicine makes sure that 75 percent of all applicants admitted to Medicine are Austrian nationals or students from South Tyrol, who are considered an Austrian minority in Italy. Hence, for Austrians and South Tyroleans it is easier to get a place at an Austrian University of Medicine, even with bad scores, than for Germans and other EU nationals because the number of EU students must not exceed 20 percent. Nevertheless, according to a study financed by the Austrian Ministry of Education, not even within the big 75 percent quota, South Tyroleans could get a place because of their mediocre entrance test results. South Tyrolean applicants had an average score of 96,5 whereas e.g. German applicants had an average score of 103,1. Interestingly, South Tyrolean applicants did even worse than Austrian applicants although Italian high school graduates are one year older. What is even worse, South Tyrolean high school graduates show a sharp gender gap with female candidates doing much worse than their male colleagues, which reflects the Italian high school system: Certain types of high schools popular with female students (e.g. ''licei linguistici, licei scienze umane'') have little mathematics and sciences. The difference between the average female and the average male South Tyrolean score in the 2007 EMS was dramatic: 94,8 versus 100,1 points.<ref>Cf. the EMS study published by the Austrian Ministry of Research in 2007: http://www.bmwf.gv.at/startseite/mini_menue/service/publikationen/wissenschaft/universitaetswesen/spiel_studie/. Retrieved 1 April 2012. Abridged version: https://docplayer.org/18915390-Evaluation-der-eignungstests-fuer-das-medizinstudium-in-oesterreich.html = Evaluation der Eignungstests für das Medizinstudium in Österreich - PDF Free Download (docplayer.org) – retrieved 10 January 2021.</ref> Later on the Swiss EMS organisation disallowed the Austrian Universities of Medicine to use its EMS test and the Austrian universities introduced a new entrance test scheme with an extra quota for female students. South Tyrolean mass media did not cover the EMS debacle at all. Within the South Tyrolean parliament, however, there have been debates as to the reasons of the poor performance of South Tyrolean high school students.<ref>E.g. in the year 2007: c.f. the parliamentary question of an opposition party: https://suedtiroler-freiheit.com/2007/08/16/landtagsanfrage-zu-den-medizinstudium-ausbildungsplaetzen. Retrieved 6 April 2021. Or the more recent article in an online newspaper: https://www.salto.bz/de/article/25082016/braucht-suedtirol-die-oesterreicher-quote. Retrieved 6 April 2021.</ref> As a consequence of the EMS debacle, the former South Tyrolean governor, Luis Durnwalder, envisaged a collaboration between the University of Medicine in Innsbruck and the Hospital in Bolzano (Bozen) which should have led to a Euregio Medical School open to all North Tyrolean (i.e. Austrian) and South Tyrolean applicants without any entrance exam. Eventually, the South Tyrolean government adopted a more pragmatic approach, it subsidized, as it were, extra places for South Tyrolean students. This seems to be a breach of the national quota system and the entrance requirements as outlined by Austrian laws. The former South Tyrolean governor, Luis Durnwalder, however, frankly admitted to resort to this solution when he complained about the high amount of money South Tyrol paid to the University of Medicine in Innsbruck for buying extra places for South Tyrolean students.<ref>Cf. the interview in the Austrian daily ''Tiroler Tageszeitung,'' 3 November 2008, p. 4.</ref>
* '''German DESI assessment of linguistic skills in German (mother tongue) and English (foreign language):''' Parallel with the 2003 PISA assessment, the South Tyrolean Education Authority commissioned a second assessment, obviously because the Education Authority expected a mediocre PISA result and had downplayed the importance of PISA. Like PISA, the DESI assessment was conducted against the technical rules because, again, all vocational students attending a vocational school (''Berufsschule'') and working part-time in a workshop or small firm, i.e. one third of the target population was excluded from the test. The South Tyrolean Education Authority also excluded the whole English test section from DESI, thereby depriving DESI of its main purpose, namely a comparison of German speaking students’ language competence in L1 and L2. The South Tyrolean Education Authority argued that in (bilingual) South Tyrol, German speaking students start with Italian as their first foreign language at elementary schools. This explanation, however, lacks credibility because in Germany, as well, many students assessed by DESI had main foreign languages other than English, e.g. French or even Latin (Bavaria), and the amount of weekly classroom teaching in English from the first form at elementary schools is the same in Germany and South Tyrol. In their official report, the German experts responsible for the DESI assessment in South Tyrol generally praised the quality of teaching, but the concrete results in the report reveal drastic shortcomings. For instance, in the semantic field of ''railway station'' (''Bahnhof''), not even one South Tyrolean student with German as his or her mother tongue knew the German word for a signal box (''Stellwerk''). Instead of marking such errors as errors, however, the German experts resorted to a methodologically disputable assumption. They claimed, without checking, that all words which South Tyrolean students did not know are, by definition, not used in the South Tyrolean variant of German and that these words must be excluded from the test (''item bias''), thus counting only correct answers. The DESI testers from Germany, however, did not check if the words excluded from DESI because of ''item bias'' actually were unknown. In fact, all words excluded from the South Tyrolean DESI questionnaire are common German words used in South Tyrol as well, i.e. for these words there are no South Tyrolean variants at all. For all the technical exceptions and modifications, the 2003 South Tyrolean DESI result was very disappointing. Only 14 percent of the South Tyrolean high school students came into the best achievement group, whereas in Germany almost half of the 15 year old students belong to this group. On the other hand, one fourth of the South Tyrolean German speaking students came into the lowest achievement group, which in Germany, in spite of all social problems in big towns, comprises only 7 percent of all students.<ref>An abridged version of the South Tyrolean DESI report was published by the Pedagogical Institute on its site: http://www.provinz.bz.it/news/de/news.asp?news_action=5&news_article_id=138926. Retrieved 8 April 2021.</ref>
* '''Italian INVALSI assessments:''' Traditionally, the annual assessments conducted by the Italian INVALSI in the pre-PISA period painted a sorry picture of the northernmost Italian ''Regione Autonoma di Trentino e Alto Adige'' comprising the two autonomous provinces called Trentino (Italian speaking) and South Tyrol (predominantly German speaking). Schools in Trentino and South Tyrol did constantly worse than those in all other North Italian regions and even lagged behind the Italian average, though it is not clear if the sample of German speaking students was representative because, again, the South Tyrolean Education Authority was entitled to eliminate bad schools from the sample, thereby manipulating the validity of the assessment.<ref>Cf. the reports published by INVALSI, or its predecessor SNV, Servizio nazionale di evaluazione: http://www.invalsi.it/invalsi/index.php?page=snv. Retrieved 27 December 2009.</ref> To some extent, two South Tyrolean anomalies may account for the traditional bad performance of South Tyrolean students in international evaluations. Until recently, no pedagogical qualifications were required for teachers, not even for permanently employed teachers (''insegnanti di ruolo / Stammrollenlehrer'') appointed by the Education Authority in Bolzano (Bozen) through a so-called ''concorso''. Besides, the admission criteria for the South Tyrolean ''concorsi'' have always been inconsistent. For example, a South Tyrolean student who studied at an Italian university German as a Foreign Language or Art History, automatically obtained the teaching license for completely unrelated subjects, like History and Latin, at South Tyrolean high schools.

===United States===
Two studies have compared high achievers in mathematics on the PISA and those on the U.S. [[National Assessment of Educational Progress]] (NAEP). Comparisons were made between those scoring at the "advanced" and "proficient" levels in mathematics on the NAEP with the corresponding performance on the PISA.  Overall, 30 nations had higher percentages than the U.S. of students at the "advanced" level of mathematics. The only [[OECD]] countries with worse results were Portugal, Greece, Turkey, and Mexico. Six percent of U.S. students were "advanced" in mathematics compared to 28 percent in Taiwan.  The highest ranked state in the U.S. (Massachusetts) was just 15th in the world if it was compared with the nations participating in the PISA. 31 nations had higher percentages of "proficient" students than the U.S. Massachusetts was again the best U.S. state, but it ranked just ninth in the world if compared with the nations participating in the PISA.<ref>[[Paul E. Peterson]], Ludger Woessmann, [[Eric Hanushek|Eric A. Hanushek]], and Carlos X. Lastra-Anadón (2011) "Are U.S. students ready to compete?  The latest on each state's international standing." ''Education Next'' 11:4 (Fall): 51–59. http://educationnext.org/are-u-s-students-ready-to-compete/</ref><ref>[[Eric Hanushek|Eric A. Hanushek]], [[Paul E. Peterson]], and Ludger Woessmann (2011) "Teaching math to the talented." ''Education Next'' 11, no. 1 (Winter): 10–18. http://educationnext.org/teaching-math-to-the-talented/</ref>

Comparisons with results for the [[Trends in International Mathematics and Science Study]] (TIMSS) appear to give different results—suggesting that the U.S. states actually do better in world rankings.<ref>Gary W. Phillips (2007) ''Chance favors the prepared mind: Mathematics and science indicators for comparing states.'' Washington: American Institutes for Research (14 November); Gary W. Phillips (2009) ''The Second Derivative:International Benchmarks in Mathematics For U.S. States and School Districts.'' Washington, DC: American Institutes for Research (June).</ref> This can likely be traced to the different material being covered and the United States teaching mathematics in a style less harmonious with the "Realistic Mathematics Education" which forms the basis of the exam.<ref>{{cite web|title=PISA Mathematics: A Teacher's Guide|date=13 August 2019 |url=https://www.education.ie/en/Publications/Inspection-Reports-Publications/Evaluation-Reports-Guidelines/insp_pisa_maths_teach_guide_pdf.pdf}}</ref>  Countries that commonly use this teaching method score higher on PISA, and less highly on TIMSS and other assessments.<ref>{{cite web|last1=Loveless|first1=Tom|title=International Tests Are Not All the Same|url=http://www.brookings.edu/research/papers/2013/01/09-timss-pisa-loveless|website=Brookings Institution}}</ref>

====Poverty====
Stephen Krassen, professor emeritus at the University of Southern California,<ref>quoted in [https://web.archive.org/web/20101219034611/http://voices.washingtonpost.com/answer-sheet/research/how-poverty-affected-us-pisa-s.html Valerie Strauss, "How poverty affected U.S. PISA scores", ''The Washington Post'', 9 December 2010.]</ref> and Mel Riddile of the [[National Association of Secondary School Principals|NASSP]] attributed the relatively low performance of students in the United States to the country's high rate of child [[Poverty in the United States|poverty]], which exceeds that of other OECD countries.<ref name=NEA>"[http://neatoday.org/2010/12/09/a-look-at-the-economic-numbers-on-international-education-rankings/ The Economics Behind International Education Rankings]" National Educational Association</ref><ref name=NSSP>{{citation|url = http://nasspblogs.org/principaldifference/2010/12/pisa_its_poverty_not_stupid_1.html|title = PISA: It's Poverty Not Stupid|date = 15 December 2010|first = Mel|last = Riddile|publisher = [[National Association of Secondary School Principals]]|access-date = 15 April 2016|archive-url = https://web.archive.org/web/20140122143137/http://nasspblogs.org/principaldifference/2010/12/pisa_its_poverty_not_stupid_1.html|archive-date = 22 January 2014|url-status = dead}}</ref> However, individual US schools with poverty rates comparable to Finland's (below 10%), as measured by reduced-price school lunch participation, outperform Finland; and US schools in the 10–24% reduced-price lunch range are not far behind.<ref>{{cite web|url=http://simplystatistics.org/2013/08/23/stratifying-pisa-scores-by-poverty-rates-suggests-imitating-finland-is-not-necessarily-the-way-to-go-for-us-schools/ |title=Stratifying PISA scores by poverty rates suggests imitating Finland is not necessarily the way to go for US schools |publisher=Simply Statistics |date=23 August 2013}}</ref>

[[Reduced price meal|Reduced school lunch]] participation is the only available intra-poverty indicator for US schoolchildren. In the United States, schools in locations in which less than 10% of the students qualified for free or reduced-price lunch averaged PISA scores of 551  (higher than any other OECD country). This can be compared with the other OECD countries (which have tabled figures on children living in relative poverty):<ref name=NSSP />

{| class="sortable wikitable" style="margin:auto;"  border="1"
|-
! Country
! Percent of reduced school lunches (US)<ref name=NSSP /> 
Percent of relative child poverty 
(Other OECD countries)<ref>"[https://www.theguardian.com/news/datablog/2009/sep/11/child-poverty-statistics-uk-countries Child poverty statistics: how the UK compares to other countries]", ''The Guardian''. The same UNICEF figures were used by Riddile.</ref> 
! PISA score<ref>''[http://nces.ed.gov/pubs2011/2011004.pdf Highlights From PISA 2009],'' Table 3.</ref>
|-
|'''United States'''
|'''< 10%'''
|'''551'''
|-
|Finland
|3.4%
|536
|-
|Netherlands
|9.0%
|508
|-
|Belgium
|6.7%
|506
|-
|'''United States'''
|'''10%–24.9%'''
|'''527'''
|-
|Canada
|13.6%
|524
|-
|New Zealand
|16.3%
|521
|-
|Japan
|14.3%
|520
|-
|Australia
|11.6%
|515
|-
|'''United States'''
|'''25–49.9%'''
|'''502'''
|-
|Estonia
|40.1%
|501
|-
|'''United States'''
|'''50–74.9%'''
|'''471'''
|-
|Russian Federation
|58.3%
|459
|-
|'''United States'''
|'''> 75%'''
|'''446'''
|}

====Sampling errors====
In 2013 Martin Carnoy of the Stanford University Graduate School of Education and Richard Rothstein of the [[Economic Policy Institute]] released a report, "What do international tests really show about U.S. student performance?", analyzing the 2009 PISA data base. Their report found that U.S. PISA test scores had been lowered by a sampling error that over-represented adolescents from the most disadvantaged American schools in the test-taking sample.<ref>See, [http://www.epi.org/publication/us-student-performance-testing/ Martin Carnoy and Richard Rothstein, "What do international tests really show about U.S. student performance?", Economic Policy Institute, 28 January 2013].</ref> The authors cautioned that international test scores are often "interpreted to show that American students perform poorly when compared to students internationally" and that school reformers then conclude that "U.S. public education is failing." Such inferences, made before the data has been carefully analyzed, they say, "are too glib"<ref>[https://www.washingtonpost.com/blogs/answer-sheet/wp/2013/01/15/u-s-scores-on-international-test-lowered-by-sampling-error-report/ Valerie Strauss, "U.S. scores on international test lowered by sampling error: report", ''Washington Post'', 15 January 2013].</ref> and "may lead policymakers to pursue inappropriate and even harmful reforms."<ref>Carnoy and Rothstein, "What do international tests really show about U.S. student performance?", Economic Policy Institute, 28 January 2013</ref>

Carnoy and Rothstein observe that in all countries, students from disadvantaged backgrounds perform worse than those from advantaged backgrounds, and the US has a greater percentage of students from disadvantaged backgrounds. The sampling error on the PISA results lowered U.S. scores for 15-year-olds even further, they say. The authors add, however, that in countries such as Finland, the scores of disadvantaged students tends to be stagnant, whereas in the U.S the scores of disadvantaged students have been steadily rising over time, albeit still lagging behind their those of their more advantaged peers. When the figures are adjusted for social class, the PISA scores of all US students would still remain behind those of the highest scoring countries, nevertheless, the scores of US students of all social backgrounds have shown a trajectory of improvement over time, notably in mathematics, a circumstance PISA's report fails to take into account.

Carnoy and Rothstein write that PISA spokesman Schleicher has been quoted saying that "international education benchmarks make disappointing reading for the U.S." and that "in the U.S. in particular, poverty was destiny. Low-income American students did (and still do) much worse than high-income ones on PISA. But poor kids in Finland and Canada do far better relative to their more privileged peers, despite their disadvantages" (Ripley 2011).<ref>Schleicher was quoted by Amanda Ripley to this effect in her 2011 book, ''The Smartest Kids in The World'' (Simon and Schuster).</ref> Carnoy and Rothstein state that their report's analysis shows Schleicher and Ripley's claims to be untrue. They further fault the way PISA's results have persistently been released to the press before experts have time to evaluate them; and they charge the OECD reports with inconsistency in explaining such factors as the role of parental education. Carnoy and Rothstein also note with alarm that the US secretary of education [[Arne Duncan]] regularly consults with PISA's Andreas Schleicher in formulating educational policy before other experts have been given a chance to analyze the results.<ref>Carnoy and Rothstein, "What do international tests really show about U.S. student performance?", EPI, 28 January 2013. Another scholar, Matthew di Carlo of the Albert Shanker Institute, criticized PISA for reporting its results in the form of national rankings, since rankings can give a misleading impression that differences between countries' scores are far larger than is actually the case. Di Carlo also faulted PISA's methodology for disregarding factors such as margin of error. See [http://shankerblog.org/?p=1535 Matthew di Carlo, "Pisa For Our Time: A Balanced Look"], [[Albert Shanker Institute]] website, 10 January 2011.</ref> Carnoy and Rothstein's report (written before the release of the 2011 database) concludes:<blockquote>We are most certain of this: To make judgments only on the basis of national average scores, on only one test, at only one point in time, without comparing trends on different tests that purport to measure the same thing, and without disaggregation by social class groups, is the worst possible choice. But, unfortunately, this is how most policymakers and analysts approach the field.
</blockquote><blockquote>The most recent test for which an international database is presently available is PISA, administered in 2009. A database for TIMSS 2011 is scheduled for release in mid-January 2013. In December 2013, PISA will announce results and make data available from its 2012 test administration. Scholars will then be able to dig into TIMSS 2011 and PISA 2012 databases so they can place the publicly promoted average national results in proper context. The analyses we have presented in this report should caution policymakers to await understanding of this context before drawing conclusions about lessons from TIMSS or PISA assessments.<ref>Carnoy and Rothstein, "What do international tests really show about U.S. student performance?", EPI, January 28, 2013.</ref></blockquote>

==References==
{{reflist}}

==Further reading==
===Official websites and reports===
*[http://www.pisa.oecd.org OECD/PISA website]
** [http://www.pisa.oecd.org/document/58/0,2340,en_32252351_32236159_33688954_1_1_1_1,00.html OECD (1999)]: Measuring Student Knowledge and Skills. A New Framework for Assessment. Paris: OECD, {{ISBN|92-64-17053-7}} [http://www.oecd.org/dataoecd/45/32/33693997.pdf]
** OECD (2001): Knowledge and Skills for Life. First Results from the OECD Programme for International Student Assessment (PISA) 2000.
** OECD (2003a): The PISA 2003 Assessment Framework. Mathematics, Reading, Science and Problem Solving Knowledge and Skills. Paris: OECD, {{ISBN|978-92-64-10172-2}} [http://www.oecd.org/dataoecd/46/14/33694881.pdf]
** OECD (2004a): Learning for Tomorrow's World. First Results from PISA 2003. Paris: OECD, {{ISBN|978-92-64-00724-6}} [http://www.oecd.org/document56/0,2340,en_2649_201185_34016248_1_1_1_1,00.html]
** OECD (2004b): Problem Solving for Tomorrow's World. First Measures of Cross-Curricular Competencies from PISA 2003. Paris: OECD, {{ISBN|978-92-64-00642-3}}
** OECD (2005): PISA 2003 Technical Report. Paris: OECD, {{ISBN|978-92-64-01053-6}}
** OECD (2007): Science Competencies for Tomorrow's World: Results from PISA 2006 [http://www.pisa.oecd.org/document/2/0,3343,en_32252351_32236191_39718850_1_1_1_1,00.html]
** OECD (2014): PISA 2012 results: Creative problem solving: Students’ skills in tackling real-life problems (Volume V) [http://www.oecd-ilibrary.org/education/pisa-2012-results-skills-for-life-volume-v_9789264208070-en]

===Reception and political consequences===
*A. P. Jakobi, K. Martens: ''Diffusion durch internationale Organisationen: Die Bildungspolitik der OECD.'' In: K. Holzinger, H. Jörgens, C. Knill: Transfer, Diffusion und Konvergenz von Politiken. VS Verlag für Sozialwissenschaften, 2007.

====France====
*N. Mons, X. Pons: ''The reception and use of Pisa in France.''

====Germany====
*[[Edelgard Bulmahn|E. Bulmahn]] [then federal secretary of education]: ''PISA: the consequences for Germany.'' OECD observer, no. 231/232, May 2002. pp.&nbsp;33–34.
*H. Ertl: ''Educational Standards and the Changing Discourse on Education: The Reception and Consequences of the PISA Study in Germany.'' Oxford Review of Education, v 32 n 5 pp 619–634 Nov 2006.

====United Kingdom====
*S. Grek, M. Lawn, J. Ozga: ''Study on the Use and Circulation of PISA in Scotland.'' [http://www.ces.ed.ac.uk/PDF%20Files/K%2BPWP12.pdf]

====Books====
* H. Brügelmann: ''Vermessene Schulen - standardisierte Schüler''. Beltz-Verlag, Weinheim (Deutsch, English summary:  https://www.academia.edu/15203894/Evidence-Based_Pedagogy ).
* S. Hopmann, G. Brinek, M. Retzl (eds.): ''PISA zufolge PISA. PISA According to PISA.'' LIT-Verlag, Wien 2007, {{ISBN|3-8258-0946-3}} (partly in German, partly in English)
* T. Jahnke, W. Meyerhöfer (eds.): ''PISA & Co&nbsp;– Kritik eines Programms.'' Franzbecker, Hildesheim 2007 (2nd edn.), {{ISBN|978-3-88120-464-4}} (in German)
* R. Münch: ''Globale Eliten, lokale Autoritäten: Bildung und Wissenschaft unter dem Regime von PISA, [[McKinsey & Company|McKinsey & Co.]]'' Frankfurt am Main : Suhrkamp, 2009. {{ISBN|978-3-518-12560-1}} (in German)

====Websites====
*J . Wuttke: [http://www.messen-und-deuten.de/pisa/biblio.htm Critical online bibliography]
*Correlation of 2009-PISA Scores with National GDP [http://www-958.ibm.com/software/data/cognos/manyeyes/visualizations/pisa-science-scores-2009-correlate  '''PISA versus GDP''']

===Video clips===
* [https://www.youtube.com/watch?v=q1I9tuScLUA#at=15 PISA: Measuring student success around the world]

[[Category:International educational organizations]]
[[Category:International rankings]]
[[Category:Student assessment and evaluation]]
[[Category:OECD]]
[[Category:Educational research]]
[[Category:Statistical service organizations]]