A Comparison Between TIMSS, SITES, and PISA.

Trends in Mathematics and Science Study (TIMSS)

The Trends in International Mathematics and Science Study (TIMMS) refers to a chain of intercontinental assessments of the knowledge of students in mathematics and science throughout the world. TIMSS aims at providing an in depth state-of-the-art assessment and as a result produce an international perspective on the learning and teaching of science and mathematics in schools worldwide. Students who participate are drawn from diverse educational systems around the globe depending on the population size, economic development and geographical location of the specific countries. There is a minimum set of students that can participate from one educational system. For instance, during the TIMSS 2006 evaluation, a minimum of 5000 to 4500 students were needed for each evaluation. Appropriate data on the learning situation of teachers and their students both in science and mathematics is collected. Teachers are interviewed with a different questionnaire.

TIMSS is the third comparison of science and mathematics achieved by the International Association for the Assessment of Educational Achievements conducted since 1995 . TIMSS reports every four years on the achievement of a fourth year and an eighth year student on both mathematics and science. Over 60 countries from around the globe participate in TIMSS evaluation. The report establishes that in Portugal achievement of students in mathematics and science have risen in recent years. In addition, the stakeholders are working hard in ensuring that the performances in these core subjects are improving to even a higher level.

As per the report by Martin et al (2004), the sample on U.S fourth grade realized an initial school response rate of 70% (weighted) with an average rate of 82 percent of the school response rate. Sampled students for the assessment were 10,795 in total and were from fourth-grade. Out of this figure, 9,829 participated in the study generating 95 percent student response. This resulted into 78% of the overall rate of response for participating students.

TIMSS was established as one of the many projects of the International Association for the Evaluation of Educational Achievement (IEA). Its main aim was to allow educational systems to make comparisons between the educational achievements of their students and also learn from other educational systems in order to design the best educational systems and policies. TIMSS’S first study was conducted in 1995 by the TIMSS International Study Center currently located in Boston, and has continued so every 4 years until 2015.

TIMSS International Results in Mathematics in 2011 summarized eighth and fourth grade students with regards to their achievement in the 63 countries which participated. TIMSS report included trends in student achievement in Mathematics from the 1995 to the 2007 assessments. It also included the performance of students at TIMSS International benchmarks. The assessment report gives a rich array of information which deeply describes mathematics educational contexts including the mathematics curriculum, student’s attitudes towards mathematics, home environment support, classroom activities and characteristics, the training and education of teachers and the contexts of the schools for instruction and learning of mathematics. The results of the achievements are presented for cognitive domains and mathematics content.

TIMSS 2015 was the sixth in the circles of the Trends in Mathematics and Science Study. It has proven to be valuable in international study of achievement trends in Science and Mathematics. TIMSS 2015 reports generally on achievement and on the outcomes of 4 global benchmarks namely high, medium, low, and advanced. These are grouped by major content domains like geometry, numbers, and algebra and so on. They are also grouped by cognitive domains which may include reasoning, knowing and applying. The TIMSS 2015 Assessment samples detailed information concerning school curriculums and curriculum implementation, school resources, and instructional practices as the assessment criteria.

The TIMSS 2015 Assessment offers a framework by which international assessments can be carried out as part of IEA’s TIMSS 2015 including TIMSS Numeracy, TIMSS Mathematics Fourth and Eighth Grade, and TIMSS Science Fourth and Eighth Grade. The content domain in TIMSS such as geometry, biology, algebra and mathematics alongside the topic areas are categorized differently for the fourth and eighth graders whereby; each topic is broadly evaluated with particular objectives. The cognitive domains on the other hand, describe the thinking aspects of students within the science and mathematical domains which are the same across grades though the level of emphasis differs. Further, the TIMSS 2015 Assessment contains the content questionnaire framework which describes the different contexts of schooling alongside the factors related with the effective learning processes, particularly in subjects of science and mathematics. The factors related with effective learning of students in science and mathematics are investigation through the questionnaires completed by the school administration, tutors and students themselves as well as the TIMSS 2015 Encyclopedia. The final element in this framework is a description of the design of the test booklet including the parameters used in development of specific items.

An article by Lynn et al (2007) analyzes the findings by Vanhanen and Lynn (2006) concerning the relationships of the national IQs to national achievement in science and mathematics among eight graders in 67 nations. The authors establish the relationship of the two aspects as being 0.92 and are considered as creating validity for the national IQs. According to the authors, this correlation is among the highest in the sense that the educational achievements and IQs can be considered as measures of similar construct. Lynn et al establishes that the country variations in educational achievements are higher when compared with the variations in IQ. This implies an implifier effect in the sense that the national variations in IQs implicate the variations in educational achievements. In terms of controlling for the country variations in IQ, slim inverse correlations of educational attainment are observed with subjective well-being, political freedom, and GDP and income inequality. Moreover, the authors noted that a public expenditure on education as a percentage of GDP was not found to be a determiner of educational attainment.

Second Information Technology in Education Study (SITES)

This is another one of IEA’s projects started in the 1990s and consists of a number of modules and projects. SITES takes to explore and establish how information technology affects the way students in schools learn. SITES 2006 was the fifth in the surveys that relate to Information Technology by IEA. SITES 2006 was a tough and ambitious study which comprised of complicated procedures for reporting and analyzing findings, drawing samples and collecting data. Technically, the implementation of SITES 2006, to a large extent, takes after IEA’S procedures in student achievement studies, particularly TIMSS. Conversely, the adult target population suggests definite operational necessities and consequences. The technical report of SITES 2006 therefore provides the full documentation of the analytical and methodological implementations of the study, which was used to make the global database fundamental to the international report. SITES 2006 is supplemented by a user guide that ensures study findings and data is handled effectively by researchers to assertively imitate the procedures used in production of international report. The user guide describes the content, proposed usage and structure of the global database. These three combined together makes it possible for analysts to replicate assuredly the procedures and undertake specific analyses precisely.

Brese and Carstens, (2009) highlights that SITES 2006 tend to target mathematics and science teachers as they teach eighth year students and schools as they offer grade eight students with instruction. This makes it evident that the study focuses on the role taken by Information Technology in learning and teaching of science and mathematics to students. It also analyzes the extent to which particular instructive traditions which are contemplated to be conducive in developing skills in the 21st century were extant as compared to conventionally significant ones. In this case, skills were defined with regards to the abilities of student to participate in lifelong learning and their ability to learn from experts and peers and to collaborate with them as well. Analyses were carried out to identify the situations at the teacher, student and school level which were associated with different educational practices and the different ways that ICT can be used in learning and teaching.

Just like TIMSS, the study utilizes questionnaires as the main instruments of gathering information from technology coordinators, school principals and mathematics and science teachers.

An article by Pelgrum (2001) sought to analyze the educational practitioner’s perception at the lower secondary level concerning things that obstructs effective achievement of ICT related objectives for schools. The authors analyzed data from a global survey among a representative sample of schools from 26 nations. The paper presents a review of the main indicators of ICT effectiveness in lower and elementary secondary schools, the main challenges affecting ICT implementation alongside an assessment of the co-variation between the relative factors and obstacles at the country level.

On their part Carstens and Pelgrum (2006) presents a technical overview of SITES 2006. According to the authors SITES 2006 is not only demanding, but ambitious study. The procedures in data collection, drawing samples, data analysis and reporting findings are complex. To a large extent, the procedures used in implementing SITES 2006 resemble those employed in PIRLS and TIMS. However, the focus populations, particularly those in the adult category present specific operational necessities and consequences.

Program for International Student Assessment (PISA)

This is a global study initiated in 2000 to evaluate systems of education throughout the world through evaluation and testing of the knowledge and skills of 15 year old students. Every three years, students studying science and mathematics are evaluated. Additionally, PISA measures general proficiencies skills such as collective problem solving among others. With regards to its design, PISA tends to focus on the functional skills acquired by students by the time they are about to clear compulsory schooling. PISA is managed by the Organization for Economic Cooperation and Development (OECD). OECD is an intergovernmental organization for industrialized countries.

An article by Goldstein (2004) raises a number of methodological concerns in regard to analysis, interpretation, and conduct of the outcome of the study by the Programme for International Student Assessment (PISA). The author notes that despite PISA representing a significant advancement of the previous attempts at international comparative studies on these assessments, there are specific problematic aspects which the study retains. Among these problems as noted by the author includes restrictions on the data analysis and modeling and the resulting interpretations. In addition, the author also raises concerns regarding the specific features of the outcomes which raise issues concerning the sufficiency of the data while emphasizing the necessity of introducing a longitudinal feature. In light of these, the authors make a suggestion on how to improve future studies in the same aspect.

In essence, PISA assesses the extent that students who are about to finish their compulsory education have gone in acquiring the skills and knowledge needed for participation in the contemporary society. All aspects of PISA cover the domain of scientific literacy and mathematical reading not merely in line with school curriculum mastery, but also in terms of the mastery of significant skills and knowledge in adult life. Just like SITES and TIMSS, students and principals are served with questionnaires to complete, which assists in the assessment.