WORKSHOPS

1st November 9AM - 5PM

How hard can it be? The practice and challenge of validation: Issues around how best to provide evidence for assessment validity, reliability and fairness

Workshop 1:

How hard can it be? The practice and challenge of validation: Issues around how best to provide evidence for assessment validity, reliability and fairness

Abstract

The primacy of validity as measurement concept has been consistently affirmed in the assessment literature. The responsibility for assessment providers to demonstrate robust and thorough validity evidence is, therefore, a long established expectation (Messick, 1992, p.89) as are warnings about the “potentially serious consequences” (Kane, 2009, p.61) of shirking such responsibilities. Even assessment providers that have limited resources will still have a responsibility to demonstrate the quality and validity of their assessments. This workshop is intended to make the complexities of validation theory and practice less challenging and more readily operational. The workshop broadly divides into two parts. The first establishes the importance and relevance of validation theory and practice. The second unpacks the complexities and challenges of conducting a validation exercise. Each part comprises sessions affording group activities and discussion. By sharing experiences through a collaborative workshop environment, greater insights will be drawn leading to an increased understanding of the validation process and how it might be routinely operationalised in differing contexts.

Honorary Professor of University College London in the Institute of Education - Curriculum, Pedagogy & Assessment, United Kingdom

Biodata

Stuart Shaw has worked for international awarding bodies for over 20 years. From 2007 to 2021, he was Head of Research at Cambridge Assessment International Education. Stuart is particularly interested in demonstrating how educational, psychological and vocational tests seek to meet the demands of validity, reliability and fairness. Stuart has a wide range of publications in English second language assessment and educational research journals (around 150). His assessment books include: Examining Writing: Research and practice in assessing second language writing (Shaw & Weir, 2007); The IELTS Writing Assessment Revision Project: towards a revised rating scale (Shaw & Falvey, 2008); Validity in Educational and Psychological Assessment (Newton & Shaw, 2014); and Language Rich: Insights from Multilingual Schools (Shaw, Imam & Hughes, 2015). His most recent book is entitled Is Assessment Fair? (Isabel Nisbet & Stuart Shaw) published by Sage (2020). Stuart is currently working on a book with Isabel Nisbet (to be published by Routledge in early 2024) entitled: Educational Assessment in a Changing World: Lessons Learned and the Path Ahead. Stuart is a Fellow of the Chartered Institute of Educational Assessors (CIEA) and a Fellow of the Association for Educational Assessment in Europe (AEA-Europe). Stuart is an elected member of the Council of AEA-Europe and is Chair of its Scientific Programme Committee. He is an elected member of the Board of Trustees of the International Association for Educational Assessment (IAEA) and Chair of the IAEA Communications Committee. Stuart is also an affiliated lecturer at the Faculty of Education, University of Cambridge, and an Honorary Professor – Curriculum, Pedagogy and Assessment, Institute of Education, University College London (UCL). Stuart has contributed to international debates on validity and fairness (see, for example, Special Issue of Assessment in Education: Principles, Policy & Practice, 2016) and more recently at The British Educational Research Association (BERA) Presidential Roundtable (2022).

$

Read More

Workshop 1:

How hard can it be? The practice and challenge of validation: Issues around how best to provide evidence for assessment validity, reliability and fairness

Abstract

The primacy of validity as measurement concept has been consistently affirmed in the assessment literature. The responsibility for assessment providers to demonstrate robust and thorough validity evidence is, therefore, a long established expectation (Messick, 1992, p.89) as are warnings about the “potentially serious consequences” (Kane, 2009, p.61) of shirking such responsibilities. Even assessment providers that have limited resources will still have a responsibility to demonstrate the quality and validity of their assessments. This workshop is intended to make the complexities of validation theory and practice less challenging and more readily operational. The workshop broadly divides into two parts. The first establishes the importance and relevance of validation theory and practice. The second unpacks the complexities and challenges of conducting a validation exercise. Each part comprises sessions affording group activities and discussion. By sharing experiences through a collaborative workshop environment, greater insights will be drawn leading to an increased understanding of the validation process and how it might be routinely operationalised in differing contexts.

Honorary Professor of University College London in the Institute of Education - Curriculum, Pedagogy & Assessment, United Kingdom

Biodata

Stuart Shaw has worked for international awarding bodies for over 20 years. From 2007 to 2021, he was Head of Research at Cambridge Assessment International Education. Stuart is particularly interested in demonstrating how educational, psychological and vocational tests seek to meet the demands of validity, reliability and fairness. Stuart has a wide range of publications in English second language assessment and educational research journals (around 150). His assessment books include: Examining Writing: Research and practice in assessing second language writing (Shaw & Weir, 2007); The IELTS Writing Assessment Revision Project: towards a revised rating scale (Shaw & Falvey, 2008); Validity in Educational and Psychological Assessment (Newton & Shaw, 2014); and Language Rich: Insights from Multilingual Schools (Shaw, Imam & Hughes, 2015). His most recent book is entitled Is Assessment Fair? (Isabel Nisbet & Stuart Shaw) published by Sage (2020). Stuart is currently working on a book with Isabel Nisbet (to be published by Routledge in early 2024) entitled: Educational Assessment in a Changing World: Lessons Learned and the Path Ahead. Stuart is a Fellow of the Chartered Institute of Educational Assessors (CIEA) and a Fellow of the Association for Educational Assessment in Europe (AEA-Europe). Stuart is an elected member of the Council of AEA-Europe and is Chair of its Scientific Programme Committee. He is an elected member of the Board of Trustees of the International Association for Educational Assessment (IAEA) and Chair of the IAEA Communications Committee. Stuart is also an affiliated lecturer at the Faculty of Education, University of Cambridge, and an Honorary Professor – Curriculum, Pedagogy and Assessment, Institute of Education, University College London (UCL). Stuart has contributed to international debates on validity and fairness (see, for example, Special Issue of Assessment in Education: Principles, Policy & Practice, 2016) and more recently at The British Educational Research Association (BERA) Presidential Roundtable (2022).

1st November 9AM - 5PM

Artificial Intelligence as a tool for Assessment Reform

Workshop 2:

Artificial Intelligence as a tool for Assessment Reform

Abstract

In 2021, the Ministry for Education, Sport, Youth, Research and Innovation in Malta embarked on a pilot project to introduce an AI-powered educational platform in Primary Mathematics. Part of the Government of Malta ‘Strategy and Vision for Artificial Intelligence in Malta’, the EducationAI pilot project aims to further move from a one-size-fits-all schooling system to a more equitable quality education for all. This is aligned with the ‘Framework for the Education Strategy for Malta 2014-2024’ and the educational reforms brought about by ‘My Journey: Achieving through different paths’. Apart from supporting students through personalised and adaptive learning, the system assists educators in building more formative assessments based on the learners’ individual needs. This complements the summative evaluation of students’ academic achievement and provides a context for implementing continuous assessment, recently introduced in the Maltese educational system as part of the Learning Outcomes Framework reform. Consequently, the platform has the potential to provide timely, quantitative and qualitative feedback to all stakeholders, as envisaged in the National Curriculum Framework for Malta. It is currently being designed and developed by the University of Malta and piloted with 50 educators and 1000 students in Mathematics across Years 4 to 6.
The proposed pre-conference workshop entitled “Artificial Intelligence as a tool for Assessment Reform” aims to provide participants with a hands-on and interactive opportunity to explore the potential of Artificial Intelligence (AI) in and for assessment reform. By creating more equitable learning environments for students and through the development of a fairer assessment system, AI has the potential to keep assessment practices apace with theoretical and technological advancements. The workshop will start with an introduction to AI, specifically focusing on education. Through a series of use cases from around the world, participants will have the opportunity to discuss the application of AI in and for various educational endeavours, ranging from teaching and learning to assessment and administration. The workshop will then feature a walkthrough of the EducationAI platform, which is currently being piloted in Maltese Primary schools, through two hands-on sessions. The first session will cover the pedagogical and technological affordances offered by the platform from the educators’ perspective, while the second session will revolve around the students’ experience in using the platform. This will enable the participants to experience AI first-hand and will underpin discussions on how AI is likely to shape and reshape teaching, learning and, particularly, assessment in the context of educational reforms. The workshop will conclude with a short presentation on the preliminary findings of the EducationAI project and a discussion on the ways forward, as well as considerations for future implementations. This will include a discussion of the current research on the use of AI in adaptive and personalised assessment and the ethical issues that need to be considered when introducing AI into the assessment process.
The target audience for the workshop includes educators, administrators, policymakers, and researchers interested in exploring AI as a tool in and for assessment reform. In terms of the required previous knowledge, participants need no prior knowledge or experience with AI, as the workshop will be designed for all levels of expertise. The hands-on nature of the workshop requires participants to make use of their own laptops or tablets. It is envisaged that by the end of the workshop, participants will have gained a deeper understanding of the potential of AI in education, particularly in the context of assessment reform. They will gain a better understanding of the possibilities of AI in education, as well as the challenges associated with its implementation. This will enable participants to critically evaluate the use of AI in their own educational settings and make informed decisions on how to implement AI-powered tools and strategies for assessment.

Biographies

Stephen Bezzina is a Research Support Officer at the Department of Artificial Intelligence at the Faculty of ICT at the University of Malta. His research interests include the application of Artificial Intelligence to playful and game-informed approaches, such as game-based methodologies and gamification. He holds a Master of Science degree in Digital Education (University of Edinburgh), specialising in games and digital assessment. In the past, he has worked as an Education Officer (responsible for diversity, cooperative learning and digital literacy) within the Ministry for Education and Employment in Malta.

Prof Alexiei Dingli is a renowned AI expert and Professor at the University of Malta. With over 20 years of experience in the field, he has helped numerous companies successfully implement AI solutions. His work has been recognized as world-class by international experts, and he has received numerous awards from organizations such as the European Space Agency, the World Intellectual Property Organization, and the United Nations. In addition to his considerable peer-reviewed publications, he has also been a critical member of the Malta.AI task force, working to position Malta as a global leader in AI.

$

Read More

Workshop 2:

Artificial Intelligence as a tool for Assessment Reform

Abstract

In 2021, the Ministry for Education, Sport, Youth, Research and Innovation in Malta embarked on a pilot project to introduce an AI-powered educational platform in Primary Mathematics. Part of the Government of Malta ‘Strategy and Vision for Artificial Intelligence in Malta’, the EducationAI pilot project aims to further move from a one-size-fits-all schooling system to a more equitable quality education for all. This is aligned with the ‘Framework for the Education Strategy for Malta 2014-2024’ and the educational reforms brought about by ‘My Journey: Achieving through different paths’. Apart from supporting students through personalised and adaptive learning, the system assists educators in building more formative assessments based on the learners’ individual needs. This complements the summative evaluation of students’ academic achievement and provides a context for implementing continuous assessment, recently introduced in the Maltese educational system as part of the Learning Outcomes Framework reform. Consequently, the platform has the potential to provide timely, quantitative and qualitative feedback to all stakeholders, as envisaged in the National Curriculum Framework for Malta. It is currently being designed and developed by the University of Malta and piloted with 50 educators and 1000 students in Mathematics across Years 4 to 6.
The proposed pre-conference workshop entitled “Artificial Intelligence as a tool for Assessment Reform” aims to provide participants with a hands-on and interactive opportunity to explore the potential of Artificial Intelligence (AI) in and for assessment reform. By creating more equitable learning environments for students and through the development of a fairer assessment system, AI has the potential to keep assessment practices apace with theoretical and technological advancements. The workshop will start with an introduction to AI, specifically focusing on education. Through a series of use cases from around the world, participants will have the opportunity to discuss the application of AI in and for various educational endeavours, ranging from teaching and learning to assessment and administration. The workshop will then feature a walkthrough of the EducationAI platform, which is currently being piloted in Maltese Primary schools, through two hands-on sessions. The first session will cover the pedagogical and technological affordances offered by the platform from the educators’ perspective, while the second session will revolve around the students’ experience in using the platform. This will enable the participants to experience AI first-hand and will underpin discussions on how AI is likely to shape and reshape teaching, learning and, particularly, assessment in the context of educational reforms. The workshop will conclude with a short presentation on the preliminary findings of the EducationAI project and a discussion on the ways forward, as well as considerations for future implementations. This will include a discussion of the current research on the use of AI in adaptive and personalised assessment and the ethical issues that need to be considered when introducing AI into the assessment process.
The target audience for the workshop includes educators, administrators, policymakers, and researchers interested in exploring AI as a tool in and for assessment reform. In terms of the required previous knowledge, participants need no prior knowledge or experience with AI, as the workshop will be designed for all levels of expertise. The hands-on nature of the workshop requires participants to make use of their own laptops or tablets. It is envisaged that by the end of the workshop, participants will have gained a deeper understanding of the potential of AI in education, particularly in the context of assessment reform. They will gain a better understanding of the possibilities of AI in education, as well as the challenges associated with its implementation. This will enable participants to critically evaluate the use of AI in their own educational settings and make informed decisions on how to implement AI-powered tools and strategies for assessment.

Biographies

Stephen Bezzina is a Research Support Officer at the Department of Artificial Intelligence at the Faculty of ICT at the University of Malta. His research interests include the application of Artificial Intelligence to playful and game-informed approaches, such as game-based methodologies and gamification. He holds a Master of Science degree in Digital Education (University of Edinburgh), specialising in games and digital assessment. In the past, he has worked as an Education Officer (responsible for diversity, cooperative learning and digital literacy) within the Ministry for Education and Employment in Malta.

Prof Alexiei Dingli is a renowned AI expert and Professor at the University of Malta. With over 20 years of experience in the field, he has helped numerous companies successfully implement AI solutions. His work has been recognized as world-class by international experts, and he has received numerous awards from organizations such as the European Space Agency, the World Intellectual Property Organization, and the United Nations. In addition to his considerable peer-reviewed publications, he has also been a critical member of the Malta.AI task force, working to position Malta as a global leader in AI.

1st November 9AM - 5PM

Moving to fully inclusive e-assessment

Workshop 3:

Moving to fully inclusive e-assessment

Short abstract

It is often the case that diversity and inclusion are afterthoughts when an organisation is evolving its e-assessment offering. This workshop will provide an engaging opportunity for collaboration with peers, to consider the perspectives of a range of stakeholders. Thought-provoking discussions will equip participants with areas to take away and integrate in their future work practices.
The premise for the workshop is that participants work for a hypothetical assessment organisation that wants to update an onscreen assessment to make greater use of onscreen interactivity. The assessment has a diverse audience, including those with various access requirements.

Small group discussion will be used to explore the premise from multiple perspectives, drawing on experiences from across the group. The presenters will provide an informative introduction to various key areas, through short presentations, and participants will investigate some of these areas in more detail for themselves during the workshop and share findings with the group.

This workshop is led by members of the AEA-Europe eAssessment and Inclusive Assessment SIGs. We aim to provide participants with a basic understanding of inclusion considerations to inform everyday practice and/or opportunity to extend existing understanding.
No prior experience of e-assessment or inclusive assessment is needed.

Presenters’ Bios

Helen Claydon is an experienced assessment developer and leader, having worked in assessment for over 25 years. She has led a range of projects developing summative and formative assessments for ages from 6 years-old to adult. Most notably she has led assessment development for national programmes in the United Kingdom, including the Scottish National Standardised Assessments, the KS1 and KS2 National Curriculum Tests in mathematics (SATs) and the Professional Skills Tests for Prospective Teachers. Her experience covers all aspects of the assessment development process, including assessment design, item writing and review, trialling, review of item and test statistics, test construction, standard setting and standards maintenance.
Helen is currently an assessment consultant, undertaking qualification and assessment design, development and review work for organisations such as the Association for Project Management (APM), GL Assessment, MH&A and Qualifications Wales.
Helen holds a Master’s degree in Education. She is a Steering Committee Member for AEA-Europe’s eAssessment Special Interest Group and a Fellow of the Chartered Institute of Educational Assessors. Helen helped to launch the annual International e-Assessment Awards in 2016 and has been a judge for the awards each year since then.

Caroline Jongkamp is currently a senior consultant for international projects at Cito, the Netherlands. She is experienced as a professional test developer in economics, financial accounting, and ICT. At Cito, she was responsible for the implementation of computer-based testing in Dutch secondary school leaving exams. She is an experienced change manager involved in change processes on item banking and computer-based testing. Caroline holds an MSc in Econometrics with a specialization in Operations Research.
Caroline is a Steering Committee member for AEA-Europe’s eAssessment Special Interest Group.

Irini-Renika Papakammenou holds a BA in English literature with English language from the university of North Wales, Bangor and an MSc in Teaching English to Speakers of other Languages (TESOL) from the University of Stirling. She has done a PhD in Linguistics with specialization in Language Testing and Assessment at the University of Cyprus. She has received an award for her PhD thesis as the best postgraduate thesis of the year. She has also published scientific papers on language testing and assessment and alternative assessment techniques in international books and journals. She has presented in numerous local and international conferences, and she has delivered training courses. She is a member of language teaching and assessment societies and research groups. Her research interests include language testing and assessment, curricula, material design (for face-to-face and online classrooms), classroom practices and teacher education and development.

Irini has been involved in EFL teaching and learning for the past 21 years. During her 21-year career she has contributed to the field in a number of different capacities such as English Language Teacher, Exam Preparation Teacher (all exams including IELTS and TOEFL), Teacher Trainer, Oral Examiner, Speaker and Researcher. She is the owner of a private institute of foreign languages, she owns a distance learning platform and leads distance learning EFL examination programs. She has created innovative classroom and online materials which have been presented in international conferences and have been published. Irini has recently received the Global Teacher Award 2020.

Thomais Rousoulioti, PhD in Applied Linguistics, works as a special teaching staff at Aristotle University of Thessaloniki, Greece. During the years 2010-2017 she worked at Department of Support and Promotion of the Greek Language of the Centre for the Greek Language, Greece, where she was involved in the design and implementation of research programs regarding the teaching and assessment of the Greek language as a second/foreign language but also with the design and editing of tests for the assessment of participants in the examinations for the Certification of Attainment in Greek.

She has also worked for the Hellenic Open University and the University of Nicosia teaching in online distance Ma programs. Her research interests include the teaching and assessment of language proficiency in multilingual settings, the design of teaching materials, adult education, distance education and inclusive assessment. She is the coordinator of a postgraduate course for the assessment in Greek as a second/foreign language and the coordinator of the module Students’ Assessment in the training program Routes of the Centre for the Greek Language which is the sole representative of Greece at the European Federation of National Institutions for Language (http://efnil.org/). She is a member of the SIG Steering Committee on Inclusive Assessment in Education (AEA-Europe), the "Psifis" laboratory of Aristotle University of Thessaloniki, Greece, the Hellenic Society of Applied Linguistics, EALTA, ALTE and OsloMet's EnA research team.

$

Read More

Workshop 3:

Moving to fully inclusive e-assessment

Short abstract

It is often the case that diversity and inclusion are afterthoughts when an organisation is evolving its e-assessment offering. This workshop will provide an engaging opportunity for collaboration with peers, to consider the perspectives of a range of stakeholders. Thought-provoking discussions will equip participants with areas to take away and integrate in their future work practices.
The premise for the workshop is that participants work for a hypothetical assessment organisation that wants to update an onscreen assessment to make greater use of onscreen interactivity. The assessment has a diverse audience, including those with various access requirements.

Small group discussion will be used to explore the premise from multiple perspectives, drawing on experiences from across the group. The presenters will provide an informative introduction to various key areas, through short presentations, and participants will investigate some of these areas in more detail for themselves during the workshop and share findings with the group.

This workshop is led by members of the AEA-Europe eAssessment and Inclusive Assessment SIGs. We aim to provide participants with a basic understanding of inclusion considerations to inform everyday practice and/or opportunity to extend existing understanding.
No prior experience of e-assessment or inclusive assessment is needed.

Presenters’ Bios

Helen Claydon is an experienced assessment developer and leader, having worked in assessment for over 25 years. She has led a range of projects developing summative and formative assessments for ages from 6 years-old to adult. Most notably she has led assessment development for national programmes in the United Kingdom, including the Scottish National Standardised Assessments, the KS1 and KS2 National Curriculum Tests in mathematics (SATs) and the Professional Skills Tests for Prospective Teachers. Her experience covers all aspects of the assessment development process, including assessment design, item writing and review, trialling, review of item and test statistics, test construction, standard setting and standards maintenance.
Helen is currently an assessment consultant, undertaking qualification and assessment design, development and review work for organisations such as the Association for Project Management (APM), GL Assessment, MH&A and Qualifications Wales.
Helen holds a Master’s degree in Education. She is a Steering Committee Member for AEA-Europe’s eAssessment Special Interest Group and a Fellow of the Chartered Institute of Educational Assessors. Helen helped to launch the annual International e-Assessment Awards in 2016 and has been a judge for the awards each year since then.

Caroline Jongkamp is currently a senior consultant for international projects at Cito, the Netherlands. She is experienced as a professional test developer in economics, financial accounting, and ICT. At Cito, she was responsible for the implementation of computer-based testing in Dutch secondary school leaving exams. She is an experienced change manager involved in change processes on item banking and computer-based testing. Caroline holds an MSc in Econometrics with a specialization in Operations Research.
Caroline is a Steering Committee member for AEA-Europe’s eAssessment Special Interest Group.

Irini-Renika Papakammenou holds a BA in English literature with English language from the university of North Wales, Bangor and an MSc in Teaching English to Speakers of other Languages (TESOL) from the University of Stirling. She has done a PhD in Linguistics with specialization in Language Testing and Assessment at the University of Cyprus. She has received an award for her PhD thesis as the best postgraduate thesis of the year. She has also published scientific papers on language testing and assessment and alternative assessment techniques in international books and journals. She has presented in numerous local and international conferences, and she has delivered training courses. She is a member of language teaching and assessment societies and research groups. Her research interests include language testing and assessment, curricula, material design (for face-to-face and online classrooms), classroom practices and teacher education and development.

Irini has been involved in EFL teaching and learning for the past 21 years. During her 21-year career she has contributed to the field in a number of different capacities such as English Language Teacher, Exam Preparation Teacher (all exams including IELTS and TOEFL), Teacher Trainer, Oral Examiner, Speaker and Researcher. She is the owner of a private institute of foreign languages, she owns a distance learning platform and leads distance learning EFL examination programs. She has created innovative classroom and online materials which have been presented in international conferences and have been published. Irini has recently received the Global Teacher Award 2020.

Thomais Rousoulioti, PhD in Applied Linguistics, works as a special teaching staff at Aristotle University of Thessaloniki, Greece. During the years 2010-2017 she worked at Department of Support and Promotion of the Greek Language of the Centre for the Greek Language, Greece, where she was involved in the design and implementation of research programs regarding the teaching and assessment of the Greek language as a second/foreign language but also with the design and editing of tests for the assessment of participants in the examinations for the Certification of Attainment in Greek.

She has also worked for the Hellenic Open University and the University of Nicosia teaching in online distance Ma programs. Her research interests include the teaching and assessment of language proficiency in multilingual settings, the design of teaching materials, adult education, distance education and inclusive assessment. She is the coordinator of a postgraduate course for the assessment in Greek as a second/foreign language and the coordinator of the module Students’ Assessment in the training program Routes of the Centre for the Greek Language which is the sole representative of Greece at the European Federation of National Institutions for Language (http://efnil.org/). She is a member of the SIG Steering Committee on Inclusive Assessment in Education (AEA-Europe), the "Psifis" laboratory of Aristotle University of Thessaloniki, Greece, the Hellenic Society of Applied Linguistics, EALTA, ALTE and OsloMet's EnA research team.

1st November 9AM - 5PM

Responding as assessment professionals to calls for reform

Workshop 4:

Responding as assessment professionals to calls for reform

Presenters

Dr. Eleanor Andressen has particular interests in technical and vocational education and training, adult and lifelong learning, and assessment cultures (with a focus on TVET and compulsory national examinations). She is in her third year as a judge for the e-Assessment Association awards and is a peer reviewer for the Higher Education and Work-based Learning journal. She is the current chair of the Steering Group for the AEA Europe Assessment Cultures special interest group.

Dr. Andrew Watts taught in secondary schools in England, and then in Singapore, where he joined a team of specialist inspectors which focused on curriculum development and in-service training. When he returned to England he worked for the Cambridge school examination board (then Cambridge Assessment) and set up the Cambridge Assessment Network, whose aim is to provide support for assessment professionals internationally.

Henning Fjørtoft is professor of Norwegian didactics at NTNU, The Norwegian University of Science and Technology, in Trondheim. His research interests include literacy education, classroom assessment, and professional development in schools. He has been involved in assessment research in Scandinavia, New Zealand, and Ethiopia, and serves on the editorial boards of Assessment in Education: Principles, Policy & PracticeEducational Assessment, Evaluation and Accountability and Teachers and Teaching: Theory and Practice.

Workshop outline

This workshop will give participants a chance to consider significant changes that are currently being discussed in their own assessment systems. Three key topics will be presented with the intention of stimulating thinking. 1) What society expects of assessment professionals? 2) How we might balance assessment of learning and assessment for learning? 3) Assessment for learning beyond schooling? (e.g. in lifelong learning and for technical and vocational skills).
Participants will have opportunities to describe their chosen system, to express their opinions about on-going thinking in their context, and to share with fellow AEA-E members their thoughts on those issues. Much of the time will be spent working within discussion groups on themes which will first be presented by workshop leaders.
Now is a time in which what it means to be employable is evolving and in which learning is increasingly experienced through technology and at a distance, even when education is delivered in traditional contexts. Attendees to the workshop will explore together how they address the changing needs of students, of the workplace and of their societies more widely. The themes which will emerge during the group discussions will be brought together in a final plenary summary introduced by our discussant, Professor Henning Fjørtoft of the Norwegian University of Science and Technology.
$

Read More

Workshop 4:

Responding as assessment professionals to calls for reform

Presenters

Dr. Eleanor Andressen has particular interests in technical and vocational education and training, adult and lifelong learning, and assessment cultures (with a focus on TVET and compulsory national examinations). She is in her third year as a judge for the e-Assessment Association awards and is a peer reviewer for the Higher Education and Work-based Learning journal. She is the current chair of the Steering Group for the AEA Europe Assessment Cultures special interest group.

Dr. Andrew Watts taught in secondary schools in England, and then in Singapore, where he joined a team of specialist inspectors which focused on curriculum development and in-service training. When he returned to England he worked for the Cambridge school examination board (then Cambridge Assessment) and set up the Cambridge Assessment Network, whose aim is to provide support for assessment professionals internationally.

Henning Fjørtoft is professor of Norwegian didactics at NTNU, The Norwegian University of Science and Technology, in Trondheim. His research interests include literacy education, classroom assessment, and professional development in schools. He has been involved in assessment research in Scandinavia, New Zealand, and Ethiopia, and serves on the editorial boards of Assessment in Education: Principles, Policy & PracticeEducational Assessment, Evaluation and Accountability and Teachers and Teaching: Theory and Practice.

Workshop outline

This workshop will give participants a chance to consider significant changes that are currently being discussed in their own assessment systems. Three key topics will be presented with the intention of stimulating thinking. 1) What society expects of assessment professionals? 2) How we might balance assessment of learning and assessment for learning? 3) Assessment for learning beyond schooling? (e.g. in lifelong learning and for technical and vocational skills).
Participants will have opportunities to describe their chosen system, to express their opinions about on-going thinking in their context, and to share with fellow AEA-E members their thoughts on those issues. Much of the time will be spent working within discussion groups on themes which will first be presented by workshop leaders.
Now is a time in which what it means to be employable is evolving and in which learning is increasingly experienced through technology and at a distance, even when education is delivered in traditional contexts. Attendees to the workshop will explore together how they address the changing needs of students, of the workplace and of their societies more widely. The themes which will emerge during the group discussions will be brought together in a final plenary summary introduced by our discussant, Professor Henning Fjørtoft of the Norwegian University of Science and Technology.

1st November 9AM - 5PM

(Re)design YOUR assessment! Designing assessment tasks with evidence-centered design

Workshop 5:

(Re)design YOUR assessment! Designing assessment tasks with evidence-centered design

Abstract

Evidence-Centered Design (ECD) is a scientifically proven framework for streamlining the process of (re)designing your assessment tasks and making your assessment practices more evidence-based (Mislevy, Almond, & Lukas, 2003). In this semi-hackathon workshop, participants from a multidisciplinary background take on the challenge to (re)design their own assessment practices, and make those more evidence-based, by working from an ECD perspective. Assessment redesign is increasingly important and necessary, as societal and technological changes require students, employees or citizens to demonstrate different universal KSA’s (knowledge, skills and abilities) to be successful in school, a job, or society. On the one hand, we, as an assessment community, need assessment practices that match these changes in society. On the other hand, there is a need for assessments that fit the specific context of a country, educational institution, or company, and reforms that take place there. In both cases, there is a profound need to align what you want to evaluate, the task characteristics that allow insights into the what, and a measurement model for interpreting task performance and providing feedback and feedforward. In this workshop, we are going to work in teams on the challenge to build a blueprint for tomorrow’s assessment, based on ECD.

Biodata

Michiel Waltman (1979) is an educational researcher at CitoLab, Cito’s research and innovation division. Michiel worked as a high school teacher, lecturer at Fontys University of Applied Sciences and University of Twente and as an assessment expert at Cito’s division of central examination. Michiel is especially interested in design and validation of assessment instruments for complex constructs like critical thinking and citizenship development. His expertise is in making validation and assessment design approaches (e.g. evidence-centered design and argument-based approach to validation) accessible and meaningful for educational practitioners. He published in educational practitioners’ journals and books and contributed to conferences.

Sebastiaan de Klerk (1986) is an educational researcher at CitoLab. CitoLab is the research and innovation division of Cito. Sebastiaan received his PhD from Twente University in 2016 for research on the design and validation of simulation-based assessment (SBA) in vocational education. In his PhD research, he focused on the measurement of procedural skills in SBA. The key element in his PhD research was to design and validate an assessment for a Dutch security profession. To that extent, evidence-centered design was used for the design. In 2016, during the AEA-Europe conference on Cyprus, he was awarded the Kathleen Tattersall New Assessment Researcher Award for his work on the design and validation of SBA. Sebastiaan also worked (2017 – 2022) as the treasurer of the Dutch Association for Educational Assessment (NVE). At CitoLab, Sebastiaan’s main focus is on research on the influence of technology on assessment. Sebastiaan has multiple scientific publications and presented at numerous conferences.

$

Read More

Workshop 5:

(Re)design YOUR assessment! Designing assessment tasks with evidence-centered design

Abstract

Evidence-Centered Design (ECD) is a scientifically proven framework for streamlining the process of (re)designing your assessment tasks and making your assessment practices more evidence-based (Mislevy, Almond, & Lukas, 2003). In this semi-hackathon workshop, participants from a multidisciplinary background take on the challenge to (re)design their own assessment practices, and make those more evidence-based, by working from an ECD perspective. Assessment redesign is increasingly important and necessary, as societal and technological changes require students, employees or citizens to demonstrate different universal KSA’s (knowledge, skills and abilities) to be successful in school, a job, or society. On the one hand, we, as an assessment community, need assessment practices that match these changes in society. On the other hand, there is a need for assessments that fit the specific context of a country, educational institution, or company, and reforms that take place there. In both cases, there is a profound need to align what you want to evaluate, the task characteristics that allow insights into the what, and a measurement model for interpreting task performance and providing feedback and feedforward. In this workshop, we are going to work in teams on the challenge to build a blueprint for tomorrow’s assessment, based on ECD.

Biodata

Michiel Waltman (1979) is an educational researcher at CitoLab, Cito’s research and innovation division. Michiel worked as a high school teacher, lecturer at Fontys University of Applied Sciences and University of Twente and as an assessment expert at Cito’s division of central examination. Michiel is especially interested in design and validation of assessment instruments for complex constructs like critical thinking and citizenship development. His expertise is in making validation and assessment design approaches (e.g. evidence-centered design and argument-based approach to validation) accessible and meaningful for educational practitioners. He published in educational practitioners’ journals and books and contributed to conferences.

Sebastiaan de Klerk (1986) is an educational researcher at CitoLab. CitoLab is the research and innovation division of Cito. Sebastiaan received his PhD from Twente University in 2016 for research on the design and validation of simulation-based assessment (SBA) in vocational education. In his PhD research, he focused on the measurement of procedural skills in SBA. The key element in his PhD research was to design and validate an assessment for a Dutch security profession. To that extent, evidence-centered design was used for the design. In 2016, during the AEA-Europe conference on Cyprus, he was awarded the Kathleen Tattersall New Assessment Researcher Award for his work on the design and validation of SBA. Sebastiaan also worked (2017 – 2022) as the treasurer of the Dutch Association for Educational Assessment (NVE). At CitoLab, Sebastiaan’s main focus is on research on the influence of technology on assessment. Sebastiaan has multiple scientific publications and presented at numerous conferences.