Joint evaluations of research and education: experiences from NOKUT - - PowerPoint PPT Presentation

Joint evaluations of research and education: experiences from NOKUT and RCN’s pilot project

Åsne Kalland Aarstad (PhD), senior advisor at NOKUT

Presentation structure

1.

Evaluation model and progress

2.

Results from the education evaluation

3.

Results from the interplay evaluation

4.

Reflections on future evaluations of research and education in Norwegian higher education

26.06.2018 | 2

Mandate

• In 2014 NOKUT and the Research Council of Norway

(RCN) were commissioned by the Norwegian Ministry of Education and Research to develop and test a model for joint evaluations of education and research.

• The model should also enable evaluations of the

interplay between research and education

12.06.2018 | 3

Evaluation model

• Three sets of evaluations
• Separate expert panels for research and education
• Joint expert panels for interplay

12.06.2018 | 4

Research evaluation (NFR) Panel

• ther

Panel econ Panel pol.sci Panel soc Educational evaluation (NOKUT)

Panel econ Panel pol.sci Panel soc

Interplay evaluation

Econ Pol.sci Soc

Project outline in cooperation with the Research Council of Norway Selection of quality dimensions in cooperation with reference group The institutions receive self- evaluation form NOKUT compiles data packages on the quality dimensions for participating institutions NOKUT receives self- evaluation from participating institutions Expert panels receive data sources and assessment tools. 1st meeting 2nd meeting Interplay meetings Draft interplay report Final interplay report Draft sub- reports/ conclusions: Political Science Sociology Economics Final sub- reports: Political Science Sociology Economics 3rd meeting

12.06.2018 | 6

Education quality indicators

1.

Initial competence

2.

Programme design

3.

Teaching and learning methods

4.

Learning environment in study programmes

5.

Educational competence

6.

Achieved learning

7.

Internationalisation

8.

Relevance

9.

Educational leadership

26.06.2018 | 7

Results: educational quality

26.06.2018 | 8

Results: educational quality

• Conclusion reached across the three panels:
• Overall picture: good
• Large variations between institutions
• Lacking international orientation
• Methods training: too little, too late

26.06.2018 | 9

Results: educational quality

• Conclusion reached across the three panels:
• Traditional teaching – and assessment methods
• Cumulative structure necessary
• Faculty size: challenging for smaller institutions

26.06.2018 | 10

Results: educational quality

• Conclusions deriving from the different

disciplines:

• Political science: name/content
• Sociology: autonomy re. admission requirements
• Economics: mathematics specialization and leave

behind integrated master programmes

26.06.2018 | 11

The interplay evaluation

26.06.2018 | 12

Why interplay?

• Why #1: Increasing curiosity about how research

quality can be said to influence educational quality

• Why #2: Trend towards large-scale evaluations of

multiple activities/dimensions

26.06.2018 | 13

What exactly did the panels evaluate?

Step 1 Step 2 Step 3 Step 4

Comparison of research quality and educational quality for each institution. Assessment of interplay dimensions for each institution. Comment on the relationship between research quality, educational quality, and level of interplay at each institution. Highlight patterns across institutions within the discipline.

26.06.2018 | 14

Interplay indicators

1.

Organizational conditions

2.

Academic staff R&D orientation

3.

Research-based curriculum

4.

Use of student active learning forms

5.

Variation of teaching and learning methods

6.

Involvement in staff research

7.

Assessment practice

12.06.2018 | 15

Results: interplay

• Not surprising: the institutions that perform well in the

research evaluation also performs well in the education evaluation, and vice-versa. No evidence of zero-sum game…

• According to all panels: decisive that the education is

provided by an active research environment, and that lecturers have research experience – or ideally – are active researchers themselves.

26.06.2018 | 16

Results: interplay

• Larger institutions have the resources to provide

research-based education across the spectre of courses and specialization offered. Smaller institutions should take measures to ensure research-based education: specialize and/or collaborate.

• The panels strongly recommends that the research
• rientation of the bachelor-level is strengthened. More

methods, more application of methods, and larger bachelor theses/projects. Important not only for the quality of education, but also for increasing the labour market relevance of the bachelor degree.

26.06.2018 | 17

The road ahead…

• NOKUT can not evaluate, audit or accredit study

programmes without a thorough assessment of the research environments in which the programmes are based

• By merging research evaluations with educational

evaluations, we can achieve a range of additional benefits:

• Institutions get a more comprehensive picture of the

quality of their core activities within the same time frame

• If executed correctly: institutions would experience

fewer and instead more comprehensive evaluations

26.06.2018 | 18

The road ahead…

• However: the future joint evaluations would need to be

joint from the very beginning: joint panels, mandate, and steering group

• Future joint evaluations in research and education would

have to concentrate on these two core activities for the sake of feasibility

26.06.2018 | 19

• The research, education and interplay reports will

be launched on Tuesday 19th of June in Oslo

• All reports will be available online at nokut.no and

at forskningsradet.no THANK YOU!

26.06.2018 | 20