Ten years ago the Helmholtz Association changed the way it allocates its research financing: since then it has organised a competition for funding. The German Council of Science and Humanities is now evaluating how well this method works. We spoke to its Chairman Manfred Prenzel
Mr Prenzel, when we asked you for this interview it was in your role as Chairman of the working group tasked with evaluating the Helmholtz Association, or more precisely, its programme-oriented funding (PoF). In the meantime you have also been appointed Chairman of the Council of Science and Humanities. With these two roles to fill you could have a stressful end to the year in store.
Stressful perhaps, but mostly exciting.
So what does your job really entail? Evaluating PoF specifically or, ultimately, Helmholtz as a whole?
Of course PoF is a key issue for the Helmholtz Association, as it secures or at least structures a large part of the financing. Our evaluation will also look at how PoF, as part of how the Helmholtz Association operates, helps it to fulfil its tasks within the research system. In that respect I can understand that many Helmholtz employees are keeping an eye on our group, perhaps even a little nervously. But I can reassure them: we certainly don’t intend to assess Helmholtz and its performance as a whole – that would be biting off more than we can chew. As for PoF, it is in the interest of all colleagues at the centres for such a process to be forward-looking, fair, reliable and transparent.
But why now in particular? Does it have something to do with the strategy paper “Helmholtz 2020” that triggered heated debates in the research world at the end of 2012?
I don’t see it that way. It’s not as if the Council of Science and Humanities says: now we have to make it clear to an association that sees itself as an architect that there are structural engineers and other people who play a part in building and landscape planning. Not in the slightest.
And yet only the processes at Helmholtz are currently being examined.
Certainly one could also look at other research organisations to see if they are applying suitable processes. What makes Helmholtz particularly interesting are the sums involved – currently €3.5 billion per year. These are large pieces of the pie when it comes to government-backed research funding. Further, PoF has now been in place for around a decade. After three complete cycles it is time to take stock. Is it still the critical process that really steers research towards future-oriented topics? To what extent does PoF run the risk of merely transferring funds in a very simple, continuous and stable manner? In 2015 the Helmholtz Association will be 20 years old. A lot has happened in this time – not only at Helmholtz, but across the entire research system. So one could touch upon many other questions here that are perhaps not best described as evaluation. In any case there are a number of good reasons that the Federal Ministry for Education and Research asked the Council for Science and Humanities to present proposals for the onward development of programme-oriented funding.
You say that the term evaluation is perhaps not entirely appropriate. In an evaluation it would primarily be a question of whether to cease or continue the process.
And precisely this simple question is not what we are considering at this time. Proposals from the Council of Science and Humanities can and should help, first and foremost, to make improvements and optimise processes. Naturally we also want to understand how PoF has changed over time – and why. Above all we are interested in the question of how PoF functions as a funding and steering tool. There is no doubt that the working group will handle its task with a great sense of responsibility. It will certainly not suggest scrapping PoF if it cannot propose a better tool. As such, it is likely to be a matter of potential ways to develop.
To develop PoF or Helmholtz?
The governance that links the two together.
Programme-oriented funding (PoF)
In 2004 the Helmholtz Association began distributing its funding resources between cross-centre research programmes in which the current total of 18 member centres are active. The researchers from the Helmholtz centres have designed these programmes, which are in competition with one another and are evaluated every five years by renowned international experts. This process aims to keep the Helmholtz research directed towards key future topics and ensure that it can compete on an international level.
With PoF we are basically talking about an administrated peer-review process: researchers evaluate researchers, in most cases specialists in the same field. Is it time to rethink peer-review processes altogether?
I think the real question is somewhat different. Is the peer-review process that has been chosen and implemented suitable for that particular case? The costs of PoF are not especially high in relation to the sums ultimately in play.
So it’s more a question of whether this peer-review process is fine-grained enough. Does it go into the necessary depth? And does it lead to sufficient consequences? Could you put that in more concrete terms?
We want to find out how the reviewers’ assessments are incorporated into the onward process. How are they interpreted or filtered? To what extent are they ultimately a reference system for the financing decisions? How do they affect the orientation of the programmes? These are the points to focus on, if you will.
Who belongs to such a working group of the Council of Science and Humanities and makes the decisions?
Above all they are well-grounded and experienced people. Such a group brings together a high degree of specialist expertise, and they are also colleagues who know the international processes well and are familiar with governance issues – people occupying top positions in research organisations and universities.
And now they are coming under pressure from all sides?
Not at all. In my experience, the research scene leaves the working groups to work in peace. Of course Helmholtz lets them know: If you need any information, please tell us, you’ll have it right away. I think that’s good, but I don’t associate that with the idea of exerting influence. And in this context there are no attempts at all by other research organisations to make contact informally.
How do the working group’s meetings proceed?
First we get an enormous amount of background information. About Helmholtz itself, about PoF, all kinds of data to give us an overview. This provides a basis for the first meetings, where we formulate our questions. Next we request additional data. Then there are hearings; we invite other parties for dialogue, Helmholtz people, partners, policy-makers. Also the fund allocators from the national and state governments. Finally we approve those proposals that are most concrete and constructive.
How many times does the group meet before a result is reached?
I would say at least five meetings of one or two days each. It’s quite an undertaking. After every meeting there are detailed minutes that we work through the next time, looking to see where there is consensus. That will form the basis of what ends up in the report.
Does the working group send delegations into the centres?
It’s a possibility, but we certainly won’t turn up on a kind of raid and start grilling individual employees. Whether or not we are present on site, naturally we are going to be interested in people who play a role at Helmholtz, whether on the level of the centres or the programmes. It’s a matter of taking a random sample, of course.
Do you also ask doctoral and post-doctoral students about their experience with Helmholtz?
I wouldn’t rule it out. Although I would question what I would gain from interviewing a post-doc student. Might I not get more precise information if I ask Helmholtz for career statistics? That might be more interesting for us than squeezing every detail from an individual in a two-hour interview.
When can the results of the working group be expected?
I hope we will be able to present our proposals in October 2015.
A few days ago you took up office as Chairman of the Council of Science and Humanities. What do you regard as your key tasks for the coming years?
For me it is firstly a matter of developing what we have inherited as effectively as possible. This inheritance includes, for instance, the paper on the “Prospects of the German Research System”. Here it is a matter of seeing whether and how the proposals, such as those on the image of the research community, have been implemented and how development can be continued. Strengthening universities is and will remain an important topic. I would like to pay particular attention to the situation and the prospects of young researchers. Another equally important task will be to take every opportunity to make it clear how central research and teaching are to our society – not only for the economy, but for everybody’s wellbeing. I fear this is sometimes forgotten.
Manfred Prenzel, 62, has been Chairman of the Council for Science and Humanities since 1 July. Prenzel is one of Germany’s leading education researchers, with roles including Project Manager of the Pisa studies in 2003 and 2006. As the Founding Dean of the School of Education at the Technical University of Munich, he established a teacher training faculty that is regarded as a national benchmark. Prenzel heads the working group of the Council of Science and Humanities to evaluate programme-oriented funding (PoF) at Helmholtz.