The First Open Science Event at ITC
On the 15th December 2020, we hosted our first Open Science Day at ITC. Well, it was an afternoon of 2,5 hours, but we still covered many aspects in the context of Open Science (OS).
Let’s take a step back and check why we attach importance to such an event at ITC. First of all, OS is becoming increasingly important. Several high-level organizations published OS statements, for example, UNESCO’s Open Science recommendation or EU’s Open Science policy. Also, bottom-up initiatives (e.g., OS Communities) are working hard to make OS the new norm. Moreover, UT’s Shaping2030 agenda is hardly achievable without OS. However, acknowledging that OS is critical does not mean it is easy. Doing OS can require quite some effort from researchers, starting already with several very fundamental questions such as:
- What does OS mean, and which OS practices exist?
- What are the benefits and limitations of these practices?
- Why is OS important for my work and science in general?
The goal of the OS Day was to shed some light on these questions. One event is not sufficient to answer these questions in detail but we aimed at raising awareness for OS and discussing its incentives as well as the cultural barriers that hinder scientists from sharing their research materials. The agenda included a brief introduction to what OS means, the role of OS at ITC, a keynote by Prof. Dr. Frank Miedema, and a panel discussion (see Materials section for the video) .
Opening
To harmonize everybody’s understanding of the OS concept, we started with a brief introduction to the OS elements (see Figure 1), including well-known practices such as Open Access and less familiar aspects such as Open Evaluation. For a more detailed definition of the individual OS practices, check our website.
Figure 1: Open Science definition based on UNESCO's Draft Recommendation on Open Science.
ITC Ph.D. Publication Award 2020
Afterwards, ITC’s dean Prof. Dr. Freek van der Meer took over the stage to present this year’s ITC Ph.D. Publication Award 2020. The three nominated papers were:
- “Detection of radioactive waste sites in the Chornobyl exclusion zone using UAV-based lidar data and multispectral imagery” by Briechle et al. (2020),
- “Understanding wheat lodging using multi-temporal Sentinel-1 and Sentinel-2 data” by Chauhan et al. (2020), and
- “UAVid: A Semantic Segmentation Dataset for UAV Imagery” by Lyu et al. (2020),
Sugandh Chauhan’s fantastic work was the chosen winner. She received not only congrats but also a nice cheque. Congratulations from our side as well!
ITC’s role in the context of Open Science
In the following, Prof. Dr. Freek van der Meer spoke about the role of OS at ITC. He acknowledged that there is no doubt regarding the importance of the ongoing OS movement at ITC and emphasized that
However, he also paid attention to the downsides of OS, for example, regarding the
To give the rest of the day some guidance, Freek concluded his lightning talk with three open questions inspired by Simon Sinek’s Golden Circle: What kind of products and services do we want to develop in the context of OS? How do we differentiate ourselves from the competitors? And most important: Why are we doing these things?
Keynote by Prof. Dr. Frank Miedema
The next point on the agenda was an insightful keynote by our guest speaker Prof. Dr. Frank Miedema from Utrecht University with a chair for Open Science (see Materials section for the slides). Frank was fortunately not reluctant to talk about his personal development from a scientist who "played the game" to an "Open Scientist". He started his talk with the credibility cycle and its flaws (see Figure 2). In short: Researchers who have scientific recognition receive funding to do their research and to hire staff. They collect data and then publish articles, which again result in more recognition.
Figure 2: Credibility cycle, adapted from Latour and Woolgar (1986).
One might assume that high recognition is the result of having published high-quality articles. However, Frank stated that the traditional evaluation criteria (e.g., number of articles, journal impact factor, citations) often focus on quantitative numbers, which are poor quality metrics.
Moreover, these metrics result in researchers focusing more on novelty and quantity than on quality, replication, relevance, and impact. Furthermore, the huge competition for funding works against good scientific practices such as team-science, multi-disciplinarity, and diversity. The "short-termism" of the research cycle of around four years for Ph.D. candidates and grant holders adds to the problem. He said that many important questions cannot be answered within such a short time. This issue keeps researchers away from the essential questions with high societal but low scientific impact. Read more about why Frank thinks that focusing on the people and the problem is more important than the scientific impact in an article published in the Guardian.
In addition, Frank explained how OS helps to bring science and society closer together (see Figure 3). Open Access, FAIR data & software, Public Engagement, and a better Recognition and Rewards system (which is probably the most difficult aspect to change) can help to create a scientific landscape that values sharing of materials, involving society, and co-operation instead of competition. This idea sounds plausible, but he also considered that the OS programme is a process that is
Figure 3: Bringing science and society together with the help of Open Access articles, FAIR data and software, Public Engagement, and Recognition and Rewards. Utrecht University Open Science Programme.
He argued that the same applies to code and data for which we need a different reward system that values sharing and reusability instead of the number of papers and where they are published. Such a reward system should help researchers make a career. In this regard, he also addressed the often mentioned fear of being scooped. In his opinion, scooping can happen but solving this issue always comes down to giving credit to researchers for what they are doing, and this not just writing papers but also collecting data and writing software. Consequently, there should be more reward for the data producer or the software developer. Then, people will share their data and code also because funders increase the pressure to share materials. Nevertheless, it is difficult to change the scientific workflow according to the OS principles, and it will take time. Or as McKiernan said:
Frank also recognized the limitations of Open Science and agreed with what Freek said about the asymmetric benefits in favour of the global north.
Countries with little money cannot afford to publish Open Access, which might discriminate against their researchers when evaluating the work based on OS metrics. This issue is a political one, and more work is needed to make OS a global initiative. UNESCO’s recommendation on OS is an initial step.
Open Science @ ITC
In the next presentation, ITC’s Open Science Officer Markus Konkol briefly presented what happened at ITC in the context of OS (see Materials section for the slides) . We demonstrated the website, including a detailed definition of what OS means, upcoming activities, and training materials. We also recently released our first blog post about the Citizen Science Conference. Attention was given to the Open Science Community Twente, a bottom-up community to promote, learn, share, and discuss OS practices. Everyone can become a member of the community. It is not required to have expertise, only enthusiasm for Open Science. Do not hesitate to JOIN. If you want to stay informed, you can also subscribe to our monthly newsletter (see the December edition) and follow us on Twitter. We were also happy to announce that the User Committee is complete. They will check our OS plans at ITC and signal the OS-related needs of their colleagues to us, or forward interesting news to their departments.
Panel Discussion
The panel discussion with Prof. Dr. Frank Miedema (Frank in the following), Prof. Dr. Karin Pfeffer (Karin in the following), and Prof. Dr. Raúl Zurita-Milla (Raúl in the following) was the final highlight of the day. We introduced the opening question with a short scenario.
The successful but “closed” scientist
A researcher is looking for a job. The applicant has an, in our traditional understanding, impressive curriculum vitae, including many frequently cited papers. Some of the papers were even published in high-ranked journals such as Science and Nature. The only problem: The researcher does not like Open Science and is not planning to become more open and transparent.
We wanted to know from the participants: Who of them would hire the candidate?
Frank was reluctant since being against OS is also against the OS policies in his university. Karin would not reject the person immediately but first, try to find out why the person is against OS. She prefers evaluating the entire package, which includes aspects beyond OS (e.g., how this person fits the institute) and argued:
Similarly, Raúl would like to learn more about why the applicant is reluctant to do OS
He would like to convince the person of the OS benefits and emphasized that it is not about imposing OS principles on researchers.
The initial discussion raised the question of how to implement OS in research environments. Karin stated that we cannot achieve a real OS environment if we impose OS principles in a top-down manner. Researchers should develop an intrinsic motivation to do OS. Frank agreed but also considered that funders increasingly require access to research materials and, thus, play an important role, too. The implementation of OS also touches the Ph.D. graduation requirements, which sometimes demand a specific number of papers but publishing OS papers takes more time. To address this issue, Karin suggested:
Raúl agreed and put supervisors as well as evaluation committees in charge to value also outputs and activities other than traditional papers. Frank considered that simply removing such rules might not be enough because there are still supervisors who stick to the old rules.
How open are scholars at ITC?
For the next question, we ran a survey with the audience and asked how open they are on a scale from 1 (fully closed science) to 5 (fully open science) regarding the listed OS practices. Figure 4 shows the results based on 24 responses.
Figure 4: Survey results to the question: How open are you? Locate yourself on the scale from 1 (fully closed science) to 5 (fully open science).
Karin started analysing what the results mean for our work at ITC. Her first key insight was that we need to create more awareness for the OS principles:
In her opinion, a culture change is required, particularly when it comes to Open Peer Review where "we still feel a bit shy" . Due to the number of survey responses, Raúl sees a lot of interest in discussing these points and concludes from the other numbers, for example, regarding pre-registration that we need hands-on skills. He also stated that we need to continue investing in open infrastructures such as the Open Research Europe platform provided by the EU. Researchers holding an EU grant can submit a paper for review and then also publish OA papers for free.
How relevant is Open Science?
The second question to the audience was about how relevant they think the Open Science practices are for their work. Figure 5 shows the results based on 20 responses.
Figure 5: Survey results to the question: How relevant are the following Open Science practices for your work? Please use the scale from 1 (not relevant at all) to 5 (fully relevant).
Frank was a bit surprised to see the low values for pre-registration and was wondering whether the researchers might not feel safe and fear that others steal their ideas. After checking the chat where some of the people from the audience started to discuss the term ‘pre-registration’, Raúl had an initial answer to Frank:
Karin added that a key success criterion for the whole idea of OS is building trust. Particularly young researchers need to be sure their work is recognized, and they can continue with their ideas and eventually graduate.
Fear of being scooped
The last question was introduced by a quote from the book Open Access by Peter Suber (2012):
We wanted to know: Is this is also true for Open Data and Open Code?
Frank agreed and said that data can be linked to ORCID, which mitigates the risk of abusing shared data. Nevertheless, he pointed out that researchers in charge (e.g., dean, department heads, supervisors) have to provide incentives and reward those who share materials. Then, people will see that they can make a career based on shared data. Besides open code and data, Frank also considered open reviews to avoid that competitors who review a paper block them for competitive reasons. Raúl generally agreed to the idea of Open Peer Review but also recognized that writing an open peer review is different from the traditional double-blind review:
Karin expects open reviews to be more constructive and less harsh. Besides, this way of reviewing articles might pave the way for a direct dialogue between the authors and the reviewers. This benefit might be true not only for scientific articles but also for research grant proposals.
New year’s resolution
The closing question asked each participant for a new year’s resolution in the context of Open Science. Frank is planning to make early-career researchers think more about what science is about (including its limitations in the system), eventually resulting in more scientific literacy. Karin’s resolutions are to publish a dataset for a paper published 15 years ago and take a look into pre-registration. Raúl would like to learn more about the process of OS and the different approaches, understand how OS changes the scientific landscape within and outside the Netherlands, and spread the word about our Open Science Community Twente.
Wrap-up
This was the report of the first Open Science Day at ITC. We heard some inspiring words from our keynote speaker Prof. Dr. Frank Miedema and ITC’s dean Prof. Dr. Freek van der Meer. We are grateful for the support from our dean, which is motivating and shows that OS is an essential point on the agenda of ITC. We hope that everyone in the audience and those reading this report could get some ideas out for their work. Maybe a new year’s resolution in these uncertain times? What we can say for sure is that this was just the first of many upcoming Open Science Days.
Materials
Video:
Slides by Prof. Dr. Frank Miedema.
Open Science at ITC presentation.
By Markus Konkol