Discussions of research having impact were for a long time limited to citation tracking, to estimate how much one piece of work influences subsequent explorations. However, with calls for research to have impact on the broader society—breaking out of the closed circle of research feeding yet more research—there’s a lot of interest in seeing how we might trace that impact pathway as it breaks through the membrane insulating the world of research. Altmetrics has held…

Continue reading

In last week’s post, I presented some of the major points of Rémi Barré’s keynote speech at STI 2017. In brief, he divides the evolution of S&T indicators into three phases. The first phase is one of indicators promising to elucidate (and thereby improve) the inner workings of science. The second phase is one of them being co-opted into the neoliberal turn, exposing scientific research to competitive pressures that Dr. Barré identifies as pushing science…

Continue reading

Continuing on from my previous discussion of impact, the second keynote speech at the 2017 Science & Technology Indicators (STI) conference in Paris was given by Rémi Barré of IFRIS, who echoed many of the points raised by Ismael Rafols. Barré’s call to action—riffing on a very traditional theme—was, “Les indicateurs sont morts! Vive les indicateurs!” Indicators are dead! Long live indicators! The call was provocative, and his talk highlighted some interesting ways in which…

Continue reading

In a blog post from earlier this year, Neil Lawrence describes some challenges to data mining projects that are familiar to many working in the domain—our team definitely included! These challenges include the availability and quality of the data available for the project. Data scientists are often faced with very detailed expectations of budgets and timelines for a project but are provided with very little information at the outset regarding what data they will have…

Continue reading

In our ongoing blog series on data mining for policy, we’ve been trying to synthesize a lot of information into short, bite-sized chunks for our audience. Invariably, well-intentioned as such efforts are, something valuable always ends up on the cutting room floor. In this case, we were a bit too hasty in providing the definition of data mining itself, which one of our readers followed up to ask about. Our initial definition was put together…

Continue reading

In our ongoing series on data mining to inform policy, we are giving the topic of data access its own post because of the implications it had for the success or failure of our case studies. The simple reality is that you can’t mine data that don’t exist (or that may as well not exist when they are functionally or realistically impossible to access). As a result, access is particularly important since it underpins the…

Continue reading

In my previous post, I laid out some history of research assessment and measurement, all so that in this post I could explore research impact assessment, which was a major topic of discussion at the 2017 Science & Technology Indicators (STI) conference in Paris. In this post, I’ll summarize the major lines of discussion I encountered at STI, use the history from the last post as a basis for diagnosing those underlying challenges, and perhaps…

Continue reading

During our data mining project for the European Commission, one of the case studies we undertook to test our framework to guide data mining for policy research explored open access (OA) publications in the European context. Specifically, the question we aimed to tackle was whether institutional OA policies have an effect on the share of an institution’s papers available in OA, and if so, to what degree. An answer to this question would provide actionable…

Continue reading

During the 2017 Science & Technology Indicators (STI) conference in Paris, a number of discussions touched on impact assessment, which has been a topic of growing interest within the research community. That researchers are increasingly aware of impact, impact pathways and impact assessments comes as no great shock, given that the research policy community is increasingly focusing on impact as the basis for funding decisions. The discussions at STI raised some substantive concerns with the…

Continue reading

In the most recent post in our ongoing data mining blog series, we explored the effect on innovation of research collaboration across disciplinary and sectoral boundaries. That topic was worth exploring because beliefs that such collaborations are effective levers to promote innovation are foundational to many policy choices, and there is scant evidence available to determine whether these levers work or not (and how powerful they are). The present post will take that line of…

Continue reading