By Prof. Mike Payne, University of Cambridge
In many countries there is increasing demand for measurable socio-economic impact from academic research. Perhaps the UK is furthest down this path with a significant fraction of the funding for Universities dependent on the ‘Impact’ (defined as impact outside of academia) of the research performed [1]. However much we might wish to ignore this trend, I am convinced that it will only increase, at least over the short to medium term. I also believe that, as a community, we should try to engage with this process and we must not pretend that this task is straightforward. For instance, some of the questions that industry might ask when assessing a computational methodology are:
1. Is it relevant – does it determine quantities of interest to me?
2. Is it accurate/predictive?
3. Is it easy to use?
4. What is the cost compared to present methodology/other solutions?
Addressing these questions is not straightforward, some of them differ markedly from the academic world view and there is a degree of conflict between them. For instance, in relation to the first question, the topics and problems that academics think are interesting are not necessarily those of most importance to industry. If academics are to engage with industry then they do need to be aware of what issues are of concern to industry.
A further complication is that planning horizons in industry tend to be shorter than in academia and priorities can shift rapidly. It is not uncommon for an academic to work on a project that is believed to be relevant to industry only to find that industry’s priorities have changed before the project comes to fruition. Similarly, it can be difficult to get industry interest, let alone engagement, for long term academic research that may take a decade to complete no matter how large a potential benefit is claimed for the work.
On the second question, the electronic structure community is, perhaps, guilty of suggesting that if a computed result does not come directly from quantum mechanics or some equally rigorous underlying model then it cannot be accurate or predictive. In contrast, industry only cares about whether a method gives useful results and is happy to accept a certain fraction of failures as long as this fraction is sufficiently small. The rapid adoption of machine learning methods throughout industry clearly shows the acceptance of this more pragmatic approach to problem solving.
The amount of theoretical or modelling work carried out in many companies, particularly in materials, is currently generally small. This is in stark contrast to many branches of engineering where the entire design cycle is performed computationally. However, we should be prepared to admit that at present and for the foreseeable future we are not capable of virtual design, testing and certification of real world materials. Until this point is reached, modelling will be a somewhat niche activity for most companies. Often a small group of modellers provide services to a whole company and, as a result, constantly shift from one problem to another, each one requiring different methods of solution. Not surprisingly, the ease of use of each modelling method then becomes paramount as the modellers do not have the time to become expert in each method they use – hence the importance of the third question.
The final issue of cost is superficially easily understood by academics. However, for industry to change methodologies there usually has to be a significant cost advantage to justify the disruption and risk associated with any change of process. While it is difficult to quantify when any company would change approaches it is much more likely to occur when there is a factor of 10 cost advantage than a factor of 2 and, unless it concerned a methodology that was used very widely by the company, then any cost advantage of less than a of 2 would probably not be sufficient to persuade the company to change. Here, computation has a significant advantage over experimental methods as the costs of compute decrease significantly over time whereas experiment tends to either increase in cost or reduce at a much slower rate. However, the computer is only one part of the cost of the ‘modelling’ effort for a company and the other costs, such as personnel costs, will not reduce with time.
So far, we have concentrated on the potential disconnect between industrial use of academic software and some of the challenges to industry and academia working together. There are, of course, many examples of long term successful collaborations between companies and academic researchers which prove that these challenges are not insurmountable. Indeed, one of the simplest ways of overcoming some of the difficulties is simply for industry to fund projects in academic groups and there is evidence for very wide use of this model in the electronic structure field [1].
Technology transfer from the academic world to industry is a difficult process in all scientific fields. Software has some particular advantages that make the process somewhat easier. In principle, it is possible for academic research software to be sold commercially. In the terminology of Technology Readiness Levels – TRLs [2], it is possible for the exactly the same piece of software that is used to develop novel scientific functionality (TRL1) to be sold commercially (TRL9). This contrasts markedly with a commercialisation of a new material discovered in the laboratory. This might involve a continuous move through the TRLs starting from the one-off academic research experiment at TRL1 all the way to industrial scale production with tight quality controls and certification at TRL9. This is a very expensive process and slow process. In the case of materials, the time to market is typically 20 years or more. Speeding up this process is a major driver of the Materials Genome Initiative [3].
So why is not far more academic software used in industry? Often the reason is that it fails under the initial questioning of relevance, accuracy, ease of use or cost. If it passes these tests, there may be other reasons for lack of adoption associated with the software itself. There may by questions of code ownership that will prevent commercialisation or the code itself may be of poor quality and/or lack documentation and/or lack a suitable test suite. The author feels that the over-emphasis on Open Source software did little to address these problems. While there are many examples of excellent quality Open Source software, unfortunately, Open Source on its own is no guarantee of quality. It is often said that research funders do not understand the intellectual challenges of software development and do not properly fund this process. Certainly, such arguments can be further extended to their inability to differentiate between good quality and poor quality software. The UK can be rather proud of recent changes in this area and, in particular, EPSRC (Engineering and Physical Sciences Research Council) have recognised the importance of Research Software Engineers (RSE) and have introduced funding for RSE Fellowships – at the same time the RSE community in the UK is organising itself into a self-help and self-support community [4]. Other countries are also responding to the challenges of scientific software, such as the US with its recently launched Computational Materials Sciences Centres [5]. If these initiatives are successful and are copied elsewhere then this will significantly enhance the degree of industrial adoption of academic software in the future.
One of the goals of E-CAM is to strengthen interactions between academic research and industry. This will be a difficult challenge and one that all the members of E-CAM must take responsibility for and make efforts to address. Over the years, I have commissioned reports from Goldbeck Consulting on the economic impact of molecular modelling [6], industry interactions of the electronic structure research community in Europe [1] and I contributed funding for a report on the economic impact of materials modelling [7] prepared for the European Materials Modelling Council [8]. These reports provide useful background information to those of us who wish to interact with industry. For those who are interested in commercialisation of software, there are a series of reports prepared by the Software Taskforce of UK E-Infrastructure Leadership Council [9] which provide guidance and advice along the whole of the technology transfer path. <!–Copies of these reports can be found on the E-CAM Website [10].–>
References
[1] Industry interactions of the electronic structure research community in Europe, G. Goldbeck, 2014 (available from https://zenodo.org/collection/user-emmc).
[2] https://en.wikipedia.org/wiki/Technology_readiness_level
[3] https://www.whitehouse.gov/mgi
[4] http://www.rse.ac.uk/who.html
[5] http://science.energy.gov/bes/funding-opportunities/closed-foas/computational-materials-sciences-awards/
[6] The economic impact of molecular modelling, G. Goldbeck, 2012 (available from https://zenodo.org/collection/user-emmc).
[7] The Economic Impact of Materials Modelling, G. Goldbeck and C. Court, 2016 (available from https://zenodo.org/collection/user-emmc).
[8] https://emmc.info/
[9] https://www.gov.uk/government/groups/e-infrastructure-leadership-council
