The Young Researcher’s Workshop on Machine Learning for Material Science took place in the Aalto Design Factory, Espoo, Finland on date 06th-10th/05/2019. Workshop programme, abstract book, and workshop material (i.e. tutorial material, registration of talks and slides) for download can be found at https://ml4ms2019.aalto.fi/.

Below we resume the highlight of the event.

ML4MS SCHOOL

The first two days of the workshop involved introductory talks and a one-day long hands-on tutorial session. The aim of this initial workshop programme was to bestow the attendees with a pedagogical and practical introduction to the most established tools and techniques exploiting machine learning algorithms employed to solve outstanding problems in physical chemistry and chemical science.

On Monday, Dr. Luca Giringhelli introduced the attendees to the nuances of material space exploration via regularized and symbolic regression, together with a didactical intro on both supervised and unsupervised learning. The key role of descriptors that need to capture the complexity of the physical system under scrutiny was highlighted. A state of the art application to the agnostic and insightful classification of binary compounds was presented. The topics of open science, reproducibility and good use of repositories were also tackled in detail.

The next talk was carried out by Dr. Nongnuch Artrith. She discussed on the development of efficient and accurate artificial neural network potentials for the simulation of complex materials. Descriptors tailored for the use of neural networks for force field generation were examined in depth.

Successively a set of cutting-edge applications of neural network force fields for the study of heterogeneous catalysts and Li-based batteries were showcased.

On Tuesday the hands on session guided by Dr. Filippo Federici Canova foresaw a set of four tutorials covering the implementation and algorithmic aspects of four topics: materials descriptors, unsupervised learning and dimensionality reduction schemes, neural networks, and kernel ridge regression.

The hands-on session thus empowered the attendees with a practical set of tools introduced in the talks by Dr. Ghiringhelli and Dr. Artrith.

Photos taken during the hands on session. Webcast from the Finnish Centre for Computational Science was streamed to the ADF location and online.

In the morning of Wednesday, Prof. Michele Ceriotti introduced the use of machine learning methods to predict properties that do not transform as scalars. Commonly used descriptors employed in machine learning algorithms for material science were formalized, together with the introduction of a Dirac Notation as a tool to understand machine learning algorithms from a physical perspective. Examples of accurate and fast prediction for complex properties such as NMR spectra, response tensors and density fields were demonstrated.

Later, following the last-minute absence of Prof. Gabor Csanyi, two of the organizers, Claudio Zeni and Aldo Glielmo, introduced the audience to the use of Gaussian process regression for force fields generation. Their talk comprised a pedagogical introduction to regression and to development process of kernels for Gaussian regression that encode the physical symmetries relevant to the prediction of forces and energies. A discussion on how to speed up machine learning potentials via mapping procedure, with a paradigmatic example for the case study of the melting of a nanoparticle, was presented.

ML4MS WORKSHOP

The second part of the workshop foresaw the alternation of 14 invited talks and 12 contributed ones, together with two 1 hour panel-discussions. The aim of the workshop was to endow the young generation of researchers in the broad community of material science, condensed matter physics and biophysics with a comprehensive and exhaustive overview of the tools and techniques emerging in the rapidly developing field of machine learning applications to physical chemistry and chemical science. The wide community of attendees, also comprising mathematicians and computer scientists, nurtured cross-subject talks and interesting debates upon transferability of solutions to outstanding problems and challenges. Engagement of Industrial partners (IBM RXN LAB, IBM POWERAI LAB, NVIDIA, CURIOUS AI) further broadened the range of problems discussed. The fertile ground for learning from and interfacing with world leaders in the field was highly appreciated by the community together with the interdisciplinary range of topics covered during the talks and the panel discussion.

Wednesday

After the opening remarks by Dr. Milica Todorovic, the workshop began with two industry talks from IBM Labs. Philippe Schwaller discussed the latest development in the IBM RXN for Chemistry Lab, where a multi-head attention Molecular Transformer model was trained to predict Chemical Reactions, demonstrating a state-of-the-art 90% accuracy on common benchmark datasets, also outperforming the chemical intuition of human experts in the field. Jukka Remes from IBM PowerAI/Watson Machine Learning Accelerator Lab instead discussed the infrastructure for open source -based scalable and productive deep learning which has been latest developed in IBM.

The Bayesian afternoon session was then started by Prof. Antonietta Mira, which introduced the Bayesian formalism and a recent technique gaining momentum in several fields of statistics: Approximate Bayesian Computation. This is a likelihood-free algorithm used to carry on statistical inference where pseudo-data by forward-simulations can be generated. A set of multidisciplinary examples were then reported together with two examples related to the force-field calibration of TIP4P models for water and LJ potentials for He to match experimental data.

Lukas Hörmann followed by discussing the use of Bayesian linear regression to extract formation energies of different arrangements of molecules adsorbed on surfaces. molecule-molecule interactions were encoded in a feature space based on atom distances separating the contributions of various molecule-fragments to the total energy of the system. In a similar fashion, Jari Järvi discussed the use of Bayesian Optimization Structure Search for detecting Stable Surface Adsorbates for the case of conformers of C10H15O deposited on a Cu(111) surface.

During the Wednesday afternoon session, Michael Sluydts also discussed active learning strategies towards locating favourable point defects in silicon. He pointed out how the complex problem of searching for relevant configurations to train machine learning algorithms should be intelligently guided, e.g. by means of uncertainty estimates.

Finally, Prof. Teemu Roos provided an overview of the goals and achieved progress in the MachQu project, dedicated to the development of novel machine learning tools for quantum-mechanics based simulations. The example of neural network force fields for the case of damaged fusion reactors was showcased, together with the use of cross-entropy methods to account for uncertainty and drive active learning.

The day found its end with the conference dinner and poster session. Here participants had the opportunity to showcase their research and interact in an informal and relaxed atmosphere while profiting from the great oriental food catered in the main hall of the building.

Snapshot from the poster presentation.

Thursday

A generative model Thursday morning was bootstrapped by Volker Roth, introducing the audience to the topics of data topology and the use of deep latent variable models for exploring the chemical space of complex systems for an application in solar energy harvesting.

On a similar note Prof. Rafael Gomez-Bombarelli demonstrated the use of variational autoencoders in drug design and coarse graining model. The accurate instruction of the latent variable spaces of semi-supervised deep autoencoders were key in guiding the identification of optimal coarse graining functions and highly sought molecules.

In the same session Niko Oinonen reported on the development of a deep learning infrastructure to match AFM images with an unique descriptor that characterizes molecular configurations. The case study of 1S-camphor on Cu(111) was discussed bas on low-temperature AFM measurements.

Configurational sampling, in the context of global minimum searches, was also discussed by Jakub Kubečka. Heuristic swarm algorithms and machine learning were here demonstrated as a powerful set of tools to explore the energetic landscape of molecular clusters relevant in atmospheric physics.

High-throughput searches and the identification of physically informed descriptors to drive machine learning method for fast and accurate screening of materials were also touched upon in the Thursday morning session, as presented by Dr. Kamal Choudhary

Industry talks took place after lunch, where two leading Finnish companies, NVIDIA and CURIOUS AI discussed the application of Machine learning outside material science, in the paradigmatic cases of autonomous driving with deep learning (NVIDIA) and autonomous reasoning task via model based learning (CURIOUS AI)

Prof. Karsten W. Jacobsen took the baton of the generative models discussion and demonstrated their use in predicting material properties without knowing where the atoms are. Here text-based approaches were introduced to the audience with a state-of-the-art application in the case of perovskite materials for energy harvest.

Dr. James Cumby presented a discussion on the use of ellipsoids as a coordination descriptors, key to rationalize phase transitions also in the case of perovskites. Following the thorough discussion of several paradigmatic examples where this novel geometric descriptors proved to be powerful order parameter, Dr. James Cumby also resulted the winner of the Young Researcher’s award for best talk, sponsored by Wiley & Co.

The late afternoon was then taken over by the discussion of machine learning methods for material science applied in the context of many-body electronic structure problems

James Nelson discussed a methodology for utilising information from small cheaply calculated systems to predict quantities for larger systems using machine learning for the case of the prediction of the ground state energy and thermodynamic functions of the disordered one-dimensional Hubbard Model.

Jonathan Schmidt showed how automatic differentiation of modern machine-learning techniques can be exploited to develop a neural network functional for the exchange-correlation energy and the exchange-correlation potential of density functional theory, which achieves good accuracy and allows for flexibility by tuning its degree of locality.

Finally Prof. Thomas Hammerschmidt spoke on the use of machine learning descriptors based on the moments of the local electronic density of states of a material. Using the first four moments is sufficient to establish robust structure-energy relation In many crystalline and non-crystalline structures. Taking into account more moments leads to highly competitive descriptors for machine learning applications, as demonstrated in the recent NOMAD open challenge on transparent semiconductors.

Friday

A biophysics flavoured Friday morning started with the talks of Dr. Michele Allegra and Dr. Simon Olsson.

Dr. Michele Allegra presented the development of a simple and robust approach to cluster regions with the same local intrinsic dimensionality in a given data landscape. The paradigmatic example of folded vs unfolded configurations in a protein, as found in long molecular dynamics trajectories, was discussed.

Dr. Simon Olsson outlined recent advancements in the modelling of molecular kinetics using neural network strategies to parametrize coarse grain force fields, as well as in the use of augmented Markov models and dynamic graphic models to improve the accuracy and breadth of investigation of large molecular systems.

Neural networks for fitting coarse grained force fields for Li-batteries were also presented by Teo Lombardo in a talk covering aspects of Battery design and production from synthesis to deployment.

Neural network were the centrepiece of the discussion of Dr. Franco Pellegrini, who presented a Python code based on TensorFlow that supports local environment descriptors and training of feed forward neural network with species-resolved architectures and is patched with LAMMPS.

Before the final talk, Alessandro Lunghi discussed the combination of ridge regression and bi-spectrum components to describe the potential energy surface of molecular compounds in a very general fashion and with chemical accuracy (below 1 kcal/mol ).

The ML4MS workshop was then concluded by the talk of Prof. Olexandr Isayev presenting ANAKIN-ME, a novel method to train neural network potentials based upon single-atom atomic environment vectors, which allows for improved accuracy, high transferability and great speed. An example of its application was demonstrated for the case of simulations investigating explicitly solvated protein folding and carbon atom-growth patterns from the vapour phase.

Discussion sessions

Beyond scientific talks, a great length time was allowed for informal discussion among participants.

Furthermore, panel discussion enabled to recap the main themes tackled by the invited researchers in a more organic narrative, while also allowing for open debates on emerging research trends in the field.

Snapshot from the panel discussion

Finally, we would like to note that the Aalto Design Factory, with its latest era Finnish design and common spaces thought for all needs, together with the organic and healthy snacks provided by the workshop, allowed to offer each participant refreshment and entertainment also beyond scientific professional development.

example of the beyond science leasure at hands of the participants

Feedback

-) 100% of the participants reported a positive or highly positive feedback on the conference organization

-) 95% of the participants judged the participation to the conference as significantly or highly beneficial to its professional development

-) 95% of the participants appreciated the quality of the workshop facilities

-) 90 % of the participants also reported a good or great enjoyment of the workshop from a personal perspective

-) The transfer of knowledge was deemed successful, with 95% of the participants reporting to have learnt about new approaches of Machine learning in Material Science, 70% of them ready to apply this newly acquired knowledge to their current work, and 66% of them also engaging in implementing new Machine Learning approaches in Material Science.

-) Correspondingly, 95% of the students expressed a positive or very positive feedback on the introductory lectures and on the overall workshop programme.

-) A similar percentage also greatly enjoyed the poster session, with a majority of the participants also mentioning this workshop as a fantastic opportunity for networking, feedback, and establishment of new collaboration.

-) A large majority of attendees also significantly appreciated the gains from the hands-on computational tutorial session, with a compound positive or greatly positive feedback of 82%.

-) Circa two thirds of the attendees also profited greatly from the industry talks and panel discussion session, with another 20% of participants reporting a neutral judgement on these activities.

Though we are scientists, sometimes words speak even better than statistic, here we report feedbacks:

“The overall organization was super great. I personally liked a lot the panel session. As far as I am concerned is the time where I learned the most. Nice job with the poster session and the diversity of the topics. Hope in the future this will grow even more.”

“Very well organized, good information about the location and facilities. The workshop website was well designed and streaming the talks online was well executed.”

ML4MS final photo

Participant List

137 participants attended the event. 19 were invited speakers (4 of which from industry), 12 were young researcher contributed speakers, while 32 were poster contributors; 6 organizers topped up the other attendees.

A 1:4 ratio of female:male participants was observed. By the same token, circa one fifth of the attendees was affiliated with a Finnish Institution. The breakdown of attendees by countries than reports UK & Germany placing each around 13% of the registered participants. A similar percentage is observed for researchers affiliated with universities outside Europe. Other significantly represented countries were France, Italy, and Switzerland, while a remaining 20% of participants were conducting research in university and research centres in other European countries.
Live streaming of the talks reached up to 50 viewers at a time, with an average of 30 per talk.

A list of participants containing name, affiliation, country of origin can be found at the end of document.

NAME SURNAME AFFILIATION COUNTRY

Kim	Nicoli	Technische Universität Berlin	Germany
Laia	Delgado Callicó	King’s College London	United Kingdom
Teng	Long	TU Darmstadt	Germany
Danilo	González Forero	Universitat Autònoma de Barcelona	Spain
Joaquin	Miranda Mena	University of Bayreuth	Mexico-Germany
Nirmal	Ganguli	IISER Bhopal	India
Carsten	Staacke	TU Munich	Germany
Soumyajyoti	Haldar	University of Kiel	Germany
Hamidreza	Hajiyani	University of Duisburg Essen	Germany
Martin	Deimel	Technical University Munich	Germany
Juan Santiago	Cingolani	Technical University Munich	Germany
Sina	Stocker	Technical University Munich	Germany
Simon	Wengert	Technical University Munich	Germany
Henri	Salmenjoki	Aalto University	Finland
Murat Cihan	Sorkun	DIFFER	Netherlands
Abhishek	Khetan	DIFFER	Netherlands
Ali	Hamedani	University of Helsinki	Finland
Jesper	Byggmästar	University of Helsinki	Finland
Matteo	Peluso	Scuola Normale Superiore Pisa	Italy
Paola	Torche	University of Southampton	United Kingdom
Luca	Bellucci	CNR-NANO	Italy
Xiong	Jingfang	Xiamen University	China
Olga	Miroshnichenko	University of Oulu	Finland
Behnam	Parsaeifard	Basel University	Switzerland
Francesco	Delfino	First Moscow State Medical University	Italy
Štěpán	Sršeň	University of Chemistry and Technology Prague	Czech Republic
James	Cumby	University of Edinburgh	United Kingdom
Eamon	McDermott	Independent researcher	Turkey
Ilia	Kichev	Sofia University	Bulgaria
Steven	Baksa	Pennsylvania State University	United States
Vitus	Besel	University of Helsinki	Finland
Yihuang	Xiong	The Pennsylvania University	United States
Karen	Dedecker	Ghent University	Belgium
Rodrigo	Pereira de Carvalho	Uppsala University	Sweden
Andrea	Silva	University of Southampton	United Kingdom
Michiel	Larmuseau	Ghent University	Belgium
Sabine	Matysik	University of Cambridge	United Kingdom
Dominika	Melicherová	Comenius University in Bratislava	Slovakia
Hossein	Kalashami	University of Antwerp	Belgium
Huzaifa	Shabbir	University of Vienna	Austria
Mario	Zauchner	Imperial College London	United Kingdom
Michele	Cutini	University of Turin	Italy
Javier	Heras-Domingo	Universitat Autònoma de Barcelona	Spain
Ivan	Yashchuk	VTT, Aalto	Finland
Marie-Pierre	Gaigeot	Université d’Evry – Paris-Saclay	France
Daria Ruth	Galimberti	Université d’Evry – Paris Saclay	Francia
Sana	Bougueroua	université d’Evry – Paris Saclay	France
Lefteri	Andritsos	King’s College London	United Kingdom
Joakim	Halldin Stenlid	Stockholm University	Sweden
Yu	Che	University of Liverpool	United Kingdom
Joris	Paret	Université de Montpellier	France
Aurelia	Li	University of Cambridge	United-Kingdom
Marta	Aragones-Anglada	University of Cambridge	United Kingdom
Matthias	Vandichel	Aalto University	Finland
Kunal	Ghosh	Aalto University	India
Lincan	Fang	Aalto University	Finland
Vaiva	Nagyte	The University of Manchester	United Kingdom
Adam	McSloy	Warwick University	United Kingdom
Nurcin	Ugur	Aalto University	Finland
Lucy	Whalley	Imperial College London	United Kingdom
Diksha	Dhawan	University of Michigan	United States
Francesco	Tavanti	University of Modena and Reggio Emilia	Italy
Zahra	Mohammadyarloo	Aalto University	Finland
Jie	Yao	university of bayreuth	germany
Djordje	Dangic	Tyndall National Institute	Ireland
Arnab	Majumdar	Uppsala University	Sweden
Andrejs	Cesnokovs	Institute of Solid State Physics	Latvia
Maria	Dimitrova	University of Helsinki	Finland
Boris	Dorado	CEA	France
Andrea	Sand	University of Helsinki	Finland
Tao	Jiang	Ecole Normale Superieure de Lyon	France
Ekaterina	Baibuz	University of Helsinki	Finland
Kasun Kalhara	Gunasooriya	Ghent University	Belgium
Sophie	Beck	ETH Zurich	Switzerland
Kai	Riedmiller	University Konstanz	Germany
Yunpei	Liu	Xiamen University	China
Feiwu	Zhang	Chinese Academy of Sciences	China
Ygor	Morais Jaques	Aalto University	Finland
Rocio	Bueno-Perez	University of Cambridge	United Kingdom
Justin	Villard	Ecole Polytechnique Fédérale de Lausanne	Switzerland
Alexander	Flachmüller	University of Constance	Germany
Hanne	Antila	Max Planck Institute of Colloids and Interfaces	Germany
Tuomas	Rossi	Chalmers University of Technology	Sweden
Fabrizio	Rovaris	University of Milano-Bicocca	Italia
Ismail Can	Oguz	ICGM-MACS	France
Fedor	Urtiev	Aalto University	Finland
Niko	Oinonen	Aalto University	Finland
Eun-Ae	Choi	Korea Institute of Materials Science	South Korea
Annika	Stuke	Aalto University	Finland
Victor	Claerbout	Czech Technical University in Prague	Czech Republic
Susi	Lehtola	University of Helsinki	Finland
Usman	Ahmed	University of Helsinki	Finland
Mikael	Johansson	University of Helsinki	Finland
Lukas	Wirz	University of Helsinki	Finland
Francesco	Marrafino	University of Salerno	Italy
Anna Maria	Nardiello	University of Salerno	Italy
Kristoffer	Simula	Aalto University	Finland
Yashasvi	Ranawat	Aalto University	Finland
Marc	Jäger	Aalto University	Finland
Prokop	Hapala	Aalto University	Finland
Anja	Aarva	Aalto University	Finland
Anais	Colibaba	Trinity College Dublin	Ireland
Konstantin	Karavaev	NUST MISIS Moscow	Russia
Lauri	Himanen	Aalto University	Finland
Timo	Roman	Nvidia	Finland
Nongnuch	Artrith	Columbia University	USA
Philippe	Schwaller	IBM Research Zurich	Switzerland
Harri	Valpola	Curious AI	Finland
Simon	Olsson	FU Berlin	Germany
Thomas	Hammerschmidt	ICAMS, Ruhr University Bochum	Germany
Rafael	Gomez-Bombarelli	MIT DMSE	United States
Jukka	Remes	IBM	Finland
Olexandr	Isayev	University of North Carolina	USA
Karsten W.	Jacobsen	Technical University of Denmark	Denmark
Teemu	Roos	University of Helsinki	Finland
Michele	Ceriotti	EPFL	Switzerland
Luca	Ghiringhelli	FHI Berlin	Germany
Filippo federici	Canova	Nanolayers ltd	Finland
Volker	Roth	Basel University	Switzerland
Michele	Allegra	Aix Marsill	France
Antonietta	Mira	USI Lugano	Switzerland
Jari	Järvi	Aalto University	Finland
Franco	Pellegrini	SISSA	Italy
Lukas	Hörmann	Graz University of Technology	Austria
Kamal	Choudhary	National Institute of Standards and Technology	United States
James	Nelson	Trinity College Dublin	Ireland
Jonathan	Schmidt	Martin-Luther Universität Halle Wittenberg	Germany
Alessandro	Lunghi	Trinity College Dublin	Ireland
Jakub	Kubecka	University of Helsinki	Finland
Michael	Sluydts	Ghent University	Belgium
Teo	Lombardo	University of Picardie Jules Verne	France
Niko	Oinonen	Aalto University	Finland
James	Cumby	University of Edinburgh	United Kingdom
Aldo	Glielmo	King’s College London	United Kingdom
Claudio	Zeni	King’s College London	United Kingdom
Kevin	Rossi	EPFL	Switzerland
Patrick	Rinke	Aalto University	Finland
Adam	Foster	Aalto University	Finland
Milica	Todorovic	Aalto University	Finland