Young Researcher’s Workshop on Machine Learning for Material Science 2019 – report

The Young Researcher’s Workshop on Machine Learning for Material Science took place in the Aalto Design Factory, Espoo, Finland on date 06th-10th/05/2019. Workshop programme, abstract book, and workshop material (i.e. tutorial material, registration of talks and slides) for download can be found at https://ml4ms2019.aalto.fi/.

Below we resume the highlight of the event.

ML4MS SCHOOL

The first two days of the workshop involved introductory talks and a one-day long hands-on tutorial session. The aim of this initial workshop programme was to bestow the attendees with a pedagogical and practical introduction to the most established tools and techniques exploiting machine learning algorithms employed to solve outstanding problems in physical chemistry and chemical science.

On Monday, Dr. Luca Giringhelli introduced the attendees to the nuances of material space exploration via regularized and symbolic regression, together with a didactical intro on both supervised and unsupervised learning. The key role of descriptors that need to capture the complexity of the physical system under scrutiny was highlighted. A state of the art application to the agnostic and insightful classification of binary compounds was presented. The topics of open science, reproducibility and good use of repositories were also tackled in detail.

The next talk was carried out by Dr. Nongnuch Artrith. She discussed on the development of efficient and accurate artificial neural network potentials for the simulation of complex materials. Descriptors tailored for the use of neural networks for force field generation were examined in depth.

Successively a set of cutting-edge applications of neural network force fields for the study of heterogeneous catalysts and Li-based batteries were showcased.

On Tuesday the hands on session guided by Dr. Filippo Federici Canova foresaw a set of four tutorials covering the implementation and algorithmic aspects of four topics: materials descriptors, unsupervised learning and dimensionality reduction schemes, neural networks, and kernel ridge regression.

The hands-on session thus empowered the attendees with a practical set of tools introduced in the talks by Dr. Ghiringhelli and Dr. Artrith.

Photos taken during the hands on session. Webcast from the Finnish Centre for Computational Science was streamed to the ADF location and online.

In the morning of Wednesday, Prof. Michele Ceriotti introduced the use of machine learning methods to predict properties that do not transform as scalars. Commonly used descriptors employed in machine learning algorithms for material science were formalized, together with the introduction of a Dirac Notation as a tool to understand machine learning algorithms from a physical perspective. Examples of accurate and fast prediction for complex properties such as NMR spectra, response tensors and density fields were demonstrated.

Later, following the last-minute absence of Prof. Gabor Csanyi, two of the organizers, Claudio Zeni and Aldo Glielmo, introduced the audience to the use of Gaussian process regression for force fields generation. Their talk comprised a pedagogical introduction to regression and to development process of kernels for Gaussian regression that encode the physical symmetries relevant to the prediction of forces and energies. A discussion on how to speed up machine learning potentials via mapping procedure, with a paradigmatic example for the case study of the melting of a nanoparticle, was presented.

ML4MS WORKSHOP

The second part of the workshop foresaw the alternation of 14 invited talks and 12 contributed ones, together with two 1 hour panel-discussions. The aim of the workshop was to endow the young generation of researchers in the broad community of material science, condensed matter physics and biophysics with a comprehensive and exhaustive overview of the tools and techniques emerging in the rapidly developing field of machine learning applications to physical chemistry and chemical science. The wide community of attendees, also comprising mathematicians and computer scientists, nurtured cross-subject talks and interesting debates upon transferability of solutions to outstanding problems and challenges. Engagement of Industrial partners (IBM RXN LAB, IBM POWERAI LAB, NVIDIA, CURIOUS AI) further broadened the range of problems discussed. The fertile ground for learning from and interfacing with world leaders in the field was highly appreciated by the community together with the interdisciplinary range of topics covered during the talks and the panel discussion.

Wednesday

After the opening remarks by Dr. Milica Todorovic, the workshop began with two industry talks from IBM Labs. Philippe Schwaller discussed the latest development in the IBM RXN for Chemistry Lab, where a multi-head attention Molecular Transformer model was trained to predict Chemical Reactions, demonstrating a state-of-the-art 90% accuracy on common benchmark datasets, also outperforming the chemical intuition of human experts in the field. Jukka Remes from IBM PowerAI/Watson Machine Learning Accelerator Lab instead discussed the infrastructure for open source -based scalable and productive deep learning which has been latest developed in IBM.

The Bayesian afternoon session was then started by Prof. Antonietta Mira, which introduced the Bayesian formalism and a recent technique gaining momentum in several fields of statistics: Approximate Bayesian Computation. This is a likelihood-free algorithm used to carry on statistical inference where pseudo-data by forward-simulations can be generated. A set of multidisciplinary examples were then reported together with two examples related to the force-field calibration of TIP4P models for water and LJ potentials for He to match experimental data.

Lukas Hörmann followed by discussing the use of Bayesian linear regression to extract formation energies of different arrangements of molecules adsorbed on surfaces. molecule-molecule interactions were encoded in a feature space based on atom distances separating the contributions of various molecule-fragments to the total energy of the system. In a similar fashion, Jari Järvi discussed the use of Bayesian Optimization Structure Search for detecting Stable Surface Adsorbates for the case of conformers of C10H15O deposited on a Cu(111) surface.

During the Wednesday afternoon session, Michael Sluydts also discussed active learning strategies towards locating favourable point defects in silicon. He pointed out how the complex problem of searching for relevant configurations to train machine learning algorithms should be intelligently guided, e.g. by means of uncertainty estimates.

Finally, Prof. Teemu Roos provided an overview of the goals and achieved progress in the MachQu project, dedicated to the development of novel machine learning tools for quantum-mechanics based simulations. The example of neural network force fields for the case of damaged fusion reactors was showcased, together with the use of cross-entropy methods to account for uncertainty and drive active learning.

The day found its end with the conference dinner and poster session. Here participants had the opportunity to showcase their research and interact in an informal and relaxed atmosphere while profiting from the great oriental food catered in the main hall of the building.

Snapshot from the poster presentation.

Thursday

A generative model Thursday morning was bootstrapped by Volker Roth, introducing the audience to the topics of data topology and the use of deep latent variable models for exploring the chemical space of complex systems for an application in solar energy harvesting.

On a similar note Prof. Rafael Gomez-Bombarelli demonstrated the use of variational autoencoders in drug design and coarse graining model. The accurate instruction of the latent variable spaces of semi-supervised deep autoencoders were key in guiding the identification of optimal coarse graining functions and highly sought molecules.

In the same session Niko Oinonen reported on the development of a deep learning infrastructure to match AFM images with an unique descriptor that characterizes molecular configurations. The case study of 1S-camphor on Cu(111) was discussed bas on low-temperature AFM measurements.

Configurational sampling, in the context of global minimum searches, was also discussed by Jakub Kubečka. Heuristic swarm algorithms and machine learning were here demonstrated as a powerful set of tools to explore the energetic landscape of molecular clusters relevant in atmospheric physics.

High-throughput searches and the identification of physically informed descriptors to drive machine learning method for fast and accurate screening of materials were also touched upon in the Thursday morning session, as presented by Dr. Kamal Choudhary

Industry talks took place after lunch, where two leading Finnish companies, NVIDIA and CURIOUS AI discussed the application of Machine learning outside material science, in the paradigmatic cases of autonomous driving with deep learning (NVIDIA) and autonomous reasoning task via model based learning (CURIOUS AI)

Prof. Karsten W. Jacobsen took the baton of the generative models discussion and demonstrated their use in predicting material properties without knowing where the atoms are. Here text-based approaches were introduced to the audience with a state-of-the-art application in the case of perovskite materials for energy harvest.

Dr. James Cumby presented a discussion on the use of ellipsoids as a coordination descriptors, key to rationalize phase transitions also in the case of perovskites. Following the thorough discussion of several paradigmatic examples where this novel geometric descriptors proved to be powerful order parameter, Dr. James Cumby also resulted the winner of the Young Researcher’s award for best talk, sponsored by Wiley & Co.

The late afternoon was then taken over by the discussion of machine learning methods for material science applied in the context of many-body electronic structure problems

James Nelson discussed a methodology for utilising information from small cheaply calculated systems to predict quantities for larger systems using machine learning for the case of the prediction of the ground state energy and thermodynamic functions of the disordered one-dimensional Hubbard Model.

Jonathan Schmidt showed how automatic differentiation of modern machine-learning techniques can be exploited to develop a neural network functional for the exchange-correlation energy and the exchange-correlation potential of density functional theory, which achieves good accuracy and allows for flexibility by tuning its degree of locality.

Finally Prof. Thomas Hammerschmidt spoke on the use of machine learning descriptors based on the moments of the local electronic density of states of a material. Using the first four moments is sufficient to establish robust structure-energy relation In many crystalline and non-crystalline structures. Taking into account more moments leads to highly competitive descriptors for machine learning applications, as demonstrated in the recent NOMAD open challenge on transparent semiconductors.

Friday

A biophysics flavoured Friday morning started with the talks of Dr. Michele Allegra and Dr. Simon Olsson.

Dr. Michele Allegra presented the development of a simple and robust approach to cluster regions with the same local intrinsic dimensionality in a given data landscape. The paradigmatic example of folded vs unfolded configurations in a protein, as found in long molecular dynamics trajectories, was discussed.

Dr. Simon Olsson outlined recent advancements in the modelling of molecular kinetics using neural network strategies to parametrize coarse grain force fields, as well as in the use of augmented Markov models and dynamic graphic models to improve the accuracy and breadth of investigation of large molecular systems.

Neural networks for fitting coarse grained force fields for Li-batteries were also presented by Teo Lombardo in a talk covering aspects of Battery design and production from synthesis to deployment.

Neural network were the centrepiece of the discussion of Dr. Franco Pellegrini, who presented a Python code based on TensorFlow that supports local environment descriptors and training of feed forward neural network with species-resolved architectures and is patched with LAMMPS.

Before the final talk, Alessandro Lunghi discussed the combination of ridge regression and bi-spectrum components to describe the potential energy surface of molecular compounds in a very general fashion and with chemical accuracy (below 1 kcal/mol ).

The ML4MS workshop was then concluded by the talk of Prof. Olexandr Isayev presenting ANAKIN-ME, a novel method to train neural network potentials based upon single-atom atomic environment vectors, which allows for improved accuracy, high transferability and great speed. An example of its application was demonstrated for the case of simulations investigating explicitly solvated protein folding and carbon atom-growth patterns from the vapour phase.

Discussion sessions

Beyond scientific talks, a great length time was allowed for informal discussion among participants.

Furthermore, panel discussion enabled to recap the main themes tackled by the invited researchers in a more organic narrative, while also allowing for open debates on emerging research trends in the field.

Snapshot from the panel discussion

Finally, we would like to note that the Aalto Design Factory, with its latest era Finnish design and common spaces thought for all needs, together with the organic and healthy snacks provided by the workshop, allowed to offer each participant refreshment and entertainment also beyond scientific professional development.

example of the beyond science leasure at hands of the participants

Feedback

-) 100% of the participants reported a positive or highly positive feedback on the conference organization

-) 95% of the participants judged the participation to the conference as significantly or highly beneficial to its professional development

-) 95% of the participants appreciated the quality of the workshop facilities

-) 90 % of the participants also reported a good or great enjoyment of the workshop from a personal perspective

-) The transfer of knowledge was deemed successful, with 95% of the participants reporting to have learnt about new approaches of Machine learning in Material Science, 70% of them ready to apply this newly acquired knowledge to their current work, and 66% of them also engaging in implementing new Machine Learning approaches in Material Science.

-) Correspondingly, 95% of the students expressed a positive or very positive feedback on the introductory lectures and on the overall workshop programme.

-) A similar percentage also greatly enjoyed the poster session, with a majority of the participants also mentioning this workshop as a fantastic opportunity for networking, feedback, and establishment of new collaboration.

-) A large majority of attendees also significantly appreciated the gains from the hands-on computational tutorial session, with a compound positive or greatly positive feedback of 82%.

-) Circa two thirds of the attendees also profited greatly from the industry talks and panel discussion session, with another 20% of participants reporting a neutral judgement on these activities.

Though we are scientists, sometimes words speak even better than statistic, here we report feedbacks:

The overall organization was super great. I personally liked a lot the panel session. As far as I am concerned is the time where I learned the most. Nice job with the poster session and the diversity of the topics. Hope in the future this will grow even more.”

Very well organized, good information about the location and facilities. The workshop website was well designed and streaming the talks online was well executed.”

ML4MS final photo

Participant List

137 participants attended the event. 19 were invited speakers (4 of which from industry), 12 were young researcher contributed speakers, while 32 were poster contributors; 6 organizers topped up the other attendees.

A 1:4 ratio of female:male participants was observed. By the same token, circa one fifth of the attendees was affiliated with a Finnish Institution. The breakdown of attendees by countries than reports UK & Germany placing each around 13% of the registered participants. A similar percentage is observed for researchers affiliated with universities outside Europe. Other significantly represented countries were France, Italy, and Switzerland, while a remaining 20% of participants were conducting research in university and research centres in other European countries.
Live streaming of the talks reached up to 50 viewers at a time, with an average of 30 per talk.

A list of participants containing name, affiliation, country of origin can be found at the end of document.

NAME SURNAME AFFILIATION COUNTRY

Kim

Nicoli

Technische Universität Berlin

Germany

Laia

Delgado Callicó

King’s College London

United Kingdom

Teng

Long

TU Darmstadt

Germany

Danilo

González Forero

Universitat Autònoma de Barcelona

Spain

Joaquin

Miranda Mena

University of Bayreuth

Mexico-Germany

Nirmal

Ganguli

IISER Bhopal

India

Carsten

Staacke

TU Munich

Germany

Soumyajyoti

Haldar

University of Kiel

Germany

Hamidreza

Hajiyani

University of Duisburg Essen

Germany

Martin

Deimel

Technical University Munich

Germany

Juan Santiago

Cingolani

Technical University Munich

Germany

Sina

Stocker

Technical University Munich

Germany

Simon

Wengert

Technical University Munich

Germany

Henri

Salmenjoki

Aalto University

Finland

Murat Cihan

Sorkun

DIFFER

Netherlands

Abhishek

Khetan

DIFFER

Netherlands

Ali

Hamedani

University of Helsinki

Finland

Jesper

Byggmästar

University of Helsinki

Finland

Matteo

Peluso

Scuola Normale Superiore Pisa

Italy

Paola

Torche

University of Southampton

United Kingdom

Luca

Bellucci

CNR-NANO

Italy

Xiong

Jingfang

Xiamen University

China

Olga

Miroshnichenko

University of Oulu

Finland

Behnam

Parsaeifard

Basel University

Switzerland

Francesco

Delfino

First Moscow State Medical University

Italy

Štěpán

Sršeň

University of Chemistry and Technology Prague

Czech Republic

James

Cumby

University of Edinburgh

United Kingdom

Eamon

McDermott

Independent researcher

Turkey

Ilia

Kichev

Sofia University

Bulgaria

Steven

Baksa

Pennsylvania State University

United States

Vitus

Besel

University of Helsinki

Finland

Yihuang

Xiong

The Pennsylvania University

United States

Karen

Dedecker

Ghent University

Belgium

Rodrigo

Pereira de Carvalho

Uppsala University

Sweden

Andrea

Silva

University of Southampton

United Kingdom

Michiel

Larmuseau

Ghent University

Belgium

Sabine

Matysik

University of Cambridge

United Kingdom

Dominika

Melicherová

Comenius University in Bratislava

Slovakia

Hossein

Kalashami

University of Antwerp

Belgium

Huzaifa

Shabbir

University of Vienna

Austria

Mario

Zauchner

Imperial College London

United Kingdom

Michele

Cutini

University of Turin

Italy

Javier

Heras-Domingo

Universitat Autònoma de Barcelona

Spain

Ivan

Yashchuk

VTT, Aalto

Finland

Marie-Pierre

Gaigeot

Université d’Evry – Paris-Saclay

France

Daria Ruth

Galimberti

Université d’Evry – Paris Saclay

Francia

Sana

Bougueroua

université d’Evry – Paris Saclay

France

Lefteri

Andritsos

King’s College London

United Kingdom

Joakim

Halldin Stenlid

Stockholm University

Sweden

Yu

Che

University of Liverpool

United Kingdom

Joris

Paret

Université de Montpellier

France

Aurelia

Li

University of Cambridge

United-Kingdom

Marta

Aragones-Anglada

University of Cambridge

United Kingdom

Matthias

Vandichel

Aalto University

Finland

Kunal

Ghosh

Aalto University

India

Lincan

Fang

Aalto University

Finland

Vaiva

Nagyte

The University of Manchester

United Kingdom

Adam

McSloy

Warwick University

United Kingdom

Nurcin

Ugur

Aalto University

Finland

Lucy

Whalley

Imperial College London

United Kingdom

Diksha

Dhawan

University of Michigan

United States

Francesco

Tavanti

University of Modena and Reggio Emilia

Italy

Zahra

Mohammadyarloo

Aalto University

Finland

Jie

Yao

university of bayreuth

germany

Djordje

Dangic

Tyndall National Institute

Ireland

Arnab

Majumdar

Uppsala University

Sweden

Andrejs

Cesnokovs

Institute of Solid State Physics

Latvia

Maria

Dimitrova

University of Helsinki

Finland

Boris

Dorado

CEA

France

Andrea

Sand

University of Helsinki

Finland

Tao

Jiang

Ecole Normale Superieure de Lyon

France

Ekaterina

Baibuz

University of Helsinki

Finland

Kasun Kalhara

Gunasooriya

Ghent University

Belgium

Sophie

Beck

ETH Zurich

Switzerland

Kai

Riedmiller

University Konstanz

Germany

Yunpei

Liu

Xiamen University

China

Feiwu

Zhang

Chinese Academy of Sciences

China

Ygor

Morais Jaques

Aalto University

Finland

Rocio

Bueno-Perez

University of Cambridge

United Kingdom

Justin

Villard

Ecole Polytechnique Fédérale de Lausanne

Switzerland

Alexander

Flachmüller

University of Constance

Germany

Hanne

Antila

Max Planck Institute of Colloids and Interfaces

Germany

Tuomas

Rossi

Chalmers University of Technology

Sweden

Fabrizio

Rovaris

University of Milano-Bicocca

Italia

Ismail Can

Oguz

ICGM-MACS

France

Fedor

Urtiev

Aalto University

Finland

Niko

Oinonen

Aalto University

Finland

Eun-Ae

Choi

Korea Institute of Materials Science

South Korea

Annika

Stuke

Aalto University

Finland

Victor

Claerbout

Czech Technical University in Prague

Czech Republic

Susi

Lehtola

University of Helsinki

Finland

Usman

Ahmed

University of Helsinki

Finland

Mikael

Johansson

University of Helsinki

Finland

Lukas

Wirz

University of Helsinki

Finland

Francesco

Marrafino

University of Salerno

Italy

Anna Maria

Nardiello

University of Salerno

Italy

Kristoffer

Simula

Aalto University

Finland

Yashasvi

Ranawat

Aalto University

Finland

Marc

Jäger

Aalto University

Finland

Prokop

Hapala

Aalto University

Finland

Anja

Aarva

Aalto University

Finland

Anais

Colibaba

Trinity College Dublin

Ireland

Konstantin

Karavaev

NUST MISIS Moscow

Russia

Lauri

Himanen

Aalto University

Finland

Timo

Roman

Nvidia

Finland

Nongnuch

Artrith

Columbia University

USA

Philippe

Schwaller

IBM Research Zurich

Switzerland

Harri

Valpola

Curious AI

Finland

Simon

Olsson

FU Berlin

Germany

Thomas

Hammerschmidt

ICAMS, Ruhr University Bochum

Germany

Rafael

Gomez-Bombarelli

MIT DMSE

United States

Jukka

Remes

IBM

Finland

Olexandr

Isayev

University of North Carolina

USA

Karsten W.

Jacobsen

Technical University of Denmark

Denmark

Teemu

Roos

University of Helsinki

Finland

Michele

Ceriotti

EPFL

Switzerland

Luca

Ghiringhelli

FHI Berlin

Germany

Filippo federici

Canova

Nanolayers ltd

Finland

Volker

Roth

Basel University

Switzerland

Michele

Allegra

Aix Marsill

France

Antonietta

Mira

USI Lugano

Switzerland

Jari

Järvi

Aalto University

Finland

Franco

Pellegrini

SISSA

Italy

Lukas

Hörmann

Graz University of Technology

Austria

Kamal

Choudhary

National Institute of Standards and Technology

United States

James

Nelson

Trinity College Dublin

Ireland

Jonathan

Schmidt

Martin-Luther Universität Halle Wittenberg

Germany

Alessandro

Lunghi

Trinity College Dublin

Ireland

Jakub

Kubecka

University of Helsinki

Finland

Michael

Sluydts

Ghent University

Belgium

Teo

Lombardo

University of Picardie Jules Verne

France

Niko

Oinonen

Aalto University

Finland

James

Cumby

University of Edinburgh

United Kingdom

Aldo

Glielmo

King’s College London

United Kingdom

Claudio

Zeni

King’s College London

United Kingdom

Kevin

Rossi

EPFL

Switzerland

Patrick

Rinke

Aalto University

Finland

Adam

Foster

Aalto University

Finland

Milica

Todorovic

Aalto University

Finland

 

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.