Inverse design of porous materials using artificial neural networks

The ability to generate optimized nanomaterials with artificial neural networks can significantly revolutionize the future of materials design in materials science. While scientists had progressively created small and simple molecules, complex crystalline porous materials remain to be generated using neural networks. In a recent report on Science Advances, Baekjun Kim and a team of researchers in the Department of Chemical and Biomolecular Engineering at the Korea Advanced Institute of Science and Technology, Republic of Korea, implemented a generative adversarial network.

They produced 121 crystalline porous materials using a training set of 31,713 known zeolites. The new neural network took input in the form of energy and materials dimensions to reliably produce zeolites with a user-desired range of 4 kJ/mol methane heat of adsorption. They designated the energy dimension in the work to be the methane potential energy. The fine-tuning of user-desired capability can potentially accelerate materials development, while demonstrating a successful case of inverse design of porous materials.

Materials scientists have conducted significant research to discover new materials using artificial intelligence in the past few years. They made considerable progress using a variety of artificial neural networks (ANNs) to generate undiscovered molecules and materials. However, ANNs remain to be successfully used to create new crystalline materials, since machine learning had thus far only predicted materials properties, compositions, bandgap energy, formation energy and gas adsorption uptakes. Crystalline porous materials contain dense arrangements of microscopic pores for higher surface area and pore volume. They are an important class of materials for a variety of diverse energy- and environment- related applications. Compared to other crystalline materials, porous materials such as zeolites, metal organic frameworks (MOFs) and covalent organic frameworks (COFs) are comparatively more challenging to generate using ANNs due to greater complexity.

In this study, Kim et al. devised an ANN to generate crystalline porous materials by specifically targeting a case study problem to produce pure silica zeolite structures, chosen for their structural simplicity. The team used a wealth of materials available on an open database of hypothetical zeolites to train the neural network. Zeolites are classically defined as aluminosilicates with open three-dimensional (3-D) frameworks containing corner-sharing TO₄ tetrahedra where T is Aluminium (Al) or Silicon (Si).

While a few previous research efforts used an evolution algorithm to target material properties, such conventional methods lead to brute-force generation of porous materials, necessitating computationally expensive screening to identify optimal materials for a given application. A majority of such generated materials have poor properties, affecting inefficient allocation of computational resources. Kim et al. designed the new neural network to represent the inputs in both material and energy dimensions. The new algorithm has a unique advantage to achieve inverse materials design using ANNs to bias the energy dimension correlating to the materials properties.

Generative adversarial network for zeolites.

The team used generative adversarial networks (GAN) to produce crystalline porous materials due to their enhanced capacity to produce realistic objects such as human faces. The GAN contained a discriminator and a generator, where the discrimination could differentiate between the real and fake data, as the generator acts to deceive the discriminator by progressively forming realistic (yet fake) objects. This setup could advance adversarial learning by generating increasingly realistic objects as a byproduct of improving the learning process for both the discriminator and generator.

Since the goal of this work was to generate materials and energy shapes, Kim et al. formed a new type of GAN named zeolite GAN (ZeoGAN). The team aimed to produce realistic zeolite materials using the generator in ZeoGAN with their corresponding energy shapes to add several features to the setup. They added periodic padding within the critic (or discriminator) to prevent generating non-realistic shapes that could lead to unrealistic bonds, and facilitated convergence for both materials and energy shapes by adding feature-matching to the ZeoGAN.

In the present experimental setup, they divided the input to the neural network into materials and energy grids, with the materials grid further sub-divided into the silicon and oxygen atom grids based on classical molecular simulations. The scientists used three grids each and kept the number of grid points small and constant to reduce the memory cost, since larger grids can lead to a very slow learning process. They represented the positions of the silicon (Si) and oxygen (O) atoms using Gaussian functions, where the peak of the Gaussian corresponded to the position of the Zeolite atoms.

Generating pure silica zeolites

The scientists used a total of 31,173 methane accessible zeolites to train the neural network. The learning process of ZeoGAN showed the evolution of material/energy shapes from their initial Gaussian noise distributions. They trained the discriminator to estimate the Earth mover's distance (EMD) between the data distribution and generator distribution, and trained the generator to minimize the EMD so as to generate realistic samples. Initially, the material/energy shapes resembled typical noise distribution but as learning progressed, they occupied separate regions in the unit cell space to morph into shapes resembling typical zeolites.

In total, they generated 1 million zeolite shapes (both material and energy) from the ZeoGAN. From these shapes, they assigned the positions of the oxygen and silicon atoms using a simple rule and calculated the Si:O ratio for each output. The zeolite shapes evolved as they successfully passed through a clean-up operation to yield an optimal Si:O ratio and 100 percent bond connectivity. From this set, they kept structures with a small number of symmetrically unique T atoms (where T is Al or Si). The final relaxed structures resembled their initial zeolite shapes, indicating that post-processing did not significantly alter the essence of new zeolite shapes. Kim et al. obtained a total of eight resulting structures after clean-up, which were not in the original training set to indicate the successful creation of new zeolites using ZeoGAN.

Using ZeoGAN for inverse design of zeolites

The zeolites thus far generated from ZeoGAN did not contain any user-desired properties. To improve the design, the research team selected to alter the methane heat of adsorption and ZeoGAN loss function to target generate zeolites with heat of adsorption values between 18 and 22 kJ/mol. The team observed a sharp change in the methane heat of adsorption distribution within the data for the 1 million newly generated- user-desired zeolite shapes indicating proper function of the user-desired criterion. The values did not correlate with the new loss function, however. The team then implemented a similar clean-up process (as before), for the 1 million user-desired zeolite shapes, to yield six new zeolites and one zeolite also previously produced within the non-user desired set. Of these six zeolites, four maintained methane heat of adsorption between 18 and 22 kJ/mol as expected, indicating successful inverse design of the zeolites.

These experiments were a first-in-study, since preceding experimental or computational models had not thus far yielded properties within this user-desired specific range. Additionally, when Kim et al. removed restrictions for the number of unique T atoms that were possible for zeolite candidates generated from ANNs, they observed a significant increase in the number of newly formed zeolites. In this way, they obtained 121 feasible zeolite structures in total using the in-house-developed ANN, to successfully extend the number of new zeolites within the pure silica zeolite material space.

This work will potentially pave the way to incorporate ANNs to target user-desired properties before material design and synthesis. Although the ANN is restricted to just silicon and oxygen atoms here for simplicity, the number of input channels can be increased to cover more complex crystalline materials such as MOFs and COFs. The scope of this work can be enhanced to affect the future design of diverse classes of materials.