When rewriting the content, the original text needs to be rewritten in Chinese, and the original English sentence does not need to appear

Not long ago, the Microsoft DeepSpeed ​​team launched a project called DeepSpeed4Science's new initiative aims to achieve scientific discovery through AI system optimization technology.

On November 13, the Microsoft team published an article titled "The Impact of Large Language Models on Scientific Discovery: A Preliminary Study Using GPT-4" on the arXiv preprint platform

The length of this article reaches 230 pages

Paper link: https://arxiv.org/abs/2311.07361

In recent years, natural language Breakthrough advances in processing have culminated in the emergence of powerful large language models (LLMs), which have demonstrated extraordinary capabilities in numerous areas, including natural language understanding, generation, and translation, even extending beyond language processing task.

In this report, Microsoft researchers take an in-depth look at the performance of LLM in a scientific discovery/research context, focusing on the state-of-the-art language model GPT-4. Research spans multiple scientific areas, including drug discovery, biology, computational chemistry (DFT and MD), materials design, and partial differential equations (PDE).

For the scientific tasks of evaluating GPT-4, it is important for tapping its potential in various research fields, validating expertise in specific fields, accelerating scientific progress, optimizing resource allocation, guiding future model development and promoting cross-border research. Subject research is very important. Exploration methods mainly include expert-driven case evaluation, which can provide qualitative insights to help the model understand complex scientific concepts and relationships, as well as occasional benchmarking to quantitatively evaluate the model's ability to solve well-defined domain-specific problems. Capabilities

Preliminary exploration shows that GPT-4 has great potential in a variety of scientific applications, demonstrating its ability to handle complex problem solving and knowledge integration tasks. Researchers analyzed the performance of GPT-4 in the above-mentioned fields (such as drug discovery, biology, computational chemistry, materials design, etc.), highlighting its advantages and limitations. A comprehensive evaluation of GPT-4's knowledge base, scientific understanding ability, scientific numerical calculation ability, and various scientific prediction abilities

GPT-4 has extensive domain knowledge in biology and materials design, helping to meet Specific requirements. In other areas such as drug discovery, GPT-4 has demonstrated strong property prediction capabilities. However, in research areas such as computational chemistry and partial differential equations, while GPT-4 is expected to help researchers make predictions and calculations, further efforts are needed to improve its accuracy. Despite its impressive capabilities, GPT-4 still offers improvements for quantitative computing tasks such as the need for fine-tuning to improve accuracy. The researchers hope this report will provide insights for those looking to leverage LLM for scientific research and Provides a valuable resource for applied researchers and practitioners, as well as those interested in advancing specific areas of natural language processing. It is important to emphasize that the field of LLM and large-scale machine learning is rapidly evolving, and future generations of LLM may have additional capabilities not mentioned in this report. It is worth noting that the integration of LLM with professional scientific tools and models and the development of basic scientific models represent two promising research directions

Drug Discovery

Drug discovery is important to the pharmaceutical industry components that play a vital role in advancing medical science. Drug discovery involves a complex multidisciplinary process, including target identification, lead optimization, and preclinical testing, ultimately leading to the development of safe and effective drugs.

GPT-4 has great potential in drug discovery, such as accelerating the discovery process, reducing search and design costs, enhancing creativity, etc. This chapter first studies GPT-4's knowledge for drug discovery through qualitative testing, and then investigates its predictive capabilities through quantitative testing on multiple key tasks, including drug-target interaction/binding affinity prediction, molecular property prediction, and retrosynthetic prediction

Rewritten content: The first example involves generating the chemical formula, IUPAC name, and SMILES of a given drug name, which is the conversion of the name into other representations of the drug. Afatinib was used as the input drug. GPT-4 successfully output the correct chemical formula C24H25ClFN5O3 and the correct IUPAC name, indicating that GPT-4 is aware of the drug Afatinib. However, the generated SMILES is incorrect. Therefore, the researchers gave guidance to let GPT-4 regenerate SMILES. Unfortunately, despite the explicit requirement that GPT-4 "pay attention to the number of atoms of each atom type" and generate SMILES based on correct IUPAC and chemical formulas, the SMILES sequences generated in several experiments were still incorrect

The first image shows the translation of the drug name and other drug representations. (Quoted from the paper)

Biology

In this chapter, the researchers deeply explore the capabilities of GPT-4 in the field of biological research, focusing mainly on its ability to understand biological language and use built-in biological knowledge to conduct research. Proficiency in reasoning, and designing biomolecules and biological experiments. Observations indicate that GPT-4 shows great potential to contribute to the field of biology by demonstrating its ability to process complex biological languages, perform bioinformatic tasks, and even serve as a scientific assistant in biological design. GPT-4's broad grasp of biological concepts and its great potential as a scientific assistant in design tasks highlight its important role in advancing the field of biology.

The ability of GPT-4 to handle biological sequence sequence symbols and text symbols was first evaluated.

The researchers asked GPT-4 to convert between biological sequences and their textual symbols: 1) Output the protein name for a given protein sequence. 2) Output the protein sequence with the given name. Before each task, the session is restarted to prevent information leakage. It turns out that GPT-4 knows about sequence-to-text symbol conversion, but it cannot directly look up (also known as BLAST sequences) itself. Meanwhile, GPT-4 prefers textual tags for biological sequences (including proteins and DNA, the latter not shown). When text symbols are given, it provides richer information, probably due to its design philosophy. It is important to note that it was also noted that generating sequences could lead to catastrophic behavior of GPT-4. As shown in the image below, although GPT-4 returned the correct UniProt ID, it encountered difficulty generating the sequence. Sequence generation crashes with several different prompts tried.

Figure 2: Conversion between sequence symbols and text symbols. (Source: Paper)

Computational Chemistry

Chemical computing is an interdisciplinary field that uses computational methods and techniques to solve complex problems in chemistry. Chemical calculations have long been an indispensable tool in the study of molecular systems, not only providing a deep understanding of atomic-level interactions but also guiding experimental work. Chemical calculations play a vital role in the understanding of molecular structures, chemical reactions and physical phenomena at the micro and macro levels

This chapter will focus on the function of GPT-4 in the field of computational chemistry. We will explore its application in electronic structure methods and molecular dynamics simulations, and demonstrate the service capabilities of GPT-4 from different perspectives by showing two practical examples. In summary, GPT-4 can assist computational chemistry researchers in a variety of ways.

The research begins with evaluating GPT-4’s ability to explain concepts in quantum chemistry and physics. The assessment covers methods commonly used in the field, such as density functional theory (DFT) and wave function theory (WFT).

Figure 3: Concept test of density functional theory. (Source: paper)

In the above example, GPT-4 provides a good understanding of the concepts of density functional theory, KohnSham density functional theory, and orbitless density functional theory.

Material Design

In this chapter, the capabilities of GPT-4 in the field of material design are studied. The researchers designed a comprehensive set of tasks covering all aspects of the materials design process, from initial conceptualization to subsequent validation and synthesis. The goal is to evaluate GPT-4's expertise and its ability to generate meaningful insights and solutions in real-world applications. The designed tasks cover various aspects, including background knowledge, design principles, candidate identification, candidate structure generation, attribute prediction, and synthetic condition prediction. By addressing the entire design process, the goal is to provide an overall assessment of GPT-4's proficiency in materials design, particularly for more complex materials such as crystalline inorganic materials, organic polymers, and metal-organic frameworks (MOFs).

It is worth noting that the evaluation focuses primarily on a qualitative assessment of GPT-4’s capabilities in this area of ​​expertise, with statistical scores being awarded only where feasible.

The researchers first investigated how current solid electrolytes are classified, which have different requirements, such as classification based on general chemistry and anion type. Additionally, they asked for examples based on these classification criteria. As shown in Figure 4, all answers are factual and most are correct. Since these classification criteria are not well represented in the literature, GPT-4 should have a relatively clear understanding of what chemistry means

Based on the source paper, it is required The rewritten content is: Figure 4: Classification of inorganic solid electrolytes

Partial Differential Equations

Partial differential equations (PDE) are an important and highly active research field in the field of mathematics, and in physics It has far-reaching applications in various disciplines such as engineering, biology and finance. Partial differential equations play a crucial role in modeling and understanding a wide range of phenomena, from fluid dynamics and heat transfer to electromagnetic fields and group dynamics.

In this chapter, GPT-4’s skills in several aspects of partial differential equations are studied: understanding the basics of partial differential equations, solving partial differential equations, and assisting AI in partial differential equation research. Researchers evaluate models on different forms of PDEs, such as linear equations, nonlinear equations, and stochastic PDEs. Research shows that GPT-4 can help researchers in a variety of ways.

The first question is about the definition and form of partial differential equations. GPT-4 provides a good explanation of partial differential equations, as shown in Figure 5. At the user's prompts, GPT-4 gives a clear concept of partial differential equations and categories of linear or nonlinear, elliptical, parabolic or hyperbolic. Newcomers to the field will benefit from these concepts and classifications.

Figure 5: Introduction to the basic concepts of PDE. (Source: Paper)

Future Outlook

In this study, we explore the capabilities and limitations of LLM in various natural science fields and cover a variety of tasks. Our main goal is to provide a preliminary assessment of the state-of-the-art LLM GPT-4 and its potential for scientific discovery, and to provide valuable resources and tools to researchers in various fields

Through extensive analysis, the study highlights GPT-4 achieves proficiency in numerous scientific tasks, from literature synthesis to property prediction and code generation. Despite its impressive capabilities, it is important to recognize the limitations of GPT-4 (and similar LLMs), such as challenges in handling specific data formats, inconsistencies in responses, and occasional hallucinations.

The researchers believe this exploration is a critical first step in understanding and recognizing the potential of GPT-4 in the natural sciences. By providing a detailed overview of its advantages and disadvantages, it is intended to help researchers make informed decisions when incorporating GPT-4 (or other LLMs) into their daily work, ensuring optimal application while being mindful of its limitations.

In addition, further exploration and development of GPT-4 and other LLMs are encouraged, aiming to improve their scientific discovery capabilities. This may involve refining training processes, merging domain-specific data and architecture, and integrating expertise tailored to different scientific disciplines.

As the field of artificial intelligence continues to develop, the integration of complex models like GPT-4 will play an increasingly important role in accelerating scientific research and innovation

Finally, the study concludes Aspects of LLM that need improvement in scientific research are discussed, and potential directions for strengthening LLM or promoting scientific breakthroughs based on this are discussed.

The above is the detailed content of GPT-4 and LLM: Microsoft team explores the impact of scientific discoveries, a 230-page article covering 5 major scientific fields. For more information, please follow other related articles on the PHP Chinese website!