Author: Yu Fan
background
Looking back at the development history of neural network architecture, we can find that symmetry has always played a hidden and core role, and geometric deep learning highlights the importance of symmetry. On the one hand, in the fields of traditional computer vision and natural language processing, we have seen that the network architecture based on Transformer has achieved amazing results in recent years. On the other hand, as a first principle, symmetry plays an important role in natural science. Therefore, we have reason to believe that from the perspective of symmetry, geometric deep learning will play an important role in the combination of artificial intelligence and science.
Figure 1: The combination of geometric deep learning in the fields of artificial intelligence and science
事实上，当前深度几何学习在AI for Science领域已经取得令人瞩目的成果，并将发挥越来越重要的作用。基于图神经网络和Transformer的Alphafold2在预测蛋白质结构中，显示出非常高的准确性。在小分子结构的预测与生成中，在图神经网络中加入额外的欧几里得空间的刚体变换对称性（E(3)对称群），可以显著提升计算精度，降低训练复杂度。在全球天气预测中，由于地球表面是一个二维球面，会涉及到流形上的卷积，坐标规范变换等问题，几何深度学习为神经网络的设计提供了系统性的理论框架。而宇宙学中，时空由于引力导致弯曲，几何深度学习为这种弯曲的黎曼流形结构与AI的结合提供了理论基础。
On the other hand, in basic science fields, such as condensed matter physics and quantum physics, the degree of integration with AI is not yet deep. This is because in these fields, human systematic theoretical knowledge is relatively well developed, but it is very difficult to obtain high-quality data. Therefore, how to "inject" known knowledge into neural networks and improve data utilization is particularly critical.
Group equivariant neural networks have shown surprising advantages in improving data utilization. For example, in the classification task of three-dimensional Tetris, there are a total of 8 configurations. The traditional approach requires a large amount of spatial rotation of each configuration of data for data enhancement, which undoubtedly greatly increases the amount of data and the complexity of training, and cannot guarantee its prediction accuracy. In the E(3) equivariant neural network, each configuration only requires one piece of data, or even less if the two configurations can be connected through an E(3) transformation. While greatly reducing the amount of data required , its prediction accuracy is guaranteed from a theoretical level. At the same time, this neural network has better interpretability and expressive capabilities .
Among all known knowledge, symmetry is a profound and basic type of knowledge, and it is also the key to explaining the laws of nature. For example, the space-time of high-energy particles has Lorentz group symmetry (SO (1, 3) group), and Lorentz group equivariant neural networks can play an important role in high-energy physics in the future. Modern physics is described using field theory, and the mathematical language behind it is also differential geometry and fiber bundles. It is foreseeable that deep geometry learning will play a leading role in the combination of modern physics and artificial intelligence.
For the large AI for Science model, based on geometric deep learning, we can make a simple and bold idea: first learn the symmetry of the input system through some kind of neural network, such as learning the Lie algebra method of the symmetry group [3]; then through A certain mechanism controls the strength of various symmetries, allowing the network to choose which symmetry group of equivariant network to use, so that the network can satisfy the approximate symmetry of the input system. Of course, there is a high probability that future large-scale models will not be so simple, and they are also full of problems that have yet to be solved. We look forward to seeing further progress in this direction in the future.
Figure 2: Lie algebra of the SO(2) group of L-conv learning data
references
[1] Bronstein, Michael M., et al. "Geometric deep learning: Grids, groups, graphs, geodesics, and gauges." arXiv preprint arXiv:2104.13478 (2021).
[2] Weiler, Maurice, et al. "Coordinate Independent Convolutional Networks--Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds." arXiv preprint arXiv:2106.06020 (2021).
[3] Dehmamy N, Walters R, Liu Y, et al. Automatic symmetry discovery with lie algebra convolutional network[J]. Advances in Neural Information Processing Systems, 2021, 34: 2503-2515.
A programmer born in the 1990s developed a video porting software and made over 7 million in less than a year. The ending was very punishing! Google confirmed layoffs, involving the "35-year-old curse" of Chinese coders in the Flutter, Dart and Python teams . Daily | Microsoft is running against Chrome; a lucky toy for impotent middle-aged people; the mysterious AI capability is too strong and is suspected of GPT-4.5; Tongyi Qianwen open source 8 models Arc Browser for Windows 1.0 in 3 months officially GA Windows 10 market share reaches 70%, Windows 11 GitHub continues to decline. GitHub releases AI native development tool GitHub Copilot Workspace JAVA is the only strong type query that can handle OLTP+OLAP. This is the best ORM. We meet each other too late.