Paper Address: A Gentle Introduction to Graph Neural Networks
Translation: Graphs are all around us; real-world objects are often defined by their connections to other things. A set of objects and the connections between them are naturally represented as a graph . For more than a decade, researchers have developed neural networks (called graph neural networks, or GNNs) that operate on graph data. Recent developments have improved their capabilities and expressiveness. We're starting to see practical applications in areas such as antimicrobial discovery, physics simulation, fake news detection, traffic prediction, and recommender systems.
This paragraph introduces the structure of the article is mainly divided into:
- 1. What data can be represented as a graph
- 2. The difference between graph and other data
- 3. Build a GNN
- 4. Provide a GNNplayground
Used to understand how each component of the GNN model contributes to the predictions it makes
The authors treat images as rectangular grids with image channels and treat them as arrays. The other treats images as graphs with a regular structure, where each pixel represents a node and is connected to adjacent pixels by edges. Each unbounded pixel has exactly 8 neighbors, and the information stored at each node is a 3D vector representing the RGB value of the pixel.
Also followed by text as graph, molecule as graph, social network as diagram
Here the author explains that there are three general types of prediction tasks on graphs: graph-level, node-level, and edge-level
Graph-level tasks: The main goal is to predict properties of the entire graph.
Node-level tasks: mainly predict the identity or role of each node in the graph
Edge-level tasks: mainly for the understanding of image scenes
This paragraph shows that when using neural networks to solve different graph tasks, the first problem is how to represent graphs compatible with neural networks
This shows that there are up to four types of information in the graph: nodes, edges, global context and connectivity
Among them, the connectivity of the graph is the most difficult and the most complicated, because using the adjacency matrix, it is easy to generate tensors and cause the adjacency matrix to be very sparse and inefficient in space. Also, there is no guarantee that these different matrices will produce the same results in a deep neural network.
Normalization stores different information about graphs:
This paragraph mainly explains that GNN (Graph Neural Networks) is an optimized conversion of all attributes of the graph (nodes, edges, and global context), which maintains graph symmetry (arrangement invariance). At the same time, the input of GNN is a graph, and the output is also a graph.
GNN will transform nodes, edges, etc., but will not change the connectivity of the graph
A simple GNN model:
Input a graph into a series of GNN layers, output an output that maintains the entire graph structure but all the attributes in it have been transformed, and finally add an appropriate output layer according to which attribute you want to predict, but if there is indeed missing information, add it Appropriate pooling layer.
This paragraph shows that we can use information passing to make more complex predictions by using pooling (pooling) in the GNN layer.
Three steps of information transfer:
- 1. For each node in the graph, collect all adjacent node embeddings (or messages), i.e. the g function as described above.
- 2. Aggregate all messages through an aggregation function (such as sum).
- 3. All aggregated messages are passed through an update function , usually a learned neural network
Here the authors say that the network we describe has a flaw: nodes in the graph that are far from each other may never be able to efficiently communicate with each other, even if we apply multiple message passing.
In order to solve the above problems, a master node (this point can be connected to all nodes and edges) is added here, which is a virtual point.
For a node, we can consider information from neighboring nodes, connection edges and global information. To condition new node embeddings on all these possible sources of information, we can simply concatenate them. In addition, we can also map them to the same space via a linear map and add them or apply a feature modulation layer, which can be considered as a featurized attention mechanism.
Sampling Graphs and Batching in GNNs
A common practice in training neural networks is to update network parameters using gradients computed on a random constant-sized (batch size) subset of the training data (mini-batch) . This approach poses challenges for graphs due to variability in the number of adjacent nodes and edges, which means we cannot have a constant batch size. The main idea of batch processing with graphs is to create subgraphs that preserve the essential properties of the larger graph. This graph sampling operation is highly context dependent and involves subselecting nodes and edges from the graph. These operations may make sense in some contexts (citation networks), while in others they may be too strong (molecules, where a subgraph simply represents a new, smaller molecule). How to sample graphs is an open research question. If we care about preserving structure at the neighborhood level, one approach is to randomly sample a uniform number of nodes, our node set . Neighboring nodes at distance k are then added to the node set neighborhood, including their edges. Each neighborhood can be viewed as a separate graph, and GNNs can be trained on batches of these subgraphs. The loss can be masked to only consider node sets, since all adjacent nodes have incomplete neighborhoods. A more efficient strategy might be to first randomly sample a single node, expand its neighborhood to distance k, and then select another node in the expanded set. These operations can terminate once a certain number of nodes, edges or subgraphs have been constructed. If the context allows, we can build constant-size neighborhoods by choosing an initial set of nodes and then subsampling a constant number of nodes (e.g. randomly, or via random walks or the Metropolis algorithm).
Graph attention network
Another way to transfer information between graph properties is through attention (weights depend on the relationship of two vertex vectors, not the position of the vertex). For example, when we consider the sum aggregation of a node and its 1-degree neighbor nodes, we can also consider using a weighted sum. The challenge then is to associate the weights in a permutation-invariant manner.
in conclusion:
First, let’s introduce what a graph is. The attributes of a graph should be represented by vectors. How to represent data in reality as a graph, and how to make predictions on graphs (points, edges, and the whole world). Briefly explain the definition of GNN, assuming that the attribute is missing to perform aggregation operation, take the attribute of the edge or the attribute of the point. Then the GNN in the true sense is illustrated. It is to transfer the information of the entire graph through aggregation in each layer. The next step is to visualize the experiment and make a beautiful visualization.