Neural networks, including both feedforward and graph neural networks (GNNs), use a combination of training data and a mathematical model to make predictions or classifications on new, unseen data. In general, the process of extrapolation involves using a trained neural network to make predictions beyond the range of the training data.
In feedforward neural networks, extrapolation can occur when the network is able to generalize from the training data to make accurate predictions on new, unseen data. This is typically achieved by training the network to minimize a loss function that measures the difference between the predicted output and the true output for each input in the training data. Once the network is trained, it can be used to make predictions on new inputs that were not included in the training data.
In graph neural networks, extrapolation involves learning representations of graphs that generalize to new, unseen graphs. GNNs typically operate on graph-structured data, where each node in the graph represents a data point and the edges represent relationships between the data points. GNNs learn a representation of the graph that can be used to make predictions on new graphs with similar structure.
One approach to achieving extrapolation in GNNs is to use a hierarchical message-passing scheme, where nodes communicate with their neighbors to update their own representations, and the updated representations are then used to update the representations of the neighbors in the next iteration. This process can be repeated multiple times until a fixed point is reached. By learning representations that capture the structural properties of the graph, GNNs can generalize to new graphs with similar structures and make predictions or classifications on new, unseen data.
Overall, the ability of neural networks to extrapolate to new, unseen data is a key feature that makes them powerful tools for a wide range of applications, from image recognition to natural language processing to scientific discovery.