Position-based Hash Embeddings For Scaling Graph Neural Networks

Abstract

Graph Neural Networks (GNNs) bring the power of deep representation learning to graph and relational data and achieve state-of-the-art performance in many applications. GNNs compute node representations by taking into account the topology of the node's ego-network and the features of the ego-network's nodes. When the nodes do not have high-quality features, GNNs learn an embedding layer to compute node embeddings and use them as input features. However, the size of the embedding layer is linear to the product of the number of nodes in the graph and the dimensionality of the embedding and does not scale to big data and graphs with hundreds of millions of nodes. To reduce the memory associated with this embedding layer, hashing-based approaches, commonly used in applications like NLP and recommender systems, can potentially be used. However, a direct application of these ideas fails to exploit the fact that in many real-world graphs, nodes that are topologically close will tend to be rel

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keykalantzi2021position

Related papers

• Embedding Compression With Hashing For Efficient Representation Learning In Large-scale Graph (2022)8.60
• Hashing-accelerated Graph Neural Networks For Link Prediction (2021)11.49
• Learning To Hash With Graph Neural Networks For Recommender Systems (2020)14.02
• Sketch-gnn: Scalable Graph Neural Networks With Sublinear Training Complexity (2024)0.00
• Hebbian Graph Embeddings (2019)0.00
• QUINT: Node Embedding Using Network Hashing (2021)5.24
• Hessian-aware Quantized Node Embeddings For Recommendation (2023)2.26
• Graphhash: Graph Clustering Enables Parameter Efficiency In Recommender Systems (2024)4.77