Stochastic Learning Of Nonstationary Kernels For Natural Language Modeling

Abstract

Natural language processing often involves computations with semantic or syntactic graphs to facilitate sophisticated reasoning based on structural relationships. While convolution kernels provide a powerful tool for comparing graph structure based on node (word) level relationships, they are difficult to customize and can be computationally expensive. We propose a generalization of convolution kernels, with a nonstationary model, for better expressibility of natural languages in supervised settings. For a scalable learning of the parameters introduced with our model, we propose a novel algorithm that leverages stochastic sampling on k-nearest neighbor graphs, along with approximations based on locality-sensitive hashing. We demonstrate the advantages of our approach on a challenging real-world (structured inference) problem of automatically extracting biological models from the text of scientific papers.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygarg2018stochastic

Related papers

• Why Do Nearest Neighbor Language Models Work? (2023)3.56
• Kernel Similarity Matching With Hebbian Neural Networks (2022)0.00
• A Two-stage Active Learning Algorithm For \(k\)-nearest Neighbors (2022)0.00
• Neural Nearest Neighbors Networks (2018)0.00
• Neurocache: Efficient Vector Retrieval For Long-range Language Modeling (2024)1.91
• Interpretable Locally Adaptive Nearest Neighbors (2020)3.58
• Leveraging Reinforcement Learning For Evaluating Robustness Of KNN Search Algorithms (2021)0.00
• Similarity Learning Via Kernel Preserving Embedding (2019)10.35