Fused Text Recogniser And Deep Embeddings Improve Word Recognition And Retrieval

Abstract

Recognition and retrieval of textual content from the large document collections have been a powerful use case for the document image analysis community. Often the word is the basic unit for recognition as well as retrieval. Systems that rely only on the text recogniser (OCR) output are not robust enough in many situations, especially when the word recognition rates are poor, as in the case of historic documents or digital libraries. An alternative has been word spotting based methods that retrieve/match words based on a holistic representation of the word. In this paper, we fuse the noisy output of text recogniser with a deep embeddings representation derived out of the entire word. We use average and max fusion for improving the ranked results in the case of retrieval. We validate our methods on a collection of Hindi documents. We improve word recognition rate by 1.4 and retrieval by 11.13 in the mAP.

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keybansal2020fused

Related papers

• Content-based Similar Document Image Retrieval Using Fusion Of CNN Features (2017)4.52
• Pattern Spotting And Image Retrieval In Historical Documents Using Deep Hashing (2022)2.26
• Cross-modal Image Retrieval With Deep Mutual Information Maximization (2021)9.59
• Matching Images And Text With Multi-modal Tensor Fusion And Re-ranking (2019)19.77
• Deep Learning Approaches For Image Retrieval And Pattern Spotting In Ancient Documents (2019)0.00
• Fine-grained Image Classification And Retrieval By Combining Visual And Locally Pooled Textual Features (2020)10.48
• F4-ITS: Fine-grained Feature Fusion For Food Image-text Search (2025)1.40
• Deep Multimodal Image-text Embeddings For Automatic Cross-media Retrieval (2020)0.00