Paecter: Patent-level Representation Learning Using Citation-informed Transformers

Abstract

PaECTER is an open-source document-level encoder specific for patents. We fine-tune BERT for Patents with examiner-added citation information to generate numerical representations for patent documents. PaECTER performs better in similarity tasks than current state-of-the-art models used in the patent domain. More specifically, our model outperforms the patent specific pre-trained language model (BERT for Patents) and general-purpose text embedding models (e.g., E5, GTE, and BGE) on our patent citation prediction test dataset on different rank evaluation metrics. PaECTER predicts at least one most similar patent at a rank of 1.32 on average when compared against 25 irrelevant patents. Numerical representations generated by PaECTER from patent text can be used for downstream tasks such as classification, tracing knowledge flows, or semantic similarity search. Semantic similarity search is especially relevant in the context of prior art search for both inventors and patent examiners.

Stats

• citations270
• S2 citations—
• github stars0
• HF likes0
• heat score18.25
• arxiv keyghosh2024paecter

Related papers

• Enhancing Patent Retrieval Using Text And Knowledge Graph Embeddings: A Technical Note (2022)10.07
• Patent Representation Learning Via Self-supervision (2025)0.00
• Learning Efficient Representations For Image-based Patent Retrieval (2023)2.26
• Llm-powered Real-time Patent Citation Recommendation For Financial Technologies (2026)0.00
• Medcpt: Contrastive Pre-trained Transformers With Large-scale Pubmed Search Logs For Zero-shot Biomedical Information Retrieval (2023)15.34
• Large Language Model Informed Patent Image Retrieval (2024)0.00
• Contrastive Learning And Mixture Of Experts Enables Precise Vector Embeddings (2024)0.00
• Refining Joint Text And Source Code Embeddings For Retrieval Task With Parameter-efficient Fine-tuning (2024)0.00