Q-adapter: Customizing Pre-trained Llms To New Preferences With Forgetting Mitigation

Abstract

Large Language Models (LLMs), trained on a large amount of corpus, have demonstrated remarkable abilities. However, it may not be sufficient to directly apply open-source LLMs like Llama to certain real-world scenarios, since most of them are trained for *general* purposes. Thus, the demands for customizing publicly available LLMs emerge, but are currently under-studied. In this work, we consider