I. Introduction
A language model's memory is based on its training data. Specifically, with longer texts, the model may forget older information because its memory is limited and is more susceptible to more recent content. The model cannot span its fixed context window and instead generates responses based on the current context.
From now on, we will use the Milvus vector database as the storage method of the long-term memory of the language model to improve the language model's ability to memorize information.
Related concept reference: Open source model application implementation-memory enhancement series-concepts (1)
2. Terminology
2.1. Lite Kite
It is a lightweight version of the Milvus vector database. Designed to provide fast and efficient vector storage and similarity search capabilities in resource-constrained environments.
Compared with the full version of Milvus, it has the following features:
- Lightweight: Milvus Lite has a small storage footprint and memory consumption, making it suitable for deployment and operation on resource-constrained devices.
- Rapid deployment: Milvus Lite provides a streamlined deployment and configuration process, making it easier to deploy and integrate on embedded devices and edge servers.
- Efficient vector indexing and search: Despite being a lightweight version, Milvus Lite still provides efficient vector indexing and similarity search capabilities to support fast vector data queries.
- Offline mode: Milvus Lite supports vector indexing and searching in offline mode without the need for a real-time connection to a remote server.