ChatGPT has limited memory, GPT-3.5-turbo has a limit of 4897 tokens, and GPT-4 has a maximum limit of 8192. If you go over 8192 tokens (about 6827 words) in a chat with GPT-4, it will start to forget. I've come up with a new trick that easily expands the conversation by a factor of 10.
This trick doesn't save every word of the dialogue into memory. When you go to a meeting, someone takes notes, right? They don't write down every word you say, just the main points.
To fix this: ask GPT to create a rolling summary of the conversation. There is such a prompt: every 5 messages, GPT will summarize the previous chat content into JSON format.
prompt：You have RollingFooterSummary mode enabled. This means thatevery time you reply, you will number your reply. Your first reply willstart as: 1/ [your reply] Every multiple of five, e.g. the 5th answer, 10th answer, you will give aquick summary of all the salient information contained in the earlierconversation, distilled into JSON format with extremely abbreviatedself-contained information, which can be understood by an LLM. Additionally, the user can manually request a footer at any time bytyping "Footer". This will prompt you to provide a summary formattedas JSON, containing extremely abbreviated, self-containedinformation that can be understood by LLMs。
This way, even if GPT forgets the beginning of the conversation, the digest only contains at most 5 previous messages. It never forgets something so recent.
Using GPT combined with the rolling summary hint above, you can have interaction log sessions that are months long.