Struggling with local LLMs losing context after 10k tokens

I’ve been trying to get a decent workflow running on my local machine using an 8GB GPU setup. I’m currently running a 7B parameter model via Ollama, specifically Llama 3, to help me refactor some legacy Python code. The problem is consistent: everything works perfectly for the first few exchanges, but once the conversation history climbs past roughly 10,000 tokens, the model starts hallucinating variable names it defined three prompts ago or simply ignores the constraints I set at the beginning of the session. It feels like it’s just forgetting the root of the thread entirely. I’ve tried increasing the context window in the config to 8192, but I suspect my VRAM is bottlenecking it, causing it to truncate the older parts of the conversation before they even reach the model’s attention. I’m wondering if I’m just hitting a hard hardware limit or if there’s a smarter way to prune these conversations without completely losing the thread. Has anyone found a reliable method for handling long-term context in local environments without needing to constantly restart the session? I’m considering switching to a smaller 4B model to gain more headroom, but I’m worried that will degrade the code quality significantly. Any advice on balancing context length with model size on consumer hardware would be appreciated. I’d really like to avoid having to pay for API calls just to manage a simple refactoring job.