Struggling with context window limits in our RAG pipeline

We are building a B2B support tool that ingests massive technical documentation sets. Currently, we are using a hybrid search approach with Pinecone and LlamaIndex, but we keep hitting walls when the user asks multi-step questions that require synthesizing info from five or six different chunks. The model starts hallucinating specific version numbers because it loses track of which document it pulled from. We tried increasing the chunk size to 1000 tokens, but then the retrieval precision dropped significantly. We are considering switching to a more complex re-ranking step using Cohere Rerank, but I am not sure if that actually solves the synthesis problem or just improves the order of the top-k results. Has anyone successfully implemented a map-reduce strategy where they first summarize each relevant chunk individually and then pass those summaries to the final LLM call? That feels like it would double our latency and cost, but maybe it's the only way to get accurate cross-reference answers. Also, are there any good open-source alternatives to LangChain that handle this state management better? We are getting really frustrated with the boilerplate code required just to maintain conversation memory across these complex multi-turn interactions. Any advice on whether we should stick with the standard RAG flow or pivot to a fine-tuned model on our specific doc structure? We have limited compute budget right now, so broad fine-tuning isn't really an option, but we are open to targeted approaches if they yield better results without the overhead.