How we choose LLMs and frameworks for AI agents

The journey from a single A100 in the cloud to an H200 cluster is not just a hardware upgrade — it’s a story of how an ML team stopped chasing “the one perfect model” and started building an ecosystem. When there are millions of lines of PostgreSQL C code under the hood and tasks range from hint-set generation to Graph-RAG, the model stops being a black box and becomes just another replaceable component. We explain how we rebuilt our stack around vLLM and MCP, why context management matters more than model weights, and how we made a 0.6B-parameter model perform on par with the giants using GRPO.


















