AI Engineering Abstractions Wanted

No good abstraction to talk to an LLM (context creation and manipulation)
No standardised LLM conversation format
Inconsistent handling of thinking tokens
No easy (or performant) way to create reproducible sandboxes/envs for agents
No good context observability tools
Poor MCP client and server implementations
Poor LLM gateway implementations
No good flexible eval annotation tool that supports stuff like semantic clustering
Poor library/SDK story outside Python and JS ecosystems