Building Engine v2: How LangChain approaches agent improvement

From the video description

Ben Tannyhill, Product Manager at LangChain, explains how his team builds and evaluates LangSmith Engine, the agent in LangSmith that finds issues in production traces, proposes fixes, and creates evals to catch regressions. He covers the per-task benchmarks behind Engine's 2x better error identification on IssueBench, how the team runs Engine on LangChain's own go-to-market and coding agents and on Engine itself, and why a smaller model is sometimes the better choice for a simple task. He then walks through what Engine v2 adds, from detecting cross-trace metric trends and inefficient agent paths to validated fixes and red teaming (both in private beta for LangSmith Deployment customers),…