Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models ...

LLMs have recently helped find solutions to a number of minor longstanding problems. But a new plan called First Proof is really putting them to the test ...