Needle In A Haystack - Pressure Testing LLMs

Greg Kamradt's blog post and open-source harness that defined the modern needle-in-a-haystack long-context evaluation methodology.

Publisher: Greg Kamradt (personal)
Year: 2023
Venue: blog
URL: github.com/gkamradt/LLMTest_NeedleInAHaystack
Code: github.com/gkamradt/LLMTest_NeedleInAHaystack
Authors: 1

Cite

Notes

Only stored in your browser.

Attribution

Abstract & full text: github.com/gkamradt/LLMTest_NeedleInAHaystack
TL;DR: Semantic Scholar
Code: github.com/gkamradt/LLMTest_NeedleInAHaystack

Attribution policy →

Introduces 2 artifacts - 2 evals

Artifacts

2

Evals

Needle in a Haystack (NIAH)Needle in a Haystack

Authors

1