Gushaka ibimenyetso muri LoCoMo
Run yuzuye ifunguye. Scores zerekana niba evidence ikenewe iri muri memories zagaruwe.
Benchmark ifunguye
Uburyo Engram ipimwa
Reba amakuru rusange, ibipimo byo gushakisha, igenzura ry'ibisubizo, ibipimo fatizo, imipaka n'amategeko y'ibivugwa inyuma y'ibisubizo byatangajwe.
Gushaka ibimenyetso muri LoCoMo92.19%
Gusuzuma ibisubizo muri LoCoMo66.33%
Uko Bikora
Gusuzuma ibisubizo muri LoCoMo
Isuzuma rya Codex/self-review ku mirimo 98 imwe. Ipima niba igisubizo cyakozwe kivuye muri context yagaruwe ari cyo.
Uko wasoma R@200 na Top-200
Evidence retrieval iba cumulative: R@200 ishobora kubamo ibyo R@50 yabonye n’ibindi, bityo ntikagombye kuba hasi. Answer accuracy iratandukanye. Context nyinshi ishobora kuzana noise cyangwa memories zivuguruzanya; Top-50 rimwe na rimwe itanga igisubizo gisukuye, Top-200 ikareba niba packaging igifasha iyo evidence ibaye nyinshi.
1,536
ibibazo bya evidence retrieval byapimwe
92.19%
Engram R@50
66.33%
answer accuracy ivuye muri top-50 context
Dutandukanya evidence retrieval na answer accuracy. Byombi bikoresha LoCoMo ifunguye, ariko bipima ibice bitandukanye bya memory loop.
LoCoMo ↗Dupima Engram mu buryo bubiri: niba ibona evidence ikwiye, kandi niba LLM isubiza neza ikoresheje iyo context.
Dutandukanya evidence retrieval na answer accuracy. Byombi bikoresha LoCoMo ifunguye, ariko bipima ibice bitandukanye bya memory loop.