Gushaka ibimenyetso muri LoCoMo
Run yuzuye ifunguye. Scores zerekana niba evidence ikenewe iri muri memories zagaruwe.
Benchmark ifunguye
Uburyo Engram ipimwa
Reba amakuru rusange, ibipimo byo gushakisha, igenzura ry'ibisubizo, ibipimo fatizo, imipaka n'amategeko y'ibivugwa inyuma y'ibisubizo byatangajwe.
Byavuguruwe 4 iminota yo gusoma
Gushaka ibimenyetso muri LoCoMo93.29%
Gusuzuma ibisubizo muri LoCoMo89.65%
Uko Bikora
Gusuzuma ibisubizo muri LoCoMo
Buri murongo ukoresha imirimo 1,536 yose ifite ibimenyetso n’umupaka umwe wa top-50. Codex yakoze ibisubizo, umusuzumyi wihariye utazi sisitemu arabipima. Iri ni isuzuma ryakozwe na Engram, si urutonde rwemewe rwa LoCoMo.
Uko wasoma imbonerahamwe zombi
Ibipimo bibiri byatangajwe bikoresha imirimo 1,536 yose ifite ibimenyetso. Gushaka ibimenyetso bisaba ibiranga inkomoko, kandi buri sisitemu ikoreshwa na model zimwe zo gusubiza no gusuzuma. Sisitemu zituzuza amasezerano amwe y’isuzuma zandikwa ukwazo aho kuvangwa ku rutonde.
1,536
ibibazo bya evidence retrieval byapimwe
93.29%
Engram R@50
89.65%
answer accuracy ivuye muri top-50 context
Dutandukanya evidence retrieval na answer accuracy. Byombi bikoresha LoCoMo ifunguye, ariko bipima ibice bitandukanye bya memory loop.
AgentMemory igarura ibisubizo bitarenze 100, ni yo mpamvu R@200 igaragazwa nka N/A. Tenure ntibika indangamimerere z’ibimenyetso bya LoCoMo, bityo ntishyirwa ku rutonde rusange; igerageza ryayo rikomeza gusobanurwa mu buryo bw’isuzuma.
Dupima Engram mu buryo bubiri: niba ibona evidence ikwiye, kandi niba LLM isubiza neza ikoresheje iyo context.
Dutandukanya evidence retrieval na answer accuracy. Byombi bikoresha LoCoMo ifunguye, ariko bipima ibice bitandukanye bya memory loop.