You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When trying to issue a length-exceeded warning while --no_cache is given, the MegatronLMClient errors out with the below stack trace.
The reason being that cache_key is None, which is not checked. The easiest solution would probably be to just omit printing the text strings if cache_key is None (i.e., if caching is disabled).
Traceback (most recent call last):
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/main.py", line 101, in <module>
main()
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/main.py", line 67, in main
results = evaluator.simple_evaluate(
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/utils.py", line 242, in _wrapper
return fn(*args, **kwargs)
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/evaluator.py", line 103, in simple_evaluate
results = evaluate(
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/utils.py", line 242, in _wrapper
return fn(*args, **kwargs)
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/evaluator.py", line 297, in evaluate
resps = getattr(lm, reqtype)([req.args for req in reqs])
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/base.py", line 210, in loglikelihood_rolling
string_nll = self._loglikelihood_tokens(
File "/p/project/opengptx-elm/ebert1/opengpt/lm-evaluation-harness/lm_eval/models/megatronlm.py", line 187, in _loglikelihood_tokens
f"WARNING: Length of concatenated context ...{repr(cache_key[0][-20:])} and continuation {repr(cache_key[1])} exceeds max length {self.max_length + 1}"
TypeError: 'NoneType' object is not subscriptable
The text was updated successfully, but these errors were encountered:
When trying to issue a length-exceeded warning while
--no_cache
is given, theMegatronLMClient
errors out with the below stack trace.The reason being that
cache_key is None
, which is not checked. The easiest solution would probably be to just omit printing the text strings ifcache_key is None
(i.e., if caching is disabled).The text was updated successfully, but these errors were encountered: