Bug: generateText is ignoring dynamic parameters due to a hard-coded model class

[FWangZil]

Is your feature request (or bug) related to a problem? Please describe.

In the generateText and generateMessageResponse function, the model class is currently hard-coded(gpt-4o), which prevents dynamically switching to different models based on incoming parameters. This reduces flexibility and can increase development complexity in scenarios where different models are needed.

When using a non-OpenAI AI provider, an error is reported because the gpt-4o model cannot be found.

Describe the solution you'd like

• Remove the hard-coded model class reference in the generateText and generateMessageResponse function and instead use a model name/type passed in as a function parameter or configuration.
• Ensure there is a sensible default value to maintain compatibility with existing functionality or cases where a specific model is not provided.

Describe alternatives you've considered

• Leaving it as is would require manual code changes to switch models in different scenarios, which is not ideal.
• Introducing a separate function to handle switching model classes could lead to redundant logic and increase maintenance overhead.

Additional context

• Risks: This change is minor, but it’s important to check if other parts of the code depend on the old hard-coded logic.
• Testing:
  • Test with multiple model types to confirm that the function correctly switches to the specified model.

[FWangZil]

I tried to directly modify the hard-coded gpt-4o to the incoming ModelClass, but encountered a new problem. The current TikToken Model only supports a limited number of models in a list. This modification alone will not solve the problem, and may even introduce new problems.

[0xCardinalError]

I had the same reverse problem actually.

was using anthropic model with generateObject and then it always failed. Worked only when I switched to using the hardcoded functions. So as this is trimTokens that is problematic, seems to me that we should need to have some custom solution for triming. As https://www.npmjs.com/package/js-tiktoken is openAI only and it will fail on each try to trim with other models.

[tcm390]

I tried to directly modify the hard-coded gpt-4o to the incoming ModelClass, but encountered a new problem. The current TikTok Model only supports a limited number of models in a list. This modification alone will not solve the problem, and may even introduce new problems.

Hello! Could you clarify the issue you encountered with the TikTok Model supporting only a limited list of models? Are you referring to the trimTokens function?

[JGTechie]

Thank you guys for flagging this. I had this issue today. GPT-4o is hardcoded and my parameter settings are ignored.

[prince981620]

similar case is with twitter agents, i am passing api key and model name in character.json but it always fall back to one api key provided in .env (gemini api key)
resulting api key gets exhausted while running multiple characters.

[FWangZil]

I tried to directly modify the hard-coded gpt-4o to the incoming ModelClass, but encountered a new problem. The current TikTok Model only supports a limited number of models in a list. This modification alone will not solve the problem, and may even introduce new problems.

Hello! Could you clarify the issue you encountered with the TikTok Model supporting only a limited list of models? Are you referring to the trimTokens function?

Yes, what I meant to say is Tiktoken. I'm sorry I wrote TikTok by mistake at first. I'm referring to the parameter model: TiktokenModel of the trimTokens method.

[impozzible]

@FWangZil I encountered the same issue today also, trying to use anthropic with generateObject, do you have a temporary fix for this?

[FWangZil]

@FWangZil I encountered the same issue today also, trying to use anthropic with generateObject, do you have a temporary fix for this?

Sorry, I haven't found a particularly good method yet, because models other than openai seem to be outside the allowed range of TiktokenModel and need to be modified or processed in other ways.

[tcm390]

@FWangZil I encountered the same issue today also, trying to use anthropic with generateObject, do you have a temporary fix for this?

Sorry, I haven't found a particularly good method yet, because models other than openai seem to be outside the allowed range of TiktokenModel and need to be modified or processed in other ways.

Apologies for the delayed response; I was busy with another task. I've created a PR: #1605. Could you test it to see if it resolves your issue?

I used the AutoTokenizer from Hugging Face's transformers library to handle non-tiktoken models. You'll need to set the environment variable for your custom tokenizer. You can find the model you need here: https://huggingface.co/models?sort=trending

For example:

env

TOKENIZER_MODEL="Xenova/claude-tokenizer"  
TOKENIZER_TYPE="auto"

[tcm390]

@impozzible, please kindly try the PR and see if it fixes your issue.

[tcm390]

Implemented in #1605

[prince981620]

@impozzible, please kindly try the PR and see if it fixes your issue.

What about twitter agent with similar problem ??

[tcm390]

@impozzible, please kindly try the PR and see if it fixes your issue.

What about twitter agent with similar problem ??

Apologies, could you please create a new issue with more details, including steps to reproduce the problem and any relevant information?

[prince981620]

@impozzible, please kindly try the PR and see if it fixes your issue.

What about twitter agent with similar problem ??

Apologies, could you please create a new issue with more details, including steps to reproduce the problem and any relevant information?

@impozzible please have a look at :
#1811 (comment)