Extension doesn't work

[bluewave41]

This has been testing on a clean install of the staging branch.

Upon clicking to "Auto-Generate Tasks" my model returns empty responses.

Endpoint response: {
  id: 'conv-1711932371199674112',
  object: 'text_completion',
  created: 1711932371,
  model: 'LLaMA2-13B-Psyfighter2.Q4_K_S.gguf',
  choices: [
    {
      index: 0,
      finish_reason: 'stop',
      text: '',
      logprobs: { top_logprobs: [ {} ] }
    }
  ],
  usage: { prompt_tokens: 1527, completion_tokens: 1, total_tokens: 1528 }
}

Before eventually crashing with the console stack trace of

Uncaught (in promise) Error: Generate circuit breaker interruption
    onSuccess http://127.0.0.1:8000/script.js:4133
    promise callback*Generate http://127.0.0.1:8000/script.js:4067
    onSuccess http://127.0.0.1:8000/script.js:4140
    Generate http://127.0.0.1:8000/script.js:4067
    onSuccess http://127.0.0.1:8000/script.js:4140
    Generate http://127.0.0.1:8000/script.js:4067
    onSuccess http://127.0.0.1:8000/script.js:4140
    Generate http://127.0.0.1:8000/script.js:4067
    onSuccess http://127.0.0.1:8000/script.js:4140
    Generate http://127.0.0.1:8000/script.js:4067
    onSuccess http://127.0.0.1:8000/script.js:4140
    Generate http://127.0.0.1:8000/script.js:4067
    generateQuietPrompt http://127.0.0.1:8000/script.js:2393
    generateTasks http://127.0.0.1:8000/scripts/extensions/third-party/Extension-Objective/index.js:76
    onGenerateObjectiveClick http://127.0.0.1:8000/scripts/extensions/third-party/Extension-Objective/index.js:602
    jQuery 8
    <anonymous> http://127.0.0.1:8000/scripts/extensions/third-party/Extension-Objective/index.js:778
    jQuery 11
    <anonymous> http://127.0.0.1:8000/scripts/extensions/third-party/Extension-Objective/index.js:732
script.js:4133:23
    onGenerateObjectiveClick http://127.0.0.1:8000/scripts/extensions/third-party/Extension-Objective/index.js:602
    AsyncFunctionThrow self-hosted:856
    jQuery 8
    <anonymous> http://127.0.0.1:8000/scripts/extensions/third-party/Extension-Objective/index.js:778
    jQuery 11
    <anonymous> http://127.0.0.1:8000/scripts/extensions/third-party/Extension-Objective/index.js:732

Adding manual tasks causes it to causes it to crash silently when the objective checking runs after X messages and it stops allowing new messages to be sent.

I'm on a Mac M1 Sonoma 14.3.1 should it be relevant.

My start script is

#!/usr/bin/env bash -l

osascript -e 'tell application "Terminal" to do script "cd ~/Desktop/llama/text-generation-webui && ./start_macos.sh"'

# SillyTavern extras
osascript -e 'tell application "Terminal" to do script "conda activate extras && cd ~/SillyTavern-extras && python server.py --enable-modules=caption,summarize,classify,sd --mps"'

# SillyTavern
osascript -e 'tell application "Terminal" to do script "cd ~/Desktop/tavern/SillyTavern && ./start.sh"'

[Kedwyn]

I noticed that with most models, if you send a message as "system" they return nothing or just repeat themselves. If you do the same but as "user" instead, it works properly. I wish I knew how to force this extension to send a prompt as "user" instead of "system" : (

[bluewave41]

I noticed that with most models, if you send a message as "system" they return nothing or just repeat themselves. If you do the same but as "user" instead, it works properly. I wish I knew how to force this extension to send a prompt as "user" instead of "system" : (

Can you provide an example of how you're sending them as "user" from the UI so I can investigate?

I'd love to get this to work but it looks like we're on our own in terms of programming it.

[Kedwyn]

Mind you, this might be a formatting issue as well, but I don't really know what the reason is anymore.

 . . . More messages above.
    {
      role: 'assistant',
      content: '**Economy of the Wastes =**\n' +
        '\n' +
        'In the aftermath of the Great War, the economy of the Wastes is a fragile and volatile entity. The standard unit of currency is the Silver Stag, a small flattened bead of pure silver.\n' +
        '\n' +
        'Basic necessities such as food, water, and shelter are scarce and expensive:\n' +
        '\n' +
        '* 1 loaf of bread: 5 Silver Stags\n' +
        '* 1 liter of clean water: 3 Silver Stags\n' +
        "* 1 night's lodging in a safehouse: 10 Silver Stags\n" +
        '\n' +
        'Wages for various professions are as follows:\n' +
        '\n' +
        '* Mercenary: 50 Silver Stags per day\n' +
        '* Skilled laborer: 20 Silver Stags per day\n' +
        '* Unskilled laborer: 10 Silver Stags per day\n' +
        '* Scholar or researcher: 30 Silver Stags per day\n' +
        '\n' +
        'Note: These prices are subject to change based on location, availability, and other factors.'
    },
    { role: 'user', content: 'create a few more prices for items.' },
    {
      role: 'assistant',
      content: 'Here are a few more prices for items in the Economy of the Wastes:\n' +
        '\n' +
        '**Weapons and Armor**\n' +
        '\n' +
        '* Rusty sword: 15 Silver Stags\n' +
        '* Leather armor: 25 Silver Stags\n' +
        '* Wooden shield: 10 Silver Stags\n' +
        '* Crossbow and 10 bolts: 40 Silver Stags\n' +
        '\n' +
        '**Potions and Scrolls**\n' +
        '\n' +
        '* Healing potion: 20 Silver Stags\n' +
        '* Mana restoration potion: 15 Silver Stags\n' +
        '* Scroll of fireball: 50 Silver Stags\n' +
        '* Scroll of healing word: 30 Silver Stags\n' +
        '\n' +
        '**Miscellaneous**\n' +
        '\n' +
        '* Horse: 100 Silver Stags\n' +
        '* Pack mule: 50 Silver Stags\n' +
        '* Map of nearby area: 10 Silver Stags\n' +
        '* Rumors of hidden treasure: 5 Silver Stags (may or may not be accurate)\n' +
        '\n' +
        "Keep in mind that these prices are subject to change based on the region, availability, and other factors. Additionally, prices may vary depending on the seller and the buyer's negotiation skills."
    },
    {
      role: 'system',     <------ MAYBE THIS IS THE PROBLEM!!!
      content: 'Pause your roleplay. Please generate a numbered list of plain text tasks to complete an objective. The objective that you must make a numbered task list for is: "A dungeon with rooms, monsters, and treasure". The tasks created should take into account the character traits of World Builder. These tasks may or may not involve Helena Orthwell directly. Include the objective as the final task.'
    }
  ],
  prompt: undefined,
  model: 'anthropic/claude-3-haiku:beta',
  temperature: 0.8,
  max_tokens: 1096,
  stream: false,
  presence_penalty: 0,
  frequency_penalty: 0,
  top_p: 0.9,
  top_k: 10,
  stop: undefined,
  logit_bias: {},
  seed: undefined,
  n: undefined,
  transforms: [ 'middle-out' ],
  min_p: 0,
  top_a: 1,
  repetition_penalty: 1.15
}
{
  id: 'gen-73SNYuUUOkzPbVsMywRx79qkyvst',
  model: 'anthropic/claude-3-haiku:beta',
  object: 'chat.completion',
  created: 1715668217,
  choices: [
    {
      index: 0,
      message: { role: 'assistant', content: '' },
      finish_reason: 'end_turn'
    }
  ],
  usage: {
    prompt_tokens: 4714,
    completion_tokens: 3,
    total_tokens: 4717,
    total_cost: 0.00118225
  }

[Haiku] Notice that the last message sent is as 'system' and not as 'user'. This returns an empty message for some reason.

Now the same last message but with 'user' instead of 'system'. I don't know how to send a message as user if not through the message box. But notice that the formatting is the same, I think...

    {
      role: 'assistant',
      content: '**Economy of the Wastes =**\n' +
        '\n' +
        'In the aftermath of the Great War, the economy of the Wastes is a fragile and volatile entity. The standard unit of currency is the Silver Stag, a small flattened bead of pure silver.\n' +
        '\n' +
        'Basic necessities such as food, water, and shelter are scarce and expensive:\n' +
        '\n' +
        '* 1 loaf of bread: 5 Silver Stags\n' +
        '* 1 liter of clean water: 3 Silver Stags\n' +
        "* 1 night's lodging in a safehouse: 10 Silver Stags\n" +
        '\n' +
        'Wages for various professions are as follows:\n' +
        '\n' +
        '* Mercenary: 50 Silver Stags per day\n' +
        '* Skilled laborer: 20 Silver Stags per day\n' +
        '* Unskilled laborer: 10 Silver Stags per day\n' +
        '* Scholar or researcher: 30 Silver Stags per day\n' +
        '\n' +
        'Note: These prices are subject to change based on location, availability, and other factors.'
    },
    { role: 'user', content: 'create a few more prices for items.' },
    {
      role: 'assistant',
      content: 'Here are a few more prices for items in the Economy of the Wastes:\n' +
        '\n' +
        '**Weapons and Armor**\n' +
        '\n' +
        '* Rusty sword: 15 Silver Stags\n' +
        '* Leather armor: 25 Silver Stags\n' +
        '* Wooden shield: 10 Silver Stags\n' +
        '* Crossbow and 10 bolts: 40 Silver Stags\n' +
        '\n' +
        '**Potions and Scrolls**\n' +
        '\n' +
        '* Healing potion: 20 Silver Stags\n' +
        '* Mana restoration potion: 15 Silver Stags\n' +
        '* Scroll of fireball: 50 Silver Stags\n' +
        '* Scroll of healing word: 30 Silver Stags\n' +
        '\n' +
        '**Miscellaneous**\n' +
        '\n' +
        '* Horse: 100 Silver Stags\n' +
        '* Pack mule: 50 Silver Stags\n' +
        '* Map of nearby area: 10 Silver Stags\n' +
        '* Rumors of hidden treasure: 5 Silver Stags (may or may not be accurate)\n' +
        '\n' +
        "Keep in mind that these prices are subject to change based on the region, availability, and other factors. Additionally, prices may vary depending on the seller and the buyer's negotiation skills."
    },
    {
      role: 'user',
      content: 'Pause your roleplay. Please generate a numbered list of plain text tasks to complete an objective. The objective that you must make a numbered task list for is: "A dungeon with rooms, monsters, and treasure". The tasks created should take into account the character traits of World Builder. These tasks may or may not involve Helena Orthwell directly. Include the objective as the final task.'
    },
    {
      role: 'assistant',
      content: 'Certainly, here is a numbered list of tasks to complete the objective of "A dungeon with rooms, monsters, and treasure":\n' +
        '\n' +
        '1. Determine the overall theme and tone of the dungeon, taking into account the dark, gritty nature of the world and the moral ambiguity of the characters.\n' +
        '2. Design the layout of the dungeon, including the number and types of rooms, as well as any environmental hazards or obstacles.\n' +
        '3. Populate the dungeon with a variety of monsters, some of which may be corrupted or mutated due to the aftermath of the Great War.\n' +
        '4. Incorporate elements of both demonic and angelic influence within the dungeon, reflecting the warring factions and their ongoing struggle for power.\n' +
        '5. Distribute valuable treasure throughout the dungeon, but ensure that it is guarded by formidable challenges and traps.\n' +
        '6. Consider the potential for moral dilemmas or difficult choices that the players may face, such as the temptation to align with a demonic or angelic entity for personal gain.\n' +
        '7. Develop a backstory for the dungeon, tying it to the larger narrative of the world and the events of the Great War.\n' +
        "8. Brainstorm potential plot hooks or adventure hooks that could draw adventurers to the dungeon, whether it's the promise of wealth, the need to uncover a hidden secret, or the desire to confront a powerful entity.\n" +
        '9. Ensure that the dungeon is challenging and engaging, with a sense of danger and uncertainty, but also the potential for great reward and personal growth.\n' +
        '10. Incorporate the objective: Create a dungeon with rooms, monsters, and treasure that reflects the dark, gritty nature of the world and the moral ambiguity of the characters.'
    }

I noticed that this is more of model thing. Llama 70b has no trouble in generating Objectives, it answers to 'system' just fine. Either way, I've tried to mess with the generateQuietPrompt function, which this extension imports, but I've been unsuccessful in changing the role. Which I assume is because I have not a clue of what I am doing.

[Kedwyn]

Oi! I was thinking about what I sent, and I realized that I might be jumping the gun, that maybe it is not system role problem. Then I remembered that you can send system messages with /sysgen (prompt), and guess what? It still returns nothing! So I think it's definitely something to do with sending a prompt as 'system' instead of 'user'.