v0.8.3

Preface:

Merry Christmas and Happy New Year! This will probably be the last release for this year (aside critical bug fixes) and I just want to say thanks to everyone who has supported the project! ChatterUI has grown from a simple curiosity to an awesome tool used by thousands, and it wouldn't have been possible without your support and feedback.

v0.8.3a

Fixes:

• SliderInput text not updating when changing Sampler presets.
• Custom API Templates incorrectly adding duplicate entries from the base OpenAI template.

v0.8.3

BREAKING CHANGE: Llama.cpp no longer supports Q4_0_4_4, Q4_0_4_8 and Q4_0_8_8. Instead, Q4_0 will automatically be aligned for ARM optimized kernels, use Q4_0 from now on!

This is a cummulative update for all beta features over the last month, including the major migration to Expo SDK 52 and the new React-Native architecture.

This update introduces the new experimental API Manager and API Configuration templates! This should allow you to add APIs missing from ChatterUI to a degree. This is mostly useful for APIs which are almost compliant with the OpenAI spec but may have extra or reduced sampler options. These configurations are sharable jsons which you can share to add new API compatibility to ChatterUI!

Known Issue:

• Softlock: If you have an Animated Ellipses visible during generations and open a menu that causes an alert, the alert will be invisible.

Features:

• Updated to Expo SDK 52. This is mostly an under-the-hood change, but should make the app feel a lot more responsive, some screens load much faster than before, and app startup should feel a lot quicker.
• Added a new API Manager!
  • This API manager functions similar to the recently added Model manager.
  • If you prefer the old API Manager, simply go to Settings > Use Legacy API
  • You can now have multiple connection presets to the same API type
  • You can now create your own API Configuration Templates! Refer to this discussion to give feedback and suggestions.
• Added an option for landscape mode in settings. This is automatically enabled for tablets.
• Added a unique component for editing string arrays such as stop_sequence.
• Added a Lock App feature which will require user authentication via PIN/Pattern/Biometrics when starting the app.
  • This is disabled by default, and can be enabled in the Settings menu.
• Added Notifications on completion, which will let you know when generations are complete while the app is in the background.
  • This is disabled by default, and can be enabled in the Settings menu.
  • You can toggle showing the message and character names from the notification or just show a generic 'Completion completed' notification.

Changes:

• Updated llama.cpp: This brings in a new feature which requantizes Q4_0 models into Q4_0_X_X for optimized arm kernels upon loading the model, removing the need for special quantizations.
  • Dry Sampling is now available in Local Mode!
• Added a new dropdown bottom sheet. This will replace a few old dropdown menus in future.
• Instruct Menu now uses new dropdown and popup menu for button functions.
• Characters and Chats will no longer have last_modified update when accessed, only when edited or chatted.
• Checkboxes have been changed to a new custom component.
• Changed animated ellipses during generations
• Updated to a new Markdown formatter! This formatter should be far better when dealing with mixed bullet types and nested lists.
• All inferencing is now done in a background task - this fixes an issue where generations are paused during the prompt building phase due to tabbing out of the app too quickly after a generation begins.

Fixes:

• Incorrect Gemma2 Instruct format.
• Exporting strings resulting is broken base64 files instead
• Cohere API being completely broken
• Fixed a softlock on Character List if you close search bar while a filter removes all characters from the list
• Fixed a crash on startup due to importing an unsupported model.

v0.8.3-beta5

v0.8.3-beta5

BREAKING FEATURE: Llama.cpp no longer supports Q4_0_4_4, Q4_0_4_8 and Q4_0_8_8. Instead, Q4_0 will automatically be aligned for ARM optimized kernels, use Q4_0 from now on.

Features:

• Reimplemented Horde for the new API system.

Changes:

• Updated Sliders which should fix buggy behavior.
  • This should also fix tapping out of the slider textbox not updating the value.
• Sync llama.cpp:
  • This removes support for Q4_0_4_4, Q4_0_4_8 and Q4_0_8_8. These models should now remain on Q4_0 only.
• ChatML is now the default instruct format on first install
• Editing chat should automatically scroll to bottom

Fixes:

• Legacy API system not working correctly.
• Incorrect Gemma2 Instruct format.

v0.8.3-beta4

v0.8.3-beta4

Just a small update with a few important fixes.

Changes:

• Instruct Menu now uses new dropdown and popup menu for button functions.

Fixes:

• The Samsung/Pixel header button bug for the API Manager.
• Exporting strings resulting is broken base64 files instead
• Incorrect response parsers for OpenAI, Text Completions and Chat Completions
• Checkboxes not updating properly

v0.8.3-beta3

v0.8.3-beta3

This update introduces the new experimental API Manager and API Configuration templates! This should allow you to add APIs missing from ChatterUI to a degree. This is mostly useful for APIs which are almost compliant with the OpenAI spec but may have extra or reduced sampler options. These configurations are sharable jsons which you can share to add new API compatibility to ChatterUI!

Features:

• Added a new API Manager!
  • This API manager functions similar to the recently added Model manager.
  • If you prefer the old API Manager, simply go to Settings > Use Legacy API
  • You can now have multiple connection presets to the same API type
  • You can now create your own API Configuration Templates! Refer to this discussion to give feedback and suggestions.
• Updated llama.cpp: This brings in a new feature which requantizes Q4_0 models into Q4_0_X_X for optimized arm kernels upon loading the model, removing the need for special quantizations.

Changes:

• Added a new dropdown bottom sheet. This will replace a few old dropdown menus in future.
• Characters and Chats will no longer have last_modified update when accessed, only when edited or chatted.
• Checkbox styles have been updated!

Fixes:

• Fixed chat displaying upside-down.
• Fixed header buttons not working on Samsung and Pixel devices.
• Fixed Markdown formatter not applying properly in specific cases.
• Fixed a crash on startup due to importing an unsupported model.

Known Issues:

• Popups in screen headers may be incorrectly placed.
• Multiline TextInput fields will not resize to full height on a blank newline.
• During generations, if AnimatedEllipsis is visible, it will break Modals such as Alerts and TextBoxModals until the generation ends.

v0.8.3-beta2

v0.8.3-beta2

Another beta with a few fixes.

Known Issues:

• Popups in screen headers may be incorrectly placed.
• Some lists may become upside-down, likely due to some react-native-reanimated bug.
• Multiline TextInput fields will not resize to full height on a blank newline.
• During generations, if AnimatedEllipsis is visible, it will break Modals such as Alerts and TextBoxModals until the generation ends.

Features:

• Added an option for landscape mode in settings. This is automatically enabled for tablets.
• Added an option to show the character name and generated text in a notification.
• Added a notification icon.

Changes:

• The last_modified value for Characters and Chats are now only updated when a Character Card is edited or a Chat is updated, it will no longer change when a card or chat is only accessed.
• Updated styles for AnimatedEllipsis

Fixes:

• Reverted to base react-native Animated values for chat text boxes. This fixes an issue where Alerts and TextBoxModals are invisible in the ChatList screen due to react-native-reanimated's useAnimatedStyle breaking Modals.
• Trailing text in markdown views not being formatted.

v0.8.3-beta1

v0.8.3-beta1

This is an unstable update to Expo SDK52 and the new React Native bridgeless architecture. As such, functionality may break due to low level changes.

Known Issues:

• Popups in screen headers may be incorrectly placed.
• Some lists may become upside-down, likely due to some react-native-reanimated bug.
• Multiline TextInput fields will not resize to full height on a blank newline.

Additions:

• Added a unique component for editing string arrays such as stop_sequence.
• Added a Lock App feature which will require user authentication via PIN/Pattern/Biometrics when starting the app.
  • This is disabled by default, and can be enabled in the Settings menu.
• Added Notifications on completion, which will let you know when generations are complete while the app is in the background.
  • This is disabled by default, and can be enabled in the Settings menu.
• Updated cui-llama.rn:
  • Dry Sampling is now available in Local Mode!

Changes:

• Added a new Ellipsis animation when waiting for responses.
• Updated Markdown package to a newer library - this also fixes the issue of unordered lists nested within ordered lists.
• All inferencing is now done in a background task - this fixes an issue where generations are paused during the prompt building phase due to tabbing out of the app too quickly after a generation begins.
• Changed Checkbox to new custom toggle component.

Fixes:

• Cohere API being completely broken
• Fixed a softlock on Character List if you close search bar while a filter removes all characters from the list

v0.8.2

v0.8.2

Release with a few critical fixes.

Changes:

• Sync to latest llama.cpp
• Filenames should now appear on model info properly. External models need to be reimported for this to display properly.

Fixes:

• Removed tfs_z due to a crash and deprecation by llama.cpp
• Fixed scenario toggle controlling addition of personality field
• Fixed Add Character button missing on app startup.

v0.8.1

v0.8.1

Features:

• Re-added purging the KV cache from previous versions
• Added check for whether or not to load kv based on previous model id
• Added settings option to bypass context_length, allowing you to send all messages regardless of whether they fit in context or not. It is never recommended to use this unless you know what you're doing!

Fixes:

• Fixed UI features such as Drawers and Alerts which use the FadeBackdrop component always closing the modal when clicking on non-button components on screen.
• Fixed OpenAI prompt builder always having 0 context length. This fix allows you to set the generated length instead
• Fixed ChatCompletions context limit being controlled by max_tokens instead of context length
  • With the two fixes above, both OpenAI and Chat Completions API options now use max_context_length to control the context builder, since the OpenAI model response does not include a context limit.

v0.8.0

v0.8.0

This is a cumulative update for the last few betas with all the new UI changes, management additions and local model features. For those who were following the 0.8.0 task list, some changes were dropped due to the scope of this update ballooning, so those will be relegated to a later update.

This is a big UI/UX overhaul for Character, Chat, Model and User card management, and adds a lot of features to further customize your experience!

Chat

• Merged Recent Chats and Character List. Now the Character List is the default view of the app.
• Moved the Chat History screen to a drawer - this lets you quickly create and manage your chats and makes this list less hidden.
• Chat logs now have names (by default, its New Chat). You can rename these as needed.
• You can Clone chats! This doesn't fully resolve the requests for proper chat forking and tree dialogue, but it has a similar use.
• Added a Portrait viewer when clicking on a character's / user's avatar in chat
• Generated text should properly update a specific chat entry, rather than replace whichever is the current last chat entry loaded.

Characters

• Added a new Character List menu:
  • You can now sort by Recent or Alphabetical order
  • Added a search bar to the character list
  • You can now clone characters
  • List now previews the most recent chat reply and the date of the most recent chat
• Improved the Character Card editor:
  • Added personality, scenario, example messages and alternate greeting fields to the editor
    • Using Personality and Scenario in generations can also now be toggled in the Instruct menu.
  • Character name is now editable
  • Added option to delete and create new alternate greetings
  • Added option to remove character portrait
  • Removed the auto-save that triggered on the editor screen
  • Added an unsaved changes check to prevent you backout out without saving or discarding
• You can now import multiple character cards at a time (Just don't import 300 in one go. You know who you are)
• Fixed UTF8 cards having broken symbols when imported.

User

• Merged the UserList and UserEditor screens into one, the User List is now a drawer in the User editor menu.
• Added ability to rename your User.
• Similar to Characters, you can now delete your User portrait.

Remote and Local Mode

The app now splits Remote and Local modes in the main Options drawer:
- Local Mode lets you customize and use your local models on your device.
- Remote Mode lets you connect to various supported APIs

Remote Mode

This mode is identical to previous versions of ChatterUI's API manager. The only difference is that the Local api option has been moved to Local Mode instead. In future, a bigger API manager rework is planned to allow users to create custom API definitions (mostly because I don't want to integrate every single API under the sun)

Local Mode

• Added a model list heavily inspired by Pocket Pal. This list will show metadata about your model extracted directly out of the GGUF file.

• Added Use External Model - this option adds a model that will load it from your device storage without needing to copy it into ChatterUI.

• Added a Model Settings Page:

  • CPU Settings (Max Context, Threads, Batch) have been moved here
  • Local Specific app settings (Autoload and Save KV) have been moved here
  • Added a Supported Quantization header to show compatibility with Q4_0_4_8 and Q4_0_4_4 models.

• Sync'd llama.cpp with a newer build. This also introduces XTC sampling to local mode.

Other Changes:

• Changed Alerts to use an in-app component instead of the default Android alerts
• Streamlined many components for future maintainability and to unify app design.
• Changed many icons to use AntDesign for uniform icon look.
• Added last_output_prefix to Instruct format
• Added a last_modified field to Chats and Characters for sorting purposes.
• Allowed user-defined SSL certs.
• Moved user card to zustand persist
• Created a unified component for showing default avatar icons when no portrait is present
• Made Drawers, SectionTitles, SwitchWithTitle / Description, FadeBackdrop and many duplicated components to standalone components.
• Updated README.md with updated app screenshots.

v0.8.0a

• Fixed an issue where migrating from 0.7.10 and a few betas would crash due to the Local Model mmkv value being incorrect.
• Made the options drawer scrollable

v0.8.0-beta5

WARNING: For those updating from 0.8.0-beta4, this update has database migrations that may break your install. Please report if your app becomes softlocked.

Updates:

• Local Mode and Remote Mode now toggled between

• New Model Manager!

  • This new model manager screen is far cleaner, allows you to rename your models, shows some metadata like parameter count, filesize and context length.
  • Experimental Feature: Added ability to Link External Model. This allows you to use a model from device storage instead of importing it into ChatterUI's app space. So far working on a few tested devices. However, app permissions and storage permissions may vary between operating systems, so this may not work on your device.

• User Manager Overhaul

  • Changed the User profile customizer to be a single screen. You can now rename your User profile or even clone it.

Style:

• Created a custom Alert component instead of using the default Android alert system. Alerts should now all look like they belong to the app.
• Unified Popup menus across the app to be uniform.