v3.0.0

After 8 months of work, I am proud to announce the new v3 release 🎉. The entire frontend and api backend is rewritten from scratch, fixing the most annoying bugs with really large channels. The new version is fast.

Not entirely bug free, but much better than v2, so let's call it stable :). As always, please report any bugs you find (or give feedback) on the issue tracker or Tyrrr's Discord Server -> #dce-frontend channel.

New features

• virtual scroll replaced with cursor based pagination to speed up the loading of large channels (and to make it possible to load channels with more than 210 000 messages)
  • old messages are not unloaded yet - if you scroll too far, click on the timestamp on any message to reload the channel view
• it looks even more like the discord desktop client (or it used to, before the discord changed its design again)
  • direct messages on homepage
  • autocomplete without using filters
  • threads split view
  • new (local) search history
  • new grouped image galleries
  • new music player
  • channel pin button
• rewritten docs to make it (hopefully) easier to understand for both users and developers
• simplified Windows build - github actions now just calls the batch script src\BUILD_RELEASE.bat, so you can build your own binaries without a complicated setup
  • python now uses venv instead of global packages on Windows
• project restructured to make dev and build folders more similar - this will allows us to create configuration files to configure the app without the need to recompile it. The configuration script was temporarily removed and will be replaced with configuration file.
• removed http-server dependency, simplifying the architecture. v3 now just won't render assets if path to them exceeds 260 characters (windows only). The fix is easy - move DCEF closer to the root of the drive. Related nginx issue. The registry tweak is still required.
• removed support for RPI 4B (RPI 4B doesn't support AVX, but RPI 5 does)
• removed themes (for now), only gray theme is available
• search results direction reversed
• force-kill all processes on exit on Windows (v2 sometimes left some processes running)
• frontend now uses svelte5 rc instead of svelte4
• and many more small changes...

Migration guide

Windows - delete everything except exports folder, then copy everything (except exports folder) from the release zip to the DCEF folder. If some assets are not loading, move DCEF closer to the root of the drive.

Linux (docker) - just pull the new image and restart it with the same command as before.

v2.9.0

New markdown renderer

• completely rewritten, now using simple-markdown library
• new rendered markdown elements: titles, subtitles, subsubtitles (discord supports only 3 levels of headings), bullet points, pretty channel links, pretty message links, pretty discord cdn links
• rewritten other markdown elements (role mentions with colors, channel mentions, timestamps, emojis, spoilers, code blocks)
• readded highlight of search terms in search results
• backend server will enrich message objects with additional information to reduce count of requests and to make rendering faster (emotes and roles from message content, channel tags)
• code blocks are now rendered using highlight.js library - the same library and theme same that discord uses
• animated emojis are now rendered correctly (without "> ending) and offline (if assets were downloaded using DCE and used somewhere as a reaction)

New features

• render lottie stickers (json animations)
• add text attachment (.txt files) preview below the message with option to expand/collapse and download it
• restyled attachment info component + add icons based on file extension
• restyled some components (again) to match changes in discord interface

Bugfixes

• fix previews of video embeds not working (broken previously by adding support for local gifs previews)
• fix bug preventing attachments without preview from showing #47
• fullscreen video is now centered in the player
• fix preprocess crash if the referenced asset file does not exist anymore
• fix inconsistent channel icon sizes in channel list

Other changes

• changed wording to be more inclusive (allowlist, denylist)

v2.8.1

New features

• render tenor gifs locally (requires exports from DiscordChatExporter v2.42.3 or newer)
• view as user - user can now be selected in bottom left corner of the app

Bugfixes

• fix audio player not shown for audio attachments
• fix user filters not working if any user is hidden using blacklist user feature using configurator script configurator.py/exe
• fix reaction_from: filter incorrectly using users from mentions: filter
• see also v2.8.0 changelog

v2.8.0

Hide users and discord servers

• added ability to hide all messages from specific user(s) - enforced on server-side. Closes issue #8

• added configurator script (windows configurator.exe, linux configurator.py) to simplify the process of configuring the server without the need to connect directly to mongodb to edit the configuration. This configurator script will allow you to more easily select which discord servers to show and which users to hide. All changes are applied immediately without the need to restart DCEF and are enforced on server side - that's why this script is CLI only. See README.md for instructions.

View messages as any user

• added ability to view discord as any user - to enable this feature, you need to right click any message and select View discord as this user. This enables you to:
  • see highlighted message reactions if the user is in the reaction list.
  • see highlighted messages containing the user's mention (doesn't include roles)

Search changes

• user search autocomplete now matches nicknames as well
• search results now have buttons to sort by old or new
• search results now have a button to show all results if a limit of 100 000 results is reached. After clicking the button, limit:0 filter is added to the search query
• after submitting a search query, the gui will show "searching..." text until the search is finished, because some searches may take a long time
• added a pin button next to search input, which will add is:pinned in:"<channelName>" filter to the search query
• fulltext search now searches in title and description of embeds

Renderer changes

• you can now select a date and time format independently in settings (before you could only select a timestamp format)
• updated both name format renderer - username is in gray color and only display name is colored
  • added name renderers where they were missing (reply author, system messages, ...)
• fix reactions not rendering in system messages
• remove pinned messages yellow background highlight

Known bugs

• is any user is blocked in server configuration, from:, mentions: and reaction_from: filters won't work.

v2.7.1

Bugfix

• fix a crash affecting new DCEF users (in database upgrade logic) introduced in v2.7.0
• see also v2.7.0 changelog

v2.7.0

Search improvements

• information doesn't leak between servers anymore (thanks to database partitioning explained below)
  • fixes many many autocomplete bugs - autocomplete counts are more accurate, all suggestions (nicknames+assets+emojis,...) are restricted strictly only to currently selected guild, etc...
• added time filters before:, during: and after:
  • supporting YYYY, YYYY-MM, and YYYY-MM-DD formats
  • behaving exactly like in Discord, the only difference is that these filters are not timezone aware (they use UTC)
  • calendar picker is not yet implemented
• new has: filters - has:sticker, has:embed and has:file
• renamed filters (to be more consistent with Discord):
  has_audio:true -> has:sound
has_image:true -> has:image
has_link:true -> has:link
• removed has_audio:false and has_image:false and has_link:false filters (you can't use negative filters now)
• removed has_other:true and has_other:false filters
• reordered filters, so the ones from Discord are first and in the same order as in Discord
• filename autocomplete now suggests only attachment names - this fixes a bug where autocomplete would suggest assets such as profile images that you cannot search for
• added autocomplete for extension: filter
  • suggested file extensions are sorted by file count, file count is shown in GUI
• changed autocomplete styling (main autocomplete shows description on second line, ... and more)

Database refactor

• objects in mongo database are now partitioned by guild id
  • this will improve performance, simplifies database queries, allows selecting which guild to show in GUI (enforces guild whitelisting at a database level), But disallows cross-guild queries (for example channel mentions mentioning channel in different guild won't work)
• added guild whitelisting
  • after preprocess phase is completed, connect to mongodb database (localhost:27017) -> dcef database -> config collection -> find document with key whitelisted_guild_ids -> set value to array of guild ids to whitelist (for example ["123456789012345678", "123456789012345679"])
    • empty list (default) means no guilds are whitelisted (shows all guilds)
    • there will be a simpler way to configure this in the future

Preprocess changes

• Fixed preprocess crash if incomplete JSON export is imported
• Percentage progress is now based on bytes processed, not number of JSON files to improve estimation accuracy

v2.6.2

IMPORTANT: Discord CDN links are now signed (you may have noticed newly added ex, is and hm query parameters in your CDN links that expire after 24 hours). If your DiscordChatExporter exports were made without --media command line option, you will lose access to all hotlinked attachments and embeds. Non-signed links still work for now, download your assets locally ASAP or reexport your channels with --media option if you still have access to them.

New DiscordChatExporter feature: DiscordChatExporter v2.41 or newer allows you to automatically export threads and forum posts. Just add --include-threads All to exportguild subcommand. Documentations in this repo was updated to reflect this change.

New project - DiscordChatExporter wrapper: I have created simple wrapper for DiscordChatExporter to simplify the process of incremental export for this frontend viewer. You declare what you want to export in a config file and the wrapper will automatically export only new messages since the last export. For more information and instructions how to use, see slatinsky/DiscordChatExporter-incrementalBackup

Changes:

• render tenor gifs (if online option is enabled in the settings)
• you can now run this frontend viewer on raspberry pi 4b
• fix wrong name+nickname order in author profile modal
• view selected reaction image in gallery, add reaction tooltips to see reaction names

pull request #39 (thanks @lolmaxz):

• add channel Icon for voice and announcement channels
• add Booster system icon
• support new system messages - Booster Level 1, Booster Level 2 and Booster Level 3
• channels without a category are now at the top of the channel list
• channel's name in channel list do not wrap anymore. Channel's name that are too long will be clipped.
• add tooltips in channel list to show the full channel name on mouse hover
• scrollbar in channel's list is now on top instead of taking a space on Hover
• ...and other adjustments

Deprecation warning:

• with the introduction of signed discord CDN links, hotlinked discord assets (exports without --media option) will no longer be supported in future versions of this frontend viewer.

v2.6.1

new features:

• new search filters - reaction_from:user and reaction_from_id:user_id

  • searches for message reactions from a specific user

• new search filters - deleted:true and deleted:false

  • searches for deleted messages
  • for deleted messages to be detected you need multiple overlapping exports of the same channel
  • deleted messages have dark red background in the UI
  • needs to load all single channel message ids from the database (requires more RAM in preprocess stage)

bugfixes:

• windows runner - preprocess logs are not delayed anymore (console is now always flushed on new line)

v2.6.0

Speed and memory usage improvements:

• added better progress logs including counts and percentage

• json exports are now streamed in batches of 10000 messages

  • this means that you can now process extra large datasets without running out of memory (only 150 MB of ram is need to preprocess json export no matter how large the dataset is)

• records are inserted into the database in batches

  • preprocessing is much much faster

• improved startup time logic

  • already processed files check is faster

    • using single database query instead of query for each file
    • BUT doesn't check now if the same file has been modified since last run

  • if no new exports are found, expensive asset finder won't run, saving more startup time

• use exportedAt field to keep in the database only the latest message

  • exportedAt field was added to DiscordChatExporter >= v2.40.1
  • exportedAt is estimated based on timestamp and timestampEdited fields if export is not created by DCE v2.40.1 or newer

bugs:

• category counts are not correctly counted, category sorting based on message count in channel selector is incorrect

v2.5.1

Bugfixes

• fix json export not detected if filename contains only characters 0123456789abcdefABCDEF
• fix extensionless 5-letter asset not detected
• fix assets beginning with a dot not detected
• fix rendering of any extensionless asset (as a workaround we are only assuming that the asset is an image, better than nothing)
• fix scroll to top/bottom/message jumping to the wrong spot
• fix scroll to bottom scrolling page by page from the top to the bottom if channel didn't load in time
• fix not being able to see more than ~330000 messages in a single channel (max height of 33554400px was reached)
  • confirmed working with 6.9 million messages in a single channel
  • if my calculations are correct, the viewer should now handle up to 41 million messages in a single channel
• render identicon if image fails to load