From 5eb292dc10b99e13c6f606b7d9f0018f59052574 Mon Sep 17 00:00:00 2001 From: Carter Anderson Date: Wed, 6 Sep 2023 19:07:27 -0700 Subject: [PATCH] Bevy Asset V2 (#8624) # Bevy Asset V2 Proposal ## Why Does Bevy Need A New Asset System? Asset pipelines are a central part of the gamedev process. Bevy's current asset system is missing a number of features that make it non-viable for many classes of gamedev. After plenty of discussions and [a long community feedback period](https://github.com/bevyengine/bevy/discussions/3972), we've identified a number missing features: * **Asset Preprocessing**: it should be possible to "preprocess" / "compile" / "crunch" assets at "development time" rather than when the game starts up. This enables offloading expensive work from deployed apps, faster asset loading, less runtime memory usage, etc. * **Per-Asset Loader Settings**: Individual assets cannot define their own loaders that override the defaults. Additionally, they cannot provide per-asset settings to their loaders. This is a huge limitation, as many asset types don't provide all information necessary for Bevy _inside_ the asset. For example, a raw PNG image says nothing about how it should be sampled (ex: linear vs nearest). * **Asset `.meta` files**: assets should have configuration files stored adjacent to the asset in question, which allows the user to configure asset-type-specific settings. These settings should be accessible during the pre-processing phase. Modifying a `.meta` file should trigger a re-processing / re-load of the asset. It should be possible to configure asset loaders from the meta file. * **Processed Asset Hot Reloading**: Changes to processed assets (or their dependencies) should result in re-processing them and re-loading the results in live Bevy Apps. * **Asset Dependency Tracking**: The current bevy_asset has no good way to wait for asset dependencies to load. It punts this as an exercise for consumers of the loader apis, which is unreasonable and error prone. There should be easy, ergonomic ways to wait for assets to load and block some logic on an asset's entire dependency tree loading. * **Runtime Asset Loading**: it should be (optionally) possible to load arbitrary assets dynamically at runtime. This necessitates being able to deploy and run the asset server alongside Bevy Apps on _all platforms_. For example, we should be able to invoke the shader compiler at runtime, stream scenes from sources like the internet, etc. To keep deployed binaries (and startup times) small, the runtime asset server configuration should be configurable with different settings compared to the "pre processor asset server". * **Multiple Backends**: It should be possible to load assets from arbitrary sources (filesystems, the internet, remote asset serves, etc). * **Asset Packing**: It should be possible to deploy assets in compressed "packs", which makes it easier and more efficient to distribute assets with Bevy Apps. * **Asset Handoff**: It should be possible to hold a "live" asset handle, which correlates to runtime data, without actually holding the asset in memory. Ex: it must be possible to hold a reference to a GPU mesh generated from a "mesh asset" without keeping the mesh data in CPU memory * **Per-Platform Processed Assets**: Different platforms and app distributions have different capabilities and requirements. Some platforms need lower asset resolutions or different asset formats to operate within the hardware constraints of the platform. It should be possible to define per-platform asset processing profiles. And it should be possible to deploy only the assets required for a given platform. These features have architectural implications that are significant enough to require a full rewrite. The current Bevy Asset implementation got us this far, but it can take us no farther. This PR defines a brand new asset system that implements most of these features, while laying the foundations for the remaining features to be built. ## Bevy Asset V2 Here is a quick overview of the features introduced in this PR. * **Asset Preprocessing**: Preprocess assets at development time into more efficient (and configurable) representations * **Dependency Aware**: Dependencies required to process an asset are tracked. If an asset's processed dependency changes, it will be reprocessed * **Hot Reprocessing/Reloading**: detect changes to asset source files, reprocess them if they have changed, and then hot-reload them in Bevy Apps. * **Only Process Changes**: Assets are only re-processed when their source file (or meta file) has changed. This uses hashing and timestamps to avoid processing assets that haven't changed. * **Transactional and Reliable**: Uses write-ahead logging (a technique commonly used by databases) to recover from crashes / forced-exits. Whenever possible it avoids full-reprocessing / only uncompleted transactions will be reprocessed. When the processor is running in parallel with a Bevy App, processor asset writes block Bevy App asset reads. Reading metadata + asset bytes is guaranteed to be transactional / correctly paired. * **Portable / Run anywhere / Database-free**: The processor does not rely on an in-memory database (although it uses some database techniques for reliability). This is important because pretty much all in-memory databases have unsupported platforms or build complications. * **Configure Processor Defaults Per File Type**: You can say "use this processor for all files of this type". * **Custom Processors**: The `Processor` trait is flexible and unopinionated. It can be implemented by downstream plugins. * **LoadAndSave Processors**: Most asset processing scenarios can be expressed as "run AssetLoader A, save the results using AssetSaver X, and then load the result using AssetLoader B". For example, load this png image using `PngImageLoader`, which produces an `Image` asset and then save it using `CompressedImageSaver` (which also produces an `Image` asset, but in a compressed format), which takes an `Image` asset as input. This means if you have an `AssetLoader` for an asset, you are already half way there! It also means that you can share AssetSavers across multiple loaders. Because `CompressedImageSaver` accepts Bevy's generic Image asset as input, it means you can also use it with some future `JpegImageLoader`. * **Loader and Saver Settings**: Asset Loaders and Savers can now define their own settings types, which are passed in as input when an asset is loaded / saved. Each asset can define its own settings. * **Asset `.meta` files**: configure asset loaders, their settings, enable/disable processing, and configure processor settings * **Runtime Asset Dependency Tracking** Runtime asset dependencies (ex: if an asset contains a `Handle`) are tracked by the asset server. An event is emitted when an asset and all of its dependencies have been loaded * **Unprocessed Asset Loading**: Assets do not require preprocessing. They can be loaded directly. A processed asset is just a "normal" asset with some extra metadata. Asset Loaders don't need to know or care about whether or not an asset was processed. * **Async Asset IO**: Asset readers/writers use async non-blocking interfaces. Note that because Rust doesn't yet support async traits, there is a bit of manual Boxing / Future boilerplate. This will hopefully be removed in the near future when Rust gets async traits. * **Pluggable Asset Readers and Writers**: Arbitrary asset source readers/writers are supported, both by the processor and the asset server. * **Better Asset Handles** * **Single Arc Tree**: Asset Handles now use a single arc tree that represents the lifetime of the asset. This makes their implementation simpler, more efficient, and allows us to cheaply attach metadata to handles. Ex: the AssetPath of a handle is now directly accessible on the handle itself! * **Const Typed Handles**: typed handles can be constructed in a const context. No more weird "const untyped converted to typed at runtime" patterns! * **Handles and Ids are Smaller / Faster To Hash / Compare**: Typed `Handle` is now much smaller in memory and `AssetId` is even smaller. * **Weak Handle Usage Reduction**: In general Handles are now considered to be "strong". Bevy features that previously used "weak `Handle`" have been ported to `AssetId`, which makes it statically clear that the features do not hold strong handles (while retaining strong type information). Currently Handle::Weak still exists, but it is very possible that we can remove that entirely. * **Efficient / Dense Asset Ids**: Assets now have efficient dense runtime asset ids, which means we can avoid expensive hash lookups. Assets are stored in Vecs instead of HashMaps. There are now typed and untyped ids, which means we no longer need to store dynamic type information in the ID for typed handles. "AssetPathId" (which was a nightmare from a performance and correctness standpoint) has been entirely removed in favor of dense ids (which are retrieved for a path on load) * **Direct Asset Loading, with Dependency Tracking**: Assets that are defined at runtime can still have their dependencies tracked by the Asset Server (ex: if you create a material at runtime, you can still wait for its textures to load). This is accomplished via the (currently optional) "asset dependency visitor" trait. This system can also be used to define a set of assets to load, then wait for those assets to load. * **Async folder loading**: Folder loading also uses this system and immediately returns a handle to the LoadedFolder asset, which means folder loading no longer blocks on directory traversals. * **Improved Loader Interface**: Loaders now have a specific "top level asset type", which makes returning the top-level asset simpler and statically typed. * **Basic Image Settings and Processing**: Image assets can now be processed into the gpu-friendly Basic Universal format. The ImageLoader now has a setting to define what format the image should be loaded as. Note that this is just a minimal MVP ... plenty of additional work to do here. To demo this, enable the `basis-universal` feature and turn on asset processing. * **Simpler Audio Play / AudioSink API**: Asset handle providers are cloneable, which means the Audio resource can mint its own handles. This means you can now do `let sink_handle = audio.play(music)` instead of `let sink_handle = audio_sinks.get_handle(audio.play(music))`. Note that this might still be replaced by https://github.com/bevyengine/bevy/pull/8424. **Removed Handle Casting From Engine Features**: Ex: FontAtlases no longer use casting between handle types ## Using The New Asset System ### Normal Unprocessed Asset Loading By default the `AssetPlugin` does not use processing. It behaves pretty much the same way as the old system. If you are defining a custom asset, first derive `Asset`: ```rust #[derive(Asset)] struct Thing { value: String, } ``` Initialize the asset: ```rust app.init_asset:() ``` Implement a new `AssetLoader` for it: ```rust #[derive(Default)] struct ThingLoader; #[derive(Serialize, Deserialize, Default)] pub struct ThingSettings { some_setting: bool, } impl AssetLoader for ThingLoader { type Asset = Thing; type Settings = ThingSettings; fn load<'a>( &'a self, reader: &'a mut Reader, settings: &'a ThingSettings, load_context: &'a mut LoadContext, ) -> BoxedFuture<'a, Result> { Box::pin(async move { let mut bytes = Vec::new(); reader.read_to_end(&mut bytes).await?; // convert bytes to value somehow Ok(Thing { value }) }) } fn extensions(&self) -> &[&str] { &["thing"] } } ``` Note that this interface will get much cleaner once Rust gets support for async traits. `Reader` is an async futures_io::AsyncRead. You can stream bytes as they come in or read them all into a `Vec`, depending on the context. You can use `let handle = load_context.load(path)` to kick off a dependency load, retrieve a handle, and register the dependency for the asset. Then just register the loader in your Bevy app: ```rust app.init_asset_loader::() ``` Now just add your `Thing` asset files into the `assets` folder and load them like this: ```rust fn system(asset_server: Res) { let handle = Handle = asset_server.load("cool.thing"); } ``` You can check load states directly via the asset server: ```rust if asset_server.load_state(&handle) == LoadState::Loaded { } ``` You can also listen for events: ```rust fn system(mut events: EventReader>, handle: Res) { for event in events.iter() { if event.is_loaded_with_dependencies(&handle) { } } } ``` Note the new `AssetEvent::LoadedWithDependencies`, which only fires when the asset is loaded _and_ all dependencies (and their dependencies) have loaded. Unlike the old asset system, for a given asset path all `Handle` values point to the same underlying Arc. This means Handles can cheaply hold more asset information, such as the AssetPath: ```rust // prints the AssetPath of the handle info!("{:?}", handle.path()) ``` ### Processed Assets Asset processing can be enabled via the `AssetPlugin`. When developing Bevy Apps with processed assets, do this: ```rust app.add_plugins(DefaultPlugins.set(AssetPlugin::processed_dev())) ``` This runs the `AssetProcessor` in the background with hot-reloading. It reads assets from the `assets` folder, processes them, and writes them to the `.imported_assets` folder. Asset loads in the Bevy App will wait for a processed version of the asset to become available. If an asset in the `assets` folder changes, it will be reprocessed and hot-reloaded in the Bevy App. When deploying processed Bevy apps, do this: ```rust app.add_plugins(DefaultPlugins.set(AssetPlugin::processed())) ``` This does not run the `AssetProcessor` in the background. It behaves like `AssetPlugin::unprocessed()`, but reads assets from `.imported_assets`. When the `AssetProcessor` is running, it will populate sibling `.meta` files for assets in the `assets` folder. Meta files for assets that do not have a processor configured look like this: ```rust ( meta_format_version: "1.0", asset: Load( loader: "bevy_render::texture::image_loader::ImageLoader", settings: ( format: FromExtension, ), ), ) ``` This is metadata for an image asset. For example, if you have `assets/my_sprite.png`, this could be the metadata stored at `assets/my_sprite.png.meta`. Meta files are totally optional. If no metadata exists, the default settings will be used. In short, this file says "load this asset with the ImageLoader and use the file extension to determine the image type". This type of meta file is supported in all AssetPlugin modes. If in `Unprocessed` mode, the asset (with the meta settings) will be loaded directly. If in `ProcessedDev` mode, the asset file will be copied directly to the `.imported_assets` folder. The meta will also be copied directly to the `.imported_assets` folder, but with one addition: ```rust ( meta_format_version: "1.0", processed_info: Some(( hash: 12415480888597742505, full_hash: 14344495437905856884, process_dependencies: [], )), asset: Load( loader: "bevy_render::texture::image_loader::ImageLoader", settings: ( format: FromExtension, ), ), ) ``` `processed_info` contains `hash` (a direct hash of the asset and meta bytes), `full_hash` (a hash of `hash` and the hashes of all `process_dependencies`), and `process_dependencies` (the `path` and `full_hash` of every process_dependency). A "process dependency" is an asset dependency that is _directly_ used when processing the asset. Images do not have process dependencies, so this is empty. When the processor is enabled, you can use the `Process` metadata config: ```rust ( meta_format_version: "1.0", asset: Process( processor: "bevy_asset::processor::process::LoadAndSave", settings: ( loader_settings: ( format: FromExtension, ), saver_settings: ( generate_mipmaps: true, ), ), ), ) ``` This configures the asset to use the `LoadAndSave` processor, which runs an AssetLoader and feeds the result into an AssetSaver (which saves the given Asset and defines a loader to load it with). (for terseness LoadAndSave will likely get a shorter/friendlier type name when [Stable Type Paths](#7184) lands). `LoadAndSave` is likely to be the most common processor type, but arbitrary processors are supported. `CompressedImageSaver` saves an `Image` in the Basis Universal format and configures the ImageLoader to load it as basis universal. The `AssetProcessor` will read this meta, run it through the LoadAndSave processor, and write the basis-universal version of the image to `.imported_assets`. The final metadata will look like this: ```rust ( meta_format_version: "1.0", processed_info: Some(( hash: 905599590923828066, full_hash: 9948823010183819117, process_dependencies: [], )), asset: Load( loader: "bevy_render::texture::image_loader::ImageLoader", settings: ( format: Format(Basis), ), ), ) ``` To try basis-universal processing out in Bevy examples, (for example `sprite.rs`), change `add_plugins(DefaultPlugins)` to `add_plugins(DefaultPlugins.set(AssetPlugin::processed_dev()))` and run with the `basis-universal` feature enabled: `cargo run --features=basis-universal --example sprite`. To create a custom processor, there are two main paths: 1. Use the `LoadAndSave` processor with an existing `AssetLoader`. Implement the `AssetSaver` trait, register the processor using `asset_processor.register_processor::>(image_saver.into())`. 2. Implement the `Process` trait directly and register it using: `asset_processor.register_processor(thing_processor)`. You can configure default processors for file extensions like this: ```rust asset_processor.set_default_processor::("thing") ``` There is one more metadata type to be aware of: ```rust ( meta_format_version: "1.0", asset: Ignore, ) ``` This will ignore the asset during processing / prevent it from being written to `.imported_assets`. The AssetProcessor stores a transaction log at `.imported_assets/log` and uses it to gracefully recover from unexpected stops. This means you can force-quit the processor (and Bevy Apps running the processor in parallel) at arbitrary times! `.imported_assets` is "local state". It should _not_ be checked into source control. It should also be considered "read only". In practice, you _can_ modify processed assets and processed metadata if you really need to test something. But those modifications will not be represented in the hashes of the assets, so the processed state will be "out of sync" with the source assets. The processor _will not_ fix this for you. Either revert the change after you have tested it, or delete the processed files so they can be re-populated. ## Open Questions There are a number of open questions to be discussed. We should decide if they need to be addressed in this PR and if so, how we will address them: ### Implied Dependencies vs Dependency Enumeration There are currently two ways to populate asset dependencies: * **Implied via AssetLoaders**: if an AssetLoader loads an asset (and retrieves a handle), a dependency is added to the list. * **Explicit via the optional Asset::visit_dependencies**: if `server.load_asset(my_asset)` is called, it will call `my_asset.visit_dependencies`, which will grab dependencies that have been manually defined for the asset via the Asset trait impl (which can be derived). This means that defining explicit dependencies is optional for "loaded assets". And the list of dependencies is always accurate because loaders can only produce Handles if they register dependencies. If an asset was loaded with an AssetLoader, it only uses the implied dependencies. If an asset was created at runtime and added with `asset_server.load_asset(MyAsset)`, it will use `Asset::visit_dependencies`. However this can create a behavior mismatch between loaded assets and equivalent "created at runtime" assets if `Assets::visit_dependencies` doesn't exactly match the dependencies produced by the AssetLoader. This behavior mismatch can be resolved by completely removing "implied loader dependencies" and requiring `Asset::visit_dependencies` to supply dependency data. But this creates two problems: * It makes defining loaded assets harder and more error prone: Devs must remember to manually annotate asset dependencies with `#[dependency]` when deriving `Asset`. For more complicated assets (such as scenes), the derive likely wouldn't be sufficient and a manual `visit_dependencies` impl would be required. * Removes the ability to immediately kick off dependency loads: When AssetLoaders retrieve a Handle, they also immediately kick off an asset load for the handle, which means it can start loading in parallel _before_ the asset finishes loading. For large assets, this could be significant. (although this could be mitigated for processed assets if we store dependencies in the processed meta file and load them ahead of time) ### Eager ProcessorDev Asset Loading I made a controversial call in the interest of fast startup times ("time to first pixel") for the "processor dev mode configuration". When initializing the AssetProcessor, current processed versions of unchanged assets are yielded immediately, even if their dependencies haven't been checked yet for reprocessing. This means that non-current-state-of-filesystem-but-previously-valid assets might be returned to the App first, then hot-reloaded if/when their dependencies change and the asset is reprocessed. Is this behavior desirable? There is largely one alternative: do not yield an asset from the processor to the app until all of its dependencies have been checked for changes. In some common cases (load dependency has not changed since last run) this will increase startup time. The main question is "by how much" and is that slower startup time worth it in the interest of only yielding assets that are true to the current state of the filesystem. Should this be configurable? I'm starting to think we should only yield an asset after its (historical) dependencies have been checked for changes + processed as necessary, but I'm curious what you all think. ### Paths Are Currently The Only Canonical ID / Do We Want Asset UUIDs? In this implementation AssetPaths are the only canonical asset identifier (just like the previous Bevy Asset system and Godot). Moving assets will result in re-scans (and currently reprocessing, although reprocessing can easily be avoided with some changes). Asset renames/moves will break code and assets that rely on specific paths, unless those paths are fixed up. Do we want / need "stable asset uuids"? Introducing them is very possible: 1. Generate a UUID and include it in .meta files 2. Support UUID in AssetPath 3. Generate "asset indices" which are loaded on startup and map UUIDs to paths. 4 (maybe). Consider only supporting UUIDs for processed assets so we can generate quick-to-load indices instead of scanning meta files. The main "pro" is that assets referencing UUIDs don't need to be migrated when a path changes. The main "con" is that UUIDs cannot be "lazily resolved" like paths. They need a full view of all assets to answer the question "does this UUID exist". Which means UUIDs require the AssetProcessor to fully finish startup scans before saying an asset doesnt exist. And they essentially require asset pre-processing to use in apps, because scanning all asset metadata files at runtime to resolve a UUID is not viable for medium-to-large apps. It really requires a pre-generated UUID index, which must be loaded before querying for assets. I personally think this should be investigated in a separate PR. Paths aren't going anywhere ... _everyone_ uses filesystems (and filesystem-like apis) to manage their asset source files. I consider them permanent canonical asset information. Additionally, they behave well for both processed and unprocessed asset modes. Given that Bevy is supporting both, this feels like the right canonical ID to start with. UUIDS (and maybe even other indexed-identifier types) can be added later as necessary. ### Folder / File Naming Conventions All asset processing config currently lives in the `.imported_assets` folder. The processor transaction log is in `.imported_assets/log`. Processed assets are added to `.imported_assets/Default`, which will make migrating to processed asset profiles (ex: a `.imported_assets/Mobile` profile) a non-breaking change. It also allows us to create top-level files like `.imported_assets/log` without it being interpreted as an asset. Meta files currently have a `.meta` suffix. Do we like these names and conventions? ### Should the `AssetPlugin::processed_dev` configuration enable `watch_for_changes` automatically? Currently it does (which I think makes sense), but it does make it the only configuration that enables watch_for_changes by default. ### Discuss on_loaded High Level Interface: This PR includes a very rough "proof of concept" `on_loaded` system adapter that uses the `LoadedWithDependencies` event in combination with `asset_server.load_asset` dependency tracking to support this pattern ```rust fn main() { App::new() .init_asset::() .add_systems(Update, on_loaded(create_array_texture)) .run(); } #[derive(Asset, Clone)] struct MyAssets { #[dependency] picture_of_my_cat: Handle, #[dependency] picture_of_my_other_cat: Handle, } impl FromWorld for ArrayTexture { fn from_world(world: &mut World) -> Self { picture_of_my_cat: server.load("meow.png"), picture_of_my_other_cat: server.load("meeeeeeeow.png"), } } fn spawn_cat(In(my_assets): In, mut commands: Commands) { commands.spawn(SpriteBundle { texture: my_assets.picture_of_my_cat.clone(), ..default() }); commands.spawn(SpriteBundle { texture: my_assets.picture_of_my_other_cat.clone(), ..default() }); } ``` The implementation is _very_ rough. And it is currently unsafe because `bevy_ecs` doesn't expose some internals to do this safely from inside `bevy_asset`. There are plenty of unanswered questions like: * "do we add a Loadable" derive? (effectively automate the FromWorld implementation above) * Should `MyAssets` even be an Asset? (largely implemented this way because it elegantly builds on `server.load_asset(MyAsset { .. })` dependency tracking). We should think hard about what our ideal API looks like (and if this is a pattern we want to support). Not necessarily something we need to solve in this PR. The current `on_loaded` impl should probably be removed from this PR before merging. ## Clarifying Questions ### What about Assets as Entities? This Bevy Asset V2 proposal implementation initially stored Assets as ECS Entities. Instead of `AssetId` + the `Assets` resource it used `Entity` as the asset id and Asset values were just ECS components. There are plenty of compelling reasons to do this: 1. Easier to inline assets in Bevy Scenes (as they are "just" normal entities + components) 2. More flexible queries: use the power of the ECS to filter assets (ex: `Query>`). 3. Extensible. Users can add arbitrary component data to assets. 4. Things like "component visualization tools" work out of the box to visualize asset data. However Assets as Entities has a ton of caveats right now: * We need to be able to allocate entity ids without a direct World reference (aka rework id allocator in Entities ... i worked around this in my prototypes by just pre allocating big chunks of entities) * We want asset change events in addition to ECS change tracking ... how do we populate them when mutations can come from anywhere? Do we use Changed queries? This would require iterating over the change data for all assets every frame. Is this acceptable or should we implement a new "event based" component change detection option? * Reconciling manually created assets with asset-system managed assets has some nuance (ex: are they "loaded" / do they also have that component metadata?) * "how do we handle "static" / default entity handles" (ties in to the Entity Indices discussion: https://github.com/bevyengine/bevy/discussions/8319). This is necessary for things like "built in" assets and default handles in things like SpriteBundle. * Storing asset information as a component makes it easy to "invalidate" asset state by removing the component (or forcing modifications). Ideally we have ways to lock this down (some combination of Rust type privacy and ECS validation) In practice, how we store and identify assets is a reasonably superficial change (porting off of Assets as Entities and implementing dedicated storage + ids took less than a day). So once we sort out the remaining challenges the flip should be straightforward. Additionally, I do still have "Assets as Entities" in my commit history, so we can reuse that work. I personally think "assets as entities" is a good endgame, but it also doesn't provide _significant_ value at the moment and it certainly isn't ready yet with the current state of things. ### Why not Distill? [Distill](https://github.com/amethyst/distill) is a high quality fully featured asset system built in Rust. It is very natural to ask "why not just use Distill?". It is also worth calling out that for awhile, [we planned on adopting Distill / I signed off on it](https://github.com/bevyengine/bevy/issues/708). However I think Bevy has a number of constraints that make Distill adoption suboptimal: * **Architectural Simplicity:** * Distill's processor requires an in-memory database (lmdb) and RPC networked API (using Cap'n Proto). Each of these introduces API complexity that increases maintenance burden and "code grokability". Ignoring tests, documentation, and examples, Distill has 24,237 lines of Rust code (including generated code for RPC + database interactions). If you ignore generated code, it has 11,499 lines. * Bevy builds the AssetProcessor and AssetServer using pluggable AssetReader/AssetWriter Rust traits with simple io interfaces. They do not necessitate databases or RPC interfaces (although Readers/Writers could use them if that is desired). Bevy Asset V2 (at the time of writing this PR) is 5,384 lines of Rust code (ignoring tests, documentation, and examples). Grain of salt: Distill does have more features currently (ex: Asset Packing, GUIDS, remote-out-of-process asset processor). I do plan to implement these features in Bevy Asset V2 and I personally highly doubt they will meaningfully close the 6115 lines-of-code gap. * This complexity gap (which while illustrated by lines of code, is much bigger than just that) is noteworthy to me. Bevy should be hackable and there are pillars of Distill that are very hard to understand and extend. This is a matter of opinion (and Bevy Asset V2 also has complicated areas), but I think Bevy Asset V2 is much more approachable for the average developer. * Necessary disclaimer: counting lines of code is an extremely rough complexity metric. Read the code and form your own opinions. * **Optional Asset Processing:** Not all Bevy Apps (or Bevy App developers) need / want asset preprocessing. Processing increases the complexity of the development environment by introducing things like meta files, imported asset storage, running processors in the background, waiting for processing to finish, etc. Distill _requires_ preprocessing to work. With Bevy Asset V2 processing is fully opt-in. The AssetServer isn't directly aware of asset processors at all. AssetLoaders only care about converting bytes to runtime Assets ... they don't know or care if the bytes were pre-processed or not. Processing is "elegantly" (forgive my self-congratulatory phrasing) layered on top and builds on the existing Asset system primitives. * **Direct Filesystem Access to Processed Asset State:** Distill stores processed assets in a database. This makes debugging / inspecting the processed outputs harder (either requires special tooling to query the database or they need to be "deployed" to be inspected). Bevy Asset V2, on the other hand, stores processed assets in the filesystem (by default ... this is configurable). This makes interacting with the processed state more natural. Note that both Godot and Unity's new asset system store processed assets in the filesystem. * **Portability**: Because Distill's processor uses lmdb and RPC networking, it cannot be run on certain platforms (ex: lmdb is a non-rust dependency that cannot run on the web, some platforms don't support running network servers). Bevy should be able to process assets everywhere (ex: run the Bevy Editor on the web, compile + process shaders on mobile, etc). Distill does partially mitigate this problem by supporting "streaming" assets via the RPC protocol, but this is not a full solve from my perspective. And Bevy Asset V2 can (in theory) also stream assets (without requiring RPC, although this isn't implemented yet) Note that I _do_ still think Distill would be a solid asset system for Bevy. But I think the approach in this PR is a better solve for Bevy's specific "asset system requirements". ### Doesn't async-fs just shim requests to "sync" `std::fs`? What is the point? "True async file io" has limited / spotty platform support. async-fs (and the rust async ecosystem generally ... ex Tokio) currently use async wrappers over std::fs that offload blocking requests to separate threads. This may feel unsatisfying, but it _does_ still provide value because it prevents our task pools from blocking on file system operations (which would prevent progress when there are many tasks to do, but all threads in a pool are currently blocking on file system ops). Additionally, using async APIs for our AssetReaders and AssetWriters also provides value because we can later add support for "true async file io" for platforms that support it. _And_ we can implement other "true async io" asset backends (such as networked asset io). ## Draft TODO - [x] Fill in missing filesystem event APIs: file removed event (which is expressed as dangling RenameFrom events in some cases), file/folder renamed event - [x] Assets without loaders are not moved to the processed folder. This breaks things like referenced `.bin` files for GLTFs. This should be configurable per-non-asset-type. - [x] Initial implementation of Reflect and FromReflect for Handle. The "deserialization" parity bar is low here as this only worked with static UUIDs in the old impl ... this is a non-trivial problem. Either we add a Handle::AssetPath variant that gets "upgraded" to a strong handle on scene load or we use a separate AssetRef type for Bevy scenes (which is converted to a runtime Handle on load). This deserves its own discussion in a different pr. - [x] Populate read_asset_bytes hash when run by the processor (a bit of a special case .. when run by the processor the processed meta will contain the hash so we don't need to compute it on the spot, but we don't want/need to read the meta when run by the main AssetServer) - [x] Delay hot reloading: currently filesystem events are handled immediately, which creates timing issues in some cases. For example hot reloading images can sometimes break because the image isn't finished writing. We should add a delay, likely similar to the [implementation in this PR](https://github.com/bevyengine/bevy/pull/8503). - [x] Port old platform-specific AssetIo implementations to the new AssetReader interface (currently missing Android and web) - [x] Resolve on_loaded unsafety (either by removing the API entirely or removing the unsafe) - [x] Runtime loader setting overrides - [x] Remove remaining unwraps that should be error-handled. There are number of TODOs here - [x] Pretty AssetPath Display impl - [x] Document more APIs - [x] Resolve spurious "reloading because it has changed" events (to repro run load_gltf with `processed_dev()`) - [x] load_dependency hot reloading currently only works for processed assets. If processing is disabled, load_dependency changes are not hot reloaded. - [x] Replace AssetInfo dependency load/fail counters with `loading_dependencies: HashSet` to prevent reloads from (potentially) breaking counters. Storing this will also enable "dependency reloaded" events (see [Next Steps](#next-steps)) - [x] Re-add filesystem watcher cargo feature gate (currently it is not optional) - [ ] Migration Guide - [ ] Changelog ## Followup TODO - [ ] Replace "eager unchanged processed asset loading" behavior with "don't returned unchanged processed asset until dependencies have been checked". - [ ] Add true `Ignore` AssetAction that does not copy the asset to the imported_assets folder. - [ ] Finish "live asset unloading" (ex: free up CPU asset memory after uploading an image to the GPU), rethink RenderAssets, and port renderer features. The `Assets` collection uses `Option` for asset storage to support its removal. (1) the Option might not actually be necessary ... might be able to just remove from the collection entirely (2) need to finalize removal apis - [ ] Try replacing the "channel based" asset id recycling with something a bit more efficient (ex: we might be able to use raw atomic ints with some cleverness) - [ ] Consider adding UUIDs to processed assets (scoped just to helping identify moved assets ... not exposed to load queries ... see [Next Steps](#next-steps)) - [ ] Store "last modified" source asset and meta timestamps in processed meta files to enable skipping expensive hashing when the file wasn't changed - [ ] Fix "slow loop" handle drop fix - [ ] Migrate to TypeName - [x] Handle "loader preregistration". See #9429 ## Next Steps * **Configurable per-type defaults for AssetMeta**: It should be possible to add configuration like "all png image meta should default to using nearest sampling" (currently this hard-coded per-loader/processor Settings::default() impls). Also see the "Folder Meta" bullet point. * **Avoid Reprocessing on Asset Renames / Moves**: See the "canonical asset ids" discussion in [Open Questions](#open-questions) and the relevant bullet point in [Draft TODO](#draft-todo). Even without canonical ids, folder renames could avoid reprocessing in some cases. * **Multiple Asset Sources**: Expand AssetPath to support "asset source names" and support multiple AssetReaders in the asset server (ex: `webserver://some_path/image.png` backed by an Http webserver AssetReader). The "default" asset reader would use normal `some_path/image.png` paths. Ideally this works in combination with multiple AssetWatchers for hot-reloading * **Stable Type Names**: this pr removes the TypeUuid requirement from assets in favor of `std::any::type_name`. This makes defining assets easier (no need to generate a new uuid / use weird proc macro syntax). It also makes reading meta files easier (because things have "friendly names"). We also use type names for components in scene files. If they are good enough for components, they are good enough for assets. And consistency across Bevy pillars is desirable. However, `std::any::type_name` is not guaranteed to be stable (although in practice it is). We've developed a [stable type path](https://github.com/bevyengine/bevy/pull/7184) to resolve this, which should be adopted when it is ready. * **Command Line Interface**: It should be possible to run the asset processor in a separate process from the command line. This will also require building a network-server-backed AssetReader to communicate between the app and the processor. We've been planning to build a "bevy cli" for awhile. This seems like a good excuse to build it. * **Asset Packing**: This is largely an additive feature, so it made sense to me to punt this until we've laid the foundations in this PR. * **Per-Platform Processed Assets**: It should be possible to generate assets for multiple platforms by supporting multiple "processor profiles" per asset (ex: compress with format X on PC and Y on iOS). I think there should probably be arbitrary "profiles" (which can be separate from actual platforms), which are then assigned to a given platform when generating the final asset distribution for that platform. Ex: maybe devs want a "Mobile" profile that is shared between iOS and Android. Or a "LowEnd" profile shared between web and mobile. * **Versioning and Migrations**: Assets, Loaders, Savers, and Processors need to have versions to determine if their schema is valid. If an asset / loader version is incompatible with the current version expected at runtime, the processor should be able to migrate them. I think we should try using Bevy Reflect for this, as it would allow us to load the old version as a dynamic Reflect type without actually having the old Rust type. It would also allow us to define "patches" to migrate between versions (Bevy Reflect devs are currently working on patching). The `.meta` file already has its own format version. Migrating that to new versions should also be possible. * **Real Copy-on-write AssetPaths**: Rust's actual Cow (clone-on-write type) currently used by AssetPath can still result in String clones that aren't actually necessary (cloning an Owned Cow clones the contents). Bevy's asset system requires cloning AssetPaths in a number of places, which result in actual clones of the internal Strings. This is not efficient. AssetPath internals should be reworked to exhibit truer cow-like-behavior that reduces String clones to the absolute minimum. * **Consider processor-less processing**: In theory the AssetServer could run processors "inline" even if the background AssetProcessor is disabled. If we decide this is actually desirable, we could add this. But I don't think its a priority in the short or medium term. * **Pre-emptive dependency loading**: We could encode dependencies in processed meta files, which could then be used by the Asset Server to kick of dependency loads as early as possible (prior to starting the actual asset load). Is this desirable? How much time would this save in practice? * **Optimize Processor With UntypedAssetIds**: The processor exclusively uses AssetPath to identify assets currently. It might be possible to swap these out for UntypedAssetIds in some places, which are smaller / cheaper to hash and compare. * **One to Many Asset Processing**: An asset source file that produces many assets currently must be processed into a single "processed" asset source. If labeled assets can be written separately they can each have their own configured savers _and_ they could be loaded more granularly. Definitely worth exploring! * **Automatically Track "Runtime-only" Asset Dependencies**: Right now, tracking "created at runtime" asset dependencies requires adding them via `asset_server.load_asset(StandardMaterial::default())`. I think with some cleverness we could also do this for `materials.add(StandardMaterial::default())`, making tracking work "everywhere". There are challenges here relating to change detection / ensuring the server is made aware of dependency changes. This could be expensive in some cases. * **"Dependency Changed" events**: Some assets have runtime artifacts that need to be re-generated when one of their dependencies change (ex: regenerate a material's bind group when a Texture needs to change). We are generating the dependency graph so we can definitely produce these events. Buuuuut generating these events will have a cost / they could be high frequency for some assets, so we might want this to be opt-in for specific cases. * **Investigate Storing More Information In Handles**: Handles can now store arbitrary information, which makes it cheaper and easier to access. How much should we move into them? Canonical asset load states (via atomics)? (`handle.is_loaded()` would be very cool). Should we store the entire asset and remove the `Assets` collection? (`Arc>>`?) * **Support processing and loading files without extensions**: This is a pretty arbitrary restriction and could be supported with very minimal changes. * **Folder Meta**: It would be nice if we could define per folder processor configuration defaults (likely in a `.meta` or `.folder_meta` file). Things like "default to linear filtering for all Images in this folder". * **Replace async_broadcast with event-listener?** This might be approximately drop-in for some uses and it feels more light weight * **Support Running the AssetProcessor on the Web**: Most of the hard work is done here, but there are some easy straggling TODOs (make the transaction log an interface instead of a direct file writer so we can write a web storage backend, implement an AssetReader/AssetWriter that reads/writes to something like LocalStorage). * **Consider identifying and preventing circular dependencies**: This is especially important for "processor dependencies", as processing will silently never finish in these cases. * **Built-in/Inlined Asset Hot Reloading**: This PR regresses "built-in/inlined" asset hot reloading (previously provided by the DebugAssetServer). I'm intentionally punting this because I think it can be cleanly implemented with "multiple asset sources" by registering a "debug asset source" (ex: `debug://bevy_pbr/src/render/pbr.wgsl` asset paths) in combination with an AssetWatcher for that asset source and support for "manually loading pats with asset bytes instead of AssetReaders". The old DebugAssetServer was quite nasty and I'd love to avoid that hackery going forward. * **Investigate ways to remove double-parsing meta files**: Parsing meta files currently involves parsing once with "minimal" versions of the meta file to extract the type name of the loader/processor config, then parsing again to parse the "full" meta. This is suboptimal. We should be able to define custom deserializers that (1) assume the loader/processor type name comes first (2) dynamically looks up the loader/processor registrations to deserialize settings in-line (similar to components in the bevy scene format). Another alternative: deserialize as dynamic Reflect objects and then convert. * **More runtime loading configuration**: Support using the Handle type as a hint to select an asset loader (instead of relying on AssetPath extensions) * **More high level Processor trait implementations**: For example, it might be worth adding support for arbitrary chains of "asset transforms" that modify an in-memory asset representation between loading and saving. (ex: load a Mesh, run a `subdivide_mesh` transform, followed by a `flip_normals` transform, then save the mesh to an efficient compressed format). * **Bevy Scene Handle Deserialization**: (see the relevant [Draft TODO item](#draft-todo) for context) * **Explore High Level Load Interfaces**: See [this discussion](#discuss-on_loaded-high-level-interface) for one prototype. * **Asset Streaming**: It would be great if we could stream Assets (ex: stream a long video file piece by piece) * **ID Exchanging**: In this PR Asset Handles/AssetIds are bigger than they need to be because they have a Uuid enum variant. If we implement an "id exchanging" system that trades Uuids for "efficient runtime ids", we can cut down on the size of AssetIds, making them more efficient. This has some open design questions, such as how to spawn entities with "default" handle values (as these wouldn't have access to the exchange api in the current system). * **Asset Path Fixup Tooling**: Assets that inline asset paths inside them will break when an asset moves. The asset system provides the functionality to detect when paths break. We should build a framework that enables formats to define "path migrations". This is especially important for scene files. For editor-generated files, we should also consider using UUIDs (see other bullet point) to avoid the need to migrate in these cases. --------- Co-authored-by: BeastLe9enD Co-authored-by: Mike Co-authored-by: Nicola Papale --- .gitignore | 4 + Cargo.toml | 30 +- crates/bevy_animation/src/lib.rs | 9 +- crates/bevy_asset/Cargo.toml | 36 +- crates/bevy_asset/macros/Cargo.toml | 19 + crates/bevy_asset/macros/src/lib.rs | 76 + crates/bevy_asset/src/asset_server.rs | 1049 -------------- crates/bevy_asset/src/assets.rs | 967 ++++++------- crates/bevy_asset/src/debug_asset_server.rs | 143 -- .../asset_count_diagnostics_plugin.rs | 60 - crates/bevy_asset/src/diagnostic/mod.rs | 4 - crates/bevy_asset/src/event.rs | 77 + crates/bevy_asset/src/filesystem_watcher.rs | 46 - crates/bevy_asset/src/folder.rs | 12 + crates/bevy_asset/src/handle.rs | 639 ++++----- crates/bevy_asset/src/id.rs | 377 +++++ crates/bevy_asset/src/info.rs | 69 - crates/bevy_asset/src/io/android.rs | 82 ++ crates/bevy_asset/src/io/android_asset_io.rs | 81 -- crates/bevy_asset/src/io/file/file_watcher.rs | 182 +++ crates/bevy_asset/src/io/file/mod.rs | 325 +++++ crates/bevy_asset/src/io/file_asset_io.rs | 227 --- crates/bevy_asset/src/io/gated.rs | 107 ++ crates/bevy_asset/src/io/memory.rs | 288 ++++ crates/bevy_asset/src/io/metadata.rs | 85 -- crates/bevy_asset/src/io/mod.rs | 326 ++++- crates/bevy_asset/src/io/processor_gated.rs | 163 +++ crates/bevy_asset/src/io/provider.rs | 190 +++ crates/bevy_asset/src/io/wasm.rs | 110 ++ crates/bevy_asset/src/io/wasm_asset_io.rs | 85 -- crates/bevy_asset/src/lib.rs | 1237 ++++++++++++++-- crates/bevy_asset/src/loader.rs | 689 ++++++--- crates/bevy_asset/src/meta.rs | 250 ++++ crates/bevy_asset/src/path.rs | 197 ++- crates/bevy_asset/src/processor/log.rs | 194 +++ crates/bevy_asset/src/processor/mod.rs | 1269 +++++++++++++++++ crates/bevy_asset/src/processor/process.rs | 260 ++++ crates/bevy_asset/src/reflect.rs | 105 +- crates/bevy_asset/src/saver.rs | 110 ++ crates/bevy_asset/src/server/info.rs | 603 ++++++++ crates/bevy_asset/src/server/mod.rs | 913 ++++++++++++ crates/bevy_audio/Cargo.toml | 1 - crates/bevy_audio/src/audio_source.rs | 36 +- crates/bevy_audio/src/lib.rs | 4 +- crates/bevy_audio/src/pitch.rs | 6 +- crates/bevy_core/src/lib.rs | 5 +- crates/bevy_core_pipeline/src/blit/mod.rs | 8 +- .../src/bloom/downsampling_pipeline.rs | 2 +- crates/bevy_core_pipeline/src/bloom/mod.rs | 6 +- .../src/bloom/upsampling_pipeline.rs | 2 +- .../src/contrast_adaptive_sharpening/mod.rs | 10 +- .../src/fullscreen_vertex_shader/mod.rs | 8 +- crates/bevy_core_pipeline/src/fxaa/mod.rs | 9 +- crates/bevy_core_pipeline/src/skybox/mod.rs | 10 +- crates/bevy_core_pipeline/src/taa/mod.rs | 9 +- .../bevy_core_pipeline/src/tonemapping/mod.rs | 13 +- crates/bevy_gizmos/src/lib.rs | 40 +- crates/bevy_gizmos/src/pipeline_2d.rs | 4 +- crates/bevy_gizmos/src/pipeline_3d.rs | 4 +- crates/bevy_gltf/Cargo.toml | 1 - crates/bevy_gltf/src/lib.rs | 31 +- crates/bevy_gltf/src/loader.rs | 417 +++--- crates/bevy_internal/Cargo.toml | 9 +- crates/bevy_internal/src/default_plugins.rs | 6 - crates/bevy_pbr/src/environment_map/mod.rs | 8 +- crates/bevy_pbr/src/lib.rs | 55 +- crates/bevy_pbr/src/material.rs | 57 +- crates/bevy_pbr/src/pbr_material.rs | 20 +- crates/bevy_pbr/src/prepass/mod.rs | 19 +- crates/bevy_pbr/src/render/fog.rs | 6 +- crates/bevy_pbr/src/render/light.rs | 2 +- crates/bevy_pbr/src/render/mesh.rs | 45 +- crates/bevy_pbr/src/ssao/mod.rs | 22 +- crates/bevy_pbr/src/wireframe.rs | 9 +- crates/bevy_reflect/src/impls/std.rs | 1 + crates/bevy_render/Cargo.toml | 1 - crates/bevy_render/src/camera/camera.rs | 12 +- crates/bevy_render/src/globals.rs | 7 +- crates/bevy_render/src/lib.rs | 23 +- crates/bevy_render/src/mesh/mesh/mod.rs | 9 +- crates/bevy_render/src/mesh/mesh/skinning.rs | 7 +- crates/bevy_render/src/mesh/mod.rs | 6 +- crates/bevy_render/src/render_asset.rs | 76 +- .../src/render_resource/pipeline_cache.rs | 98 +- .../bevy_render/src/render_resource/shader.rs | 54 +- crates/bevy_render/src/texture/basis.rs | 3 +- .../src/texture/compressed_image_saver.rs | 56 + .../src/texture/exr_texture_loader.rs | 26 +- .../src/texture/hdr_texture_loader.rs | 24 +- crates/bevy_render/src/texture/image.rs | 17 +- ...mage_texture_loader.rs => image_loader.rs} | 60 +- .../src/texture/image_texture_conversion.rs | 8 +- crates/bevy_render/src/texture/mod.rs | 29 +- crates/bevy_render/src/view/mod.rs | 7 +- .../bevy_render/src/view/window/screenshot.rs | 10 +- crates/bevy_scene/Cargo.toml | 1 - crates/bevy_scene/src/dynamic_scene.rs | 10 +- crates/bevy_scene/src/lib.rs | 8 +- crates/bevy_scene/src/scene.rs | 9 +- crates/bevy_scene/src/scene_loader.rs | 25 +- crates/bevy_scene/src/scene_spawner.rs | 109 +- crates/bevy_scene/src/serde.rs | 1 - crates/bevy_sprite/src/bundle.rs | 17 +- crates/bevy_sprite/src/lib.rs | 8 +- .../bevy_sprite/src/mesh2d/color_material.rs | 32 +- crates/bevy_sprite/src/mesh2d/material.rs | 54 +- crates/bevy_sprite/src/mesh2d/mesh.rs | 29 +- crates/bevy_sprite/src/render/mod.rs | 53 +- crates/bevy_sprite/src/texture_atlas.rs | 14 +- .../bevy_sprite/src/texture_atlas_builder.rs | 18 +- crates/bevy_text/Cargo.toml | 1 - crates/bevy_text/src/font.rs | 6 +- crates/bevy_text/src/font_atlas_set.rs | 31 +- crates/bevy_text/src/font_loader.rs | 20 +- crates/bevy_text/src/glyph_brush.rs | 29 +- crates/bevy_text/src/lib.rs | 28 +- crates/bevy_text/src/pipeline.rs | 26 +- crates/bevy_text/src/text.rs | 4 +- crates/bevy_text/src/text2d.rs | 13 +- crates/bevy_ui/src/lib.rs | 26 +- crates/bevy_ui/src/render/mod.rs | 59 +- crates/bevy_ui/src/render/pipeline.rs | 4 +- crates/bevy_ui/src/render/render_pass.rs | 10 +- crates/bevy_ui/src/ui_node.rs | 17 +- crates/bevy_ui/src/widget/text.rs | 12 +- docs/cargo_features.md | 3 +- examples/2d/custom_gltf_vertex_attribute.rs | 19 +- examples/2d/mesh2d_manual.rs | 14 +- examples/2d/texture_atlas.rs | 41 +- examples/3d/lines.rs | 5 +- examples/3d/load_gltf.rs | 3 +- examples/3d/pbr.rs | 4 +- examples/3d/skybox.rs | 7 +- examples/3d/tonemapping.rs | 17 +- examples/README.md | 3 +- examples/asset/asset_loading.rs | 18 +- examples/asset/custom_asset.rs | 26 +- examples/asset/custom_asset_io.rs | 92 -- examples/asset/custom_asset_reader.rs | 88 ++ examples/asset/hot_asset_reloading.rs | 8 +- examples/asset/processing/assets/a.cool.ron | 8 + .../asset/processing/assets/a.cool.ron.meta | 12 + examples/asset/processing/assets/d.cool.ron | 8 + .../asset/processing/assets/d.cool.ron.meta | 12 + .../asset/processing/assets/foo/b.cool.ron | 5 + .../processing/assets/foo/b.cool.ron.meta | 12 + .../asset/processing/assets/foo/c.cool.ron | 5 + .../processing/assets/foo/c.cool.ron.meta | 12 + examples/asset/processing/processing.rs | 210 +++ examples/audio/decodable.rs | 5 +- examples/scene/scene.rs | 9 +- examples/shader/animate_shader.rs | 7 +- examples/shader/array_texture.rs | 7 +- .../shader/compute_shader_game_of_life.rs | 2 +- examples/shader/custom_vertex_attribute.rs | 5 +- examples/shader/fallback_image.rs | 5 +- examples/shader/post_processing.rs | 8 +- examples/shader/shader_defs.rs | 5 +- examples/shader/shader_material.rs | 5 +- examples/shader/shader_material_glsl.rs | 5 +- .../shader_material_screenspace_texture.rs | 5 +- examples/shader/shader_prepass.rs | 8 +- examples/shader/texture_binding_array.rs | 7 +- .../tools/scene_viewer/animation_plugin.rs | 3 +- examples/tools/scene_viewer/main.rs | 14 +- .../tools/scene_viewer/scene_viewer_plugin.rs | 2 +- examples/ui/font_atlas_debug.rs | 6 +- tools/publish.sh | 1 + 168 files changed, 9961 insertions(+), 4567 deletions(-) create mode 100644 crates/bevy_asset/macros/Cargo.toml create mode 100644 crates/bevy_asset/macros/src/lib.rs delete mode 100644 crates/bevy_asset/src/asset_server.rs delete mode 100644 crates/bevy_asset/src/debug_asset_server.rs delete mode 100644 crates/bevy_asset/src/diagnostic/asset_count_diagnostics_plugin.rs delete mode 100644 crates/bevy_asset/src/diagnostic/mod.rs create mode 100644 crates/bevy_asset/src/event.rs delete mode 100644 crates/bevy_asset/src/filesystem_watcher.rs create mode 100644 crates/bevy_asset/src/folder.rs create mode 100644 crates/bevy_asset/src/id.rs delete mode 100644 crates/bevy_asset/src/info.rs create mode 100644 crates/bevy_asset/src/io/android.rs delete mode 100644 crates/bevy_asset/src/io/android_asset_io.rs create mode 100644 crates/bevy_asset/src/io/file/file_watcher.rs create mode 100644 crates/bevy_asset/src/io/file/mod.rs delete mode 100644 crates/bevy_asset/src/io/file_asset_io.rs create mode 100644 crates/bevy_asset/src/io/gated.rs create mode 100644 crates/bevy_asset/src/io/memory.rs delete mode 100644 crates/bevy_asset/src/io/metadata.rs create mode 100644 crates/bevy_asset/src/io/processor_gated.rs create mode 100644 crates/bevy_asset/src/io/provider.rs create mode 100644 crates/bevy_asset/src/io/wasm.rs delete mode 100644 crates/bevy_asset/src/io/wasm_asset_io.rs create mode 100644 crates/bevy_asset/src/meta.rs create mode 100644 crates/bevy_asset/src/processor/log.rs create mode 100644 crates/bevy_asset/src/processor/mod.rs create mode 100644 crates/bevy_asset/src/processor/process.rs create mode 100644 crates/bevy_asset/src/saver.rs create mode 100644 crates/bevy_asset/src/server/info.rs create mode 100644 crates/bevy_asset/src/server/mod.rs create mode 100644 crates/bevy_render/src/texture/compressed_image_saver.rs rename crates/bevy_render/src/texture/{image_texture_loader.rs => image_loader.rs} (62%) delete mode 100644 examples/asset/custom_asset_io.rs create mode 100644 examples/asset/custom_asset_reader.rs create mode 100644 examples/asset/processing/assets/a.cool.ron create mode 100644 examples/asset/processing/assets/a.cool.ron.meta create mode 100644 examples/asset/processing/assets/d.cool.ron create mode 100644 examples/asset/processing/assets/d.cool.ron.meta create mode 100644 examples/asset/processing/assets/foo/b.cool.ron create mode 100644 examples/asset/processing/assets/foo/b.cool.ron.meta create mode 100644 examples/asset/processing/assets/foo/c.cool.ron create mode 100644 examples/asset/processing/assets/foo/c.cool.ron.meta create mode 100644 examples/asset/processing/processing.rs diff --git a/.gitignore b/.gitignore index 0dc1de0df93cf..c338b9f8fac69 100644 --- a/.gitignore +++ b/.gitignore @@ -12,3 +12,7 @@ dxil.dll # Generated by "examples/scene/scene.rs" assets/scenes/load_scene_example-new.scn.ron + +assets/**/*.meta +crates/bevy_asset/imported_assets +imported_assets diff --git a/Cargo.toml b/Cargo.toml index 9030800ffd39e..b0f2d6f610912 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -51,7 +51,6 @@ default = [ "zstd", "vorbis", "x11", - "filesystem_watcher", "bevy_gizmos", "android_shared_stdcxx", "tonemapping_luts", @@ -194,9 +193,6 @@ symphonia-vorbis = ["bevy_internal/symphonia-vorbis"] # WAV audio format support (through symphonia) symphonia-wav = ["bevy_internal/symphonia-wav"] -# Enable watching file system for asset hot reload -filesystem_watcher = ["bevy_internal/filesystem_watcher"] - # Enable serialization support through serde serialize = ["bevy_internal/serialize"] @@ -215,9 +211,6 @@ subpixel_glyph_atlas = ["bevy_internal/subpixel_glyph_atlas"] # Enable systems that allow for automated testing on CI bevy_ci_testing = ["bevy_internal/bevy_ci_testing"] -# Enable the "debug asset server" for hot reloading internal assets -debug_asset_server = ["bevy_internal/debug_asset_server"] - # Enable animation support, and glTF animation loading animation = ["bevy_internal/animation", "bevy_animation"] @@ -248,6 +241,9 @@ shader_format_spirv = ["bevy_internal/shader_format_spirv"] # Enable some limitations to be able to use WebGL2. If not enabled, it will default to WebGPU in Wasm webgl2 = ["bevy_internal/webgl"] +# Enables watching the filesystem for Bevy Asset hot-reloading +filesystem_watcher = ["bevy_internal/filesystem_watcher"] + [dependencies] bevy_dylib = { path = "crates/bevy_dylib", version = "0.12.0-dev", default-features = false, optional = true } bevy_internal = { path = "crates/bevy_internal", version = "0.12.0-dev", default-features = false } @@ -1022,13 +1018,13 @@ category = "Assets" wasm = true [[example]] -name = "custom_asset_io" -path = "examples/asset/custom_asset_io.rs" +name = "custom_asset_reader" +path = "examples/asset/custom_asset_reader.rs" doc-scrape-examples = true -[package.metadata.example.custom_asset_io] +[package.metadata.example.custom_asset_reader] name = "Custom Asset IO" -description = "Implements a custom asset io loader" +description = "Implements a custom AssetReader" category = "Assets" wasm = true @@ -1043,6 +1039,18 @@ description = "Demonstrates automatic reloading of assets when modified on disk" category = "Assets" wasm = true +[[example]] +name = "asset_processing" +path = "examples/asset/processing/processing.rs" +doc-scrape-examples = true +required-features = ["filesystem_watcher"] + +[package.metadata.example.asset_processing] +name = "Asset Processing" +description = "Demonstrates how to process and load custom assets" +category = "Assets" +wasm = false + # Async Tasks [[example]] name = "async_compute" diff --git a/crates/bevy_animation/src/lib.rs b/crates/bevy_animation/src/lib.rs index 5e2b535c800af..d108782119cc5 100644 --- a/crates/bevy_animation/src/lib.rs +++ b/crates/bevy_animation/src/lib.rs @@ -7,12 +7,12 @@ use std::ops::Deref; use std::time::Duration; use bevy_app::{App, Plugin, PostUpdate}; -use bevy_asset::{AddAsset, Assets, Handle}; +use bevy_asset::{Asset, AssetApp, Assets, Handle}; use bevy_core::Name; use bevy_ecs::prelude::*; use bevy_hierarchy::{Children, Parent}; use bevy_math::{Quat, Vec3}; -use bevy_reflect::{Reflect, TypeUuid}; +use bevy_reflect::Reflect; use bevy_render::mesh::morph::MorphWeights; use bevy_time::Time; use bevy_transform::{prelude::Transform, TransformSystem}; @@ -65,8 +65,7 @@ pub struct EntityPath { } /// A list of [`VariableCurve`], and the [`EntityPath`] to which they apply. -#[derive(Reflect, Clone, TypeUuid, Debug, Default)] -#[uuid = "d81b7179-0448-4eb0-89fe-c067222725bf"] +#[derive(Asset, Reflect, Clone, Debug, Default)] pub struct AnimationClip { curves: Vec>, paths: HashMap, @@ -734,7 +733,7 @@ pub struct AnimationPlugin; impl Plugin for AnimationPlugin { fn build(&self, app: &mut App) { - app.add_asset::() + app.init_asset::() .register_asset_reflect::() .register_type::() .add_systems( diff --git a/crates/bevy_asset/Cargo.toml b/crates/bevy_asset/Cargo.toml index 6f7d0886f3489..05174c810cdf7 100644 --- a/crates/bevy_asset/Cargo.toml +++ b/crates/bevy_asset/Cargo.toml @@ -8,31 +8,35 @@ repository = "https://github.com/bevyengine/bevy" license = "MIT OR Apache-2.0" keywords = ["bevy"] +# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html + [features] -default = [] -filesystem_watcher = ["notify"] -debug_asset_server = ["filesystem_watcher"] +filesystem_watcher = ["notify-debouncer-full"] +multi-threaded = ["bevy_tasks/multi-threaded"] [dependencies] -# bevy bevy_app = { path = "../bevy_app", version = "0.12.0-dev" } -bevy_diagnostic = { path = "../bevy_diagnostic", version = "0.12.0-dev" } +bevy_asset_macros = { path = "macros", version = "0.12.0-dev" } bevy_ecs = { path = "../bevy_ecs", version = "0.12.0-dev" } bevy_log = { path = "../bevy_log", version = "0.12.0-dev" } -bevy_reflect = { path = "../bevy_reflect", version = "0.12.0-dev", features = ["bevy"] } +bevy_reflect = { path = "../bevy_reflect", version = "0.12.0-dev" } bevy_tasks = { path = "../bevy_tasks", version = "0.12.0-dev" } bevy_utils = { path = "../bevy_utils", version = "0.12.0-dev" } -# other +anyhow = "1.0" +async-broadcast = "0.5" +async-fs = "1.5" +async-lock = "2.8" +crossbeam-channel = "0.5" +downcast-rs = "1.2" +futures-io = "0.3" +futures-lite = "1.12" +md5 = "0.7" +parking_lot = { version = "0.12", features = ["arc_lock", "send_guard"] } +ron = "0.8" serde = { version = "1", features = ["derive"] } -crossbeam-channel = "0.5.0" -anyhow = "1.0.4" thiserror = "1.0" -downcast-rs = "1.2.0" -fastrand = "1.7.0" -notify = { version = "6.0.0", optional = true } -parking_lot = "0.12.1" -async-channel = "1.4.2" +notify-debouncer-full = { version = "0.2.0", optional = true } [target.'cfg(target_os = "android")'.dependencies] bevy_winit = { path = "../bevy_winit", version = "0.12.0-dev" } @@ -44,6 +48,4 @@ wasm-bindgen-futures = "0.4" js-sys = "0.3" [dev-dependencies] -futures-lite = "1.4.0" -tempfile = "3.2.0" -bevy_core = { path = "../bevy_core", version = "0.12.0-dev" } +bevy_core = { path = "../bevy_core", version = "0.12.0-dev" } \ No newline at end of file diff --git a/crates/bevy_asset/macros/Cargo.toml b/crates/bevy_asset/macros/Cargo.toml new file mode 100644 index 0000000000000..b2f9930def52a --- /dev/null +++ b/crates/bevy_asset/macros/Cargo.toml @@ -0,0 +1,19 @@ +[package] +name = "bevy_asset_macros" +version = "0.12.0-dev" +edition = "2021" +description = "Derive implementations for bevy_asset" +homepage = "https://bevyengine.org" +repository = "https://github.com/bevyengine/bevy" +license = "MIT OR Apache-2.0" +keywords = ["bevy"] + +[lib] +proc-macro = true + +[dependencies] +bevy_macro_utils = { path = "../../bevy_macro_utils", version = "0.12.0-dev" } + +syn = "2.0" +proc-macro2 = "1.0" +quote = "1.0" diff --git a/crates/bevy_asset/macros/src/lib.rs b/crates/bevy_asset/macros/src/lib.rs new file mode 100644 index 0000000000000..0dee6e24ab727 --- /dev/null +++ b/crates/bevy_asset/macros/src/lib.rs @@ -0,0 +1,76 @@ +use bevy_macro_utils::BevyManifest; +use proc_macro::{Span, TokenStream}; +use quote::quote; +use syn::{parse_macro_input, Data, DeriveInput, Path}; + +pub(crate) fn bevy_asset_path() -> syn::Path { + BevyManifest::default().get_path("bevy_asset") +} + +const DEPENDENCY_ATTRIBUTE: &str = "dependency"; + +#[proc_macro_derive(Asset, attributes(dependency))] +pub fn derive_asset(input: TokenStream) -> TokenStream { + let ast = parse_macro_input!(input as DeriveInput); + let bevy_asset_path: Path = bevy_asset_path(); + + let struct_name = &ast.ident; + let (impl_generics, type_generics, where_clause) = &ast.generics.split_for_impl(); + let dependency_visitor = match derive_dependency_visitor_internal(&ast, &bevy_asset_path) { + Ok(dependency_visitor) => dependency_visitor, + Err(err) => return err.into_compile_error().into(), + }; + + TokenStream::from(quote! { + impl #impl_generics #bevy_asset_path::Asset for #struct_name #type_generics #where_clause { } + #dependency_visitor + }) +} + +#[proc_macro_derive(VisitAssetDependencies, attributes(dependency))] +pub fn derive_asset_dependency_visitor(input: TokenStream) -> TokenStream { + let ast = parse_macro_input!(input as DeriveInput); + let bevy_asset_path: Path = bevy_asset_path(); + match derive_dependency_visitor_internal(&ast, &bevy_asset_path) { + Ok(dependency_visitor) => TokenStream::from(dependency_visitor), + Err(err) => err.into_compile_error().into(), + } +} + +fn derive_dependency_visitor_internal( + ast: &DeriveInput, + bevy_asset_path: &Path, +) -> Result { + let mut field_visitors = Vec::new(); + if let Data::Struct(data_struct) = &ast.data { + for field in data_struct.fields.iter() { + if field + .attrs + .iter() + .any(|a| a.path().is_ident(DEPENDENCY_ATTRIBUTE)) + { + if let Some(field_ident) = &field.ident { + field_visitors.push(quote! { + #bevy_asset_path::VisitAssetDependencies::visit_dependencies(&self.#field_ident, visit); + }); + } + } + } + } else { + return Err(syn::Error::new( + Span::call_site().into(), + "Asset derive currently only works on structs", + )); + } + + let struct_name = &ast.ident; + let (impl_generics, type_generics, where_clause) = &ast.generics.split_for_impl(); + + Ok(quote! { + impl #impl_generics #bevy_asset_path::VisitAssetDependencies for #struct_name #type_generics #where_clause { + fn visit_dependencies(&self, visit: &mut impl FnMut(#bevy_asset_path::UntypedAssetId)) { + #(#field_visitors)* + } + } + }) +} diff --git a/crates/bevy_asset/src/asset_server.rs b/crates/bevy_asset/src/asset_server.rs deleted file mode 100644 index 11dbd21b37e28..0000000000000 --- a/crates/bevy_asset/src/asset_server.rs +++ /dev/null @@ -1,1049 +0,0 @@ -use crate::{ - path::{AssetPath, AssetPathId, SourcePathId}, - Asset, AssetIo, AssetIoError, AssetLifecycle, AssetLifecycleChannel, AssetLifecycleEvent, - AssetLoader, Assets, Handle, HandleId, HandleUntyped, LabelId, LoadContext, LoadState, - RefChange, RefChangeChannel, SourceInfo, SourceMeta, -}; -use anyhow::Result; -use bevy_ecs::system::{Res, ResMut, Resource}; -use bevy_log::warn; -use bevy_tasks::IoTaskPool; -use bevy_utils::{Entry, HashMap, Uuid}; -use crossbeam_channel::TryRecvError; -use parking_lot::{Mutex, RwLock}; -use std::{path::Path, sync::Arc}; -use thiserror::Error; - -/// Errors that occur while loading assets with an [`AssetServer`]. -#[derive(Error, Debug)] -pub enum AssetServerError { - /// Asset folder is not a directory. - #[error("asset folder path is not a directory: {0}")] - AssetFolderNotADirectory(String), - - /// No asset loader was found for the specified extensions. - #[error("no `AssetLoader` found{}", format_missing_asset_ext(.extensions))] - MissingAssetLoader { - /// The list of extensions detected on the asset source path that failed to load. - /// - /// The list may be empty if the asset path is invalid or doesn't have an extension. - extensions: Vec, - }, - - /// The handle type does not match the type of the loaded asset. - #[error("the given type does not match the type of the loaded asset")] - IncorrectHandleType, - - /// Encountered an error while processing an asset. - #[error("encountered an error while loading an asset: {0}")] - AssetLoaderError(anyhow::Error), - - /// Encountered an error while reading an asset from disk. - #[error("encountered an error while reading an asset: {0}")] - AssetIoError(#[from] AssetIoError), -} - -fn format_missing_asset_ext(exts: &[String]) -> String { - if !exts.is_empty() { - format!( - " for the following extension{}: {}", - if exts.len() > 1 { "s" } else { "" }, - exts.join(", ") - ) - } else { - String::new() - } -} - -#[derive(Default)] -pub(crate) struct AssetRefCounter { - pub(crate) channel: Arc, - pub(crate) ref_counts: Arc>>, - pub(crate) mark_unused_assets: Arc>>, -} - -#[derive(Clone)] -enum MaybeAssetLoader { - Ready(Arc), - Pending { - sender: async_channel::Sender<()>, - receiver: async_channel::Receiver<()>, - }, -} - -/// Internal data for the asset server. -/// -/// [`AssetServer`] is the public API for interacting with the asset server. -pub struct AssetServerInternal { - pub(crate) asset_io: Box, - pub(crate) asset_ref_counter: AssetRefCounter, - pub(crate) asset_sources: Arc>>, - pub(crate) asset_lifecycles: Arc>>>, - loaders: RwLock>, - extension_to_loader_index: RwLock>, - handle_to_path: Arc>>>, -} - -/// Loads assets from the filesystem in the background. -/// -/// The asset server is the primary way of loading assets in bevy. It keeps track of the load state -/// of the assets it manages and can even reload them from the filesystem with -/// ``` -/// # use bevy_asset::*; -/// # use bevy_app::*; -/// # use bevy_utils::Duration; -/// # let mut app = App::new(); -/// // The asset plugin can be configured to watch for asset changes. -/// app.add_plugins(AssetPlugin { -/// watch_for_changes: ChangeWatcher::with_delay(Duration::from_millis(200)), -/// ..Default::default() -/// }); -/// ``` -/// -/// The asset server is a _resource_, so in order to access it in a system you need a `Res` -/// accessor, like this: -/// -/// ```rust,no_run -/// use bevy_asset::{AssetServer, Handle}; -/// use bevy_ecs::prelude::{Commands, Res}; -/// -/// # #[derive(Debug, bevy_reflect::TypeUuid, bevy_reflect::TypePath)] -/// # #[uuid = "00000000-0000-0000-0000-000000000000"] -/// # struct Image; -/// -/// fn my_system(mut commands: Commands, asset_server: Res) -/// { -/// // Now you can do whatever you want with the asset server, such as loading an asset: -/// let asset_handle: Handle = asset_server.load("cool_picture.png"); -/// } -/// ``` -/// -/// See the [`asset_loading`] example for more information. -/// -/// [`asset_loading`]: https://github.com/bevyengine/bevy/tree/latest/examples/asset/asset_loading.rs -#[derive(Clone, Resource)] -pub struct AssetServer { - pub(crate) server: Arc, -} - -impl AssetServer { - /// Creates a new asset server with the provided asset I/O. - pub fn new(source_io: T) -> Self { - Self::with_boxed_io(Box::new(source_io)) - } - - /// Creates a new asset server with a boxed asset I/O. - pub fn with_boxed_io(asset_io: Box) -> Self { - AssetServer { - server: Arc::new(AssetServerInternal { - loaders: Default::default(), - extension_to_loader_index: Default::default(), - asset_sources: Default::default(), - asset_ref_counter: Default::default(), - handle_to_path: Default::default(), - asset_lifecycles: Default::default(), - asset_io, - }), - } - } - - /// Returns the associated asset I/O. - pub fn asset_io(&self) -> &dyn AssetIo { - &*self.server.asset_io - } - - pub(crate) fn register_asset_type(&self) -> Assets { - if self - .server - .asset_lifecycles - .write() - .insert(T::TYPE_UUID, Box::>::default()) - .is_some() - { - panic!("Error while registering new asset type: {:?} with UUID: {:?}. Another type with the same UUID is already registered. Can not register new asset type with the same UUID", - std::any::type_name::(), T::TYPE_UUID); - } - Assets::new(self.server.asset_ref_counter.channel.sender.clone()) - } - - /// Pre-register a loader that will later be added. - /// - /// Assets loaded with matching extensions will be blocked until the - /// real loader is added. - pub fn preregister_loader(&self, extensions: &[&str]) { - let mut loaders = self.server.loaders.write(); - let loader_index = loaders.len(); - for extension in extensions { - if self - .server - .extension_to_loader_index - .write() - .insert(extension.to_string(), loader_index) - .is_some() - { - warn!("duplicate preregistration for `{extension}`, any assets loaded with the previous loader will never complete."); - } - } - let (sender, receiver) = async_channel::bounded(1); - loaders.push(MaybeAssetLoader::Pending { sender, receiver }); - } - - /// Adds the provided asset loader to the server. - /// - /// If `loader` has one or more supported extensions in conflict with loaders that came before - /// it, it will replace them. - pub fn add_loader(&self, loader: T) - where - T: AssetLoader, - { - let mut loaders = self.server.loaders.write(); - let next_loader_index = loaders.len(); - let mut maybe_existing_loader_index = None; - let mut loader_map = self.server.extension_to_loader_index.write(); - let mut maybe_sender = None; - - for extension in loader.extensions() { - if let Some(&extension_index) = loader_map.get(*extension) { - // replacing an existing entry - match maybe_existing_loader_index { - None => { - match &loaders[extension_index] { - MaybeAssetLoader::Ready(_) => { - // replacing an existing loader, nothing special to do - } - MaybeAssetLoader::Pending { sender, .. } => { - // the loader was pre-registered, store the channel to notify pending assets - maybe_sender = Some(sender.clone()); - } - } - } - Some(index) => { - // ensure the loader extensions are consistent - if index != extension_index { - warn!("inconsistent extensions between loader preregister_loader and add_loader, \ - loading `{extension}` assets will never complete."); - } - } - } - - maybe_existing_loader_index = Some(extension_index); - } else { - loader_map.insert(extension.to_string(), next_loader_index); - } - } - - if let Some(existing_index) = maybe_existing_loader_index { - loaders[existing_index] = MaybeAssetLoader::Ready(Arc::new(loader)); - if let Some(sender) = maybe_sender { - // notify after replacing the loader - let _ = sender.close(); - } - } else { - loaders.push(MaybeAssetLoader::Ready(Arc::new(loader))); - } - } - - /// Gets a strong handle for an asset with the provided id. - pub fn get_handle>(&self, id: I) -> Handle { - let sender = self.server.asset_ref_counter.channel.sender.clone(); - Handle::strong(id.into(), sender) - } - - /// Gets an untyped strong handle for an asset with the provided id. - pub fn get_handle_untyped>(&self, id: I) -> HandleUntyped { - let sender = self.server.asset_ref_counter.channel.sender.clone(); - HandleUntyped::strong(id.into(), sender) - } - - fn get_asset_loader(&self, extension: &str) -> Result { - let index = { - // scope map to drop lock as soon as possible - let map = self.server.extension_to_loader_index.read(); - map.get(extension).copied() - }; - index - .map(|index| self.server.loaders.read()[index].clone()) - .ok_or_else(|| AssetServerError::MissingAssetLoader { - extensions: vec![extension.to_string()], - }) - } - - fn get_path_asset_loader>( - &self, - path: P, - include_pending: bool, - ) -> Result { - let s = path - .as_ref() - .file_name() - .ok_or(AssetServerError::MissingAssetLoader { - extensions: Vec::new(), - })? - .to_str() - .map(|s| s.to_lowercase()) - .ok_or(AssetServerError::MissingAssetLoader { - extensions: Vec::new(), - })?; - - let mut exts = Vec::new(); - let mut ext = s.as_str(); - while let Some(idx) = ext.find('.') { - ext = &ext[idx + 1..]; - exts.push(ext); - if let Ok(loader) = self.get_asset_loader(ext) { - if include_pending || matches!(loader, MaybeAssetLoader::Ready(_)) { - return Ok(loader); - } - } - } - Err(AssetServerError::MissingAssetLoader { - extensions: exts.into_iter().map(String::from).collect(), - }) - } - - /// Gets the source path of an asset from the provided handle. - pub fn get_handle_path>(&self, handle: H) -> Option> { - self.server - .handle_to_path - .read() - .get(&handle.into()) - .cloned() - } - - /// Gets the load state of an asset from the provided handle. - pub fn get_load_state>(&self, handle: H) -> LoadState { - match handle.into() { - HandleId::AssetPathId(id) => { - let asset_sources = self.server.asset_sources.read(); - asset_sources - .get(&id.source_path_id()) - .map_or(LoadState::NotLoaded, |info| info.load_state) - } - HandleId::Id(_, _) => LoadState::NotLoaded, - } - } - - /// Gets the overall load state of a group of assets from the provided handles. - /// - /// This method will only return [`LoadState::Loaded`] if all assets in the - /// group were loaded successfully. - pub fn get_group_load_state(&self, handles: impl IntoIterator) -> LoadState { - let mut load_state = LoadState::Loaded; - for handle_id in handles { - match handle_id { - HandleId::AssetPathId(id) => match self.get_load_state(id) { - LoadState::Loaded => continue, - LoadState::Loading => { - load_state = LoadState::Loading; - } - LoadState::Failed => return LoadState::Failed, - LoadState::NotLoaded => return LoadState::NotLoaded, - LoadState::Unloaded => return LoadState::Unloaded, - }, - HandleId::Id(_, _) => return LoadState::NotLoaded, - } - } - - load_state - } - - /// Queues an [`Asset`] at the provided relative path for asynchronous loading. - /// - /// The absolute path to the asset is `"ROOT/ASSET_FOLDER_NAME/path"`. Its extension is then - /// extracted to search for an [asset loader]. If an asset path contains multiple dots (e.g. - /// `foo.bar.baz`), each level is considered a separate extension and the asset server will try - /// to look for loaders of `bar.baz` and `baz` assets. - /// - /// By default the `ROOT` is the directory of the Application, but this can be overridden by - /// setting the `"BEVY_ASSET_ROOT"` or `"CARGO_MANIFEST_DIR"` environment variable - /// (see ) - /// to another directory. When the application is run through Cargo, then - /// `"CARGO_MANIFEST_DIR"` is automatically set to the root folder of your crate (workspace). - /// - /// The name of the asset folder is set inside the - /// [`AssetPlugin`](crate::AssetPlugin). The default name is - /// `"assets"`. - /// - /// The asset is loaded asynchronously, and will generally not be available by the time - /// this calls returns. Use [`AssetServer::get_load_state`] to determine when the asset is - /// effectively loaded and available in the [`Assets`] collection. The asset will always fail to - /// load if the provided path doesn't contain an extension. - /// - /// [asset loader]: AssetLoader - #[must_use = "not using the returned strong handle may result in the unexpected release of the asset"] - pub fn load<'a, T: Asset, P: Into>>(&self, path: P) -> Handle { - self.load_untyped(path).typed() - } - - async fn load_async( - &self, - asset_path: AssetPath<'_>, - force: bool, - ) -> Result { - let asset_path_id: AssetPathId = asset_path.get_id(); - - // load metadata and update source info. this is done in a scope to ensure we release the - // locks before loading - let version = { - let mut asset_sources = self.server.asset_sources.write(); - let source_info = match asset_sources.entry(asset_path_id.source_path_id()) { - Entry::Occupied(entry) => entry.into_mut(), - Entry::Vacant(entry) => entry.insert(SourceInfo { - asset_types: Default::default(), - committed_assets: Default::default(), - load_state: LoadState::NotLoaded, - meta: None, - path: asset_path.path().to_owned(), - version: 0, - }), - }; - - // if asset is already loaded or is loading, don't load again - if !force - && (source_info - .committed_assets - .contains(&asset_path_id.label_id()) - || source_info.load_state == LoadState::Loading) - { - return Ok(asset_path_id); - } - - source_info.load_state = LoadState::Loading; - source_info.committed_assets.clear(); - source_info.version += 1; - source_info.meta = None; - source_info.version - }; - - let set_asset_failed = || { - let mut asset_sources = self.server.asset_sources.write(); - let source_info = asset_sources - .get_mut(&asset_path_id.source_path_id()) - .expect("`AssetSource` should exist at this point."); - source_info.load_state = LoadState::Failed; - }; - - // get the according asset loader - let mut maybe_asset_loader = self.get_path_asset_loader(asset_path.path(), true); - - // if it's still pending, block until notified and refetch the new asset loader - if let Ok(MaybeAssetLoader::Pending { receiver, .. }) = maybe_asset_loader { - let _ = receiver.recv().await; - maybe_asset_loader = self.get_path_asset_loader(asset_path.path(), false); - } - - let asset_loader = match maybe_asset_loader { - Ok(MaybeAssetLoader::Ready(loader)) => loader, - Err(err) => { - set_asset_failed(); - return Err(err); - } - Ok(MaybeAssetLoader::Pending { .. }) => unreachable!(), - }; - - // load the asset bytes - let bytes = match self.asset_io().load_path(asset_path.path()).await { - Ok(bytes) => bytes, - Err(err) => { - set_asset_failed(); - return Err(AssetServerError::AssetIoError(err)); - } - }; - - // load the asset source using the corresponding AssetLoader - let mut load_context = LoadContext::new( - asset_path.path(), - &self.server.asset_ref_counter.channel, - self.asset_io(), - version, - ); - - if let Err(err) = asset_loader - .load(&bytes, &mut load_context) - .await - .map_err(AssetServerError::AssetLoaderError) - { - set_asset_failed(); - return Err(err); - } - - // if version has changed since we loaded and grabbed a lock, return. there is a newer - // version being loaded - let mut asset_sources = self.server.asset_sources.write(); - let source_info = asset_sources - .get_mut(&asset_path_id.source_path_id()) - .expect("`AssetSource` should exist at this point."); - if version != source_info.version { - return Ok(asset_path_id); - } - - // if all assets have been committed already (aka there were 0), set state to "Loaded" - if source_info.is_loaded() { - source_info.load_state = LoadState::Loaded; - } - - // reset relevant SourceInfo fields - source_info.committed_assets.clear(); - // TODO: queue free old assets - source_info.asset_types.clear(); - - source_info.meta = Some(SourceMeta { - assets: load_context.get_asset_metas(), - }); - - // load asset dependencies and prepare asset type hashmap - for (label, loaded_asset) in &mut load_context.labeled_assets { - let label_id = LabelId::from(label.as_ref().map(|label| label.as_str())); - let type_uuid = loaded_asset.value.as_ref().unwrap().type_uuid(); - source_info.asset_types.insert(label_id, type_uuid); - for dependency in &loaded_asset.dependencies { - self.load_untracked(dependency.clone(), false); - } - } - - self.asset_io() - .watch_path_for_changes(asset_path.path(), None) - .unwrap(); - self.create_assets_in_load_context(&mut load_context); - Ok(asset_path_id) - } - - /// Queues the [`Asset`] at the provided path for loading and returns an untyped handle. - /// - /// See [`load`](AssetServer::load). - #[must_use = "not using the returned strong handle may result in the unexpected release of the asset"] - pub fn load_untyped<'a, P: Into>>(&self, path: P) -> HandleUntyped { - let handle_id = self.load_untracked(path.into(), false); - self.get_handle_untyped(handle_id) - } - - /// Force an [`Asset`] to be reloaded. - /// - /// This is useful for custom hot-reloading or for supporting `watch_for_changes` - /// in custom [`AssetIo`] implementations. - pub fn reload_asset<'a, P: Into>>(&self, path: P) { - self.load_untracked(path.into(), true); - } - - pub(crate) fn load_untracked(&self, asset_path: AssetPath<'_>, force: bool) -> HandleId { - let server = self.clone(); - let owned_path = asset_path.to_owned(); - IoTaskPool::get() - .spawn(async move { - if let Err(err) = server.load_async(owned_path, force).await { - warn!("{}", err); - } - }) - .detach(); - - let handle_id = asset_path.get_id().into(); - self.server - .handle_to_path - .write() - .entry(handle_id) - .or_insert_with(|| asset_path.to_owned()); - - asset_path.into() - } - - /// Loads assets from the specified folder recursively. - /// - /// # Errors - /// - /// - If the provided path is not a directory, it will fail with - /// [`AssetServerError::AssetFolderNotADirectory`]. - /// - If something unexpected happened while loading an asset, other - /// [`AssetServerError`]s may be returned. - #[must_use = "not using the returned strong handles may result in the unexpected release of the assets"] - pub fn load_folder>( - &self, - path: P, - ) -> Result, AssetServerError> { - let path = path.as_ref(); - if !self.asset_io().is_dir(path) { - return Err(AssetServerError::AssetFolderNotADirectory( - path.to_str().unwrap().to_string(), - )); - } - - let mut handles = Vec::new(); - for child_path in self.asset_io().read_directory(path.as_ref())? { - if self.asset_io().is_dir(&child_path) { - handles.extend(self.load_folder(&child_path)?); - } else { - if self.get_path_asset_loader(&child_path, true).is_err() { - continue; - } - let handle = - self.load_untyped(child_path.to_str().expect("Path should be a valid string.")); - handles.push(handle); - } - } - - Ok(handles) - } - - /// Frees unused assets, unloading them from memory. - pub fn free_unused_assets(&self) { - let mut potential_frees = self.server.asset_ref_counter.mark_unused_assets.lock(); - - if !potential_frees.is_empty() { - let ref_counts = self.server.asset_ref_counter.ref_counts.read(); - let asset_sources = self.server.asset_sources.read(); - let asset_lifecycles = self.server.asset_lifecycles.read(); - for potential_free in potential_frees.drain(..) { - if let Some(&0) = ref_counts.get(&potential_free) { - let type_uuid = match potential_free { - HandleId::Id(type_uuid, _) => Some(type_uuid), - HandleId::AssetPathId(id) => asset_sources - .get(&id.source_path_id()) - .and_then(|source_info| source_info.get_asset_type(id.label_id())), - }; - - if let Some(type_uuid) = type_uuid { - if let Some(asset_lifecycle) = asset_lifecycles.get(&type_uuid) { - asset_lifecycle.free_asset(potential_free); - } - } - } - } - } - } - - /// Iterates through asset references and marks assets with no active handles as unused. - pub fn mark_unused_assets(&self) { - let receiver = &self.server.asset_ref_counter.channel.receiver; - let mut ref_counts = self.server.asset_ref_counter.ref_counts.write(); - let mut potential_frees = None; - loop { - let ref_change = match receiver.try_recv() { - Ok(ref_change) => ref_change, - Err(TryRecvError::Empty) => break, - Err(TryRecvError::Disconnected) => panic!("RefChange channel disconnected."), - }; - match ref_change { - RefChange::Increment(handle_id) => *ref_counts.entry(handle_id).or_insert(0) += 1, - RefChange::Decrement(handle_id) => { - let entry = ref_counts.entry(handle_id).or_insert(0); - *entry -= 1; - if *entry == 0 { - potential_frees - .get_or_insert_with(|| { - self.server.asset_ref_counter.mark_unused_assets.lock() - }) - .push(handle_id); - } - } - } - } - } - - fn create_assets_in_load_context(&self, load_context: &mut LoadContext) { - let asset_lifecycles = self.server.asset_lifecycles.read(); - for (label, asset) in &mut load_context.labeled_assets { - let asset_value = asset - .value - .take() - .expect("Asset should exist at this point."); - if let Some(asset_lifecycle) = asset_lifecycles.get(&asset_value.type_uuid()) { - let asset_path = - AssetPath::new_ref(load_context.path, label.as_ref().map(|l| l.as_str())); - asset_lifecycle.create_asset(asset_path.into(), asset_value, load_context.version); - } else { - panic!( - "Failed to find AssetLifecycle for label '{:?}', which has an asset type {} (UUID {:?}). \ - Are you sure this asset type has been added to your app builder?", - label, - asset_value.type_name(), - asset_value.type_uuid(), - ); - } - } - } - - // Note: this takes a `ResMut>` to ensure change detection does not get - // triggered unless the `Assets` collection is actually updated. - pub(crate) fn update_asset_storage(&self, mut assets: ResMut>) { - let asset_lifecycles = self.server.asset_lifecycles.read(); - let asset_lifecycle = asset_lifecycles.get(&T::TYPE_UUID).unwrap(); - let mut asset_sources_guard = None; - let channel = asset_lifecycle - .downcast_ref::>() - .unwrap(); - - loop { - match channel.receiver.try_recv() { - Ok(AssetLifecycleEvent::Create(result)) => { - // update SourceInfo if this asset was loaded from an AssetPath - if let HandleId::AssetPathId(id) = result.id { - let asset_sources = asset_sources_guard - .get_or_insert_with(|| self.server.asset_sources.write()); - if let Some(source_info) = asset_sources.get_mut(&id.source_path_id()) { - if source_info.version == result.version { - source_info.committed_assets.insert(id.label_id()); - if source_info.is_loaded() { - source_info.load_state = LoadState::Loaded; - } - } - } - } - - assets.set_untracked(result.id, *result.asset); - } - Ok(AssetLifecycleEvent::Free(handle_id)) => { - if let HandleId::AssetPathId(id) = handle_id { - let asset_sources = asset_sources_guard - .get_or_insert_with(|| self.server.asset_sources.write()); - if let Some(source_info) = asset_sources.get_mut(&id.source_path_id()) { - source_info.committed_assets.remove(&id.label_id()); - source_info.load_state = LoadState::Unloaded; - } - } - assets.remove(handle_id); - } - Err(TryRecvError::Empty) => { - break; - } - Err(TryRecvError::Disconnected) => panic!("AssetChannel disconnected."), - } - } - } -} - -fn free_unused_assets_system_impl(asset_server: &AssetServer) { - asset_server.free_unused_assets(); - asset_server.mark_unused_assets(); -} - -/// A system for freeing assets that have no active handles. -pub fn free_unused_assets_system(asset_server: Res) { - free_unused_assets_system_impl(&asset_server); -} - -#[cfg(test)] -mod test { - use super::*; - use crate::{loader::LoadedAsset, update_asset_storage_system}; - use bevy_app::{App, Update}; - use bevy_ecs::prelude::*; - use bevy_reflect::{TypePath, TypeUuid}; - use bevy_utils::BoxedFuture; - - #[derive(Debug, TypeUuid, TypePath)] - #[uuid = "a5189b72-0572-4290-a2e0-96f73a491c44"] - struct PngAsset; - - struct FakePngLoader; - impl AssetLoader for FakePngLoader { - fn load<'a>( - &'a self, - _: &'a [u8], - ctx: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<(), anyhow::Error>> { - ctx.set_default_asset(LoadedAsset::new(PngAsset)); - Box::pin(async move { Ok(()) }) - } - - fn extensions(&self) -> &[&str] { - &["png"] - } - } - - struct FailingLoader; - impl AssetLoader for FailingLoader { - fn load<'a>( - &'a self, - _: &'a [u8], - _: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<(), anyhow::Error>> { - Box::pin(async { anyhow::bail!("failed") }) - } - - fn extensions(&self) -> &[&str] { - &["fail"] - } - } - - struct FakeMultipleDotLoader; - impl AssetLoader for FakeMultipleDotLoader { - fn load<'a>( - &'a self, - _: &'a [u8], - _: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<(), anyhow::Error>> { - Box::pin(async move { Ok(()) }) - } - - fn extensions(&self) -> &[&str] { - &["test.png"] - } - } - - fn setup(asset_path: impl AsRef) -> AssetServer { - use crate::FileAssetIo; - IoTaskPool::init(Default::default); - AssetServer::new(FileAssetIo::new(asset_path, &None)) - } - - #[test] - fn extensions() { - let asset_server = setup("."); - asset_server.add_loader(FakePngLoader); - - let Ok(MaybeAssetLoader::Ready(t)) = asset_server.get_path_asset_loader("test.png", true) - else { - panic!(); - }; - - assert_eq!(t.extensions()[0], "png"); - } - - #[test] - fn case_insensitive_extensions() { - let asset_server = setup("."); - asset_server.add_loader(FakePngLoader); - - let Ok(MaybeAssetLoader::Ready(t)) = asset_server.get_path_asset_loader("test.PNG", true) - else { - panic!(); - }; - assert_eq!(t.extensions()[0], "png"); - } - - #[test] - fn no_loader() { - let asset_server = setup("."); - let t = asset_server.get_path_asset_loader("test.pong", true); - assert!(t.is_err()); - } - - #[test] - fn multiple_extensions_no_loader() { - let asset_server = setup("."); - - assert!( - match asset_server.get_path_asset_loader("test.v1.2.3.pong", true) { - Err(AssetServerError::MissingAssetLoader { extensions }) => - extensions == vec!["v1.2.3.pong", "2.3.pong", "3.pong", "pong"], - _ => false, - } - ); - } - - #[test] - fn missing_asset_loader_error_messages() { - assert_eq!( - AssetServerError::MissingAssetLoader { extensions: vec![] }.to_string(), - "no `AssetLoader` found" - ); - assert_eq!( - AssetServerError::MissingAssetLoader { - extensions: vec!["png".into()] - } - .to_string(), - "no `AssetLoader` found for the following extension: png" - ); - assert_eq!( - AssetServerError::MissingAssetLoader { - extensions: vec!["1.2.png".into(), "2.png".into(), "png".into()] - } - .to_string(), - "no `AssetLoader` found for the following extensions: 1.2.png, 2.png, png" - ); - } - - #[test] - fn filename_with_dots() { - let asset_server = setup("."); - asset_server.add_loader(FakePngLoader); - - let Ok(MaybeAssetLoader::Ready(t)) = - asset_server.get_path_asset_loader("test-v1.2.3.png", true) - else { - panic!(); - }; - assert_eq!(t.extensions()[0], "png"); - } - - #[test] - fn multiple_extensions() { - let asset_server = setup("."); - asset_server.add_loader(FakeMultipleDotLoader); - - let Ok(MaybeAssetLoader::Ready(t)) = - asset_server.get_path_asset_loader("test.test.png", true) - else { - panic!(); - }; - assert_eq!(t.extensions()[0], "test.png"); - } - - fn create_dir_and_file(file: impl AsRef) -> tempfile::TempDir { - let asset_dir = tempfile::tempdir().unwrap(); - std::fs::write(asset_dir.path().join(file), []).unwrap(); - asset_dir - } - - #[test] - fn test_missing_loader() { - let dir = create_dir_and_file("file.not-a-real-extension"); - let asset_server = setup(dir.path()); - - let path: AssetPath = "file.not-a-real-extension".into(); - let handle = asset_server.get_handle_untyped(path.get_id()); - - let err = futures_lite::future::block_on(asset_server.load_async(path.clone(), true)) - .unwrap_err(); - assert!(match err { - AssetServerError::MissingAssetLoader { extensions } => { - extensions == ["not-a-real-extension"] - } - _ => false, - }); - - assert_eq!(asset_server.get_load_state(handle), LoadState::Failed); - } - - #[test] - fn test_invalid_asset_path() { - let asset_server = setup("."); - asset_server.add_loader(FakePngLoader); - - let path: AssetPath = "an/invalid/path.png".into(); - let handle = asset_server.get_handle_untyped(path.get_id()); - - let err = futures_lite::future::block_on(asset_server.load_async(path.clone(), true)) - .unwrap_err(); - assert!(matches!(err, AssetServerError::AssetIoError(_))); - - assert_eq!(asset_server.get_load_state(handle), LoadState::Failed); - } - - #[test] - fn test_failing_loader() { - let dir = create_dir_and_file("fake.fail"); - let asset_server = setup(dir.path()); - asset_server.add_loader(FailingLoader); - - let path: AssetPath = "fake.fail".into(); - let handle = asset_server.get_handle_untyped(path.get_id()); - - let err = futures_lite::future::block_on(asset_server.load_async(path.clone(), true)) - .unwrap_err(); - assert!(matches!(err, AssetServerError::AssetLoaderError(_))); - - assert_eq!(asset_server.get_load_state(handle), LoadState::Failed); - } - - #[test] - fn test_asset_lifecycle() { - let dir = create_dir_and_file("fake.png"); - let asset_server = setup(dir.path()); - asset_server.add_loader(FakePngLoader); - let assets = asset_server.register_asset_type::(); - - #[derive(SystemSet, Clone, Hash, Debug, PartialEq, Eq)] - struct FreeUnusedAssets; - let mut app = App::new(); - app.insert_resource(assets); - app.insert_resource(asset_server); - app.add_systems( - Update, - ( - free_unused_assets_system.in_set(FreeUnusedAssets), - update_asset_storage_system::.after(FreeUnusedAssets), - ), - ); - - fn load_asset(path: AssetPath, world: &World) -> HandleUntyped { - let asset_server = world.resource::(); - let id = futures_lite::future::block_on(asset_server.load_async(path.clone(), true)) - .unwrap(); - asset_server.get_handle_untyped(id) - } - - fn get_asset<'world>( - id: &Handle, - world: &'world World, - ) -> Option<&'world PngAsset> { - world.resource::>().get(id) - } - - fn get_load_state(id: impl Into, world: &World) -> LoadState { - world.resource::().get_load_state(id.into()) - } - - // --- - // Start of the actual lifecycle test - // --- - - let path: AssetPath = "fake.png".into(); - assert_eq!( - LoadState::NotLoaded, - get_load_state(path.get_id(), &app.world) - ); - - // load the asset - let handle = load_asset(path.clone(), &app.world).typed(); - let weak_handle = handle.clone_weak(); - - // asset is loading - assert_eq!(LoadState::Loading, get_load_state(&handle, &app.world)); - - app.update(); - // asset should exist and be loaded at this point - assert_eq!(LoadState::Loaded, get_load_state(&handle, &app.world)); - assert!(get_asset(&handle, &app.world).is_some()); - - // after dropping the handle, next call to `tick` will prepare the assets for removal. - drop(handle); - app.update(); - assert_eq!(LoadState::Loaded, get_load_state(&weak_handle, &app.world)); - assert!(get_asset(&weak_handle, &app.world).is_some()); - - // second call to tick will actually remove the asset. - app.update(); - assert_eq!( - LoadState::Unloaded, - get_load_state(&weak_handle, &app.world) - ); - assert!(get_asset(&weak_handle, &app.world).is_none()); - - // finally, reload the asset - let handle = load_asset(path.clone(), &app.world).typed(); - assert_eq!(LoadState::Loading, get_load_state(&handle, &app.world)); - app.update(); - assert_eq!(LoadState::Loaded, get_load_state(&handle, &app.world)); - assert!(get_asset(&handle, &app.world).is_some()); - } - - #[test] - fn test_get_handle_path() { - const PATH: &str = "path/file.png"; - - // valid handle - let server = setup("."); - let handle = server.load_untyped(PATH); - let handle_path = server.get_handle_path(&handle).unwrap(); - - assert_eq!(handle_path.path(), Path::new(PATH)); - assert!(handle_path.label().is_none()); - - let handle_id: HandleId = handle.into(); - let path_id: HandleId = handle_path.get_id().into(); - assert_eq!(handle_id, path_id); - - // invalid handle (not loaded through server) - let mut assets = server.register_asset_type::(); - let handle = assets.add(PngAsset); - assert!(server.get_handle_path(&handle).is_none()); - - // invalid HandleId - let invalid_id = HandleId::new(Uuid::new_v4(), 42); - assert!(server.get_handle_path(invalid_id).is_none()); - - // invalid AssetPath - let invalid_path = AssetPath::new("some/path.ext".into(), None); - assert!(server.get_handle_path(invalid_path).is_none()); - } -} diff --git a/crates/bevy_asset/src/assets.rs b/crates/bevy_asset/src/assets.rs index b586acaeb56ff..a01a93c4dd71d 100644 --- a/crates/bevy_asset/src/assets.rs +++ b/crates/bevy_asset/src/assets.rs @@ -1,583 +1,532 @@ -use crate::{ - update_asset_storage_system, Asset, AssetEvents, AssetLoader, AssetServer, Handle, HandleId, - LoadAssets, RefChange, ReflectAsset, ReflectHandle, +use crate::{Asset, AssetEvent, AssetHandleProvider, AssetId, AssetServer, Handle}; +use bevy_ecs::{ + prelude::EventWriter, + system::{Res, ResMut, Resource}, }; -use bevy_app::App; -use bevy_ecs::prelude::*; -use bevy_ecs::reflect::AppTypeRegistry; -use bevy_reflect::{FromReflect, GetTypeRegistration, Reflect}; +use bevy_reflect::{Reflect, Uuid}; use bevy_utils::HashMap; -use crossbeam_channel::Sender; -use std::fmt::Debug; +use crossbeam_channel::{Receiver, Sender}; +use serde::{Deserialize, Serialize}; +use std::{ + any::TypeId, + iter::Enumerate, + marker::PhantomData, + sync::{atomic::AtomicU32, Arc}, +}; +use thiserror::Error; + +/// A generational runtime-only identifier for a specific [`Asset`] stored in [`Assets`]. This is optimized for efficient runtime +/// usage and is not suitable for identifying assets across app runs. +#[derive( + Debug, Copy, Clone, Eq, PartialEq, Hash, Ord, PartialOrd, Reflect, Serialize, Deserialize, +)] +pub struct AssetIndex { + pub(crate) generation: u32, + pub(crate) index: u32, +} -/// Events that involve assets of type `T`. -/// -/// Events sent via the [`Assets`] struct will always be sent with a _Weak_ handle, because the -/// asset may not exist by the time the event is handled. -#[derive(Event)] -pub enum AssetEvent { - #[allow(missing_docs)] - Created { handle: Handle }, - #[allow(missing_docs)] - Modified { handle: Handle }, - #[allow(missing_docs)] - Removed { handle: Handle }, +/// Allocates generational [`AssetIndex`] values and facilitates their reuse. +pub(crate) struct AssetIndexAllocator { + /// A monotonically increasing index. + next_index: AtomicU32, + recycled_queue_sender: Sender, + /// This receives every recycled AssetIndex. It serves as a buffer/queue to store indices ready for reuse. + recycled_queue_receiver: Receiver, + recycled_sender: Sender, + recycled_receiver: Receiver, } -impl Debug for AssetEvent { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - match self { - AssetEvent::Created { handle } => f - .debug_struct(&format!( - "AssetEvent<{}>::Created", - std::any::type_name::() - )) - .field("handle", &handle.id()) - .finish(), - AssetEvent::Modified { handle } => f - .debug_struct(&format!( - "AssetEvent<{}>::Modified", - std::any::type_name::() - )) - .field("handle", &handle.id()) - .finish(), - AssetEvent::Removed { handle } => f - .debug_struct(&format!( - "AssetEvent<{}>::Removed", - std::any::type_name::() - )) - .field("handle", &handle.id()) - .finish(), +impl Default for AssetIndexAllocator { + fn default() -> Self { + let (recycled_queue_sender, recycled_queue_receiver) = crossbeam_channel::unbounded(); + let (recycled_sender, recycled_receiver) = crossbeam_channel::unbounded(); + Self { + recycled_queue_sender, + recycled_queue_receiver, + recycled_sender, + recycled_receiver, + next_index: Default::default(), } } } -/// Stores Assets of a given type and tracks changes to them. -/// -/// Each asset is mapped by a unique [`HandleId`], allowing any [`Handle`] with the same -/// [`HandleId`] to access it. These assets remain loaded for as long as a Strong handle to that -/// asset exists. -/// -/// To store a reference to an asset without forcing it to stay loaded, you can use a Weak handle. -/// To make a Weak handle a Strong one, use [`Assets::get_handle`] or pass the `Assets` collection -/// into the handle's [`make_strong`](Handle::make_strong) method. -/// -/// Remember, if there are no Strong handles for an asset (i.e. they have all been dropped), the -/// asset will unload. Make sure you always have a Strong handle when you want to keep an asset -/// loaded! -#[derive(Debug, Resource)] -pub struct Assets { - assets: HashMap, - events: Events>, - pub(crate) ref_change_sender: Sender, +impl AssetIndexAllocator { + /// Reserves a new [`AssetIndex`], either by reusing a recycled index (with an incremented generation), or by creating a new index + /// by incrementing the index counter for a given asset type `A`. + pub fn reserve(&self) -> AssetIndex { + if let Ok(mut recycled) = self.recycled_queue_receiver.try_recv() { + recycled.generation += 1; + self.recycled_sender.send(recycled).unwrap(); + recycled + } else { + AssetIndex { + index: self + .next_index + .fetch_add(1, std::sync::atomic::Ordering::Relaxed), + generation: 0, + } + } + } + + /// Queues the given `index` for reuse. This should only be done if the `index` is no longer being used. + pub fn recycle(&self, index: AssetIndex) { + self.recycled_queue_sender.send(index).unwrap(); + } } -impl Assets { - pub(crate) fn new(ref_change_sender: Sender) -> Self { - Assets { - assets: HashMap::default(), - events: Events::default(), - ref_change_sender, +// PERF: do we actually need this to be an enum? Can we just use an "invalid" generation instead +#[derive(Default)] +enum Entry { + #[default] + None, + Some { + value: Option, + generation: u32, + }, +} + +/// Stores [`Asset`] values in a Vec-like storage identified by [`AssetIndex`]. +struct DenseAssetStorage { + storage: Vec>, + len: u32, + allocator: Arc, +} + +impl Default for DenseAssetStorage { + fn default() -> Self { + Self { + len: 0, + storage: Default::default(), + allocator: Default::default(), } } +} - /// Adds an asset to the collection, returning a Strong handle to that asset. - /// - /// # Events - /// - /// * [`AssetEvent::Created`] - pub fn add(&mut self, asset: T) -> Handle { - let id = HandleId::random::(); - self.assets.insert(id, asset); - self.events.send(AssetEvent::Created { - handle: Handle::weak(id), - }); - self.get_handle(id) +impl DenseAssetStorage { + // Returns the number of assets stored. + pub(crate) fn len(&self) -> usize { + self.len as usize } - /// Add/modify the asset pointed to by the given handle. - /// - /// Unless there exists another Strong handle for this asset, it's advised to use the returned - /// Strong handle. Not doing so may result in the unexpected release of the asset. - /// - /// See [`set_untracked`](Assets::set_untracked) for more info. - #[must_use = "not using the returned strong handle may result in the unexpected release of the asset"] - pub fn set>(&mut self, handle: H, asset: T) -> Handle { - let id: HandleId = handle.into(); - self.set_untracked(id, asset); - self.get_handle(id) + // Returns `true` if there are no assets stored. + pub(crate) fn is_empty(&self) -> bool { + self.len == 0 } - /// Add/modify the asset pointed to by the given handle. - /// - /// If an asset already exists with the given [`HandleId`], it will be modified. Otherwise the - /// new asset will be inserted. - /// - /// # Events - /// - /// * [`AssetEvent::Created`]: Sent if the asset did not yet exist with the given handle. - /// * [`AssetEvent::Modified`]: Sent if the asset with given handle already existed. - pub fn set_untracked>(&mut self, handle: H, asset: T) { - let id: HandleId = handle.into(); - if self.assets.insert(id, asset).is_some() { - self.events.send(AssetEvent::Modified { - handle: Handle::weak(id), - }); + /// Insert the value at the given index. Returns true if a value already exists (and was replaced) + pub(crate) fn insert( + &mut self, + index: AssetIndex, + asset: A, + ) -> Result { + self.flush(); + let entry = &mut self.storage[index.index as usize]; + if let Entry::Some { value, generation } = entry { + if *generation == index.generation { + let exists = value.is_some(); + if !exists { + self.len += 1; + } + *value = Some(asset); + Ok(exists) + } else { + Err(InvalidGenerationError { + index, + current_generation: *generation, + }) + } } else { - self.events.send(AssetEvent::Created { - handle: Handle::weak(id), - }); + unreachable!("entries should always be valid after a flush"); } } - /// Gets the asset for the given handle. - /// - /// This is the main method for accessing asset data from an [Assets] collection. If you need - /// mutable access to the asset, use [`get_mut`](Assets::get_mut). - pub fn get(&self, handle: &Handle) -> Option<&T> { - self.assets.get::(&handle.into()) - } - - /// Checks if an asset exists for the given handle - pub fn contains(&self, handle: &Handle) -> bool { - self.assets.contains_key::(&handle.into()) + /// Removes the asset stored at the given `index` and returns it as [`Some`] (if the asset exists). + pub(crate) fn remove(&mut self, index: AssetIndex) -> Option { + self.flush(); + let value = match &mut self.storage[index.index as usize] { + Entry::None => return None, + Entry::Some { value, generation } => { + if *generation == index.generation { + value.take().map(|value| { + self.len -= 1; + value + }) + } else { + return None; + } + } + }; + self.storage[index.index as usize] = Entry::None; + self.allocator.recycle(index); + value } - /// Get mutable access to the asset for the given handle. - /// - /// This is the main method for mutably accessing asset data from an [Assets] collection. If you - /// do not need mutable access to the asset, you may also use [`get`](Assets::get). - pub fn get_mut(&mut self, handle: &Handle) -> Option<&mut T> { - let id: HandleId = handle.into(); - self.events.send(AssetEvent::Modified { - handle: Handle::weak(id), - }); - self.assets.get_mut(&id) + pub(crate) fn get(&self, index: AssetIndex) -> Option<&A> { + let entry = self.storage.get(index.index as usize)?; + match entry { + Entry::None => None, + Entry::Some { value, generation } => { + if *generation == index.generation { + value.as_ref() + } else { + None + } + } + } } - /// Gets a _Strong_ handle pointing to the same asset as the given one. - pub fn get_handle>(&self, handle: H) -> Handle { - Handle::strong(handle.into(), self.ref_change_sender.clone()) + pub(crate) fn get_mut(&mut self, index: AssetIndex) -> Option<&mut A> { + let entry = self.storage.get_mut(index.index as usize)?; + match entry { + Entry::None => None, + Entry::Some { value, generation } => { + if *generation == index.generation { + value.as_mut() + } else { + None + } + } + } } - /// Gets mutable access to an asset for the given handle, inserting a new value if none exists. - /// - /// # Events - /// - /// * [`AssetEvent::Created`]: Sent if the asset did not yet exist with the given handle. - pub fn get_or_insert_with>( - &mut self, - handle: H, - insert_fn: impl FnOnce() -> T, - ) -> &mut T { - let mut event = None; - let id: HandleId = handle.into(); - let borrowed = self.assets.entry(id).or_insert_with(|| { - event = Some(AssetEvent::Created { - handle: Handle::weak(id), - }); - insert_fn() + pub(crate) fn flush(&mut self) { + // NOTE: this assumes the allocator index is monotonically increasing. + let new_len = self + .allocator + .next_index + .load(std::sync::atomic::Ordering::Relaxed); + self.storage.resize_with(new_len as usize, || Entry::Some { + value: None, + generation: 0, }); - - if let Some(event) = event { - self.events.send(event); + while let Ok(recycled) = self.allocator.recycled_receiver.try_recv() { + let entry = &mut self.storage[recycled.index as usize]; + *entry = Entry::Some { + value: None, + generation: recycled.generation, + }; } - borrowed } - /// Gets an iterator over all assets in the collection. - pub fn iter(&self) -> impl Iterator { - self.assets.iter().map(|(k, v)| (*k, v)) + pub(crate) fn get_index_allocator(&self) -> Arc { + self.allocator.clone() } - /// Gets a mutable iterator over all assets in the collection. - pub fn iter_mut(&mut self) -> impl Iterator { - self.assets.iter_mut().map(|(k, v)| { - self.events.send(AssetEvent::Modified { - handle: Handle::weak(*k), - }); - (*k, v) - }) + pub(crate) fn ids(&self) -> impl Iterator> + '_ { + self.storage + .iter() + .enumerate() + .filter_map(|(i, v)| match v { + Entry::None => None, + Entry::Some { value, generation } => { + if value.is_some() { + Some(AssetId::from(AssetIndex { + index: i as u32, + generation: *generation, + })) + } else { + None + } + } + }) } +} - /// Gets an iterator over all [`HandleId`]'s in the collection. - pub fn ids(&self) -> impl Iterator + '_ { - self.assets.keys().cloned() - } +/// Stores [`Asset`] values identified by their [`AssetId`]. +/// +/// Assets identified by [`AssetId::Index`] will be stored in a "dense" vec-like storage. This is more efficient, but it means that +/// the assets can only be identified at runtime. This is the default behavior. +/// +/// Assets identified by [`AssetId::Uuid`] will be stored in a hashmap. This is less efficient, but it means that the assets can be referenced +/// at compile time. +/// +/// This tracks (and queues) [`AssetEvent`] events whenever changes to the collection occur. +#[derive(Resource)] +pub struct Assets { + dense_storage: DenseAssetStorage, + hash_map: HashMap, + handle_provider: AssetHandleProvider, + queued_events: Vec>, +} - /// Removes an asset for the given handle. - /// - /// The asset is returned if it existed in the collection, otherwise `None`. - /// - /// # Events - /// - /// * [`AssetEvent::Removed`] - pub fn remove>(&mut self, handle: H) -> Option { - let id: HandleId = handle.into(); - let asset = self.assets.remove(&id); - if asset.is_some() { - self.events.send(AssetEvent::Removed { - handle: Handle::weak(id), - }); +impl Default for Assets { + fn default() -> Self { + let dense_storage = DenseAssetStorage::default(); + let handle_provider = + AssetHandleProvider::new(TypeId::of::(), dense_storage.get_index_allocator()); + Self { + dense_storage, + handle_provider, + hash_map: Default::default(), + queued_events: Default::default(), } - asset } +} - /// Clears the inner asset map, removing all key-value pairs. - /// - /// Keeps the allocated memory for reuse. - pub fn clear(&mut self) { - self.assets.clear(); +impl Assets { + /// Retrieves an [`AssetHandleProvider`] capable of reserving new [`Handle`] values for assets that will be stored in this + /// collection. + pub fn get_handle_provider(&self) -> AssetHandleProvider { + self.handle_provider.clone() } - /// Reserves capacity for at least additional more elements to be inserted into the assets. - /// - /// The collection may reserve more space to avoid frequent reallocations. - pub fn reserve(&mut self, additional: usize) { - self.assets.reserve(additional); + /// Inserts the given `asset`, identified by the given `id`. If an asset already exists for `id`, it will be replaced. + pub fn insert(&mut self, id: impl Into>, asset: A) { + let id: AssetId = id.into(); + match id { + AssetId::Index { index, .. } => { + self.insert_with_index(index, asset).unwrap(); + } + AssetId::Uuid { uuid } => { + self.insert_with_uuid(uuid, asset); + } + } } - /// Shrinks the capacity of the asset map as much as possible. - /// - /// It will drop down as much as possible while maintaining the internal rules and possibly - /// leaving some space in accordance with the resize policy. - pub fn shrink_to_fit(&mut self) { - self.assets.shrink_to_fit(); - } - - /// A system that creates [`AssetEvent`]s at the end of the frame based on changes in the - /// asset storage. - pub fn asset_event_system( - mut events: EventWriter>, - mut assets: ResMut>, - ) { - // Check if the events are empty before calling `drain`. - // As `drain` triggers change detection. - if !assets.events.is_empty() { - events.send_batch(assets.events.drain()); + /// Retrieves an [`Asset`] stored for the given `id` if it exists. If it does not exist, it will be inserted using `insert_fn`. + // PERF: Optimize this or remove it + pub fn get_or_insert_with( + &mut self, + id: impl Into>, + insert_fn: impl FnOnce() -> A, + ) -> &mut A { + let id: AssetId = id.into(); + if self.get(id).is_none() { + self.insert(id, (insert_fn)()); } + self.get_mut(id).unwrap() } - /// Gets the number of assets in the collection. - pub fn len(&self) -> usize { - self.assets.len() + /// Returns `true` if the `id` exists in this collection. Otherwise it returns `false`. + // PERF: Optimize this or remove it + pub fn contains(&self, id: impl Into>) -> bool { + self.get(id).is_some() } - /// Returns `true` if there are no stored assets. - pub fn is_empty(&self) -> bool { - self.assets.is_empty() + pub(crate) fn insert_with_uuid(&mut self, uuid: Uuid, asset: A) -> Option { + let result = self.hash_map.insert(uuid, asset); + if result.is_some() { + self.queued_events + .push(AssetEvent::Modified { id: uuid.into() }); + } else { + self.queued_events + .push(AssetEvent::Added { id: uuid.into() }); + } + result + } + pub(crate) fn insert_with_index( + &mut self, + index: AssetIndex, + asset: A, + ) -> Result { + let replaced = self.dense_storage.insert(index, asset)?; + if replaced { + self.queued_events + .push(AssetEvent::Modified { id: index.into() }); + } else { + self.queued_events + .push(AssetEvent::Added { id: index.into() }); + } + Ok(replaced) } -} - -/// [`App`] extension methods for adding new asset types. -pub trait AddAsset { - /// Registers `T` as a supported asset in the application. - /// - /// Adding the same type again after it has been added does nothing. - fn add_asset(&mut self) -> &mut Self - where - T: Asset; - - /// Registers the asset type `T` using [`App::register_type`], - /// and adds [`ReflectAsset`] type data to `T` and [`ReflectHandle`] type data to [`Handle`] in the type registry. - /// - /// This enables reflection code to access assets. For detailed information, see the docs on [`ReflectAsset`] and [`ReflectHandle`]. - fn register_asset_reflect(&mut self) -> &mut Self - where - T: Asset + Reflect + FromReflect + GetTypeRegistration; - - /// Registers `T` as a supported internal asset in the application. - /// - /// Internal assets (e.g. shaders) are bundled directly into the app and can't be hot reloaded - /// using the conventional API. See `DebugAssetServerPlugin`. - /// - /// Adding the same type again after it has been added does nothing. - fn add_debug_asset(&mut self) -> &mut Self - where - T: Asset; - - /// Adds an asset loader `T` using default values. - /// - /// The default values may come from the [`World`] or from `T::default()`. - fn init_asset_loader(&mut self) -> &mut Self - where - T: AssetLoader + FromWorld; - /// Adds an asset loader `T` for internal assets using default values. - /// - /// Internal assets (e.g. shaders) are bundled directly into the app and can't be hot reloaded - /// using the conventional API. See `DebugAssetServerPlugin`. - /// - /// The default values may come from the [`World`] or from `T::default()`. - fn init_debug_asset_loader(&mut self) -> &mut Self - where - T: AssetLoader + FromWorld; - - /// Adds the provided asset loader to the application. - fn add_asset_loader(&mut self, loader: T) -> &mut Self - where - T: AssetLoader; - - /// Preregisters a loader for the given extensions, that will block asset loads until a real loader - /// is registered. - fn preregister_asset_loader(&mut self, extensions: &[&str]) -> &mut Self; -} + /// Adds the given `asset` and allocates a new strong [`Handle`] for it. + #[inline] + pub fn add(&mut self, asset: A) -> Handle { + let index = self.dense_storage.allocator.reserve(); + self.insert_with_index(index, asset).unwrap(); + Handle::Strong( + self.handle_provider + .get_handle(index.into(), false, None, None), + ) + } -impl AddAsset for App { - fn add_asset(&mut self) -> &mut Self - where - T: Asset, - { - if self.world.contains_resource::>() { - return self; + /// Retrieves a reference to the [`Asset`] with the given `id`, if its exists. + /// Note that this supports anything that implements `Into>`, which includes [`Handle`] and [`AssetId`]. + #[inline] + pub fn get(&self, id: impl Into>) -> Option<&A> { + let id: AssetId = id.into(); + match id { + AssetId::Index { index, .. } => self.dense_storage.get(index), + AssetId::Uuid { uuid } => self.hash_map.get(&uuid), } - let assets = { - let asset_server = self.world.resource::(); - asset_server.register_asset_type::() - }; + } - self.insert_resource(assets) - .add_systems(LoadAssets, update_asset_storage_system::) - .add_systems(AssetEvents, Assets::::asset_event_system) - .register_type::>() - .add_event::>() - } - - fn register_asset_reflect(&mut self) -> &mut Self - where - T: Asset + Reflect + FromReflect + GetTypeRegistration, - { - let type_registry = self.world.resource::(); - { - let mut type_registry = type_registry.write(); - - type_registry.register::(); - type_registry.register::>(); - type_registry.register_type_data::(); - type_registry.register_type_data::, ReflectHandle>(); + /// Retrieves a mutable reference to the [`Asset`] with the given `id`, if its exists. + /// Note that this supports anything that implements `Into>`, which includes [`Handle`] and [`AssetId`]. + #[inline] + pub fn get_mut(&mut self, id: impl Into>) -> Option<&mut A> { + let id: AssetId = id.into(); + let result = match id { + AssetId::Index { index, .. } => self.dense_storage.get_mut(index), + AssetId::Uuid { uuid } => self.hash_map.get_mut(&uuid), + }; + if result.is_some() { + self.queued_events.push(AssetEvent::Modified { id }); } + result + } - self - } - - fn add_debug_asset(&mut self) -> &mut Self - where - T: Asset, - { - #[cfg(feature = "debug_asset_server")] - { - self.add_systems( - bevy_app::Update, - crate::debug_asset_server::sync_debug_assets::, - ); - let mut app = self - .world - .non_send_resource_mut::(); - app.add_asset::() - .init_resource::>(); + /// Removes (and returns) the [`Asset`] with the given `id`, if its exists. + /// Note that this supports anything that implements `Into>`, which includes [`Handle`] and [`AssetId`]. + pub fn remove(&mut self, id: impl Into>) -> Option { + let id: AssetId = id.into(); + let result = self.remove_untracked(id); + if result.is_some() { + self.queued_events.push(AssetEvent::Removed { id }); } - self - } - - fn init_asset_loader(&mut self) -> &mut Self - where - T: AssetLoader + FromWorld, - { - let result = T::from_world(&mut self.world); - self.add_asset_loader(result) - } - - fn init_debug_asset_loader(&mut self) -> &mut Self - where - T: AssetLoader + FromWorld, - { - #[cfg(feature = "debug_asset_server")] - { - let mut app = self - .world - .non_send_resource_mut::(); - app.init_asset_loader::(); + result + } + + /// Removes (and returns) the [`Asset`] with the given `id`, if its exists. This skips emitting [`AssetEvent::Removed`]. + /// Note that this supports anything that implements `Into>`, which includes [`Handle`] and [`AssetId`]. + pub fn remove_untracked(&mut self, id: impl Into>) -> Option { + let id: AssetId = id.into(); + match id { + AssetId::Index { index, .. } => self.dense_storage.remove(index), + AssetId::Uuid { uuid } => self.hash_map.remove(&uuid), } - self } - fn add_asset_loader(&mut self, loader: T) -> &mut Self - where - T: AssetLoader, - { - self.world.resource_mut::().add_loader(loader); - self + /// Returns `true` if there are no assets in this collection. + pub fn is_empty(&self) -> bool { + self.dense_storage.is_empty() && self.hash_map.is_empty() } - fn preregister_asset_loader(&mut self, extensions: &[&str]) -> &mut Self { - self.world - .resource_mut::() - .preregister_loader(extensions); - self + /// Returns the number of assets currently stored in the collection. + pub fn len(&self) -> usize { + self.dense_storage.len() + self.hash_map.len() } -} -/// Loads an internal asset from a project source file. -/// the file and its path are passed to the loader function, together with any additional parameters. -/// the resulting asset is stored in the app's asset server. -/// -/// Internal assets (e.g. shaders) are bundled directly into the app and can't be hot reloaded -/// using the conventional API. See [`DebugAssetServerPlugin`](crate::debug_asset_server::DebugAssetServerPlugin). -#[cfg(feature = "debug_asset_server")] -#[macro_export] -macro_rules! load_internal_asset { - ($app: ident, $handle: ident, $path_str: expr, $loader: expr) => {{ - { - let mut debug_app = $app - .world - .non_send_resource_mut::<$crate::debug_asset_server::DebugAssetApp>(); - $crate::debug_asset_server::register_handle_with_loader::<_, &'static str>( - $loader, - &mut debug_app, - $handle, - file!(), - $path_str, - ); - } - let mut assets = $app.world.resource_mut::<$crate::Assets<_>>(); - assets.set_untracked( - $handle, - ($loader)( - include_str!($path_str), - std::path::Path::new(file!()) - .parent() - .unwrap() - .join($path_str) - .to_string_lossy(), + /// Returns an iterator over the [`AssetId`] of every [`Asset`] stored in this collection. + pub fn ids(&self) -> impl Iterator> + '_ { + self.dense_storage + .ids() + .chain(self.hash_map.keys().map(|uuid| AssetId::from(*uuid))) + } + + /// Returns an iterator over the [`AssetId`] and [`Asset`] ref of every asset in this collection. + // PERF: this could be accelerated if we implement a skip list. Consider the cost/benefits + pub fn iter(&self) -> impl Iterator, &A)> { + self.dense_storage + .storage + .iter() + .enumerate() + .filter_map(|(i, v)| match v { + Entry::None => None, + Entry::Some { value, generation } => value.as_ref().map(|v| { + let id = AssetId::Index { + index: AssetIndex { + generation: *generation, + index: i as u32, + }, + marker: PhantomData, + }; + (id, v) + }), + }) + .chain( + self.hash_map + .iter() + .map(|(i, v)| (AssetId::Uuid { uuid: *i }, v)), ) - ); - }}; - // we can't support params without variadic arguments, so internal assets with additional params can't be hot-reloaded - ($app: ident, $handle: ident, $path_str: expr, $loader: expr $(, $param:expr)+) => {{ - let mut assets = $app.world.resource_mut::<$crate::Assets<_>>(); - assets.set_untracked( - $handle, - ($loader)( - include_str!($path_str), - std::path::Path::new(file!()) - .parent() - .unwrap() - .join($path_str) - .to_string_lossy(), - $($param),+ - ), - ); - }}; -} + } -/// Loads an internal asset from a project source file. -/// the file and its path are passed to the loader function, together with any additional parameters. -/// the resulting asset is stored in the app's asset server. -/// -/// Internal assets (e.g. shaders) are bundled directly into the app and can't be hot reloaded -/// using the conventional API. See `DebugAssetServerPlugin`. -#[cfg(not(feature = "debug_asset_server"))] -#[macro_export] -macro_rules! load_internal_asset { - ($app: ident, $handle: ident, $path_str: expr, $loader: expr $(, $param:expr)*) => {{ - let mut assets = $app.world.resource_mut::<$crate::Assets<_>>(); - assets.set_untracked( - $handle, - ($loader)( - include_str!($path_str), - std::path::Path::new(file!()) - .parent() - .unwrap() - .join($path_str) - .to_string_lossy(), - $($param),* - ), - ); - }}; -} + /// Returns an iterator over the [`AssetId`] and mutable [`Asset`] ref of every asset in this collection. + // PERF: this could be accelerated if we implement a skip list. Consider the cost/benefits + pub fn iter_mut(&mut self) -> AssetsMutIterator<'_, A> { + AssetsMutIterator { + dense_storage: self.dense_storage.storage.iter_mut().enumerate(), + hash_map: self.hash_map.iter_mut(), + queued_events: &mut self.queued_events, + } + } -/// Loads an internal binary asset. -/// -/// Internal binary assets (e.g. spir-v shaders) are bundled directly into the app and can't be hot reloaded -/// using the conventional API. See `DebugAssetServerPlugin`. -#[cfg(feature = "debug_asset_server")] -#[macro_export] -macro_rules! load_internal_binary_asset { - ($app: ident, $handle: ident, $path_str: expr, $loader: expr) => {{ - { - let mut debug_app = $app - .world - .non_send_resource_mut::<$crate::debug_asset_server::DebugAssetApp>(); - $crate::debug_asset_server::register_handle_with_loader::<_, &'static [u8]>( - $loader, - &mut debug_app, - $handle, - file!(), - $path_str, - ); + /// A system that synchronizes the state of assets in this collection with the [`AssetServer`]. This manages + /// [`Handle`] drop events and adds queued [`AssetEvent`] values to their [`Events`] resource. + /// + /// [`Events`]: bevy_ecs::event::Events + pub fn track_assets(mut assets: ResMut, asset_server: Res) { + let assets = &mut *assets; + // note that we must hold this lock for the entire duration of this function to ensure + // that `asset_server.load` calls that occur during it block, which ensures that + // re-loads are kicked off appropriately. This function must be "transactional" relative + // to other asset info operations + let mut infos = asset_server.data.infos.write(); + let mut not_ready = Vec::new(); + while let Ok(drop_event) = assets.handle_provider.drop_receiver.try_recv() { + let id = drop_event.id; + if !assets.contains(id.typed()) { + not_ready.push(drop_event); + continue; + } + if drop_event.asset_server_managed { + if infos.process_handle_drop(id.untyped(TypeId::of::())) { + assets.remove(id.typed()); + } + } else { + assets.remove(id.typed()); + } } - let mut assets = $app.world.resource_mut::<$crate::Assets<_>>(); - assets.set_untracked( - $handle, - ($loader)( - include_bytes!($path_str).as_ref(), - std::path::Path::new(file!()) - .parent() - .unwrap() - .join($path_str) - .to_string_lossy() - .into(), - ), - ); - }}; + // TODO: this is _extremely_ inefficient find a better fix + // This will also loop failed assets indefinitely. Is that ok? + for event in not_ready { + assets.handle_provider.drop_sender.send(event).unwrap(); + } + } + + /// A system that applies accumulated asset change events to the [`Events`] resource. + /// + /// [`Events`]: bevy_ecs::event::Events + pub fn asset_events(mut assets: ResMut, mut events: EventWriter>) { + events.send_batch(assets.queued_events.drain(..)); + } } -/// Loads an internal binary asset. -/// -/// Internal binary assets (e.g. spir-v shaders) are bundled directly into the app and can't be hot reloaded -/// using the conventional API. See `DebugAssetServerPlugin`. -#[cfg(not(feature = "debug_asset_server"))] -#[macro_export] -macro_rules! load_internal_binary_asset { - ($app: ident, $handle: ident, $path_str: expr, $loader: expr) => {{ - let mut assets = $app.world.resource_mut::<$crate::Assets<_>>(); - assets.set_untracked( - $handle, - ($loader)( - include_bytes!($path_str).as_ref(), - std::path::Path::new(file!()) - .parent() - .unwrap() - .join($path_str) - .to_string_lossy() - .into(), - ), - ); - }}; +/// A mutable iterator over [`Assets`]. +pub struct AssetsMutIterator<'a, A: Asset> { + queued_events: &'a mut Vec>, + dense_storage: Enumerate>>, + hash_map: bevy_utils::hashbrown::hash_map::IterMut<'a, Uuid, A>, } -#[cfg(test)] -mod tests { - use bevy_app::App; - - use crate::{AddAsset, Assets}; - - #[test] - fn asset_overwriting() { - #[derive(bevy_reflect::TypeUuid, bevy_reflect::TypePath)] - #[uuid = "44115972-f31b-46e5-be5c-2b9aece6a52f"] - struct MyAsset; - let mut app = App::new(); - app.add_plugins(( - bevy_core::TaskPoolPlugin::default(), - bevy_core::TypeRegistrationPlugin, - crate::AssetPlugin::default(), - )); - app.add_asset::(); - let mut assets_before = app.world.resource_mut::>(); - let handle = assets_before.add(MyAsset); - app.add_asset::(); // Ensure this doesn't overwrite the Asset - let assets_after = app.world.resource_mut::>(); - assert!(assets_after.get(&handle).is_some()); +impl<'a, A: Asset> Iterator for AssetsMutIterator<'a, A> { + type Item = (AssetId, &'a mut A); + + fn next(&mut self) -> Option { + for (i, entry) in &mut self.dense_storage { + match entry { + Entry::None => { + continue; + } + Entry::Some { value, generation } => { + let id = AssetId::Index { + index: AssetIndex { + generation: *generation, + index: i as u32, + }, + marker: PhantomData, + }; + self.queued_events.push(AssetEvent::Modified { id }); + if let Some(value) = value { + return Some((id, value)); + } + } + } + } + if let Some((key, value)) = self.hash_map.next() { + let id = AssetId::Uuid { uuid: *key }; + self.queued_events.push(AssetEvent::Modified { id }); + Some((id, value)) + } else { + None + } } } + +#[derive(Error, Debug)] +#[error("AssetIndex {index:?} has an invalid generation. The current generation is: '{current_generation}'.")] +pub struct InvalidGenerationError { + index: AssetIndex, + current_generation: u32, +} diff --git a/crates/bevy_asset/src/debug_asset_server.rs b/crates/bevy_asset/src/debug_asset_server.rs deleted file mode 100644 index 3819b1005a312..0000000000000 --- a/crates/bevy_asset/src/debug_asset_server.rs +++ /dev/null @@ -1,143 +0,0 @@ -//! Support for hot reloading internal assets. -//! -//! Internal assets (e.g. shaders) are bundled directly into an application and can't be hot -//! reloaded using the conventional API. -use bevy_app::{App, Plugin, Update}; -use bevy_ecs::{prelude::*, system::SystemState}; -use bevy_tasks::{IoTaskPool, TaskPoolBuilder}; -use bevy_utils::{Duration, HashMap}; -use std::{ - ops::{Deref, DerefMut}, - path::Path, -}; - -use crate::{ - Asset, AssetEvent, AssetPlugin, AssetServer, Assets, ChangeWatcher, FileAssetIo, Handle, - HandleUntyped, -}; - -/// A helper [`App`] used for hot reloading internal assets, which are compiled-in to Bevy plugins. -pub struct DebugAssetApp(App); - -impl Deref for DebugAssetApp { - type Target = App; - - fn deref(&self) -> &Self::Target { - &self.0 - } -} - -impl DerefMut for DebugAssetApp { - fn deref_mut(&mut self) -> &mut Self::Target { - &mut self.0 - } -} - -/// A set containing the system that runs [`DebugAssetApp`]. -#[derive(SystemSet, Debug, Clone, PartialEq, Eq, Hash)] -pub struct DebugAssetAppRun; - -/// Facilitates the creation of a "debug asset app", whose sole responsibility is hot reloading -/// assets that are "internal" / compiled-in to Bevy Plugins. -/// -/// Pair with the [`load_internal_asset`](crate::load_internal_asset) macro to load hot-reloadable -/// assets. The `debug_asset_server` feature flag must also be enabled for hot reloading to work. -/// Currently only hot reloads assets stored in the `crates` folder. -#[derive(Default)] -pub struct DebugAssetServerPlugin; - -/// A collection that maps internal assets in a [`DebugAssetApp`]'s asset server to their mirrors in -/// the main [`App`]. -#[derive(Resource)] -pub struct HandleMap { - /// The collection of asset handles. - pub handles: HashMap, Handle>, -} - -impl Default for HandleMap { - fn default() -> Self { - Self { - handles: Default::default(), - } - } -} - -impl Plugin for DebugAssetServerPlugin { - fn build(&self, app: &mut bevy_app::App) { - IoTaskPool::init(|| { - TaskPoolBuilder::default() - .num_threads(2) - .thread_name("Debug Asset Server IO Task Pool".to_string()) - .build() - }); - let mut debug_asset_app = App::new(); - debug_asset_app.add_plugins(AssetPlugin { - asset_folder: "crates".to_string(), - watch_for_changes: ChangeWatcher::with_delay(Duration::from_millis(200)), - }); - app.insert_non_send_resource(DebugAssetApp(debug_asset_app)); - app.add_systems(Update, run_debug_asset_app); - } -} - -fn run_debug_asset_app(mut debug_asset_app: NonSendMut) { - debug_asset_app.0.update(); -} - -pub(crate) fn sync_debug_assets( - mut debug_asset_app: NonSendMut, - mut assets: ResMut>, -) { - let world = &mut debug_asset_app.0.world; - let mut state = SystemState::<( - Res>>, - Res>, - Res>, - )>::new(world); - let (changed_shaders, handle_map, debug_assets) = state.get_mut(world); - for changed in changed_shaders.iter_current_update_events() { - let debug_handle = match changed { - AssetEvent::Created { handle } | AssetEvent::Modified { handle } => handle, - AssetEvent::Removed { .. } => continue, - }; - if let Some(handle) = handle_map.handles.get(debug_handle) { - if let Some(debug_asset) = debug_assets.get(debug_handle) { - assets.set_untracked(handle, debug_asset.clone()); - } - } - } -} - -/// Uses the return type of the given loader to register the given handle with the appropriate type -/// and load the asset with the given `path` and parent `file_path`. -/// -/// If this feels a bit odd ... that's because it is. This was built to improve the UX of the -/// `load_internal_asset` macro. -pub fn register_handle_with_loader( - _loader: fn(T, String) -> A, - app: &mut DebugAssetApp, - handle: HandleUntyped, - file_path: &str, - path: &'static str, -) { - let mut state = SystemState::<(ResMut>, Res)>::new(&mut app.world); - let manifest_dir = std::env::var("CARGO_MANIFEST_DIR").unwrap(); - let manifest_dir_path = Path::new(&manifest_dir); - let (mut handle_map, asset_server) = state.get_mut(&mut app.world); - let asset_io = asset_server - .asset_io() - .downcast_ref::() - .expect("The debug AssetServer only works with FileAssetIo-backed AssetServers"); - let absolute_file_path = manifest_dir_path.join( - Path::new(file_path) - .parent() - .expect("file path must have a parent"), - ); - let asset_folder_relative_path = absolute_file_path - .strip_prefix(asset_io.root_path()) - .expect("The AssetIo root path should be a prefix of the absolute file path"); - handle_map.handles.insert( - asset_server.load(asset_folder_relative_path.join(path)), - handle.clone_weak().typed::(), - ); -} diff --git a/crates/bevy_asset/src/diagnostic/asset_count_diagnostics_plugin.rs b/crates/bevy_asset/src/diagnostic/asset_count_diagnostics_plugin.rs deleted file mode 100644 index dafd21567347b..0000000000000 --- a/crates/bevy_asset/src/diagnostic/asset_count_diagnostics_plugin.rs +++ /dev/null @@ -1,60 +0,0 @@ -use crate::{Asset, Assets}; -use bevy_app::prelude::*; -use bevy_diagnostic::{ - Diagnostic, DiagnosticId, Diagnostics, DiagnosticsStore, MAX_DIAGNOSTIC_NAME_WIDTH, -}; -use bevy_ecs::prelude::*; - -/// Adds an asset count diagnostic to an [`App`] for assets of type `T`. -pub struct AssetCountDiagnosticsPlugin { - marker: std::marker::PhantomData, -} - -impl Default for AssetCountDiagnosticsPlugin { - fn default() -> Self { - Self { - marker: std::marker::PhantomData, - } - } -} - -impl Plugin for AssetCountDiagnosticsPlugin { - fn build(&self, app: &mut App) { - app.add_systems(Startup, Self::setup_system) - .add_systems(Update, Self::diagnostic_system); - } -} - -impl AssetCountDiagnosticsPlugin { - /// Gets unique id of this diagnostic. - /// - /// The diagnostic id is the type uuid of `T`. - pub fn diagnostic_id() -> DiagnosticId { - DiagnosticId(T::TYPE_UUID) - } - - /// Registers the asset count diagnostic for the current application. - pub fn setup_system(mut diagnostics: ResMut) { - let asset_type_name = std::any::type_name::(); - let max_length = MAX_DIAGNOSTIC_NAME_WIDTH - "asset_count ".len(); - diagnostics.add(Diagnostic::new( - Self::diagnostic_id(), - format!( - "asset_count {}", - if asset_type_name.len() > max_length { - asset_type_name - .split_at(asset_type_name.len() - max_length + 1) - .1 - } else { - asset_type_name - } - ), - 20, - )); - } - - /// Updates the asset count of `T` assets. - pub fn diagnostic_system(mut diagnostics: Diagnostics, assets: Res>) { - diagnostics.add_measurement(Self::diagnostic_id(), || assets.len() as f64); - } -} diff --git a/crates/bevy_asset/src/diagnostic/mod.rs b/crates/bevy_asset/src/diagnostic/mod.rs deleted file mode 100644 index 6bc1ed390c881..0000000000000 --- a/crates/bevy_asset/src/diagnostic/mod.rs +++ /dev/null @@ -1,4 +0,0 @@ -//! Diagnostic providers for `bevy_diagnostic`. - -mod asset_count_diagnostics_plugin; -pub use asset_count_diagnostics_plugin::AssetCountDiagnosticsPlugin; diff --git a/crates/bevy_asset/src/event.rs b/crates/bevy_asset/src/event.rs new file mode 100644 index 0000000000000..96f10a9c6f5ab --- /dev/null +++ b/crates/bevy_asset/src/event.rs @@ -0,0 +1,77 @@ +use crate::{Asset, AssetId}; +use bevy_ecs::event::Event; +use std::fmt::Debug; + +/// Events that occur for a specific [`Asset`], such as "value changed" events and "dependency" events. +#[derive(Event)] +pub enum AssetEvent { + /// Emitted whenever an [`Asset`] is added. + Added { id: AssetId }, + /// Emitted whenever an [`Asset`] value is modified. + Modified { id: AssetId }, + /// Emitted whenever an [`Asset`] is removed. + Removed { id: AssetId }, + /// Emitted whenever an [`Asset`] has been fully loaded (including its dependencies and all "recursive dependencies"). + LoadedWithDependencies { id: AssetId }, +} + +impl AssetEvent { + /// Returns `true` if this event is [`AssetEvent::LoadedWithDependencies`] and matches the given `id`. + pub fn is_loaded_with_dependencies(&self, asset_id: impl Into>) -> bool { + matches!(self, AssetEvent::LoadedWithDependencies { id } if *id == asset_id.into()) + } + + /// Returns `true` if this event is [`AssetEvent::Added`] and matches the given `id`. + pub fn is_added(&self, asset_id: impl Into>) -> bool { + matches!(self, AssetEvent::Added { id } if *id == asset_id.into()) + } + + /// Returns `true` if this event is [`AssetEvent::Modified`] and matches the given `id`. + pub fn is_modified(&self, asset_id: impl Into>) -> bool { + matches!(self, AssetEvent::Modified { id } if *id == asset_id.into()) + } + + /// Returns `true` if this event is [`AssetEvent::Removed`] and matches the given `id`. + pub fn is_removed(&self, asset_id: impl Into>) -> bool { + matches!(self, AssetEvent::Removed { id } if *id == asset_id.into()) + } +} + +impl Clone for AssetEvent { + fn clone(&self) -> Self { + *self + } +} + +impl Copy for AssetEvent {} + +impl Debug for AssetEvent { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::Added { id } => f.debug_struct("Added").field("id", id).finish(), + Self::Modified { id } => f.debug_struct("Modified").field("id", id).finish(), + Self::Removed { id } => f.debug_struct("Removed").field("id", id).finish(), + Self::LoadedWithDependencies { id } => f + .debug_struct("LoadedWithDependencies") + .field("id", id) + .finish(), + } + } +} + +impl PartialEq for AssetEvent { + fn eq(&self, other: &Self) -> bool { + match (self, other) { + (Self::Added { id: l_id }, Self::Added { id: r_id }) + | (Self::Modified { id: l_id }, Self::Modified { id: r_id }) + | (Self::Removed { id: l_id }, Self::Removed { id: r_id }) + | ( + Self::LoadedWithDependencies { id: l_id }, + Self::LoadedWithDependencies { id: r_id }, + ) => l_id == r_id, + _ => false, + } + } +} + +impl Eq for AssetEvent {} diff --git a/crates/bevy_asset/src/filesystem_watcher.rs b/crates/bevy_asset/src/filesystem_watcher.rs deleted file mode 100644 index fa1b9c173e89b..0000000000000 --- a/crates/bevy_asset/src/filesystem_watcher.rs +++ /dev/null @@ -1,46 +0,0 @@ -use bevy_utils::{default, Duration, HashMap, HashSet}; -use crossbeam_channel::Receiver; -use notify::{Event, RecommendedWatcher, RecursiveMode, Result, Watcher}; -use std::path::{Path, PathBuf}; - -use crate::ChangeWatcher; - -/// Watches for changes to files on the local filesystem. -/// -/// When hot-reloading is enabled, the [`AssetServer`](crate::AssetServer) uses this to reload -/// assets when their source files are modified. -pub struct FilesystemWatcher { - pub watcher: RecommendedWatcher, - pub receiver: Receiver>, - pub path_map: HashMap>, - pub delay: Duration, -} - -impl FilesystemWatcher { - pub fn new(configuration: &ChangeWatcher) -> Self { - let (sender, receiver) = crossbeam_channel::unbounded(); - let watcher: RecommendedWatcher = RecommendedWatcher::new( - move |res| { - sender.send(res).expect("Watch event send failure."); - }, - default(), - ) - .expect("Failed to create filesystem watcher."); - FilesystemWatcher { - watcher, - receiver, - path_map: default(), - delay: configuration.delay, - } - } - - /// Watch for changes recursively at the provided path. - pub fn watch>(&mut self, to_watch: P, to_reload: PathBuf) -> Result<()> { - self.path_map - .entry(to_watch.as_ref().to_owned()) - .or_default() - .insert(to_reload); - self.watcher - .watch(to_watch.as_ref(), RecursiveMode::Recursive) - } -} diff --git a/crates/bevy_asset/src/folder.rs b/crates/bevy_asset/src/folder.rs new file mode 100644 index 0000000000000..e2c6b0ca2309c --- /dev/null +++ b/crates/bevy_asset/src/folder.rs @@ -0,0 +1,12 @@ +use crate as bevy_asset; +use crate::{Asset, UntypedHandle}; +use bevy_reflect::TypePath; + +/// A "loaded folder" containing handles for all assets stored in a given [`AssetPath`]. +/// +/// [`AssetPath`]: crate::AssetPath +#[derive(Asset, TypePath)] +pub struct LoadedFolder { + #[dependency] + pub handles: Vec, +} diff --git a/crates/bevy_asset/src/handle.rs b/crates/bevy_asset/src/handle.rs index b0da1df00d45c..81de4456b4a78 100644 --- a/crates/bevy_asset/src/handle.rs +++ b/crates/bevy_asset/src/handle.rs @@ -1,472 +1,387 @@ +use crate::{ + meta::MetaTransform, Asset, AssetId, AssetIndexAllocator, AssetPath, InternalAssetId, + UntypedAssetId, +}; +use bevy_ecs::prelude::*; +use bevy_reflect::{Reflect, TypePath, Uuid}; +use bevy_utils::get_short_name; +use crossbeam_channel::{Receiver, Sender}; use std::{ - cmp::Ordering, - fmt::Debug, + any::TypeId, hash::{Hash, Hasher}, - marker::PhantomData, + sync::Arc, }; -use crate::{ - path::{AssetPath, AssetPathId}, - Asset, Assets, -}; -use bevy_ecs::{component::Component, reflect::ReflectComponent}; -use bevy_reflect::{std_traits::ReflectDefault, Reflect, ReflectDeserialize, ReflectSerialize}; -use bevy_utils::Uuid; -use crossbeam_channel::{Receiver, Sender}; -use serde::{Deserialize, Serialize}; - -/// A unique, stable asset id. -#[derive( - Debug, Clone, Copy, Eq, PartialEq, Hash, Ord, PartialOrd, Serialize, Deserialize, Reflect, -)] -#[reflect_value(Serialize, Deserialize, PartialEq, Hash)] -pub enum HandleId { - /// A handle id of a loaded asset. - Id(Uuid, u64), - - /// A handle id of a pending asset. - AssetPathId(AssetPathId), +/// Provides [`Handle`] and [`UntypedHandle`] _for a specific asset type_. +/// This should _only_ be used for one specific asset type. +#[derive(Clone)] +pub struct AssetHandleProvider { + pub(crate) allocator: Arc, + pub(crate) drop_sender: Sender, + pub(crate) drop_receiver: Receiver, + pub(crate) type_id: TypeId, } -impl From for HandleId { - fn from(value: AssetPathId) -> Self { - HandleId::AssetPathId(value) - } +pub(crate) struct DropEvent { + pub(crate) id: InternalAssetId, + pub(crate) asset_server_managed: bool, } -impl<'a> From> for HandleId { - fn from(value: AssetPath<'a>) -> Self { - HandleId::AssetPathId(AssetPathId::from(value)) +impl AssetHandleProvider { + pub(crate) fn new(type_id: TypeId, allocator: Arc) -> Self { + let (drop_sender, drop_receiver) = crossbeam_channel::unbounded(); + Self { + type_id, + allocator, + drop_sender, + drop_receiver, + } } -} -impl<'a, 'b> From<&'a AssetPath<'b>> for HandleId { - fn from(value: &'a AssetPath<'b>) -> Self { - HandleId::AssetPathId(AssetPathId::from(value)) + /// Reserves a new strong [`UntypedHandle`] (with a new [`UntypedAssetId`]). The stored [`Asset`] [`TypeId`] in the + /// [`UntypedHandle`] will match the [`Asset`] [`TypeId`] assigned to this [`AssetHandleProvider`]. + pub fn reserve_handle(&self) -> UntypedHandle { + let index = self.allocator.reserve(); + UntypedHandle::Strong(self.get_handle(InternalAssetId::Index(index), false, None, None)) + } + + pub(crate) fn get_handle( + &self, + id: InternalAssetId, + asset_server_managed: bool, + path: Option>, + meta_transform: Option, + ) -> Arc { + Arc::new(StrongHandle { + id: id.untyped(self.type_id), + drop_sender: self.drop_sender.clone(), + meta_transform, + path, + asset_server_managed, + }) + } + + pub(crate) fn reserve_handle_internal( + &self, + asset_server_managed: bool, + path: Option>, + meta_transform: Option, + ) -> Arc { + let index = self.allocator.reserve(); + self.get_handle( + InternalAssetId::Index(index), + asset_server_managed, + path, + meta_transform, + ) + } +} + +/// The internal "strong" [`Asset`] handle storage for [`Handle::Strong`] and [`UntypedHandle::Strong`]. When this is dropped, +/// the [`Asset`] will be freed. It also stores some asset metadata for easy access from handles. +#[derive(TypePath)] +pub struct StrongHandle { + pub(crate) id: UntypedAssetId, + pub(crate) asset_server_managed: bool, + pub(crate) path: Option>, + /// Modifies asset meta. This is stored on the handle because it is: + /// 1. configuration tied to the lifetime of a specific asset load + /// 2. configuration that must be repeatable when the asset is hot-reloaded + pub(crate) meta_transform: Option, + pub(crate) drop_sender: Sender, +} + +impl Drop for StrongHandle { + fn drop(&mut self) { + if let Err(err) = self.drop_sender.send(DropEvent { + id: self.id.internal(), + asset_server_managed: self.asset_server_managed, + }) { + println!("Failed to send DropEvent for StrongHandle {:?}", err); + } } } -impl HandleId { - /// Creates a random id for an asset of type `T`. - #[inline] - pub fn random() -> Self { - HandleId::Id(T::TYPE_UUID, fastrand::u64(..)) - } - - /// Creates the default id for an asset of type `T`. - #[inline] - #[allow(clippy::should_implement_trait)] // `Default` is not implemented for `HandleId`, the default value depends on the asset type - pub fn default() -> Self { - HandleId::Id(T::TYPE_UUID, 0) - } - - /// Creates an arbitrary asset id without an explicit type bound. - #[inline] - pub const fn new(type_uuid: Uuid, id: u64) -> Self { - HandleId::Id(type_uuid, id) +impl std::fmt::Debug for StrongHandle { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("StrongHandle") + .field("id", &self.id) + .field("asset_server_managed", &self.asset_server_managed) + .field("path", &self.path) + .field("drop_sender", &self.drop_sender) + .finish() } } -/// A handle into a specific [`Asset`] of type `T`. -/// -/// Handles contain a unique id that corresponds to a specific asset in the [`Assets`] collection. -/// -/// # Accessing the Asset -/// -/// A handle is _not_ the asset itself, but should be seen as a pointer to the asset. Modifying a -/// handle's `id` only modifies which asset is being pointed to. To get the actual asset, try using -/// [`Assets::get`] or [`Assets::get_mut`]. +/// A strong or weak handle to a specific [`Asset`]. If a [`Handle`] is [`Handle::Strong`], the [`Asset`] will be kept +/// alive until the [`Handle`] is dropped. If a [`Handle`] is [`Handle::Weak`], it does not necessarily reference a live [`Asset`], +/// nor will it keep assets alive. /// -/// # Strong and Weak -/// -/// A handle can be either "Strong" or "Weak". Simply put: Strong handles keep the asset loaded, -/// while Weak handles do not affect the loaded status of assets. This is due to a type of -/// _reference counting_. When the number of Strong handles that exist for any given asset reach -/// zero, the asset is dropped and becomes unloaded. In some cases, you might want a reference to an -/// asset but don't want to take the responsibility of keeping it loaded that comes with a Strong handle. -/// This is where a Weak handle can be very useful. -/// -/// For example, imagine you have a `Sprite` component and a `Collider` component. The `Collider` uses -/// the `Sprite`'s image size to check for collisions. It does so by keeping a Weak copy of the -/// `Sprite`'s Strong handle to the image asset. -/// -/// If the `Sprite` is removed, its Strong handle to the image is dropped with it. And since it was the -/// only Strong handle for that asset, the asset is unloaded. Our `Collider` component still has a Weak -/// handle to the unloaded asset, but it will not be able to retrieve the image data, resulting in -/// collisions no longer being detected for that entity. +/// [`Handle`] can be cloned. If a [`Handle::Strong`] is cloned, the referenced [`Asset`] will not be freed until _all_ instances +/// of the [`Handle`] are dropped. /// +/// [`Handle::Strong`] also provides access to useful [`Asset`] metadata, such as the [`AssetPath`] (if it exists). #[derive(Component, Reflect)] -#[reflect(Component, Default)] -pub struct Handle -where - T: Asset, -{ - id: HandleId, - #[reflect(ignore)] - handle_type: HandleType, - #[reflect(ignore)] - // NOTE: PhantomData T> gives this safe Send/Sync impls - marker: PhantomData T>, -} - -#[derive(Default)] -enum HandleType { - #[default] - Weak, - Strong(Sender), +#[reflect(Component)] +pub enum Handle { + /// A "strong" reference to a live (or loading) [`Asset`]. If a [`Handle`] is [`Handle::Strong`], the [`Asset`] will be kept + /// alive until the [`Handle`] is dropped. Strong handles also provide access to additional asset metadata. + Strong(Arc), + /// A "weak" reference to an [`Asset`]. If a [`Handle`] is [`Handle::Weak`], it does not necessarily reference a live [`Asset`], + /// nor will it keep assets alive. + Weak(AssetId), } -impl Debug for HandleType { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { +impl Clone for Handle { + fn clone(&self) -> Self { match self { - HandleType::Weak => f.write_str("Weak"), - HandleType::Strong(_) => f.write_str("Strong"), + Handle::Strong(handle) => Handle::Strong(handle.clone()), + Handle::Weak(id) => Handle::Weak(*id), } } } -impl Handle { - pub(crate) fn strong(id: HandleId, ref_change_sender: Sender) -> Self { - ref_change_sender.send(RefChange::Increment(id)).unwrap(); - Self { - id, - handle_type: HandleType::Strong(ref_change_sender), - marker: PhantomData, - } +impl Handle { + /// Create a new [`Handle::Weak`] with the given [`u128`] encoding of a [`Uuid`]. + pub const fn weak_from_u128(value: u128) -> Self { + Handle::Weak(AssetId::Uuid { + uuid: Uuid::from_u128(value), + }) } - /// Creates a weak handle into an Asset identified by `id`. + /// Returns the [`AssetId`] of this [`Asset`]. #[inline] - pub fn weak(id: HandleId) -> Self { - Self { - id, - handle_type: HandleType::Weak, - marker: PhantomData, + pub fn id(&self) -> AssetId { + match self { + Handle::Strong(handle) => handle.id.typed_unchecked(), + Handle::Weak(id) => *id, } } - /// The ID of the asset as contained within its respective [`Assets`] collection. + /// Returns the path if this is (1) a strong handle and (2) the asset has a path #[inline] - pub fn id(&self) -> HandleId { - self.id - } - - /// Recasts this handle as a weak handle of an Asset `U`. - pub fn cast_weak(&self) -> Handle { - let id = if let HandleId::Id(_, id) = self.id { - HandleId::Id(U::TYPE_UUID, id) - } else { - self.id - }; - - Handle { - id, - handle_type: HandleType::Weak, - marker: PhantomData, + pub fn path(&self) -> Option<&AssetPath<'static>> { + match self { + Handle::Strong(handle) => handle.path.as_ref(), + Handle::Weak(_) => None, } } /// Returns `true` if this is a weak handle. + #[inline] pub fn is_weak(&self) -> bool { - matches!(self.handle_type, HandleType::Weak) + matches!(self, Handle::Weak(_)) } /// Returns `true` if this is a strong handle. + #[inline] pub fn is_strong(&self) -> bool { - matches!(self.handle_type, HandleType::Strong(_)) + matches!(self, Handle::Strong(_)) } - /// Makes this handle Strong if it wasn't already. - /// - /// This method requires the corresponding [`Assets`](crate::Assets) collection. - pub fn make_strong(&mut self, assets: &Assets) { - if self.is_strong() { - return; - } - let sender = assets.ref_change_sender.clone(); - sender.send(RefChange::Increment(self.id)).unwrap(); - self.handle_type = HandleType::Strong(sender); - } - - /// Creates a weak copy of this handle. + /// Creates a [`Handle::Weak`] clone of this [`Handle`], which will not keep the referenced [`Asset`] alive. #[inline] - #[must_use] pub fn clone_weak(&self) -> Self { - Self::weak(self.id) - } - - /// Creates an untyped copy of this handle. - pub fn clone_untyped(&self) -> HandleUntyped { - match &self.handle_type { - HandleType::Strong(sender) => HandleUntyped::strong(self.id, sender.clone()), - HandleType::Weak => HandleUntyped::weak(self.id), + match self { + Handle::Strong(handle) => Handle::Weak(handle.id.typed_unchecked::()), + Handle::Weak(id) => Handle::Weak(*id), } } - /// Creates a weak, untyped copy of this handle. - pub fn clone_weak_untyped(&self) -> HandleUntyped { - HandleUntyped::weak(self.id) - } -} - -impl Drop for Handle { - fn drop(&mut self) { - match self.handle_type { - HandleType::Strong(ref sender) => { - // ignore send errors because this means the channel is shut down / the game has - // stopped - let _ = sender.send(RefChange::Decrement(self.id)); - } - HandleType::Weak => {} + /// Converts this [`Handle`] to an "untyped" / "generic-less" [`UntypedHandle`], which stores the [`Asset`] type information + /// _inside_ [`UntypedHandle`]. This will return [`UntypedHandle::Strong`] for [`Handle::Strong`] and [`UntypedHandle::Weak`] for + /// [`Handle::Weak`]. + #[inline] + pub fn untyped(self) -> UntypedHandle { + match self { + Handle::Strong(handle) => UntypedHandle::Strong(handle), + Handle::Weak(id) => UntypedHandle::Weak(id.untyped()), } } } -impl From> for HandleId { - fn from(value: Handle) -> Self { - value.id - } -} - -impl From for HandleId { - fn from(value: HandleUntyped) -> Self { - value.id - } -} - -impl From<&str> for HandleId { - fn from(value: &str) -> Self { - AssetPathId::from(value).into() - } -} - -impl From<&String> for HandleId { - fn from(value: &String) -> Self { - AssetPathId::from(value).into() - } -} - -impl From for HandleId { - fn from(value: String) -> Self { - AssetPathId::from(&value).into() +impl Default for Handle { + fn default() -> Self { + Handle::Weak(AssetId::default()) } } -impl From<&Handle> for HandleId { - fn from(value: &Handle) -> Self { - value.id +impl std::fmt::Debug for Handle { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let name = get_short_name(std::any::type_name::()); + match self { + Handle::Strong(handle) => { + write!( + f, + "StrongHandle<{name}>{{ id: {:?}, path: {:?} }}", + handle.id.internal(), + handle.path + ) + } + Handle::Weak(id) => write!(f, "WeakHandle<{name}>({:?})", id.internal()), + } } } -impl Hash for Handle { +impl Hash for Handle { + #[inline] fn hash(&self, state: &mut H) { - Hash::hash(&self.id, state); - } -} - -impl PartialEq for Handle { - fn eq(&self, other: &Self) -> bool { - self.id == other.id + Hash::hash(&self.id(), state); } } -impl Eq for Handle {} - -impl PartialOrd for Handle { - fn partial_cmp(&self, other: &Self) -> Option { - Some(self.cmp(other)) +impl PartialOrd for Handle { + fn partial_cmp(&self, other: &Self) -> Option { + Some(self.id().cmp(&other.id())) } } -impl Ord for Handle { - fn cmp(&self, other: &Self) -> Ordering { - self.id.cmp(&other.id) +impl Ord for Handle { + fn cmp(&self, other: &Self) -> std::cmp::Ordering { + self.id().cmp(&other.id()) } } -impl Default for Handle { - fn default() -> Self { - Handle::weak(HandleId::default::()) +impl PartialEq for Handle { + #[inline] + fn eq(&self, other: &Self) -> bool { + self.id() == other.id() } } -impl Debug for Handle { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::result::Result<(), std::fmt::Error> { - let name = std::any::type_name::().split("::").last().unwrap(); - write!(f, "{:?}Handle<{name}>({:?})", self.handle_type, self.id) - } -} +impl Eq for Handle {} -impl Clone for Handle { - fn clone(&self) -> Self { - match self.handle_type { - HandleType::Strong(ref sender) => Handle::strong(self.id, sender.clone()), - HandleType::Weak => Handle::weak(self.id), - } - } -} - -/// A non-generic version of [`Handle`]. +/// An untyped variant of [`Handle`], which internally stores the [`Asset`] type information at runtime +/// as a [`TypeId`] instead of encoding it in the compile-time type. This allows handles across [`Asset`] types +/// to be stored together and compared. /// -/// This allows handles to be mingled in a cross asset context. For example, storing `Handle` and -/// `Handle` in the same `HashSet`. -/// -/// To convert back to a typed handle, use the [typed](HandleUntyped::typed) method. -#[derive(Debug)] -pub struct HandleUntyped { - id: HandleId, - handle_type: HandleType, +/// See [`Handle`] for more information. +#[derive(Clone)] +pub enum UntypedHandle { + Strong(Arc), + Weak(UntypedAssetId), } -impl HandleUntyped { - /// Creates a weak untyped handle with an arbitrary id. - pub const fn weak_from_u64(uuid: Uuid, id: u64) -> Self { - Self { - id: HandleId::new(uuid, id), - handle_type: HandleType::Weak, +impl UntypedHandle { + /// Returns the [`UntypedAssetId`] for the referenced asset. + #[inline] + pub fn id(&self) -> UntypedAssetId { + match self { + UntypedHandle::Strong(handle) => handle.id, + UntypedHandle::Weak(id) => *id, } } - pub(crate) fn strong(id: HandleId, ref_change_sender: Sender) -> Self { - ref_change_sender.send(RefChange::Increment(id)).unwrap(); - Self { - id, - handle_type: HandleType::Strong(ref_change_sender), + /// Returns the path if this is (1) a strong handle and (2) the asset has a path + #[inline] + pub fn path(&self) -> Option<&AssetPath<'static>> { + match self { + UntypedHandle::Strong(handle) => handle.path.as_ref(), + UntypedHandle::Weak(_) => None, } } - /// Create a weak, untyped handle into an Asset identified by `id`. - pub fn weak(id: HandleId) -> Self { - Self { - id, - handle_type: HandleType::Weak, + /// Creates an [`UntypedHandle::Weak`] clone of this [`UntypedHandle`], which will not keep the referenced [`Asset`] alive. + #[inline] + pub fn clone_weak(&self) -> UntypedHandle { + match self { + UntypedHandle::Strong(handle) => UntypedHandle::Weak(handle.id), + UntypedHandle::Weak(id) => UntypedHandle::Weak(*id), } } - /// The ID of the asset. + /// Returns the [`TypeId`] of the referenced [`Asset`]. #[inline] - pub fn id(&self) -> HandleId { - self.id - } - - /// Creates a weak copy of this handle. - #[must_use] - pub fn clone_weak(&self) -> Self { - Self::weak(self.id) - } - - /// Returns `true` if this is a weak handle. - pub fn is_weak(&self) -> bool { - matches!(self.handle_type, HandleType::Weak) - } - - /// Returns `true` if this is a strong handle. - pub fn is_strong(&self) -> bool { - matches!(self.handle_type, HandleType::Strong(_)) - } - - /// Create a weak typed [`Handle`] from this handle. - /// - /// If this handle is strong and dropped, there is no guarantee that the asset - /// will still be available (if only the returned handle is kept) - pub fn typed_weak(&self) -> Handle { - self.clone_weak().typed() - } - - /// Converts this handle into a typed [`Handle`] of an [`Asset`] `T`. - /// - /// The new handle will maintain the Strong or Weak status of the current handle. - /// - /// # Panics - /// - /// Will panic if type `T` doesn't match this handle's actual asset type. - pub fn typed(mut self) -> Handle { - if let HandleId::Id(type_uuid, _) = self.id { - assert!( - T::TYPE_UUID == type_uuid, - "Attempted to convert handle to invalid type." - ); - } - let handle_type = match &self.handle_type { - HandleType::Strong(sender) => HandleType::Strong(sender.clone()), - HandleType::Weak => HandleType::Weak, - }; - // ensure we don't send the RefChange event when "self" is dropped - self.handle_type = HandleType::Weak; - Handle { - handle_type, - id: self.id, - marker: PhantomData, + pub fn type_id(&self) -> TypeId { + match self { + UntypedHandle::Strong(handle) => handle.id.type_id(), + UntypedHandle::Weak(id) => id.type_id(), } } -} -impl Drop for HandleUntyped { - fn drop(&mut self) { - match self.handle_type { - HandleType::Strong(ref sender) => { - // ignore send errors because this means the channel is shut down / the game has - // stopped - let _ = sender.send(RefChange::Decrement(self.id)); - } - HandleType::Weak => {} + /// Converts to a typed Handle. This _will not check if the target Handle type matches_. + #[inline] + pub fn typed_unchecked(self) -> Handle { + match self { + UntypedHandle::Strong(handle) => Handle::Strong(handle), + UntypedHandle::Weak(id) => Handle::Weak(id.typed_unchecked::()), } } -} -impl From> for HandleUntyped { - fn from(mut handle: Handle) -> Self { - let handle_type = std::mem::replace(&mut handle.handle_type, HandleType::Weak); - HandleUntyped { - id: handle.id, - handle_type, + /// Converts to a typed Handle. This will check the type when compiled with debug asserts, but it + /// _will not check if the target Handle type matches in release builds_. Use this as an optimization + /// when you want some degree of validation at dev-time, but you are also very certain that the type + /// actually matches. + #[inline] + pub fn typed_debug_checked(self) -> Handle { + debug_assert_eq!( + self.type_id(), + TypeId::of::(), + "The target Handle's TypeId does not match the TypeId of this UntypedHandle" + ); + match self { + UntypedHandle::Strong(handle) => Handle::Strong(handle), + UntypedHandle::Weak(id) => Handle::Weak(id.typed_unchecked::()), } } -} -impl From<&HandleUntyped> for HandleId { - fn from(value: &HandleUntyped) -> Self { - value.id + /// Converts to a typed Handle. This will panic if the internal [`TypeId`] does not match the given asset type `A` + #[inline] + pub fn typed(self) -> Handle { + assert_eq!( + self.type_id(), + TypeId::of::(), + "The target Handle's TypeId does not match the TypeId of this UntypedHandle" + ); + self.typed_unchecked() } -} -impl Hash for HandleUntyped { - fn hash(&self, state: &mut H) { - Hash::hash(&self.id, state); + /// The "meta transform" for the strong handle. This will only be [`Some`] if the handle is strong and there is a meta transform + /// associated with it. + #[inline] + pub fn meta_transform(&self) -> Option<&MetaTransform> { + match self { + UntypedHandle::Strong(handle) => handle.meta_transform.as_ref(), + UntypedHandle::Weak(_) => None, + } } } -impl PartialEq for HandleUntyped { +impl PartialEq for UntypedHandle { + #[inline] fn eq(&self, other: &Self) -> bool { - self.id == other.id + self.id() == other.id() && self.type_id() == other.type_id() } } -impl Eq for HandleUntyped {} +impl Eq for UntypedHandle {} -impl Clone for HandleUntyped { - fn clone(&self) -> Self { - match self.handle_type { - HandleType::Strong(ref sender) => HandleUntyped::strong(self.id, sender.clone()), - HandleType::Weak => HandleUntyped::weak(self.id), - } +impl Hash for UntypedHandle { + #[inline] + fn hash(&self, state: &mut H) { + self.id().hash(state); + self.type_id().hash(state); } } -pub(crate) enum RefChange { - Increment(HandleId), - Decrement(HandleId), -} - -#[derive(Clone)] -pub(crate) struct RefChangeChannel { - pub sender: Sender, - pub receiver: Receiver, -} - -impl Default for RefChangeChannel { - fn default() -> Self { - let (sender, receiver) = crossbeam_channel::unbounded(); - RefChangeChannel { sender, receiver } +impl std::fmt::Debug for UntypedHandle { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + UntypedHandle::Strong(handle) => { + write!( + f, + "StrongHandle{{ type_id: {:?}, id: {:?}, path: {:?} }}", + handle.id.type_id(), + handle.id.internal(), + handle.path + ) + } + UntypedHandle::Weak(id) => write!( + f, + "WeakHandle{{ type_id: {:?}, id: {:?} }}", + id.type_id(), + id.internal() + ), + } } } diff --git a/crates/bevy_asset/src/id.rs b/crates/bevy_asset/src/id.rs new file mode 100644 index 0000000000000..428b992ca0742 --- /dev/null +++ b/crates/bevy_asset/src/id.rs @@ -0,0 +1,377 @@ +use crate::{Asset, AssetIndex, Handle, UntypedHandle}; +use bevy_reflect::{Reflect, Uuid}; +use std::{ + any::TypeId, + fmt::{Debug, Display}, + hash::Hash, + marker::PhantomData, +}; + +/// A unique runtime-only identifier for an [`Asset`]. This is cheap to [`Copy`]/[`Clone`] and is not directly tied to the +/// lifetime of the Asset. This means it _can_ point to an [`Asset`] that no longer exists. +/// +/// For an identifier tied to the lifetime of an asset, see [`Handle`]. +/// +/// For an "untyped" / "generic-less" id, see [`UntypedAssetId`]. +#[derive(Reflect)] +pub enum AssetId { + /// A small / efficient runtime identifier that can be used to efficiently look up an asset stored in [`Assets`]. This is + /// the "default" identifier used for assets. The alternative(s) (ex: [`AssetId::Uuid`]) will only be used if assets are + /// explicitly registered that way. + /// + /// [`Assets`]: crate::Assets + Index { + index: AssetIndex, + #[reflect(ignore)] + marker: PhantomData A>, + }, + /// A stable-across-runs / const asset identifier. This will only be used if an asset is explicitly registered in [`Assets`] + /// with one. + /// + /// [`Assets`]: crate::Assets + Uuid { uuid: Uuid }, +} + +impl AssetId { + /// The uuid for the default [`AssetId`]. It is valid to assign a value to this in [`Assets`](crate::Assets) + /// and by convention (where appropriate) assets should support this pattern. + pub const DEFAULT_UUID: Uuid = Uuid::from_u128(200809721996911295814598172825939264631); + + /// This asset id _should_ never be valid. Assigning a value to this in [`Assets`](crate::Assets) will + /// produce undefined behavior, so don't do it! + pub const INVALID_UUID: Uuid = Uuid::from_u128(108428345662029828789348721013522787528); + + /// Returns an [`AssetId`] with [`Self::INVALID_UUID`], which _should_ never be assigned to. + #[inline] + pub const fn invalid() -> Self { + Self::Uuid { + uuid: Self::INVALID_UUID, + } + } + + /// Converts this to an "untyped" / "generic-less" [`Asset`] identifier that stores the type information + /// _inside_ the [`UntypedAssetId`]. + #[inline] + pub fn untyped(self) -> UntypedAssetId { + match self { + AssetId::Index { index, .. } => UntypedAssetId::Index { + index, + type_id: TypeId::of::(), + }, + AssetId::Uuid { uuid } => UntypedAssetId::Uuid { + uuid, + type_id: TypeId::of::(), + }, + } + } + + #[inline] + pub(crate) fn internal(self) -> InternalAssetId { + match self { + AssetId::Index { index, .. } => InternalAssetId::Index(index), + AssetId::Uuid { uuid } => InternalAssetId::Uuid(uuid), + } + } +} + +impl Default for AssetId { + fn default() -> Self { + AssetId::Uuid { + uuid: Self::DEFAULT_UUID, + } + } +} + +impl Clone for AssetId { + fn clone(&self) -> Self { + *self + } +} + +impl Copy for AssetId {} + +impl Display for AssetId { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + Debug::fmt(self, f) + } +} +impl Debug for AssetId { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + AssetId::Index { index, .. } => { + write!( + f, + "AssetId<{}>{{ index: {}, generation: {}}}", + std::any::type_name::(), + index.index, + index.generation + ) + } + AssetId::Uuid { uuid } => { + write!( + f, + "AssetId<{}>{{uuid: {}}}", + std::any::type_name::(), + uuid + ) + } + } + } +} + +impl Hash for AssetId { + #[inline] + fn hash(&self, state: &mut H) { + self.internal().hash(state); + } +} + +impl PartialEq for AssetId { + #[inline] + fn eq(&self, other: &Self) -> bool { + self.internal().eq(&other.internal()) + } +} + +impl Eq for AssetId {} + +impl PartialOrd for AssetId { + fn partial_cmp(&self, other: &Self) -> Option { + Some(self.cmp(other)) + } +} + +impl Ord for AssetId { + fn cmp(&self, other: &Self) -> std::cmp::Ordering { + self.internal().cmp(&other.internal()) + } +} + +impl From for AssetId { + #[inline] + fn from(value: AssetIndex) -> Self { + Self::Index { + index: value, + marker: PhantomData, + } + } +} + +impl From for AssetId { + #[inline] + fn from(value: Uuid) -> Self { + Self::Uuid { uuid: value } + } +} + +impl From> for AssetId { + #[inline] + fn from(value: Handle) -> Self { + value.id() + } +} + +impl From<&Handle> for AssetId { + #[inline] + fn from(value: &Handle) -> Self { + value.id() + } +} + +impl From for AssetId { + #[inline] + fn from(value: UntypedHandle) -> Self { + value.id().typed() + } +} + +impl From<&UntypedHandle> for AssetId { + #[inline] + fn from(value: &UntypedHandle) -> Self { + value.id().typed() + } +} + +impl From for AssetId { + #[inline] + fn from(value: UntypedAssetId) -> Self { + value.typed() + } +} + +impl From<&UntypedAssetId> for AssetId { + #[inline] + fn from(value: &UntypedAssetId) -> Self { + value.typed() + } +} + +/// An "untyped" / "generic-less" [`Asset`] identifier that behaves much like [`AssetId`], but stores the [`Asset`] type +/// information at runtime instead of compile-time. This increases the size of the type, but it enables storing asset ids +/// across asset types together and enables comparisons between them. +#[derive(Debug, Copy, Clone, Eq, PartialEq, Hash)] +pub enum UntypedAssetId { + /// A small / efficient runtime identifier that can be used to efficiently look up an asset stored in [`Assets`]. This is + /// the "default" identifier used for assets. The alternative(s) (ex: [`UntypedAssetId::Uuid`]) will only be used if assets are + /// explicitly registered that way. + /// + /// [`Assets`]: crate::Assets + Index { type_id: TypeId, index: AssetIndex }, + /// A stable-across-runs / const asset identifier. This will only be used if an asset is explicitly registered in [`Assets`] + /// with one. + /// + /// [`Assets`]: crate::Assets + Uuid { type_id: TypeId, uuid: Uuid }, +} + +impl UntypedAssetId { + /// Converts this to a "typed" [`AssetId`] without checking the stored type to see if it matches the target `A` [`Asset`] type. + /// This should only be called if you are _absolutely certain_ the asset type matches the stored type. And even then, you should + /// consider using [`UntypedAssetId::typed_debug_checked`] instead. + #[inline] + pub fn typed_unchecked(self) -> AssetId { + match self { + UntypedAssetId::Index { index, .. } => AssetId::Index { + index, + marker: PhantomData, + }, + UntypedAssetId::Uuid { uuid, .. } => AssetId::Uuid { uuid }, + } + } + + /// Converts this to a "typed" [`AssetId`]. When compiled in debug-mode it will check to see if the stored type + /// matches the target `A` [`Asset`] type. When compiled in release-mode, this check will be skipped. + /// + /// # Panics + /// + /// Panics if compiled in debug mode and the [`TypeId`] of `A` does not match the stored [`TypeId`]. + #[inline] + pub fn typed_debug_checked(self) -> AssetId { + debug_assert_eq!( + self.type_id(), + TypeId::of::(), + "The target AssetId<{}>'s TypeId does not match the TypeId of this UntypedAssetId", + std::any::type_name::() + ); + self.typed_unchecked() + } + + /// Converts this to a "typed" [`AssetId`]. + /// + /// # Panics + /// + /// Panics if the [`TypeId`] of `A` does not match the stored type id. + #[inline] + pub fn typed(self) -> AssetId { + assert_eq!( + self.type_id(), + TypeId::of::(), + "The target AssetId<{}>'s TypeId does not match the TypeId of this UntypedAssetId", + std::any::type_name::() + ); + self.typed_unchecked() + } + + /// Returns the stored [`TypeId`] of the referenced [`Asset`]. + #[inline] + pub fn type_id(&self) -> TypeId { + match self { + UntypedAssetId::Index { type_id, .. } | UntypedAssetId::Uuid { type_id, .. } => { + *type_id + } + } + } + + #[inline] + pub(crate) fn internal(self) -> InternalAssetId { + match self { + UntypedAssetId::Index { index, .. } => InternalAssetId::Index(index), + UntypedAssetId::Uuid { uuid, .. } => InternalAssetId::Uuid(uuid), + } + } +} + +impl Display for UntypedAssetId { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let mut writer = f.debug_struct("UntypedAssetId"); + match self { + UntypedAssetId::Index { index, type_id } => { + writer + .field("type_id", type_id) + .field("index", &index.index) + .field("generation", &index.generation); + } + UntypedAssetId::Uuid { uuid, type_id } => { + writer.field("type_id", type_id).field("uuid", uuid); + } + } + writer.finish() + } +} + +impl From> for UntypedAssetId { + #[inline] + fn from(value: AssetId) -> Self { + value.untyped() + } +} + +impl From> for UntypedAssetId { + #[inline] + fn from(value: Handle) -> Self { + value.id().untyped() + } +} + +impl From<&Handle> for UntypedAssetId { + #[inline] + fn from(value: &Handle) -> Self { + value.id().untyped() + } +} + +/// An asset id without static or dynamic types associated with it. +/// This exist to support efficient type erased id drop tracking. We +/// could use [`UntypedAssetId`] for this, but the [`TypeId`] is unnecessary. +/// +/// Do not _ever_ use this across asset types for comparison. +/// [`InternalAssetId`] contains no type information and will happily collide +/// with indices across types. +#[derive(Debug, Copy, Clone, Eq, PartialEq, Hash, PartialOrd, Ord)] +pub(crate) enum InternalAssetId { + Index(AssetIndex), + Uuid(Uuid), +} + +impl InternalAssetId { + #[inline] + pub(crate) fn typed(self) -> AssetId { + match self { + InternalAssetId::Index(index) => AssetId::Index { + index, + marker: PhantomData, + }, + InternalAssetId::Uuid(uuid) => AssetId::Uuid { uuid }, + } + } + + #[inline] + pub(crate) fn untyped(self, type_id: TypeId) -> UntypedAssetId { + match self { + InternalAssetId::Index(index) => UntypedAssetId::Index { index, type_id }, + InternalAssetId::Uuid(uuid) => UntypedAssetId::Uuid { uuid, type_id }, + } + } +} + +impl From for InternalAssetId { + fn from(value: AssetIndex) -> Self { + Self::Index(value) + } +} + +impl From for InternalAssetId { + fn from(value: Uuid) -> Self { + Self::Uuid(value) + } +} diff --git a/crates/bevy_asset/src/info.rs b/crates/bevy_asset/src/info.rs deleted file mode 100644 index fc757b70e7eb4..0000000000000 --- a/crates/bevy_asset/src/info.rs +++ /dev/null @@ -1,69 +0,0 @@ -use crate::{path::AssetPath, LabelId}; -use bevy_utils::{HashMap, HashSet, Uuid}; -use serde::{Deserialize, Serialize}; -use std::path::PathBuf; - -/// Metadata for an asset source. -#[derive(Clone, Debug, Serialize, Deserialize)] -pub struct SourceMeta { - /// A collection of asset metadata. - pub assets: Vec, -} - -/// Metadata for an asset. -#[derive(Clone, Debug, Serialize, Deserialize)] -pub struct AssetMeta { - /// Asset label. - pub label: Option, - /// Asset dependencies. - pub dependencies: Vec>, - /// An unique identifier for an asset type. - pub type_uuid: Uuid, -} - -/// Information about an asset source, such as its path, load state and asset metadata. -#[derive(Clone, Debug)] -pub struct SourceInfo { - /// Metadata for the source. - pub meta: Option, - /// The path of the source. - pub path: PathBuf, - /// A map of assets and their type identifiers. - pub asset_types: HashMap, - /// The load state of the source. - pub load_state: LoadState, - /// A collection to track which assets were sent to their asset storages. - pub committed_assets: HashSet, - /// Current version of the source. - pub version: usize, -} - -impl SourceInfo { - /// Returns `true` if all assets tracked by the source were loaded into their asset storages. - pub fn is_loaded(&self) -> bool { - self.meta.as_ref().map_or(false, |meta| { - self.committed_assets.len() == meta.assets.len() - }) - } - - /// Gets the type identifier for an asset identified by `label_id`. - pub fn get_asset_type(&self, label_id: LabelId) -> Option { - self.asset_types.get(&label_id).cloned() - } -} - -/// The load state of an asset. -#[derive(Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd)] -pub enum LoadState { - /// The asset has not been loaded. - NotLoaded, - /// The asset is in the process of loading. - Loading, - /// The asset has been loaded and is living inside an [`Assets`](crate::Assets) collection. - Loaded, - /// The asset failed to load. - Failed, - /// The asset was previously loaded, however all handles were dropped and the asset was removed - /// from the [`Assets`](crate::Assets) collection. - Unloaded, -} diff --git a/crates/bevy_asset/src/io/android.rs b/crates/bevy_asset/src/io/android.rs new file mode 100644 index 0000000000000..7ee1f5a1c927a --- /dev/null +++ b/crates/bevy_asset/src/io/android.rs @@ -0,0 +1,82 @@ +use crate::io::{ + get_meta_path, AssetReader, AssetReaderError, AssetWatcher, EmptyPathStream, PathStream, + Reader, VecReader, +}; +use anyhow::Result; +use bevy_log::error; +use bevy_utils::BoxedFuture; +use std::{ffi::CString, path::Path}; + +/// [`AssetReader`] implementation for Android devices, built on top of Android's [`AssetManager`]. +/// +/// Implementation details: +/// +/// - [`load_path`](AssetIo::load_path) uses the [`AssetManager`] to load files. +/// - [`read_directory`](AssetIo::read_directory) always returns an empty iterator. +/// - Watching for changes is not supported. The watcher method will do nothing. +/// +/// [AssetManager]: https://developer.android.com/reference/android/content/res/AssetManager +pub struct AndroidAssetReader; + +impl AssetReader for AndroidAssetReader { + fn read<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + Box::pin(async move { + let asset_manager = bevy_winit::ANDROID_APP + .get() + .expect("Bevy must be setup with the #[bevy_main] macro on Android") + .asset_manager(); + let mut opened_asset = asset_manager + .open(&CString::new(path.to_str().unwrap()).unwrap()) + .ok_or(AssetReaderError::NotFound(path.to_path_buf()))?; + let bytes = opened_asset.get_buffer()?; + let reader: Box = Box::new(VecReader::new(bytes.to_vec())); + Ok(reader) + }) + } + + fn read_meta<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + Box::pin(async move { + let meta_path = get_meta_path(path); + let asset_manager = bevy_winit::ANDROID_APP + .get() + .expect("Bevy must be setup with the #[bevy_main] macro on Android") + .asset_manager(); + let mut opened_asset = asset_manager + .open(&CString::new(meta_path.to_str().unwrap()).unwrap()) + .ok_or(AssetReaderError::NotFound(meta_path))?; + let bytes = opened_asset.get_buffer()?; + let reader: Box = Box::new(VecReader::new(bytes.to_vec())); + Ok(reader) + }) + } + + fn read_directory<'a>( + &'a self, + _path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetReaderError>> { + let stream: Box = Box::new(EmptyPathStream); + error!("Reading directories is not supported with the AndroidAssetReader"); + Box::pin(async move { Ok(stream) }) + } + + fn is_directory<'a>( + &'a self, + _path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result> { + error!("Reading directories is not supported with the AndroidAssetReader"); + Box::pin(async move { Ok(false) }) + } + + fn watch_for_changes( + &self, + _event_sender: crossbeam_channel::Sender, + ) -> Option> { + None + } +} diff --git a/crates/bevy_asset/src/io/android_asset_io.rs b/crates/bevy_asset/src/io/android_asset_io.rs deleted file mode 100644 index a90187b8f294e..0000000000000 --- a/crates/bevy_asset/src/io/android_asset_io.rs +++ /dev/null @@ -1,81 +0,0 @@ -use crate::{AssetIo, AssetIoError, ChangeWatcher, Metadata}; -use anyhow::Result; -use bevy_utils::BoxedFuture; -use std::{ - convert::TryFrom, - ffi::CString, - path::{Path, PathBuf}, -}; - -/// I/O implementation for Android devices. -/// -/// Implementation details: -/// -/// - [`load_path`](AssetIo::load_path) uses the [`AssetManager`] to load files. -/// - [`read_directory`](AssetIo::read_directory) always returns an empty iterator. -/// - [`get_metadata`](AssetIo::get_metadata) will probably return an error. -/// - Watching for changes is not supported. The watcher methods will do nothing. -/// -/// [AssetManager]: https://developer.android.com/reference/android/content/res/AssetManager -pub struct AndroidAssetIo { - root_path: PathBuf, -} - -impl AndroidAssetIo { - /// Creates a new [`AndroidAssetIo`] at the given root path - pub fn new>(path: P) -> Self { - AndroidAssetIo { - root_path: path.as_ref().to_owned(), - } - } -} - -impl AssetIo for AndroidAssetIo { - fn load_path<'a>(&'a self, path: &'a Path) -> BoxedFuture<'a, Result, AssetIoError>> { - Box::pin(async move { - let asset_manager = bevy_winit::ANDROID_APP - .get() - .expect("Bevy must be setup with the #[bevy_main] macro on Android") - .asset_manager(); - let mut opened_asset = asset_manager - .open(&CString::new(path.to_str().unwrap()).unwrap()) - .ok_or(AssetIoError::NotFound(path.to_path_buf()))?; - let bytes = opened_asset.get_buffer()?; - Ok(bytes.to_vec()) - }) - } - - fn read_directory( - &self, - _path: &Path, - ) -> Result>, AssetIoError> { - Ok(Box::new(std::iter::empty::())) - } - - fn watch_path_for_changes( - &self, - _to_watch: &Path, - _to_reload: Option, - ) -> Result<(), AssetIoError> { - Ok(()) - } - - fn watch_for_changes(&self, _configuration: &ChangeWatcher) -> Result<(), AssetIoError> { - bevy_log::warn!("Watching for changes is not supported on Android"); - Ok(()) - } - - fn get_metadata(&self, path: &Path) -> Result { - let full_path = self.root_path.join(path); - full_path - .metadata() - .and_then(Metadata::try_from) - .map_err(|e| { - if e.kind() == std::io::ErrorKind::NotFound { - AssetIoError::NotFound(full_path) - } else { - e.into() - } - }) - } -} diff --git a/crates/bevy_asset/src/io/file/file_watcher.rs b/crates/bevy_asset/src/io/file/file_watcher.rs new file mode 100644 index 0000000000000..1a4bc4cffa427 --- /dev/null +++ b/crates/bevy_asset/src/io/file/file_watcher.rs @@ -0,0 +1,182 @@ +use crate::io::{AssetSourceEvent, AssetWatcher}; +use anyhow::Result; +use bevy_log::error; +use bevy_utils::Duration; +use crossbeam_channel::Sender; +use notify_debouncer_full::{ + new_debouncer, + notify::{ + self, + event::{AccessKind, AccessMode, CreateKind, ModifyKind, RemoveKind, RenameMode}, + RecommendedWatcher, RecursiveMode, Watcher, + }, + DebounceEventResult, Debouncer, FileIdMap, +}; +use std::path::{Path, PathBuf}; + +pub struct FileWatcher { + _watcher: Debouncer, +} + +impl FileWatcher { + pub fn new( + root: PathBuf, + sender: Sender, + debounce_wait_time: Duration, + ) -> Result { + let owned_root = root.clone(); + let mut debouncer = new_debouncer( + debounce_wait_time, + None, + move |result: DebounceEventResult| { + match result { + Ok(events) => { + for event in events.iter() { + match event.kind { + notify::EventKind::Create(CreateKind::File) => { + let (path, is_meta) = + get_asset_path(&owned_root, &event.paths[0]); + if is_meta { + sender.send(AssetSourceEvent::AddedMeta(path)).unwrap(); + } else { + sender.send(AssetSourceEvent::AddedAsset(path)).unwrap(); + } + } + notify::EventKind::Create(CreateKind::Folder) => { + let (path, _) = get_asset_path(&owned_root, &event.paths[0]); + sender.send(AssetSourceEvent::AddedFolder(path)).unwrap(); + } + notify::EventKind::Modify(ModifyKind::Any) => { + let (path, is_meta) = + get_asset_path(&owned_root, &event.paths[0]); + if event.paths[0].is_dir() { + // modified folder means nothing in this case + } else if is_meta { + sender.send(AssetSourceEvent::ModifiedMeta(path)).unwrap(); + } else { + sender.send(AssetSourceEvent::ModifiedAsset(path)).unwrap(); + }; + } + notify::EventKind::Access(AccessKind::Close(AccessMode::Write)) => { + let (path, is_meta) = + get_asset_path(&owned_root, &event.paths[0]); + if is_meta { + sender.send(AssetSourceEvent::ModifiedMeta(path)).unwrap(); + } else { + sender.send(AssetSourceEvent::ModifiedAsset(path)).unwrap(); + } + } + notify::EventKind::Remove(RemoveKind::Any) | + // Because this is debounced over a reasonable period of time, "From" events are assumed to be "dangling" without + // a follow up "To" event. Without debouncing, "From" -> "To" -> "Both" events are emitted for renames. + // If a From is dangling, it is assumed to be "removed" from the context of the asset system. + notify::EventKind::Modify(ModifyKind::Name(RenameMode::From)) => { + let (path, is_meta) = + get_asset_path(&owned_root, &event.paths[0]); + sender + .send(AssetSourceEvent::RemovedUnknown { path, is_meta }) + .unwrap(); + } + notify::EventKind::Create(CreateKind::Any) + | notify::EventKind::Modify(ModifyKind::Name(RenameMode::To)) => { + let (path, is_meta) = + get_asset_path(&owned_root, &event.paths[0]); + let event = if event.paths[0].is_dir() { + AssetSourceEvent::AddedFolder(path) + } else if is_meta { + AssetSourceEvent::AddedMeta(path) + } else { + AssetSourceEvent::AddedAsset(path) + }; + sender.send(event).unwrap(); + } + notify::EventKind::Modify(ModifyKind::Name(RenameMode::Both)) => { + let (old_path, old_is_meta) = + get_asset_path(&owned_root, &event.paths[0]); + let (new_path, new_is_meta) = + get_asset_path(&owned_root, &event.paths[1]); + // only the new "real" path is considered a directory + if event.paths[1].is_dir() { + sender + .send(AssetSourceEvent::RenamedFolder { + old: old_path, + new: new_path, + }) + .unwrap(); + } else { + match (old_is_meta, new_is_meta) { + (true, true) => { + sender + .send(AssetSourceEvent::RenamedMeta { + old: old_path, + new: new_path, + }) + .unwrap(); + } + (false, false) => { + sender + .send(AssetSourceEvent::RenamedAsset { + old: old_path, + new: new_path, + }) + .unwrap(); + } + (true, false) => { + error!( + "Asset metafile {old_path:?} was changed to asset file {new_path:?}, which is not supported. Try restarting your app to see if configuration is still valid" + ); + } + (false, true) => { + error!( + "Asset file {old_path:?} was changed to meta file {new_path:?}, which is not supported. Try restarting your app to see if configuration is still valid" + ); + } + } + } + } + notify::EventKind::Remove(RemoveKind::File) => { + let (path, is_meta) = + get_asset_path(&owned_root, &event.paths[0]); + if is_meta { + sender.send(AssetSourceEvent::RemovedMeta(path)).unwrap(); + } else { + sender.send(AssetSourceEvent::RemovedAsset(path)).unwrap(); + } + } + notify::EventKind::Remove(RemoveKind::Folder) => { + let (path, _) = get_asset_path(&owned_root, &event.paths[0]); + sender.send(AssetSourceEvent::RemovedFolder(path)).unwrap(); + } + _ => {} + } + } + } + Err(errors) => errors.iter().for_each(|error| { + error!("Encountered a filesystem watcher error {error:?}"); + }), + } + }, + )?; + debouncer.watcher().watch(&root, RecursiveMode::Recursive)?; + debouncer.cache().add_root(&root, RecursiveMode::Recursive); + Ok(Self { + _watcher: debouncer, + }) + } +} + +impl AssetWatcher for FileWatcher {} + +pub(crate) fn get_asset_path(root: &Path, absolute_path: &Path) -> (PathBuf, bool) { + let relative_path = absolute_path.strip_prefix(root).unwrap(); + let is_meta = relative_path + .extension() + .map(|e| e == "meta") + .unwrap_or(false); + let asset_path = if is_meta { + relative_path.with_extension("") + } else { + relative_path.to_owned() + }; + (asset_path, is_meta) +} diff --git a/crates/bevy_asset/src/io/file/mod.rs b/crates/bevy_asset/src/io/file/mod.rs new file mode 100644 index 0000000000000..3e53981a643d2 --- /dev/null +++ b/crates/bevy_asset/src/io/file/mod.rs @@ -0,0 +1,325 @@ +#[cfg(feature = "filesystem_watcher")] +mod file_watcher; + +use crate::io::{ + get_meta_path, AssetReader, AssetReaderError, AssetWatcher, AssetWriter, AssetWriterError, + PathStream, Reader, Writer, +}; +use anyhow::Result; +use async_fs::{read_dir, File}; +use bevy_utils::BoxedFuture; +use futures_lite::StreamExt; + +use std::{ + env, + path::{Path, PathBuf}, +}; + +pub(crate) fn get_base_path() -> PathBuf { + if let Ok(manifest_dir) = env::var("BEVY_ASSET_ROOT") { + PathBuf::from(manifest_dir) + } else if let Ok(manifest_dir) = env::var("CARGO_MANIFEST_DIR") { + PathBuf::from(manifest_dir) + } else { + env::current_exe() + .map(|path| { + path.parent() + .map(|exe_parent_path| exe_parent_path.to_owned()) + .unwrap() + }) + .unwrap() + } +} + +/// I/O implementation for the local filesystem. +/// +/// This asset I/O is fully featured but it's not available on `android` and `wasm` targets. +pub struct FileAssetReader { + root_path: PathBuf, +} + +impl FileAssetReader { + /// Creates a new `FileAssetIo` at a path relative to the executable's directory, optionally + /// watching for changes. + /// + /// See `get_base_path` below. + pub fn new>(path: P) -> Self { + let root_path = Self::get_base_path().join(path.as_ref()); + std::fs::create_dir_all(&root_path).unwrap_or_else(|e| { + panic!( + "Failed to create root directory {:?} for file asset reader: {:?}", + root_path, e + ) + }); + Self { root_path } + } + + /// Returns the base path of the assets directory, which is normally the executable's parent + /// directory. + /// + /// If the `CARGO_MANIFEST_DIR` environment variable is set, then its value will be used + /// instead. It's set by cargo when running with `cargo run`. + pub fn get_base_path() -> PathBuf { + get_base_path() + } + + /// Returns the root directory where assets are loaded from. + /// + /// See `get_base_path`. + pub fn root_path(&self) -> &PathBuf { + &self.root_path + } +} + +impl AssetReader for FileAssetReader { + fn read<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + Box::pin(async move { + let full_path = self.root_path.join(path); + match File::open(&full_path).await { + Ok(file) => { + let reader: Box = Box::new(file); + Ok(reader) + } + Err(e) => { + if e.kind() == std::io::ErrorKind::NotFound { + Err(AssetReaderError::NotFound(full_path)) + } else { + Err(e.into()) + } + } + } + }) + } + + fn read_meta<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + let meta_path = get_meta_path(path); + Box::pin(async move { + let full_path = self.root_path.join(meta_path); + match File::open(&full_path).await { + Ok(file) => { + let reader: Box = Box::new(file); + Ok(reader) + } + Err(e) => { + if e.kind() == std::io::ErrorKind::NotFound { + Err(AssetReaderError::NotFound(full_path)) + } else { + Err(e.into()) + } + } + } + }) + } + + fn read_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetReaderError>> { + Box::pin(async move { + let full_path = self.root_path.join(path); + match read_dir(&full_path).await { + Ok(read_dir) => { + let root_path = self.root_path.clone(); + let mapped_stream = read_dir.filter_map(move |f| { + f.ok().and_then(|dir_entry| { + let path = dir_entry.path(); + // filter out meta files as they are not considered assets + if let Some(ext) = path.extension().and_then(|e| e.to_str()) { + if ext.eq_ignore_ascii_case("meta") { + return None; + } + } + let relative_path = path.strip_prefix(&root_path).unwrap(); + Some(relative_path.to_owned()) + }) + }); + let read_dir: Box = Box::new(mapped_stream); + Ok(read_dir) + } + Err(e) => { + if e.kind() == std::io::ErrorKind::NotFound { + Err(AssetReaderError::NotFound(full_path)) + } else { + Err(e.into()) + } + } + } + }) + } + + fn is_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result> { + Box::pin(async move { + let full_path = self.root_path.join(path); + let metadata = full_path + .metadata() + .map_err(|_e| AssetReaderError::NotFound(path.to_owned()))?; + Ok(metadata.file_type().is_dir()) + }) + } + + fn watch_for_changes( + &self, + _event_sender: crossbeam_channel::Sender, + ) -> Option> { + #[cfg(feature = "filesystem_watcher")] + return Some(Box::new( + file_watcher::FileWatcher::new( + self.root_path.clone(), + _event_sender, + std::time::Duration::from_millis(300), + ) + .unwrap(), + )); + #[cfg(not(feature = "filesystem_watcher"))] + return None; + } +} + +pub struct FileAssetWriter { + root_path: PathBuf, +} + +impl FileAssetWriter { + /// Creates a new `FileAssetIo` at a path relative to the executable's directory, optionally + /// watching for changes. + /// + /// See `get_base_path` below. + pub fn new>(path: P) -> Self { + Self { + root_path: get_base_path().join(path.as_ref()), + } + } +} + +impl AssetWriter for FileAssetWriter { + fn write<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetWriterError>> { + Box::pin(async move { + let full_path = self.root_path.join(path); + if let Some(parent) = full_path.parent() { + async_fs::create_dir_all(parent).await?; + } + let file = File::create(&full_path).await?; + let writer: Box = Box::new(file); + Ok(writer) + }) + } + + fn write_meta<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetWriterError>> { + Box::pin(async move { + let meta_path = get_meta_path(path); + let full_path = self.root_path.join(meta_path); + if let Some(parent) = full_path.parent() { + async_fs::create_dir_all(parent).await?; + } + let file = File::create(&full_path).await?; + let writer: Box = Box::new(file); + Ok(writer) + }) + } + + fn remove<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result<(), AssetWriterError>> { + Box::pin(async move { + let full_path = self.root_path.join(path); + async_fs::remove_file(full_path).await?; + Ok(()) + }) + } + + fn remove_meta<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result<(), AssetWriterError>> { + Box::pin(async move { + let meta_path = get_meta_path(path); + let full_path = self.root_path.join(meta_path); + async_fs::remove_file(full_path).await?; + Ok(()) + }) + } + + fn remove_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result<(), AssetWriterError>> { + Box::pin(async move { + let full_path = self.root_path.join(path); + async_fs::remove_dir_all(full_path).await?; + Ok(()) + }) + } + + fn remove_empty_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result<(), AssetWriterError>> { + Box::pin(async move { + let full_path = self.root_path.join(path); + async_fs::remove_dir(full_path).await?; + Ok(()) + }) + } + + fn remove_assets_in_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result<(), AssetWriterError>> { + Box::pin(async move { + let full_path = self.root_path.join(path); + async_fs::remove_dir_all(&full_path).await?; + async_fs::create_dir_all(&full_path).await?; + Ok(()) + }) + } + + fn rename<'a>( + &'a self, + old_path: &'a Path, + new_path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result<(), AssetWriterError>> { + Box::pin(async move { + let full_old_path = self.root_path.join(old_path); + let full_new_path = self.root_path.join(new_path); + if let Some(parent) = full_new_path.parent() { + async_fs::create_dir_all(parent).await?; + } + async_fs::rename(full_old_path, full_new_path).await?; + Ok(()) + }) + } + + fn rename_meta<'a>( + &'a self, + old_path: &'a Path, + new_path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result<(), AssetWriterError>> { + Box::pin(async move { + let old_meta_path = get_meta_path(old_path); + let new_meta_path = get_meta_path(new_path); + let full_old_path = self.root_path.join(old_meta_path); + let full_new_path = self.root_path.join(new_meta_path); + if let Some(parent) = full_new_path.parent() { + async_fs::create_dir_all(parent).await?; + } + async_fs::rename(full_old_path, full_new_path).await?; + Ok(()) + }) + } +} diff --git a/crates/bevy_asset/src/io/file_asset_io.rs b/crates/bevy_asset/src/io/file_asset_io.rs deleted file mode 100644 index ced308064074a..0000000000000 --- a/crates/bevy_asset/src/io/file_asset_io.rs +++ /dev/null @@ -1,227 +0,0 @@ -#[cfg(feature = "filesystem_watcher")] -use crate::{filesystem_watcher::FilesystemWatcher, AssetServer}; -use crate::{AssetIo, AssetIoError, ChangeWatcher, Metadata}; -use anyhow::Result; -#[cfg(feature = "filesystem_watcher")] -use bevy_ecs::system::{Local, Res}; -use bevy_utils::BoxedFuture; -#[cfg(feature = "filesystem_watcher")] -use bevy_utils::{default, HashMap, Instant}; -#[cfg(feature = "filesystem_watcher")] -use crossbeam_channel::TryRecvError; -use fs::File; -#[cfg(feature = "filesystem_watcher")] -use parking_lot::RwLock; -#[cfg(feature = "filesystem_watcher")] -use std::sync::Arc; -use std::{ - convert::TryFrom, - env, fs, - io::Read, - path::{Path, PathBuf}, -}; - -/// I/O implementation for the local filesystem. -/// -/// This asset I/O is fully featured but it's not available on `android` and `wasm` targets. -pub struct FileAssetIo { - root_path: PathBuf, - #[cfg(feature = "filesystem_watcher")] - filesystem_watcher: Arc>>, -} - -impl FileAssetIo { - /// Creates a new `FileAssetIo` at a path relative to the executable's directory, optionally - /// watching for changes. - /// - /// See `get_base_path` below. - pub fn new>(path: P, watch_for_changes: &Option) -> Self { - let file_asset_io = FileAssetIo { - #[cfg(feature = "filesystem_watcher")] - filesystem_watcher: default(), - root_path: Self::get_base_path().join(path.as_ref()), - }; - if let Some(configuration) = watch_for_changes { - #[cfg(any( - not(feature = "filesystem_watcher"), - target_arch = "wasm32", - target_os = "android" - ))] - panic!( - "Watch for changes requires the filesystem_watcher feature and cannot be used on \ - wasm32 / android targets" - ); - #[cfg(feature = "filesystem_watcher")] - file_asset_io.watch_for_changes(configuration).unwrap(); - } - file_asset_io - } - - /// Returns the base path of the assets directory, which is normally the executable's parent - /// directory. - /// - /// If a `BEVY_ASSET_ROOT` environment variable is set, then its value will be used. - /// - /// Else if the `CARGO_MANIFEST_DIR` environment variable is set, then its value will be used - /// instead. It's set by cargo when running with `cargo run`. - pub fn get_base_path() -> PathBuf { - if let Ok(env_bevy_asset_root) = env::var("BEVY_ASSET_ROOT") { - PathBuf::from(env_bevy_asset_root) - } else if let Ok(manifest_dir) = env::var("CARGO_MANIFEST_DIR") { - PathBuf::from(manifest_dir) - } else { - env::current_exe() - .map(|path| { - path.parent() - .map(|exe_parent_path| exe_parent_path.to_owned()) - .unwrap() - }) - .unwrap() - } - } - - /// Returns the root directory where assets are loaded from. - /// - /// See [`get_base_path`](FileAssetIo::get_base_path). - pub fn root_path(&self) -> &PathBuf { - &self.root_path - } -} - -impl AssetIo for FileAssetIo { - fn load_path<'a>(&'a self, path: &'a Path) -> BoxedFuture<'a, Result, AssetIoError>> { - Box::pin(async move { - let mut bytes = Vec::new(); - let full_path = self.root_path.join(path); - match File::open(&full_path) { - Ok(mut file) => { - file.read_to_end(&mut bytes)?; - } - Err(e) => { - return if e.kind() == std::io::ErrorKind::NotFound { - Err(AssetIoError::NotFound(full_path)) - } else { - Err(e.into()) - } - } - } - Ok(bytes) - }) - } - - fn read_directory( - &self, - path: &Path, - ) -> Result>, AssetIoError> { - let root_path = self.root_path.to_owned(); - let path = path.to_owned(); - Ok(Box::new(fs::read_dir(root_path.join(&path))?.map( - move |entry| { - let file_name = entry.unwrap().file_name(); - path.join(file_name) - }, - ))) - } - - fn watch_path_for_changes( - &self, - to_watch: &Path, - to_reload: Option, - ) -> Result<(), AssetIoError> { - #![allow(unused_variables)] - #[cfg(feature = "filesystem_watcher")] - { - let to_reload = to_reload.unwrap_or_else(|| to_watch.to_owned()); - let to_watch = self.root_path.join(to_watch); - let mut watcher = self.filesystem_watcher.write(); - if let Some(ref mut watcher) = *watcher { - watcher - .watch(&to_watch, to_reload) - .map_err(|_error| AssetIoError::PathWatchError(to_watch))?; - } - } - - Ok(()) - } - - fn watch_for_changes(&self, configuration: &ChangeWatcher) -> Result<(), AssetIoError> { - #[cfg(feature = "filesystem_watcher")] - { - *self.filesystem_watcher.write() = Some(FilesystemWatcher::new(configuration)); - } - #[cfg(not(feature = "filesystem_watcher"))] - bevy_log::warn!("Watching for changes is not supported when the `filesystem_watcher` feature is disabled"); - - Ok(()) - } - - fn get_metadata(&self, path: &Path) -> Result { - let full_path = self.root_path.join(path); - full_path - .metadata() - .and_then(Metadata::try_from) - .map_err(|e| { - if e.kind() == std::io::ErrorKind::NotFound { - AssetIoError::NotFound(full_path) - } else { - e.into() - } - }) - } -} - -/// Watches for file changes in the local file system. -#[cfg(all( - feature = "filesystem_watcher", - all(not(target_arch = "wasm32"), not(target_os = "android")) -))] -pub fn filesystem_watcher_system( - asset_server: Res, - mut changed: Local>, -) { - let asset_io = - if let Some(asset_io) = asset_server.server.asset_io.downcast_ref::() { - asset_io - } else { - return; - }; - let watcher = asset_io.filesystem_watcher.read(); - - if let Some(ref watcher) = *watcher { - loop { - let event = match watcher.receiver.try_recv() { - Ok(result) => result.unwrap(), - Err(TryRecvError::Empty) => break, - Err(TryRecvError::Disconnected) => panic!("FilesystemWatcher disconnected."), - }; - - if let notify::event::Event { - kind: notify::event::EventKind::Modify(_), - paths, - .. - } = event - { - for path in &paths { - let Some(set) = watcher.path_map.get(path) else { - continue; - }; - for to_reload in set { - // When an asset is modified, note down the timestamp (overriding any previous modification events) - changed.insert(to_reload.to_owned(), Instant::now()); - } - } - } - } - - // Reload all assets whose last modification was at least 50ms ago. - // - // When changing and then saving a shader, several modification events are sent in short succession. - // Unless we wait until we are sure the shader is finished being modified (and that there will be no more events coming), - // we will sometimes get a crash when trying to reload a partially-modified shader. - for (to_reload, _) in - changed.extract_if(|_, last_modified| last_modified.elapsed() >= watcher.delay) - { - let _ = asset_server.load_untracked(to_reload.as_path().into(), true); - } - } -} diff --git a/crates/bevy_asset/src/io/gated.rs b/crates/bevy_asset/src/io/gated.rs new file mode 100644 index 0000000000000..cc8bd9ab117ff --- /dev/null +++ b/crates/bevy_asset/src/io/gated.rs @@ -0,0 +1,107 @@ +use crate::io::{AssetReader, AssetReaderError, PathStream, Reader}; +use anyhow::Result; +use bevy_utils::{BoxedFuture, HashMap}; +use crossbeam_channel::{Receiver, Sender}; +use parking_lot::RwLock; +use std::{ + path::{Path, PathBuf}, + sync::Arc, +}; + +/// A "gated" reader that will prevent asset reads from returning until +/// a given path has been "opened" using [`GateOpener`]. +/// +/// This is built primarily for unit tests. +pub struct GatedReader { + reader: R, + gates: Arc, Receiver<()>)>>>, +} + +impl Clone for GatedReader { + fn clone(&self) -> Self { + Self { + reader: self.reader.clone(), + gates: self.gates.clone(), + } + } +} + +/// Opens path "gates" for a [`GatedReader`]. +pub struct GateOpener { + gates: Arc, Receiver<()>)>>>, +} + +impl GateOpener { + /// Opens the `path` "gate", allowing a _single_ [`AssetReader`] operation to return for that path. + /// If multiple operations are expected, call `open` the expected number of calls. + pub fn open>(&self, path: P) { + let mut gates = self.gates.write(); + let gates = gates + .entry(path.as_ref().to_path_buf()) + .or_insert_with(crossbeam_channel::unbounded); + gates.0.send(()).unwrap(); + } +} + +impl GatedReader { + /// Creates a new [`GatedReader`], which wraps the given `reader`. Also returns a [`GateOpener`] which + /// can be used to open "path gates" for this [`GatedReader`]. + pub fn new(reader: R) -> (Self, GateOpener) { + let gates = Arc::new(RwLock::new(HashMap::new())); + ( + Self { + reader, + gates: gates.clone(), + }, + GateOpener { gates }, + ) + } +} + +impl AssetReader for GatedReader { + fn read<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + let receiver = { + let mut gates = self.gates.write(); + let gates = gates + .entry(path.to_path_buf()) + .or_insert_with(crossbeam_channel::unbounded); + gates.1.clone() + }; + Box::pin(async move { + receiver.recv().unwrap(); + let result = self.reader.read(path).await?; + Ok(result) + }) + } + + fn read_meta<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + self.reader.read_meta(path) + } + + fn read_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetReaderError>> { + self.reader.read_directory(path) + } + + fn is_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result> { + self.reader.is_directory(path) + } + + fn watch_for_changes( + &self, + event_sender: Sender, + ) -> Option> { + self.reader.watch_for_changes(event_sender) + } +} diff --git a/crates/bevy_asset/src/io/memory.rs b/crates/bevy_asset/src/io/memory.rs new file mode 100644 index 0000000000000..9ca193e08d6ee --- /dev/null +++ b/crates/bevy_asset/src/io/memory.rs @@ -0,0 +1,288 @@ +use crate::io::{AssetReader, AssetReaderError, PathStream, Reader}; +use anyhow::Result; +use bevy_utils::{BoxedFuture, HashMap}; +use futures_io::AsyncRead; +use futures_lite::{ready, Stream}; +use parking_lot::RwLock; +use std::{ + path::{Path, PathBuf}, + pin::Pin, + sync::Arc, + task::Poll, +}; + +#[derive(Default, Debug)] +struct DirInternal { + assets: HashMap, + metadata: HashMap, + dirs: HashMap, + path: PathBuf, +} + +/// A clone-able (internally Arc-ed) / thread-safe "in memory" filesystem. +/// This is built for [`MemoryAssetReader`] and is primarily intended for unit tests. +#[derive(Default, Clone, Debug)] +pub struct Dir(Arc>); + +impl Dir { + /// Creates a new [`Dir`] for the given `path`. + pub fn new(path: PathBuf) -> Self { + Self(Arc::new(RwLock::new(DirInternal { + path, + ..Default::default() + }))) + } + + pub fn insert_asset_text(&self, path: &Path, asset: &str) { + self.insert_asset(path, asset.as_bytes().to_vec()); + } + + pub fn insert_meta_text(&self, path: &Path, asset: &str) { + self.insert_meta(path, asset.as_bytes().to_vec()); + } + + pub fn insert_asset(&self, path: &Path, asset: Vec) { + let mut dir = self.clone(); + if let Some(parent) = path.parent() { + dir = self.get_or_insert_dir(parent); + } + dir.0.write().assets.insert( + path.file_name().unwrap().to_string_lossy().to_string(), + Data(Arc::new((asset, path.to_owned()))), + ); + } + + pub fn insert_meta(&self, path: &Path, asset: Vec) { + let mut dir = self.clone(); + if let Some(parent) = path.parent() { + dir = self.get_or_insert_dir(parent); + } + dir.0.write().metadata.insert( + path.file_name().unwrap().to_string_lossy().to_string(), + Data(Arc::new((asset, path.to_owned()))), + ); + } + + pub fn get_or_insert_dir(&self, path: &Path) -> Dir { + let mut dir = self.clone(); + let mut full_path = PathBuf::new(); + for c in path.components() { + full_path.push(c); + let name = c.as_os_str().to_string_lossy().to_string(); + dir = { + let dirs = &mut dir.0.write().dirs; + dirs.entry(name) + .or_insert_with(|| Dir::new(full_path.clone())) + .clone() + }; + } + + dir + } + + pub fn get_dir(&self, path: &Path) -> Option { + let mut dir = self.clone(); + for p in path.components() { + let component = p.as_os_str().to_str().unwrap(); + let next_dir = dir.0.read().dirs.get(component)?.clone(); + dir = next_dir; + } + Some(dir) + } + + pub fn get_asset(&self, path: &Path) -> Option { + let mut dir = self.clone(); + if let Some(parent) = path.parent() { + dir = dir.get_dir(parent)?; + } + + path.file_name() + .and_then(|f| dir.0.read().assets.get(f.to_str().unwrap()).cloned()) + } + + pub fn get_metadata(&self, path: &Path) -> Option { + let mut dir = self.clone(); + if let Some(parent) = path.parent() { + dir = dir.get_dir(parent)?; + } + + path.file_name() + .and_then(|f| dir.0.read().metadata.get(f.to_str().unwrap()).cloned()) + } + + pub fn path(&self) -> PathBuf { + self.0.read().path.to_owned() + } +} + +pub struct DirStream { + dir: Dir, + index: usize, +} + +impl DirStream { + fn new(dir: Dir) -> Self { + Self { dir, index: 0 } + } +} + +impl Stream for DirStream { + type Item = PathBuf; + + fn poll_next( + self: Pin<&mut Self>, + _cx: &mut std::task::Context<'_>, + ) -> Poll> { + let this = self.get_mut(); + let index = this.index; + this.index += 1; + let dir = this.dir.0.read(); + Poll::Ready(dir.assets.values().nth(index).map(|d| d.path().to_owned())) + } +} + +/// In-memory [`AssetReader`] implementation. +/// This is primarily intended for unit tests. +#[derive(Default, Clone)] +pub struct MemoryAssetReader { + pub root: Dir, +} + +/// Asset data stored in a [`Dir`]. +#[derive(Clone, Debug)] +pub struct Data(Arc<(Vec, PathBuf)>); + +impl Data { + fn path(&self) -> &Path { + &self.0 .1 + } + fn data(&self) -> &[u8] { + &self.0 .0 + } +} + +struct DataReader { + data: Data, + bytes_read: usize, +} + +impl AsyncRead for DataReader { + fn poll_read( + mut self: std::pin::Pin<&mut Self>, + cx: &mut std::task::Context<'_>, + buf: &mut [u8], + ) -> std::task::Poll> { + if self.bytes_read >= self.data.data().len() { + Poll::Ready(Ok(0)) + } else { + let n = ready!(Pin::new(&mut &self.data.data()[self.bytes_read..]).poll_read(cx, buf))?; + self.bytes_read += n; + Poll::Ready(Ok(n)) + } + } +} + +impl AssetReader for MemoryAssetReader { + fn read<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + Box::pin(async move { + self.root + .get_asset(path) + .map(|data| { + let reader: Box = Box::new(DataReader { + data, + bytes_read: 0, + }); + reader + }) + .ok_or(AssetReaderError::NotFound(PathBuf::new())) + }) + } + + fn read_meta<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + Box::pin(async move { + self.root + .get_metadata(path) + .map(|data| { + let reader: Box = Box::new(DataReader { + data, + bytes_read: 0, + }); + reader + }) + .ok_or(AssetReaderError::NotFound(PathBuf::new())) + }) + } + + fn read_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetReaderError>> { + Box::pin(async move { + self.root + .get_dir(path) + .map(|dir| { + let stream: Box = Box::new(DirStream::new(dir)); + stream + }) + .ok_or(AssetReaderError::NotFound(PathBuf::new())) + }) + } + + fn is_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result> { + Box::pin(async move { Ok(self.root.get_dir(path).is_some()) }) + } + + fn watch_for_changes( + &self, + _event_sender: crossbeam_channel::Sender, + ) -> Option> { + None + } +} + +#[cfg(test)] +pub mod test { + use super::Dir; + use std::path::Path; + + #[test] + fn memory_dir() { + let dir = Dir::default(); + let a_path = Path::new("a.txt"); + let a_data = "a".as_bytes().to_vec(); + let a_meta = "ameta".as_bytes().to_vec(); + + dir.insert_asset(a_path, a_data.clone()); + let asset = dir.get_asset(a_path).unwrap(); + assert_eq!(asset.path(), a_path); + assert_eq!(asset.data(), a_data); + + dir.insert_meta(a_path, a_meta.clone()); + let meta = dir.get_metadata(a_path).unwrap(); + assert_eq!(meta.path(), a_path); + assert_eq!(meta.data(), a_meta); + + let b_path = Path::new("x/y/b.txt"); + let b_data = "b".as_bytes().to_vec(); + let b_meta = "meta".as_bytes().to_vec(); + dir.insert_asset(b_path, b_data.clone()); + dir.insert_meta(b_path, b_meta.clone()); + + let asset = dir.get_asset(b_path).unwrap(); + assert_eq!(asset.path(), b_path); + assert_eq!(asset.data(), b_data); + + let meta = dir.get_metadata(b_path).unwrap(); + assert_eq!(meta.path(), b_path); + assert_eq!(meta.data(), b_meta); + } +} diff --git a/crates/bevy_asset/src/io/metadata.rs b/crates/bevy_asset/src/io/metadata.rs deleted file mode 100644 index 466c30590b702..0000000000000 --- a/crates/bevy_asset/src/io/metadata.rs +++ /dev/null @@ -1,85 +0,0 @@ -use std::convert::{TryFrom, TryInto}; - -/// A enum representing a type of file. -#[non_exhaustive] -#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash)] -pub enum FileType { - /// A directory. - Directory, - /// A file. - File, -} - -impl FileType { - /// Returns `true` if the entry is a directory. - #[inline] - pub const fn is_dir(&self) -> bool { - matches!(self, Self::Directory) - } - - #[inline] - /// Returns `true` if the entry is a file. - pub const fn is_file(&self) -> bool { - matches!(self, Self::File) - } -} - -impl TryFrom for FileType { - type Error = std::io::Error; - - fn try_from(file_type: std::fs::FileType) -> Result { - if file_type.is_dir() { - Ok(Self::Directory) - } else if file_type.is_file() { - Ok(Self::File) - } else { - Err(std::io::Error::new( - std::io::ErrorKind::Other, - "unknown file type", - )) - } - } -} - -/// Metadata information about a file. -/// -/// This structure is returned from the [`AssetIo::get_metadata`](crate::AssetIo) method. -#[derive(Debug, Clone)] -pub struct Metadata { - file_type: FileType, -} - -impl Metadata { - /// Creates new metadata information. - pub fn new(file_type: FileType) -> Self { - Self { file_type } - } - - /// Returns the file type. - #[inline] - pub const fn file_type(&self) -> FileType { - self.file_type - } - - /// Returns `true` if the entry is a directory. - #[inline] - pub const fn is_dir(&self) -> bool { - self.file_type.is_dir() - } - - /// Returns `true` if the entry is a file. - #[inline] - pub const fn is_file(&self) -> bool { - self.file_type.is_file() - } -} - -impl TryFrom for Metadata { - type Error = std::io::Error; - - fn try_from(metadata: std::fs::Metadata) -> Result { - Ok(Self { - file_type: metadata.file_type().try_into()?, - }) - } -} diff --git a/crates/bevy_asset/src/io/mod.rs b/crates/bevy_asset/src/io/mod.rs index 081076f4ed8d1..a29902c5837b2 100644 --- a/crates/bevy_asset/src/io/mod.rs +++ b/crates/bevy_asset/src/io/mod.rs @@ -1,105 +1,281 @@ #[cfg(target_os = "android")] -mod android_asset_io; -#[cfg(all(not(target_arch = "wasm32"), not(target_os = "android")))] -mod file_asset_io; +pub mod android; +#[cfg(not(target_arch = "wasm32"))] +pub mod file; +pub mod gated; +pub mod memory; +pub mod processor_gated; #[cfg(target_arch = "wasm32")] -mod wasm_asset_io; +pub mod wasm; -mod metadata; +mod provider; -#[cfg(target_os = "android")] -pub use android_asset_io::*; -#[cfg(all(not(target_arch = "wasm32"), not(target_os = "android")))] -pub use file_asset_io::*; -#[cfg(target_arch = "wasm32")] -pub use wasm_asset_io::*; - -pub use metadata::*; +pub use futures_lite::{AsyncReadExt, AsyncWriteExt}; +pub use provider::*; -use anyhow::Result; use bevy_utils::BoxedFuture; -use downcast_rs::{impl_downcast, Downcast}; +use crossbeam_channel::Sender; +use futures_io::{AsyncRead, AsyncWrite}; +use futures_lite::{ready, Stream}; use std::{ - io, path::{Path, PathBuf}, + pin::Pin, + task::Poll, }; use thiserror::Error; -use crate::ChangeWatcher; - /// Errors that occur while loading assets. #[derive(Error, Debug)] -pub enum AssetIoError { +pub enum AssetReaderError { /// Path not found. #[error("path not found: {0}")] NotFound(PathBuf), /// Encountered an I/O error while loading an asset. #[error("encountered an io error while loading asset: {0}")] - Io(#[from] io::Error), - - /// Failed to watch path. - #[error("failed to watch path: {0}")] - PathWatchError(PathBuf), + Io(#[from] std::io::Error), } -/// A storage provider for an [`AssetServer`]. -/// -/// An asset I/O is the backend actually providing data for the asset loaders managed by the asset -/// server. An average user will probably be just fine with the default [`FileAssetIo`], but you -/// can easily use your own custom I/O to, for example, load assets from cloud storage or create a -/// seamless VFS layout using custom containers. -/// -/// See the [`custom_asset_io`] example in the repository for more details. +pub type Reader<'a> = dyn AsyncRead + Unpin + Send + Sync + 'a; + +/// Performs read operations on an asset storage. [`AssetReader`] exposes a "virtual filesystem" +/// API, where asset bytes and asset metadata bytes are both stored and accessible for a given +/// `path`. /// -/// [`AssetServer`]: struct.AssetServer.html -/// [`custom_asset_io`]: https://github.com/bevyengine/bevy/tree/latest/examples/asset/custom_asset_io.rs -pub trait AssetIo: Downcast + Send + Sync + 'static { +/// Also see [`AssetWriter`]. +pub trait AssetReader: Send + Sync + 'static { /// Returns a future to load the full file data at the provided path. - fn load_path<'a>(&'a self, path: &'a Path) -> BoxedFuture<'a, Result, AssetIoError>>; - + fn read<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>>; + /// Returns a future to load the full file data at the provided path. + fn read_meta<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>>; /// Returns an iterator of directory entry names at the provided path. - fn read_directory( - &self, - path: &Path, - ) -> Result>, AssetIoError>; - - /// Returns metadata about the filesystem entry at the provided path. - fn get_metadata(&self, path: &Path) -> Result; - - /// Tells the asset I/O to watch for changes recursively at the provided path. - /// - /// No-op if [`watch_for_changes`](AssetIo::watch_for_changes) hasn't been called yet. - /// Otherwise triggers a reload each time `to_watch` changes. - /// In most cases the asset found at the watched path should be changed, - /// but when an asset depends on data at another path, the asset's path - /// is provided in `to_reload`. - /// Note that there may be a many-to-many correspondence between - /// `to_watch` and `to_reload` paths. - fn watch_path_for_changes( + fn read_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetReaderError>>; + /// Returns an iterator of directory entry names at the provided path. + fn is_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>; + + /// Returns an Asset watcher that will send events on the given channel. + /// If this reader does not support watching for changes, this will return [`None`]. + fn watch_for_changes( &self, - to_watch: &Path, - to_reload: Option, - ) -> Result<(), AssetIoError>; - - /// Enables change tracking in this asset I/O. - fn watch_for_changes(&self, configuration: &ChangeWatcher) -> Result<(), AssetIoError>; - - /// Returns `true` if the path is a directory. - fn is_dir(&self, path: &Path) -> bool { - self.get_metadata(path) - .as_ref() - .map(Metadata::is_dir) - .unwrap_or(false) + event_sender: Sender, + ) -> Option>; + + /// Reads asset metadata bytes at the given `path` into a [`Vec`]. This is a convenience + /// function that wraps [`AssetReader::read_meta`] by default. + fn read_meta_bytes<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetReaderError>> { + Box::pin(async move { + let mut meta_reader = self.read_meta(path).await?; + let mut meta_bytes = Vec::new(); + meta_reader.read_to_end(&mut meta_bytes).await?; + Ok(meta_bytes) + }) } +} + +pub type Writer = dyn AsyncWrite + Unpin + Send + Sync; + +pub type PathStream = dyn Stream + Unpin + Send; - /// Returns `true` if the path is a file. - fn is_file(&self, path: &Path) -> bool { - self.get_metadata(path) - .as_ref() - .map(Metadata::is_file) - .unwrap_or(false) +/// Errors that occur while loading assets. +#[derive(Error, Debug)] +pub enum AssetWriterError { + /// Encountered an I/O error while loading an asset. + #[error("encountered an io error while loading asset: {0}")] + Io(#[from] std::io::Error), +} + +/// Preforms write operations on an asset storage. [`AssetWriter`] exposes a "virtual filesystem" +/// API, where asset bytes and asset metadata bytes are both stored and accessible for a given +/// `path`. +/// +/// Also see [`AssetReader`]. +pub trait AssetWriter: Send + Sync + 'static { + /// Writes the full asset bytes at the provided path. + fn write<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetWriterError>>; + /// Writes the full asset meta bytes at the provided path. + /// This _should not_ include storage specific extensions like `.meta`. + fn write_meta<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetWriterError>>; + /// Removes the asset stored at the given path. + fn remove<'a>(&'a self, path: &'a Path) -> BoxedFuture<'a, Result<(), AssetWriterError>>; + /// Removes the asset meta stored at the given path. + /// This _should not_ include storage specific extensions like `.meta`. + fn remove_meta<'a>(&'a self, path: &'a Path) -> BoxedFuture<'a, Result<(), AssetWriterError>>; + /// Renames the asset at `old_path` to `new_path` + fn rename<'a>( + &'a self, + old_path: &'a Path, + new_path: &'a Path, + ) -> BoxedFuture<'a, Result<(), AssetWriterError>>; + /// Renames the asset meta for the asset at `old_path` to `new_path`. + /// This _should not_ include storage specific extensions like `.meta`. + fn rename_meta<'a>( + &'a self, + old_path: &'a Path, + new_path: &'a Path, + ) -> BoxedFuture<'a, Result<(), AssetWriterError>>; + /// Removes the directory at the given path, including all assets _and_ directories in that directory. + fn remove_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result<(), AssetWriterError>>; + /// Removes the directory at the given path, but only if it is completely empty. This will return an error if the + /// directory is not empty. + fn remove_empty_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result<(), AssetWriterError>>; + /// Removes all assets (and directories) in this directory, resulting in an empty directory. + fn remove_assets_in_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result<(), AssetWriterError>>; + /// Writes the asset `bytes` to the given `path`. + fn write_bytes<'a>( + &'a self, + path: &'a Path, + bytes: &'a [u8], + ) -> BoxedFuture<'a, Result<(), AssetWriterError>> { + Box::pin(async move { + let mut writer = self.write(path).await?; + writer.write_all(bytes).await?; + writer.flush().await?; + Ok(()) + }) + } + /// Writes the asset meta `bytes` to the given `path`. + fn write_meta_bytes<'a>( + &'a self, + path: &'a Path, + bytes: &'a [u8], + ) -> BoxedFuture<'a, Result<(), AssetWriterError>> { + Box::pin(async move { + let mut meta_writer = self.write_meta(path).await?; + meta_writer.write_all(bytes).await?; + meta_writer.flush().await?; + Ok(()) + }) } } -impl_downcast!(AssetIo); +/// An "asset source change event" that occurs whenever asset (or asset metadata) is created/added/removed +#[derive(Clone, Debug)] +pub enum AssetSourceEvent { + /// An asset at this path was added. + AddedAsset(PathBuf), + /// An asset at this path was modified. + ModifiedAsset(PathBuf), + /// An asset at this path was removed. + RemovedAsset(PathBuf), + /// An asset at this path was renamed. + RenamedAsset { old: PathBuf, new: PathBuf }, + /// Asset metadata at this path was added. + AddedMeta(PathBuf), + /// Asset metadata at this path was modified. + ModifiedMeta(PathBuf), + /// Asset metadata at this path was removed. + RemovedMeta(PathBuf), + /// Asset metadata at this path was renamed. + RenamedMeta { old: PathBuf, new: PathBuf }, + /// A folder at the given path was added. + AddedFolder(PathBuf), + /// A folder at the given path was removed. + RemovedFolder(PathBuf), + /// A folder at the given path was renamed. + RenamedFolder { old: PathBuf, new: PathBuf }, + /// Something of unknown type was removed. It is the job of the event handler to determine the type. + /// This exists because notify-rs produces "untyped" rename events without destination paths for unwatched folders, so we can't determine the type of + /// the rename. + RemovedUnknown { + /// The path of the removed asset or folder (undetermined). This could be an asset path or a folder. This will not be a "meta file" path. + path: PathBuf, + /// This field is only relevant if `path` is determined to be an asset path (and therefore not a folder). If this field is `true`, + /// then this event corresponds to a meta removal (not an asset removal) . If `false`, then this event corresponds to an asset removal + /// (not a meta removal). + is_meta: bool, + }, +} + +/// A handle to an "asset watcher" process, that will listen for and emit [`AssetSourceEvent`] values for as long as +/// [`AssetWatcher`] has not been dropped. +/// +/// See [`AssetReader::watch_for_changes`]. +pub trait AssetWatcher: Send + Sync + 'static {} + +/// An [`AsyncRead`] implementation capable of reading a [`Vec`]. +pub struct VecReader { + bytes: Vec, + bytes_read: usize, +} + +impl VecReader { + /// Create a new [`VecReader`] for `bytes`. + pub fn new(bytes: Vec) -> Self { + Self { + bytes_read: 0, + bytes, + } + } +} + +impl AsyncRead for VecReader { + fn poll_read( + mut self: std::pin::Pin<&mut Self>, + cx: &mut std::task::Context<'_>, + buf: &mut [u8], + ) -> std::task::Poll> { + if self.bytes_read >= self.bytes.len() { + Poll::Ready(Ok(0)) + } else { + let n = ready!(Pin::new(&mut &self.bytes[self.bytes_read..]).poll_read(cx, buf))?; + self.bytes_read += n; + Poll::Ready(Ok(n)) + } + } +} + +/// Appends `.meta` to the given path. +pub(crate) fn get_meta_path(path: &Path) -> PathBuf { + let mut meta_path = path.to_path_buf(); + let mut extension = path + .extension() + .expect("asset paths must have extensions") + .to_os_string(); + extension.push(".meta"); + meta_path.set_extension(extension); + meta_path +} + +/// A [`PathBuf`] [`Stream`] implementation that immediately returns nothing. +struct EmptyPathStream; + +impl Stream for EmptyPathStream { + type Item = PathBuf; + + fn poll_next( + self: Pin<&mut Self>, + _cx: &mut std::task::Context<'_>, + ) -> Poll> { + Poll::Ready(None) + } +} diff --git a/crates/bevy_asset/src/io/processor_gated.rs b/crates/bevy_asset/src/io/processor_gated.rs new file mode 100644 index 0000000000000..fcf891a8f7174 --- /dev/null +++ b/crates/bevy_asset/src/io/processor_gated.rs @@ -0,0 +1,163 @@ +use crate::{ + io::{AssetReader, AssetReaderError, PathStream, Reader}, + processor::{AssetProcessorData, ProcessStatus}, + AssetPath, +}; +use anyhow::Result; +use async_lock::RwLockReadGuardArc; +use bevy_log::trace; +use bevy_utils::BoxedFuture; +use futures_io::AsyncRead; +use std::{path::Path, pin::Pin, sync::Arc}; + +/// An [`AssetReader`] that will prevent asset (and asset metadata) read futures from returning for a +/// given path until that path has been processed by [`AssetProcessor`]. +/// +/// [`AssetProcessor`]: crate::processor::AssetProcessor +pub struct ProcessorGatedReader { + reader: Box, + processor_data: Arc, +} + +impl ProcessorGatedReader { + /// Creates a new [`ProcessorGatedReader`]. + pub fn new(reader: Box, processor_data: Arc) -> Self { + Self { + processor_data, + reader, + } + } + + /// Gets a "transaction lock" that can be used to ensure no writes to asset or asset meta occur + /// while it is held. + async fn get_transaction_lock( + &self, + path: &Path, + ) -> Result, AssetReaderError> { + let infos = self.processor_data.asset_infos.read().await; + let info = infos + .get(&AssetPath::new(path.to_owned(), None)) + .ok_or_else(|| AssetReaderError::NotFound(path.to_owned()))?; + Ok(info.file_transaction_lock.read_arc().await) + } +} + +impl AssetReader for ProcessorGatedReader { + fn read<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + Box::pin(async move { + trace!("Waiting for processing to finish before reading {:?}", path); + let process_result = self.processor_data.wait_until_processed(path).await; + match process_result { + ProcessStatus::Processed => {} + ProcessStatus::Failed | ProcessStatus::NonExistent => { + return Err(AssetReaderError::NotFound(path.to_owned())) + } + } + trace!( + "Processing finished with {:?}, reading {:?}", + process_result, + path + ); + let lock = self.get_transaction_lock(path).await?; + let asset_reader = self.reader.read(path).await?; + let reader: Box> = + Box::new(TransactionLockedReader::new(asset_reader, lock)); + Ok(reader) + }) + } + + fn read_meta<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + Box::pin(async move { + trace!( + "Waiting for processing to finish before reading meta {:?}", + path + ); + let process_result = self.processor_data.wait_until_processed(path).await; + match process_result { + ProcessStatus::Processed => {} + ProcessStatus::Failed | ProcessStatus::NonExistent => { + return Err(AssetReaderError::NotFound(path.to_owned())); + } + } + trace!( + "Processing finished with {:?}, reading meta {:?}", + process_result, + path + ); + let lock = self.get_transaction_lock(path).await?; + let meta_reader = self.reader.read_meta(path).await?; + let reader: Box> = Box::new(TransactionLockedReader::new(meta_reader, lock)); + Ok(reader) + }) + } + + fn read_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetReaderError>> { + Box::pin(async move { + trace!( + "Waiting for processing to finish before reading directory {:?}", + path + ); + self.processor_data.wait_until_finished().await; + trace!("Processing finished, reading directory {:?}", path); + let result = self.reader.read_directory(path).await?; + Ok(result) + }) + } + + fn is_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result> { + Box::pin(async move { + trace!( + "Waiting for processing to finish before reading directory {:?}", + path + ); + self.processor_data.wait_until_finished().await; + trace!("Processing finished, getting directory status {:?}", path); + let result = self.reader.is_directory(path).await?; + Ok(result) + }) + } + + fn watch_for_changes( + &self, + event_sender: crossbeam_channel::Sender, + ) -> Option> { + self.reader.watch_for_changes(event_sender) + } +} + +/// An [`AsyncRead`] impl that will hold its asset's transaction lock until [`TransactionLockedReader`] is dropped. +pub struct TransactionLockedReader<'a> { + reader: Box>, + _file_transaction_lock: RwLockReadGuardArc<()>, +} + +impl<'a> TransactionLockedReader<'a> { + fn new(reader: Box>, file_transaction_lock: RwLockReadGuardArc<()>) -> Self { + Self { + reader, + _file_transaction_lock: file_transaction_lock, + } + } +} + +impl<'a> AsyncRead for TransactionLockedReader<'a> { + fn poll_read( + mut self: std::pin::Pin<&mut Self>, + cx: &mut std::task::Context<'_>, + buf: &mut [u8], + ) -> std::task::Poll> { + Pin::new(&mut self.reader).poll_read(cx, buf) + } +} diff --git a/crates/bevy_asset/src/io/provider.rs b/crates/bevy_asset/src/io/provider.rs new file mode 100644 index 0000000000000..d41d8248ce042 --- /dev/null +++ b/crates/bevy_asset/src/io/provider.rs @@ -0,0 +1,190 @@ +use bevy_ecs::system::Resource; +use bevy_utils::HashMap; + +use crate::{ + io::{AssetReader, AssetWriter}, + AssetPlugin, +}; + +/// A reference to an "asset provider", which maps to an [`AssetReader`] and/or [`AssetWriter`]. +#[derive(Default, Clone, Debug)] +pub enum AssetProvider { + /// The default asset provider + #[default] + Default, + /// A custom / named asset provider + Custom(String), +} + +/// A [`Resource`] that hold (repeatable) functions capable of producing new [`AssetReader`] and [`AssetWriter`] instances +/// for a given [`AssetProvider`]. +#[derive(Resource, Default)] +pub struct AssetProviders { + readers: HashMap Box + Send + Sync>>, + writers: HashMap Box + Send + Sync>>, + default_file_source: Option, + default_file_destination: Option, +} + +impl AssetProviders { + /// Inserts a new `get_reader` function with the given `provider` name. This function will be used to create new [`AssetReader`]s + /// when they are requested for the given `provider`. + pub fn insert_reader( + &mut self, + provider: &str, + get_reader: impl FnMut() -> Box + Send + Sync + 'static, + ) { + self.readers + .insert(provider.to_string(), Box::new(get_reader)); + } + /// Inserts a new `get_reader` function with the given `provider` name. This function will be used to create new [`AssetReader`]s + /// when they are requested for the given `provider`. + pub fn with_reader( + mut self, + provider: &str, + get_reader: impl FnMut() -> Box + Send + Sync + 'static, + ) -> Self { + self.insert_reader(provider, get_reader); + self + } + /// Inserts a new `get_writer` function with the given `provider` name. This function will be used to create new [`AssetWriter`]s + /// when they are requested for the given `provider`. + pub fn insert_writer( + &mut self, + provider: &str, + get_writer: impl FnMut() -> Box + Send + Sync + 'static, + ) { + self.writers + .insert(provider.to_string(), Box::new(get_writer)); + } + /// Inserts a new `get_writer` function with the given `provider` name. This function will be used to create new [`AssetWriter`]s + /// when they are requested for the given `provider`. + pub fn with_writer( + mut self, + provider: &str, + get_writer: impl FnMut() -> Box + Send + Sync + 'static, + ) -> Self { + self.insert_writer(provider, get_writer); + self + } + /// Returns the default "asset source" path for the [`FileAssetReader`] and [`FileAssetWriter`]. + /// + /// [`FileAssetReader`]: crate::io::file::FileAssetReader + /// [`FileAssetWriter`]: crate::io::file::FileAssetWriter + pub fn default_file_source(&self) -> &str { + self.default_file_source + .as_deref() + .unwrap_or(AssetPlugin::DEFAULT_FILE_SOURCE) + } + + /// Sets the default "asset source" path for the [`FileAssetReader`] and [`FileAssetWriter`]. + /// + /// [`FileAssetReader`]: crate::io::file::FileAssetReader + /// [`FileAssetWriter`]: crate::io::file::FileAssetWriter + pub fn with_default_file_source(mut self, path: String) -> Self { + self.default_file_source = Some(path); + self + } + + /// Sets the default "asset destination" path for the [`FileAssetReader`] and [`FileAssetWriter`]. + /// + /// [`FileAssetReader`]: crate::io::file::FileAssetReader + /// [`FileAssetWriter`]: crate::io::file::FileAssetWriter + pub fn with_default_file_destination(mut self, path: String) -> Self { + self.default_file_destination = Some(path); + self + } + + /// Returns the default "asset destination" path for the [`FileAssetReader`] and [`FileAssetWriter`]. + /// + /// [`FileAssetReader`]: crate::io::file::FileAssetReader + /// [`FileAssetWriter`]: crate::io::file::FileAssetWriter + pub fn default_file_destination(&self) -> &str { + self.default_file_destination + .as_deref() + .unwrap_or(AssetPlugin::DEFAULT_FILE_DESTINATION) + } + + /// Returns a new "source" [`AssetReader`] for the given [`AssetProvider`]. + pub fn get_source_reader(&mut self, provider: &AssetProvider) -> Box { + match provider { + AssetProvider::Default => { + #[cfg(all(not(target_arch = "wasm32"), not(target_os = "android")))] + let reader = super::file::FileAssetReader::new(self.default_file_source()); + #[cfg(target_arch = "wasm32")] + let reader = super::wasm::HttpWasmAssetReader::new(self.default_file_source()); + #[cfg(target_os = "android")] + let reader = super::android::AndroidAssetReader; + Box::new(reader) + } + AssetProvider::Custom(provider) => { + let get_reader = self + .readers + .get_mut(provider) + .unwrap_or_else(|| panic!("Asset Provider {} does not exist", provider)); + (get_reader)() + } + } + } + /// Returns a new "destination" [`AssetReader`] for the given [`AssetProvider`]. + pub fn get_destination_reader(&mut self, provider: &AssetProvider) -> Box { + match provider { + AssetProvider::Default => { + #[cfg(all(not(target_arch = "wasm32"), not(target_os = "android")))] + let reader = super::file::FileAssetReader::new(self.default_file_destination()); + #[cfg(target_arch = "wasm32")] + let reader = super::wasm::HttpWasmAssetReader::new(self.default_file_destination()); + #[cfg(target_os = "android")] + let reader = super::android::AndroidAssetReader; + Box::new(reader) + } + AssetProvider::Custom(provider) => { + let get_reader = self + .readers + .get_mut(provider) + .unwrap_or_else(|| panic!("Asset Provider {} does not exist", provider)); + (get_reader)() + } + } + } + /// Returns a new "source" [`AssetWriter`] for the given [`AssetProvider`]. + pub fn get_source_writer(&mut self, provider: &AssetProvider) -> Box { + match provider { + AssetProvider::Default => { + #[cfg(all(not(target_arch = "wasm32"), not(target_os = "android")))] + return Box::new(super::file::FileAssetWriter::new( + self.default_file_source(), + )); + #[cfg(any(target_arch = "wasm32", target_os = "android"))] + panic!("Writing assets isn't supported on this platform yet"); + } + AssetProvider::Custom(provider) => { + let get_writer = self + .writers + .get_mut(provider) + .unwrap_or_else(|| panic!("Asset Provider {} does not exist", provider)); + (get_writer)() + } + } + } + /// Returns a new "destination" [`AssetWriter`] for the given [`AssetProvider`]. + pub fn get_destination_writer(&mut self, provider: &AssetProvider) -> Box { + match provider { + AssetProvider::Default => { + #[cfg(all(not(target_arch = "wasm32"), not(target_os = "android")))] + return Box::new(super::file::FileAssetWriter::new( + self.default_file_destination(), + )); + #[cfg(any(target_arch = "wasm32", target_os = "android"))] + panic!("Writing assets isn't supported on this platform yet"); + } + AssetProvider::Custom(provider) => { + let get_writer = self + .writers + .get_mut(provider) + .unwrap_or_else(|| panic!("Asset Provider {} does not exist", provider)); + (get_writer)() + } + } + } +} diff --git a/crates/bevy_asset/src/io/wasm.rs b/crates/bevy_asset/src/io/wasm.rs new file mode 100644 index 0000000000000..a90a5a0569379 --- /dev/null +++ b/crates/bevy_asset/src/io/wasm.rs @@ -0,0 +1,110 @@ +use crate::io::{ + get_meta_path, AssetReader, AssetReaderError, AssetWatcher, EmptyPathStream, PathStream, + Reader, VecReader, +}; +use anyhow::Result; +use bevy_log::error; +use bevy_utils::BoxedFuture; +use js_sys::{Uint8Array, JSON}; +use std::path::{Path, PathBuf}; +use wasm_bindgen::{JsCast, JsValue}; +use wasm_bindgen_futures::JsFuture; +use web_sys::Response; + +/// Reader implementation for loading assets via HTTP in WASM. +pub struct HttpWasmAssetReader { + root_path: PathBuf, +} + +impl HttpWasmAssetReader { + /// Creates a new `WasmAssetReader`. The path provided will be used to build URLs to query for assets. + pub fn new>(path: P) -> Self { + Self { + root_path: path.as_ref().to_owned(), + } + } +} + +fn js_value_to_err<'a>(context: &'a str) -> impl FnOnce(JsValue) -> std::io::Error + 'a { + move |value| { + let message = match JSON::stringify(&value) { + Ok(js_str) => format!("Failed to {context}: {js_str}"), + Err(_) => { + format!("Failed to {context} and also failed to stringify the JSValue of the error") + } + }; + + std::io::Error::new(std::io::ErrorKind::Other, message) + } +} + +impl HttpWasmAssetReader { + async fn fetch_bytes<'a>(&self, path: PathBuf) -> Result>, AssetReaderError> { + let window = web_sys::window().unwrap(); + let resp_value = JsFuture::from(window.fetch_with_str(path.to_str().unwrap())) + .await + .map_err(js_value_to_err("fetch path"))?; + let resp = resp_value + .dyn_into::() + .map_err(js_value_to_err("convert fetch to Response"))?; + match resp.status() { + 200 => { + let data = JsFuture::from(resp.array_buffer().unwrap()).await.unwrap(); + let bytes = Uint8Array::new(&data).to_vec(); + let reader: Box = Box::new(VecReader::new(bytes)); + Ok(reader) + } + 404 => Err(AssetReaderError::NotFound(path)), + status => Err(AssetReaderError::Io(std::io::Error::new( + std::io::ErrorKind::Other, + format!("Encountered unexpected HTTP status {status}"), + ))), + } + } +} + +impl AssetReader for HttpWasmAssetReader { + fn read<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + Box::pin(async move { + let path = self.root_path.join(path); + self.fetch_bytes(path).await + }) + } + + fn read_meta<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + Box::pin(async move { + let meta_path = get_meta_path(path); + Ok(self.fetch_bytes(meta_path).await?) + }) + } + + fn read_directory<'a>( + &'a self, + _path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetReaderError>> { + let stream: Box = Box::new(EmptyPathStream); + error!("Reading directories is not supported with the HttpWasmAssetReader"); + Box::pin(async move { Ok(stream) }) + } + + fn is_directory<'a>( + &'a self, + _path: &'a Path, + ) -> BoxedFuture<'a, std::result::Result> { + error!("Reading directories is not supported with the HttpWasmAssetReader"); + Box::pin(async move { Ok(false) }) + } + + fn watch_for_changes( + &self, + _event_sender: crossbeam_channel::Sender, + ) -> Option> { + None + } +} diff --git a/crates/bevy_asset/src/io/wasm_asset_io.rs b/crates/bevy_asset/src/io/wasm_asset_io.rs deleted file mode 100644 index 00afb398ea5e6..0000000000000 --- a/crates/bevy_asset/src/io/wasm_asset_io.rs +++ /dev/null @@ -1,85 +0,0 @@ -use crate::{AssetIo, AssetIoError, ChangeWatcher, Metadata}; -use anyhow::Result; -use bevy_utils::BoxedFuture; -use js_sys::Uint8Array; -use std::{ - convert::TryFrom, - path::{Path, PathBuf}, -}; -use wasm_bindgen::JsCast; -use wasm_bindgen_futures::JsFuture; -use web_sys::Response; - -/// I/O implementation for web builds. -/// -/// Implementation details: -/// -/// - `load_path` makes [fetch()] requests. -/// - `read_directory` always returns an empty iterator. -/// - `get_metadata` will always return an error. -/// - Watching for changes is not supported. The watcher methods will do nothing. -/// -/// [fetch()]: https://developer.mozilla.org/en-US/docs/Web/API/fetch -pub struct WasmAssetIo { - root_path: PathBuf, -} - -impl WasmAssetIo { - /// Creates a new `WasmAssetIo`. The path provided will be used to build URLs to query for assets. - pub fn new>(path: P) -> Self { - WasmAssetIo { - root_path: path.as_ref().to_owned(), - } - } -} - -impl AssetIo for WasmAssetIo { - fn load_path<'a>(&'a self, path: &'a Path) -> BoxedFuture<'a, Result, AssetIoError>> { - Box::pin(async move { - let path = self.root_path.join(path); - let window = web_sys::window().unwrap(); - let resp_value = JsFuture::from(window.fetch_with_str(path.to_str().unwrap())) - .await - .unwrap(); - let resp: Response = resp_value.dyn_into().unwrap(); - let data = JsFuture::from(resp.array_buffer().unwrap()).await.unwrap(); - let bytes = Uint8Array::new(&data).to_vec(); - Ok(bytes) - }) - } - - fn read_directory( - &self, - _path: &Path, - ) -> Result>, AssetIoError> { - bevy_log::warn!("Loading folders is not supported in WASM"); - Ok(Box::new(std::iter::empty::())) - } - - fn watch_path_for_changes( - &self, - _to_watch: &Path, - _to_reload: Option, - ) -> Result<(), AssetIoError> { - Ok(()) - } - - fn watch_for_changes(&self, _configuration: &ChangeWatcher) -> Result<(), AssetIoError> { - bevy_log::warn!("Watching for changes is not supported in WASM"); - Ok(()) - } - - fn get_metadata(&self, path: &Path) -> Result { - let full_path = self.root_path.join(path); - full_path - .metadata() - .and_then(Metadata::try_from) - .map_err(|e| { - if e.kind() == std::io::ErrorKind::NotFound { - AssetIoError::NotFound(full_path) - } else { - e.into() - } - }) - } -} diff --git a/crates/bevy_asset/src/lib.rs b/crates/bevy_asset/src/lib.rs index cc325d26c4dd5..a8cdc776743b6 100644 --- a/crates/bevy_asset/src/lib.rs +++ b/crates/bevy_asset/src/lib.rs @@ -1,149 +1,1168 @@ -//! Built-in plugin for asset support. -//! -//! This plugin allows a bevy app to work with assets from the filesystem (or [another source]), -//! providing an [asset server] for loading and processing [`Asset`]s and storing them in an -//! [asset storage] to be accessed by systems. -//! -//! [another source]: trait.AssetIo.html -//! [asset server]: struct.AssetServer.html -//! [asset storage]: struct.Assets.html - -#![warn(missing_docs)] -#![allow(clippy::type_complexity)] - -mod asset_server; -mod assets; -#[cfg(feature = "debug_asset_server")] -pub mod debug_asset_server; -pub mod diagnostic; -#[cfg(all( - feature = "filesystem_watcher", - all(not(target_arch = "wasm32"), not(target_os = "android")) -))] -mod filesystem_watcher; -mod handle; -mod info; -mod io; -mod loader; -mod path; -mod reflect; +pub mod io; +pub mod meta; +pub mod processor; +pub mod saver; -/// The `bevy_asset` prelude. pub mod prelude { #[doc(hidden)] pub use crate::{ - AddAsset, AssetEvent, AssetPlugin, AssetServer, Assets, Handle, HandleUntyped, + Asset, AssetApp, AssetEvent, AssetId, AssetPlugin, AssetServer, Assets, Handle, + UntypedHandle, }; } -pub use anyhow::Error; -pub use asset_server::*; +mod assets; +mod event; +mod folder; +mod handle; +mod id; +mod loader; +mod path; +mod reflect; +mod server; + pub use assets::*; -pub use bevy_utils::BoxedFuture; +pub use bevy_asset_macros::Asset; +pub use event::*; +pub use folder::*; +pub use futures_lite::{AsyncReadExt, AsyncWriteExt}; pub use handle::*; -pub use info::*; -pub use io::*; +pub use id::*; pub use loader::*; pub use path::*; pub use reflect::*; +pub use server::*; -use bevy_app::{prelude::*, MainScheduleOrder}; -use bevy_ecs::schedule::ScheduleLabel; -use bevy_utils::Duration; +pub use anyhow; -/// Asset storages are updated. -#[derive(Debug, Hash, PartialEq, Eq, Clone, ScheduleLabel)] -pub struct LoadAssets; -/// Asset events are generated. -#[derive(Debug, Hash, PartialEq, Eq, Clone, ScheduleLabel)] -pub struct AssetEvents; +use crate::{ + io::{processor_gated::ProcessorGatedReader, AssetProvider, AssetProviders}, + processor::{AssetProcessor, Process}, +}; +use bevy_app::{App, First, MainScheduleOrder, Plugin, PostUpdate, Startup}; +use bevy_ecs::{ + reflect::AppTypeRegistry, + schedule::{IntoSystemConfigs, IntoSystemSetConfigs, ScheduleLabel, SystemSet}, + world::FromWorld, +}; +use bevy_reflect::{FromReflect, GetTypeRegistration, Reflect, TypePath}; +use std::{any::TypeId, sync::Arc}; -/// Configuration for hot reloading assets by watching for changes. -#[derive(Debug, Clone)] -pub struct ChangeWatcher { - /// Minimum delay after which a file change will trigger a reload. +/// Provides "asset" loading and processing functionality. An [`Asset`] is a "runtime value" that is loaded from an [`AssetProvider`], +/// which can be something like a filesystem, a network, etc. +/// +/// Supports flexible "modes", such as [`AssetPlugin::Processed`] and +/// [`AssetPlugin::Unprocessed`] that enable using the asset workflow that best suits your project. +pub enum AssetPlugin { + /// Loads assets without any "preprocessing" from the configured asset `source` (defaults to the `assets` folder). + Unprocessed { + source: AssetProvider, + watch_for_changes: bool, + }, + /// Loads "processed" assets from a given `destination` source (defaults to the `imported_assets/Default` folder). This should + /// generally only be used when distributing apps. Use [`AssetPlugin::ProcessedDev`] to develop apps that process assets, + /// then switch to [`AssetPlugin::Processed`] when deploying the apps. + Processed { + destination: AssetProvider, + watch_for_changes: bool, + }, + /// Starts an [`AssetProcessor`] in the background that reads assets from the `source` provider (defaults to the `assets` folder), + /// processes them according to their [`AssetMeta`], and writes them to the `destination` provider (defaults to the `imported_assets/Default` folder). /// - /// The change watcher will wait for this duration after a file change before reloading the - /// asset. This is useful to avoid reloading an asset multiple times when it is changed - /// multiple times in a short period of time, or to avoid reloading an asset that is still - /// being written to. + /// By default this will hot reload changes to the `source` provider, resulting in reprocessing the asset and reloading it in the [`App`]. /// - /// If you have a slow hard drive or expect to reload large assets, you may want to increase - /// this value. - pub delay: Duration, + /// [`AssetMeta`]: crate::meta::AssetMeta + ProcessedDev { + source: AssetProvider, + destination: AssetProvider, + watch_for_changes: bool, + }, } -impl ChangeWatcher { - /// Enable change watching with the given delay when a file is changed. - /// - /// See [`Self::delay`] for more details on how this value is used. - pub fn with_delay(delay: Duration) -> Option { - Some(Self { delay }) +impl Default for AssetPlugin { + fn default() -> Self { + Self::unprocessed() } } -/// Adds support for [`Assets`] to an App. -/// -/// Assets are typed collections with change tracking, which are added as App Resources. Examples of -/// assets: textures, sounds, 3d models, maps, scenes -#[derive(Debug, Clone)] -pub struct AssetPlugin { - /// The base folder where assets are loaded from, relative to the executable. - pub asset_folder: String, - /// Whether to watch for changes in asset files. Requires the `filesystem_watcher` feature, - /// and cannot be supported on the wasm32 arch nor android os. - pub watch_for_changes: Option, +impl AssetPlugin { + const DEFAULT_FILE_SOURCE: &str = "assets"; + /// NOTE: this is in the Default sub-folder to make this forward compatible with "import profiles" + /// and to allow us to put the "processor transaction log" at `imported_assets/log` + const DEFAULT_FILE_DESTINATION: &str = "imported_assets/Default"; + + /// Returns the default [`AssetPlugin::Processed`] configuration + pub fn processed() -> Self { + Self::Processed { + destination: Default::default(), + watch_for_changes: false, + } + } + + /// Returns the default [`AssetPlugin::ProcessedDev`] configuration + pub fn processed_dev() -> Self { + Self::ProcessedDev { + source: Default::default(), + destination: Default::default(), + watch_for_changes: true, + } + } + + /// Returns the default [`AssetPlugin::Unprocessed`] configuration + pub fn unprocessed() -> Self { + Self::Unprocessed { + source: Default::default(), + watch_for_changes: false, + } + } + + /// Enables watching for changes, which will hot-reload assets when they change. + pub fn watch_for_changes(mut self) -> Self { + match &mut self { + AssetPlugin::Unprocessed { + watch_for_changes, .. + } + | AssetPlugin::Processed { + watch_for_changes, .. + } + | AssetPlugin::ProcessedDev { + watch_for_changes, .. + } => *watch_for_changes = true, + }; + self + } } -impl Default for AssetPlugin { - fn default() -> Self { - Self { - asset_folder: "assets".to_string(), - watch_for_changes: None, +impl Plugin for AssetPlugin { + fn build(&self, app: &mut App) { + app.init_schedule(UpdateAssets) + .init_schedule(AssetEvents) + .init_resource::(); + { + match self { + AssetPlugin::Unprocessed { + source, + watch_for_changes, + } => { + let source_reader = app + .world + .resource_mut::() + .get_source_reader(source); + app.insert_resource(AssetServer::new(source_reader, *watch_for_changes)); + } + AssetPlugin::Processed { + destination, + watch_for_changes, + } => { + let destination_reader = app + .world + .resource_mut::() + .get_destination_reader(destination); + app.insert_resource(AssetServer::new(destination_reader, *watch_for_changes)); + } + AssetPlugin::ProcessedDev { + source, + destination, + watch_for_changes, + } => { + let mut asset_providers = app.world.resource_mut::(); + let processor = AssetProcessor::new(&mut asset_providers, source, destination); + let destination_reader = asset_providers.get_destination_reader(source); + // the main asset server gates loads based on asset state + let gated_reader = + ProcessorGatedReader::new(destination_reader, processor.data.clone()); + // the main asset server shares loaders with the processor asset server + app.insert_resource(AssetServer::new_with_loaders( + Box::new(gated_reader), + processor.server().data.loaders.clone(), + *watch_for_changes, + )) + .insert_resource(processor) + .add_systems(Startup, AssetProcessor::start); + } + } } + app.init_asset::() + .init_asset::<()>() + .configure_sets( + UpdateAssets, + TrackAssets.after(server::handle_internal_asset_events), + ) + .add_systems(UpdateAssets, server::handle_internal_asset_events); + + let mut order = app.world.resource_mut::(); + order.insert_after(First, UpdateAssets); + order.insert_after(PostUpdate, AssetEvents); } } -impl AssetPlugin { - /// Creates an instance of the platform's default [`AssetIo`]. +pub trait Asset: VisitAssetDependencies + TypePath + Send + Sync + 'static {} + +pub trait VisitAssetDependencies { + fn visit_dependencies(&self, visit: &mut impl FnMut(UntypedAssetId)); +} + +impl VisitAssetDependencies for Handle { + fn visit_dependencies(&self, visit: &mut impl FnMut(UntypedAssetId)) { + visit(self.id().untyped()); + } +} + +impl VisitAssetDependencies for Option> { + fn visit_dependencies(&self, visit: &mut impl FnMut(UntypedAssetId)) { + if let Some(handle) = self { + visit(handle.id().untyped()); + } + } +} + +impl VisitAssetDependencies for UntypedHandle { + fn visit_dependencies(&self, visit: &mut impl FnMut(UntypedAssetId)) { + visit(self.id()); + } +} + +impl VisitAssetDependencies for Option { + fn visit_dependencies(&self, visit: &mut impl FnMut(UntypedAssetId)) { + if let Some(handle) = self { + visit(handle.id()); + } + } +} + +impl VisitAssetDependencies for Vec> { + fn visit_dependencies(&self, visit: &mut impl FnMut(UntypedAssetId)) { + for dependency in self.iter() { + visit(dependency.id().untyped()); + } + } +} + +impl VisitAssetDependencies for Vec { + fn visit_dependencies(&self, visit: &mut impl FnMut(UntypedAssetId)) { + for dependency in self.iter() { + visit(dependency.id()); + } + } +} + +/// Adds asset-related builder methods to [`App`]. +pub trait AssetApp { + /// Registers the given `loader` in the [`App`]'s [`AssetServer`]. + fn register_asset_loader(&mut self, loader: L) -> &mut Self; + /// Registers the given `processor` in the [`App`]'s [`AssetProcessor`]. + fn register_asset_processor(&mut self, processor: P) -> &mut Self; + /// Sets the default asset processor for the given `extension`. + fn set_default_asset_processor(&mut self, extension: &str) -> &mut Self; + /// Initializes the given loader in the [`App`]'s [`AssetServer`]. + fn init_asset_loader(&mut self) -> &mut Self; + /// Initializes the given [`Asset`] in the [`App`] by: + /// * Registering the [`Asset`] in the [`AssetServer`] + /// * Initializing the [`AssetEvent`] resource for the [`Asset`] + /// * Adding other relevant systems and resources for the [`Asset`] + fn init_asset(&mut self) -> &mut Self; + /// Registers the asset type `T` using `[App::register]`, + /// and adds [`ReflectAsset`] type data to `T` and [`ReflectHandle`] type data to [`Handle`] in the type registry. /// - /// This is useful when providing a custom `AssetIo` instance that needs to - /// delegate to the default `AssetIo` for the platform. - pub fn create_platform_default_asset_io(&self) -> Box { - #[cfg(all(not(target_arch = "wasm32"), not(target_os = "android")))] - let source = FileAssetIo::new(&self.asset_folder, &self.watch_for_changes); - #[cfg(target_arch = "wasm32")] - let source = WasmAssetIo::new(&self.asset_folder); - #[cfg(target_os = "android")] - let source = AndroidAssetIo::new(&self.asset_folder); + /// This enables reflection code to access assets. For detailed information, see the docs on [`ReflectAsset`] and [`ReflectHandle`]. + fn register_asset_reflect(&mut self) -> &mut Self + where + A: Asset + Reflect + FromReflect + GetTypeRegistration; + /// Preregisters a loader for the given extensions, that will block asset loads until a real loader + /// is registered. + fn preregister_asset_loader(&mut self, extensions: &[&str]) -> &mut Self; +} + +impl AssetApp for App { + fn register_asset_loader(&mut self, loader: L) -> &mut Self { + self.world.resource::().register_loader(loader); + self + } + + fn init_asset_loader(&mut self) -> &mut Self { + let loader = L::from_world(&mut self.world); + self.register_asset_loader(loader) + } + + fn init_asset(&mut self) -> &mut Self { + let assets = Assets::::default(); + self.world.resource::().register_asset(&assets); + if self.world.contains_resource::() { + let processor = self.world.resource::(); + // The processor should have its own handle provider separate from the Asset storage + // to ensure the id spaces are entirely separate. Not _strictly_ necessary, but + // desirable. + processor + .server() + .register_handle_provider(AssetHandleProvider::new( + TypeId::of::(), + Arc::new(AssetIndexAllocator::default()), + )); + } + self.insert_resource(assets) + .add_event::>() + .register_type::>() + .register_type::>() + .add_systems(AssetEvents, Assets::::asset_events) + .add_systems(UpdateAssets, Assets::::track_assets.in_set(TrackAssets)) + } - Box::new(source) + fn register_asset_reflect(&mut self) -> &mut Self + where + A: Asset + Reflect + FromReflect + GetTypeRegistration, + { + let type_registry = self.world.resource::(); + { + let mut type_registry = type_registry.write(); + + type_registry.register::(); + type_registry.register::>(); + type_registry.register_type_data::(); + type_registry.register_type_data::, ReflectHandle>(); + } + + self + } + + fn preregister_asset_loader(&mut self, extensions: &[&str]) -> &mut Self { + self.world + .resource_mut::() + .preregister_loader::(extensions); + self + } + + fn register_asset_processor(&mut self, processor: P) -> &mut Self { + if let Some(asset_processor) = self.world.get_resource::() { + asset_processor.register_processor(processor); + } + self + } + + fn set_default_asset_processor(&mut self, extension: &str) -> &mut Self { + if let Some(asset_processor) = self.world.get_resource::() { + asset_processor.set_default_processor::

(extension); + } + self } } -impl Plugin for AssetPlugin { - fn build(&self, app: &mut App) { - if !app.world.contains_resource::() { - let source = self.create_platform_default_asset_io(); - let asset_server = AssetServer::with_boxed_io(source); - app.insert_resource(asset_server); +/// A system set that holds all "track asset" operations. +#[derive(SystemSet, Hash, Debug, PartialEq, Eq, Clone)] +pub struct TrackAssets; + +/// Schedule where [`Assets`] resources are updated. +#[derive(Debug, Hash, PartialEq, Eq, Clone, ScheduleLabel)] +pub struct UpdateAssets; + +/// Schedule where events accumulated in [`Assets`] are applied to the [`AssetEvent`] [`Events`] resource. +/// +/// [`Events`]: bevy_ecs::event::Events +#[derive(Debug, Hash, PartialEq, Eq, Clone, ScheduleLabel)] +pub struct AssetEvents; + +/// Loads an "internal" asset by embedding the string stored in the given `path_str` and associates it with the given handle. +#[macro_export] +macro_rules! load_internal_asset { + ($app: ident, $handle: expr, $path_str: expr, $loader: expr) => {{ + let mut assets = $app.world.resource_mut::<$crate::Assets<_>>(); + assets.insert($handle, ($loader)( + include_str!($path_str), + std::path::Path::new(file!()) + .parent() + .unwrap() + .join($path_str) + .to_string_lossy() + )); + }}; + // we can't support params without variadic arguments, so internal assets with additional params can't be hot-reloaded + ($app: ident, $handle: ident, $path_str: expr, $loader: expr $(, $param:expr)+) => {{ + let mut assets = $app.world.resource_mut::<$crate::Assets<_>>(); + assets.insert($handle, ($loader)( + include_str!($path_str), + std::path::Path::new(file!()) + .parent() + .unwrap() + .join($path_str) + .to_string_lossy(), + $($param),+ + )); + }}; +} + +/// Loads an "internal" binary asset by embedding the bytes stored in the given `path_str` and associates it with the given handle. +#[macro_export] +macro_rules! load_internal_binary_asset { + ($app: ident, $handle: expr, $path_str: expr, $loader: expr) => {{ + let mut assets = $app.world.resource_mut::<$crate::Assets<_>>(); + assets.insert( + $handle, + ($loader)( + include_bytes!($path_str).as_ref(), + std::path::Path::new(file!()) + .parent() + .unwrap() + .join($path_str) + .to_string_lossy() + .into(), + ), + ); + }}; +} + +#[cfg(test)] +mod tests { + use crate::{ + self as bevy_asset, + folder::LoadedFolder, + handle::Handle, + io::{ + gated::{GateOpener, GatedReader}, + memory::{Dir, MemoryAssetReader}, + Reader, + }, + loader::{AssetLoader, LoadContext}, + Asset, AssetApp, AssetEvent, AssetId, AssetPlugin, AssetProvider, AssetProviders, + AssetServer, Assets, DependencyLoadState, LoadState, RecursiveDependencyLoadState, + }; + use bevy_app::{App, Update}; + use bevy_core::TaskPoolPlugin; + use bevy_ecs::event::ManualEventReader; + use bevy_ecs::prelude::*; + use bevy_log::LogPlugin; + use bevy_reflect::TypePath; + use bevy_utils::BoxedFuture; + use futures_lite::AsyncReadExt; + use serde::{Deserialize, Serialize}; + use std::path::Path; + + #[derive(Asset, TypePath, Debug)] + pub struct CoolText { + text: String, + embedded: String, + #[dependency] + dependencies: Vec>, + #[dependency] + sub_texts: Vec>, + } + + #[derive(Asset, TypePath, Debug)] + pub struct SubText { + text: String, + } + + #[derive(Serialize, Deserialize)] + pub struct CoolTextRon { + text: String, + dependencies: Vec, + embedded_dependencies: Vec, + sub_texts: Vec, + } + + #[derive(Default)] + struct CoolTextLoader; + + impl AssetLoader for CoolTextLoader { + type Asset = CoolText; + + type Settings = (); + + fn load<'a>( + &'a self, + reader: &'a mut Reader, + _settings: &'a Self::Settings, + load_context: &'a mut LoadContext, + ) -> BoxedFuture<'a, Result> { + Box::pin(async move { + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; + let mut ron: CoolTextRon = ron::de::from_bytes(&bytes)?; + let mut embedded = String::new(); + for dep in ron.embedded_dependencies { + let loaded = load_context.load_direct(&dep).await?; + let cool = loaded.get::().unwrap(); + embedded.push_str(&cool.text); + } + Ok(CoolText { + text: ron.text, + embedded, + dependencies: ron + .dependencies + .iter() + .map(|p| load_context.load(p)) + .collect(), + sub_texts: ron + .sub_texts + .drain(..) + .map(|text| load_context.add_labeled_asset(text.clone(), SubText { text })) + .collect(), + }) + }) + } + + fn extensions(&self) -> &[&str] { + &["cool.ron"] } + } - app.register_type::(); - app.register_type::(); + fn test_app(dir: Dir) -> (App, GateOpener) { + let mut app = App::new(); + let (gated_memory_reader, gate_opener) = GatedReader::new(MemoryAssetReader { root: dir }); + app.insert_resource( + AssetProviders::default() + .with_reader("Test", move || Box::new(gated_memory_reader.clone())), + ) + .add_plugins(( + TaskPoolPlugin::default(), + LogPlugin::default(), + AssetPlugin::Unprocessed { + source: AssetProvider::Custom("Test".to_string()), + watch_for_changes: false, + }, + )); + (app, gate_opener) + } - app.add_systems(PreUpdate, asset_server::free_unused_assets_system); - app.init_schedule(LoadAssets); - app.init_schedule(AssetEvents); + fn run_app_until(app: &mut App, mut predicate: impl FnMut(&mut World) -> Option<()>) { + for _ in 0..LARGE_ITERATION_COUNT { + app.update(); + if (predicate)(&mut app.world).is_some() { + return; + } + } - #[cfg(all( - feature = "filesystem_watcher", - all(not(target_arch = "wasm32"), not(target_os = "android")) - ))] - app.add_systems(LoadAssets, io::filesystem_watcher_system); + panic!("Ran out of loops to return `Some` from `predicate`"); + } - let mut order = app.world.resource_mut::(); - order.insert_after(First, LoadAssets); - order.insert_after(PostUpdate, AssetEvents); + const LARGE_ITERATION_COUNT: usize = 50; + + fn get(world: &World, id: AssetId) -> Option<&A> { + world.resource::>().get(id) + } + + #[derive(Resource, Default)] + struct StoredEvents(Vec>); + + fn store_asset_events( + mut reader: EventReader>, + mut storage: ResMut, + ) { + storage.0.extend(reader.read().cloned()); + } + + #[test] + fn load_dependencies() { + let dir = Dir::default(); + + let a_path = "a.cool.ron"; + let a_ron = r#" +( + text: "a", + dependencies: [ + "foo/b.cool.ron", + "c.cool.ron", + ], + embedded_dependencies: [], + sub_texts: [], +)"#; + let b_path = "foo/b.cool.ron"; + let b_ron = r#" +( + text: "b", + dependencies: [], + embedded_dependencies: [], + sub_texts: [], +)"#; + + let c_path = "c.cool.ron"; + let c_ron = r#" +( + text: "c", + dependencies: [ + "d.cool.ron", + ], + embedded_dependencies: ["a.cool.ron", "foo/b.cool.ron"], + sub_texts: ["hello"], +)"#; + + let d_path = "d.cool.ron"; + let d_ron = r#" +( + text: "d", + dependencies: [], + embedded_dependencies: [], + sub_texts: [], +)"#; + + dir.insert_asset_text(Path::new(a_path), a_ron); + dir.insert_asset_text(Path::new(b_path), b_ron); + dir.insert_asset_text(Path::new(c_path), c_ron); + dir.insert_asset_text(Path::new(d_path), d_ron); + + #[derive(Resource)] + struct IdResults { + b_id: AssetId, + c_id: AssetId, + d_id: AssetId, + } + + let (mut app, gate_opener) = test_app(dir); + app.init_asset::() + .init_asset::() + .init_resource::() + .register_asset_loader(CoolTextLoader) + .add_systems(Update, store_asset_events); + let asset_server = app.world.resource::().clone(); + let handle: Handle = asset_server.load(a_path); + let a_id = handle.id(); + let entity = app.world.spawn(handle).id(); + app.update(); + { + let a_text = get::(&app.world, a_id); + let (a_load, a_deps, a_rec_deps) = asset_server.get_load_states(a_id).unwrap(); + assert!(a_text.is_none(), "a's asset should not exist yet"); + assert_eq!(a_load, LoadState::Loading, "a should still be loading"); + assert_eq!( + a_deps, + DependencyLoadState::Loading, + "a deps should still be loading" + ); + assert_eq!( + a_rec_deps, + RecursiveDependencyLoadState::Loading, + "a recursive deps should still be loading" + ); + } + + // Allow "a" to load ... wait for it to finish loading and validate results + // Dependencies are still gated so they should not be loaded yet + gate_opener.open(a_path); + run_app_until(&mut app, |world| { + let a_text = get::(world, a_id)?; + let (a_load, a_deps, a_rec_deps) = asset_server.get_load_states(a_id).unwrap(); + assert_eq!(a_text.text, "a"); + assert_eq!(a_text.dependencies.len(), 2); + assert_eq!(a_load, LoadState::Loaded, "a is loaded"); + assert_eq!(a_deps, DependencyLoadState::Loading); + assert_eq!(a_rec_deps, RecursiveDependencyLoadState::Loading); + + let b_id = a_text.dependencies[0].id(); + let b_text = get::(world, b_id); + let (b_load, b_deps, b_rec_deps) = asset_server.get_load_states(b_id).unwrap(); + assert!(b_text.is_none(), "b component should not exist yet"); + assert_eq!(b_load, LoadState::Loading); + assert_eq!(b_deps, DependencyLoadState::Loading); + assert_eq!(b_rec_deps, RecursiveDependencyLoadState::Loading); + + let c_id = a_text.dependencies[1].id(); + let c_text = get::(world, c_id); + let (c_load, c_deps, c_rec_deps) = asset_server.get_load_states(c_id).unwrap(); + assert!(c_text.is_none(), "c component should not exist yet"); + assert_eq!(c_load, LoadState::Loading); + assert_eq!(c_deps, DependencyLoadState::Loading); + assert_eq!(c_rec_deps, RecursiveDependencyLoadState::Loading); + Some(()) + }); + + // Allow "b" to load ... wait for it to finish loading and validate results + // "c" should not be loaded yet + gate_opener.open(b_path); + run_app_until(&mut app, |world| { + let a_text = get::(world, a_id)?; + let (a_load, a_deps, a_rec_deps) = asset_server.get_load_states(a_id).unwrap(); + assert_eq!(a_text.text, "a"); + assert_eq!(a_text.dependencies.len(), 2); + assert_eq!(a_load, LoadState::Loaded); + assert_eq!(a_deps, DependencyLoadState::Loading); + assert_eq!(a_rec_deps, RecursiveDependencyLoadState::Loading); + + let b_id = a_text.dependencies[0].id(); + let b_text = get::(world, b_id)?; + let (b_load, b_deps, b_rec_deps) = asset_server.get_load_states(b_id).unwrap(); + assert_eq!(b_text.text, "b"); + assert_eq!(b_load, LoadState::Loaded); + assert_eq!(b_deps, DependencyLoadState::Loaded); + assert_eq!(b_rec_deps, RecursiveDependencyLoadState::Loaded); + + let c_id = a_text.dependencies[1].id(); + let c_text = get::(world, c_id); + let (c_load, c_deps, c_rec_deps) = asset_server.get_load_states(c_id).unwrap(); + assert!(c_text.is_none(), "c component should not exist yet"); + assert_eq!(c_load, LoadState::Loading); + assert_eq!(c_deps, DependencyLoadState::Loading); + assert_eq!(c_rec_deps, RecursiveDependencyLoadState::Loading); + Some(()) + }); + + // Allow "c" to load ... wait for it to finish loading and validate results + // all "a" dependencies should be loaded now + gate_opener.open(c_path); + + // Re-open a and b gates to allow c to load embedded deps (gates are closed after each load) + gate_opener.open(a_path); + gate_opener.open(b_path); + run_app_until(&mut app, |world| { + let a_text = get::(world, a_id)?; + let (a_load, a_deps, a_rec_deps) = asset_server.get_load_states(a_id).unwrap(); + assert_eq!(a_text.text, "a"); + assert_eq!(a_text.embedded, ""); + assert_eq!(a_text.dependencies.len(), 2); + assert_eq!(a_load, LoadState::Loaded); + + let b_id = a_text.dependencies[0].id(); + let b_text = get::(world, b_id)?; + let (b_load, b_deps, b_rec_deps) = asset_server.get_load_states(b_id).unwrap(); + assert_eq!(b_text.text, "b"); + assert_eq!(b_text.embedded, ""); + assert_eq!(b_load, LoadState::Loaded); + assert_eq!(b_deps, DependencyLoadState::Loaded); + assert_eq!(b_rec_deps, RecursiveDependencyLoadState::Loaded); + + let c_id = a_text.dependencies[1].id(); + let c_text = get::(world, c_id)?; + let (c_load, c_deps, c_rec_deps) = asset_server.get_load_states(c_id).unwrap(); + assert_eq!(c_text.text, "c"); + assert_eq!(c_text.embedded, "ab"); + assert_eq!(c_load, LoadState::Loaded); + assert_eq!( + c_deps, + DependencyLoadState::Loading, + "c deps should not be loaded yet because d has not loaded" + ); + assert_eq!( + c_rec_deps, + RecursiveDependencyLoadState::Loading, + "c rec deps should not be loaded yet because d has not loaded" + ); + + let sub_text_id = c_text.sub_texts[0].id(); + let sub_text = get::(world, sub_text_id) + .expect("subtext should exist if c exists. it came from the same loader"); + assert_eq!(sub_text.text, "hello"); + let (sub_text_load, sub_text_deps, sub_text_rec_deps) = + asset_server.get_load_states(sub_text_id).unwrap(); + assert_eq!(sub_text_load, LoadState::Loaded); + assert_eq!(sub_text_deps, DependencyLoadState::Loaded); + assert_eq!(sub_text_rec_deps, RecursiveDependencyLoadState::Loaded); + + let d_id = c_text.dependencies[0].id(); + let d_text = get::(world, d_id); + let (d_load, d_deps, d_rec_deps) = asset_server.get_load_states(d_id).unwrap(); + assert!(d_text.is_none(), "d component should not exist yet"); + assert_eq!(d_load, LoadState::Loading); + assert_eq!(d_deps, DependencyLoadState::Loading); + assert_eq!(d_rec_deps, RecursiveDependencyLoadState::Loading); + + assert_eq!( + a_deps, + DependencyLoadState::Loaded, + "If c has been loaded, the a deps should all be considered loaded" + ); + assert_eq!( + a_rec_deps, + RecursiveDependencyLoadState::Loading, + "d is not loaded, so a's recursive deps should still be loading" + ); + world.insert_resource(IdResults { b_id, c_id, d_id }); + Some(()) + }); + + gate_opener.open(d_path); + run_app_until(&mut app, |world| { + let a_text = get::(world, a_id)?; + let (_a_load, _a_deps, a_rec_deps) = asset_server.get_load_states(a_id).unwrap(); + let c_id = a_text.dependencies[1].id(); + let c_text = get::(world, c_id)?; + let (c_load, c_deps, c_rec_deps) = asset_server.get_load_states(c_id).unwrap(); + assert_eq!(c_text.text, "c"); + assert_eq!(c_text.embedded, "ab"); + + let d_id = c_text.dependencies[0].id(); + let d_text = get::(world, d_id)?; + let (d_load, d_deps, d_rec_deps) = asset_server.get_load_states(d_id).unwrap(); + assert_eq!(d_text.text, "d"); + assert_eq!(d_text.embedded, ""); + + assert_eq!(c_load, LoadState::Loaded); + assert_eq!(c_deps, DependencyLoadState::Loaded); + assert_eq!(c_rec_deps, RecursiveDependencyLoadState::Loaded); + + assert_eq!(d_load, LoadState::Loaded); + assert_eq!(d_deps, DependencyLoadState::Loaded); + assert_eq!(d_rec_deps, RecursiveDependencyLoadState::Loaded); + + assert_eq!( + a_rec_deps, + RecursiveDependencyLoadState::Loaded, + "d is loaded, so a's recursive deps should be loaded" + ); + Some(()) + }); + + { + let mut texts = app.world.resource_mut::>(); + let a = texts.get_mut(a_id).unwrap(); + a.text = "Changed".to_string(); + } + + app.world.despawn(entity); + app.update(); + assert_eq!( + app.world.resource::>().len(), + 0, + "CoolText asset entities should be despawned when no more handles exist" + ); + app.update(); + // this requires a second update because the parent asset was freed in the previous app.update() + assert_eq!( + app.world.resource::>().len(), + 0, + "SubText asset entities should be despawned when no more handles exist" + ); + let events = app.world.remove_resource::().unwrap(); + let id_results = app.world.remove_resource::().unwrap(); + let expected_events = vec![ + AssetEvent::Added { id: a_id }, + AssetEvent::LoadedWithDependencies { + id: id_results.b_id, + }, + AssetEvent::Added { + id: id_results.b_id, + }, + AssetEvent::Added { + id: id_results.c_id, + }, + AssetEvent::LoadedWithDependencies { + id: id_results.d_id, + }, + AssetEvent::LoadedWithDependencies { + id: id_results.c_id, + }, + AssetEvent::LoadedWithDependencies { id: a_id }, + AssetEvent::Added { + id: id_results.d_id, + }, + AssetEvent::Modified { id: a_id }, + AssetEvent::Removed { id: a_id }, + AssetEvent::Removed { + id: id_results.b_id, + }, + AssetEvent::Removed { + id: id_results.c_id, + }, + AssetEvent::Removed { + id: id_results.d_id, + }, + ]; + assert_eq!(events.0, expected_events); + } + + #[test] + fn failure_load_states() { + let dir = Dir::default(); + + let a_path = "a.cool.ron"; + let a_ron = r#" +( + text: "a", + dependencies: [ + "b.cool.ron", + "c.cool.ron", + ], + embedded_dependencies: [], + sub_texts: [] +)"#; + let b_path = "b.cool.ron"; + let b_ron = r#" +( + text: "b", + dependencies: [], + embedded_dependencies: [], + sub_texts: [] +)"#; + + let c_path = "c.cool.ron"; + let c_ron = r#" +( + text: "c", + dependencies: [ + "d.cool.ron", + ], + embedded_dependencies: [], + sub_texts: [] +)"#; + + let d_path = "d.cool.ron"; + let d_ron = r#" +( + text: "d", + dependencies: [], + OH NO THIS ASSET IS MALFORMED + embedded_dependencies: [], + sub_texts: [] +)"#; + + dir.insert_asset_text(Path::new(a_path), a_ron); + dir.insert_asset_text(Path::new(b_path), b_ron); + dir.insert_asset_text(Path::new(c_path), c_ron); + dir.insert_asset_text(Path::new(d_path), d_ron); + + let (mut app, gate_opener) = test_app(dir); + app.init_asset::() + .register_asset_loader(CoolTextLoader); + let asset_server = app.world.resource::().clone(); + let handle: Handle = asset_server.load(a_path); + let a_id = handle.id(); + { + let other_handle: Handle = asset_server.load(a_path); + assert_eq!( + other_handle, handle, + "handles from consecutive load calls should be equal" + ); + assert_eq!( + other_handle.id(), + handle.id(), + "handle ids from consecutive load calls should be equal" + ); + } + + app.world.spawn(handle); + gate_opener.open(a_path); + gate_opener.open(b_path); + gate_opener.open(c_path); + gate_opener.open(d_path); + + run_app_until(&mut app, |world| { + let a_text = get::(world, a_id)?; + let (a_load, a_deps, a_rec_deps) = asset_server.get_load_states(a_id).unwrap(); + + let b_id = a_text.dependencies[0].id(); + let b_text = get::(world, b_id)?; + let (b_load, b_deps, b_rec_deps) = asset_server.get_load_states(b_id).unwrap(); + + let c_id = a_text.dependencies[1].id(); + let c_text = get::(world, c_id)?; + let (c_load, c_deps, c_rec_deps) = asset_server.get_load_states(c_id).unwrap(); + + let d_id = c_text.dependencies[0].id(); + let d_text = get::(world, d_id); + let (d_load, d_deps, d_rec_deps) = asset_server.get_load_states(d_id).unwrap(); + if d_load != LoadState::Failed { + // wait until d has exited the loading state + return None; + } + + assert!(d_text.is_none()); + assert_eq!(d_load, LoadState::Failed); + assert_eq!(d_deps, DependencyLoadState::Failed); + assert_eq!(d_rec_deps, RecursiveDependencyLoadState::Failed); + + assert_eq!(a_text.text, "a"); + assert_eq!(a_load, LoadState::Loaded); + assert_eq!(a_deps, DependencyLoadState::Loaded); + assert_eq!(a_rec_deps, RecursiveDependencyLoadState::Failed); + + assert_eq!(b_text.text, "b"); + assert_eq!(b_load, LoadState::Loaded); + assert_eq!(b_deps, DependencyLoadState::Loaded); + assert_eq!(b_rec_deps, RecursiveDependencyLoadState::Loaded); + + assert_eq!(c_text.text, "c"); + assert_eq!(c_load, LoadState::Loaded); + assert_eq!(c_deps, DependencyLoadState::Failed); + assert_eq!(c_rec_deps, RecursiveDependencyLoadState::Failed); + + Some(()) + }); + } + + #[test] + fn manual_asset_management() { + let dir = Dir::default(); + + let dep_path = "dep.cool.ron"; + let dep_ron = r#" +( + text: "dep", + dependencies: [], + embedded_dependencies: [], + sub_texts: [], +)"#; + + dir.insert_asset_text(Path::new(dep_path), dep_ron); + + let (mut app, gate_opener) = test_app(dir); + app.init_asset::() + .init_asset::() + .init_resource::() + .register_asset_loader(CoolTextLoader) + .add_systems(Update, store_asset_events); + + let hello = "hello".to_string(); + let empty = "".to_string(); + + let id = { + let handle = { + let mut texts = app.world.resource_mut::>(); + texts.add(CoolText { + text: hello.clone(), + embedded: empty.clone(), + dependencies: vec![], + sub_texts: Vec::new(), + }) + }; + + app.update(); + + { + let text = app + .world + .resource::>() + .get(&handle) + .unwrap(); + assert_eq!(text.text, hello); + } + handle.id() + }; + // handle is dropped + app.update(); + assert!( + app.world.resource::>().get(id).is_none(), + "asset has no handles, so it should have been dropped last update" + ); + // remove event is emitted + app.update(); + let events = std::mem::take(&mut app.world.resource_mut::().0); + let expected_events = vec![AssetEvent::Added { id }, AssetEvent::Removed { id }]; + assert_eq!(events, expected_events); + + let dep_handle = app.world.resource::().load(dep_path); + let a = CoolText { + text: "a".to_string(), + embedded: empty, + // this dependency is behind a manual load gate, which should prevent 'a' from emitting a LoadedWithDependencies event + dependencies: vec![dep_handle.clone()], + sub_texts: Vec::new(), + }; + let a_handle = app.world.resource::().load_asset(a); + app.update(); + // TODO: ideally it doesn't take two updates for the added event to emit + app.update(); + + let events = std::mem::take(&mut app.world.resource_mut::().0); + let expected_events = vec![AssetEvent::Added { id: a_handle.id() }]; + assert_eq!(events, expected_events); + + gate_opener.open(dep_path); + loop { + app.update(); + let events = std::mem::take(&mut app.world.resource_mut::().0); + if events.is_empty() { + continue; + } + let expected_events = vec![ + AssetEvent::LoadedWithDependencies { + id: dep_handle.id(), + }, + AssetEvent::LoadedWithDependencies { id: a_handle.id() }, + ]; + assert_eq!(events, expected_events); + break; + } + app.update(); + let events = std::mem::take(&mut app.world.resource_mut::().0); + let expected_events = vec![AssetEvent::Added { + id: dep_handle.id(), + }]; + assert_eq!(events, expected_events); + } + + #[test] + fn load_folder() { + let dir = Dir::default(); + + let a_path = "text/a.cool.ron"; + let a_ron = r#" +( + text: "a", + dependencies: [ + "b.cool.ron", + ], + embedded_dependencies: [], + sub_texts: [], +)"#; + let b_path = "b.cool.ron"; + let b_ron = r#" +( + text: "b", + dependencies: [], + embedded_dependencies: [], + sub_texts: [], +)"#; + + let c_path = "text/c.cool.ron"; + let c_ron = r#" +( + text: "c", + dependencies: [ + ], + embedded_dependencies: [], + sub_texts: [], +)"#; + dir.insert_asset_text(Path::new(a_path), a_ron); + dir.insert_asset_text(Path::new(b_path), b_ron); + dir.insert_asset_text(Path::new(c_path), c_ron); + + let (mut app, gate_opener) = test_app(dir); + app.init_asset::() + .init_asset::() + .register_asset_loader(CoolTextLoader); + let asset_server = app.world.resource::().clone(); + let handle: Handle = asset_server.load_folder("text"); + gate_opener.open(a_path); + gate_opener.open(b_path); + gate_opener.open(c_path); + + let mut reader = ManualEventReader::default(); + run_app_until(&mut app, |world| { + let events = world.resource::>>(); + let asset_server = world.resource::(); + let loaded_folders = world.resource::>(); + let cool_texts = world.resource::>(); + for event in reader.read(events) { + if let AssetEvent::LoadedWithDependencies { id } = event { + if *id == handle.id() { + let loaded_folder = loaded_folders.get(&handle).unwrap(); + let a_handle: Handle = + asset_server.get_handle("text/a.cool.ron").unwrap(); + let c_handle: Handle = + asset_server.get_handle("text/c.cool.ron").unwrap(); + + let mut found_a = false; + let mut found_c = false; + for asset_handle in &loaded_folder.handles { + if asset_handle.id() == a_handle.id().untyped() { + found_a = true; + } else if asset_handle.id() == c_handle.id().untyped() { + found_c = true; + } + } + assert!(found_a); + assert!(found_c); + assert_eq!(loaded_folder.handles.len(), 2); + + let a_text = cool_texts.get(&a_handle).unwrap(); + let b_text = cool_texts.get(&a_text.dependencies[0]).unwrap(); + let c_text = cool_texts.get(&c_handle).unwrap(); + + assert_eq!("a", a_text.text); + assert_eq!("b", b_text.text); + assert_eq!("c", c_text.text); + + return Some(()); + } + } + } + None + }); } } diff --git a/crates/bevy_asset/src/loader.rs b/crates/bevy_asset/src/loader.rs index 6b2536e43c44f..261ef22a08d1a 100644 --- a/crates/bevy_asset/src/loader.rs +++ b/crates/bevy_asset/src/loader.rs @@ -1,282 +1,547 @@ use crate::{ - path::AssetPath, AssetIo, AssetIoError, AssetMeta, AssetServer, Assets, Handle, HandleId, - HandleUntyped, RefChangeChannel, + io::{AssetReaderError, Reader}, + meta::{ + loader_settings_meta_transform, AssetHash, AssetMeta, AssetMetaDyn, ProcessedInfoMinimal, + Settings, + }, + path::AssetPath, + Asset, AssetLoadError, AssetServer, Assets, Handle, UntypedAssetId, UntypedHandle, }; -use anyhow::Error; -use anyhow::Result; -use bevy_ecs::system::{Res, ResMut}; -use bevy_reflect::TypePath; -use bevy_reflect::{TypeUuid, TypeUuidDynamic}; -use bevy_utils::{BoxedFuture, HashMap}; -use crossbeam_channel::{Receiver, Sender}; +use bevy_ecs::world::World; +use bevy_utils::{BoxedFuture, HashMap, HashSet}; use downcast_rs::{impl_downcast, Downcast}; -use std::path::Path; +use futures_lite::AsyncReadExt; +use ron::error::SpannedError; +use serde::{Deserialize, Serialize}; +use std::{ + any::{Any, TypeId}, + path::Path, +}; +use thiserror::Error; -/// A loader for an asset source. -/// -/// Types implementing this trait are used by the [`AssetServer`] to load assets -/// into their respective asset storages. +/// Loads an [`Asset`] from a given byte [`Reader`]. This can accept [`AssetLoader::Settings`], which configure how the [`Asset`] +/// should be loaded. pub trait AssetLoader: Send + Sync + 'static { - /// Processes the asset in an asynchronous closure. + /// The top level [`Asset`] loaded by this [`AssetLoader`]. + type Asset: crate::Asset; + /// The settings type used by this [`AssetLoader`]. + type Settings: Settings + Default + Serialize + for<'a> Deserialize<'a>; + /// Asynchronously loads [`AssetLoader::Asset`] (and any other labeled assets) from the bytes provided by [`Reader`]. fn load<'a>( &'a self, - bytes: &'a [u8], + reader: &'a mut Reader, + settings: &'a Self::Settings, load_context: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<(), Error>>; + ) -> BoxedFuture<'a, Result>; /// Returns a list of extensions supported by this asset loader, without the preceding dot. fn extensions(&self) -> &[&str]; } -/// An essential piece of data of an application. -/// -/// Assets are the building blocks of games. They can be anything, from images and sounds to scenes -/// and scripts. In Bevy, an asset is any struct that has an unique type id, as shown below: -/// -/// ```rust -/// use bevy_reflect::{TypePath, TypeUuid}; -/// use serde::Deserialize; -/// -/// #[derive(Debug, Deserialize, TypeUuid, TypePath)] -/// #[uuid = "39cadc56-aa9c-4543-8640-a018b74b5052"] -/// pub struct CustomAsset { -/// pub value: i32, -/// } -/// # fn is_asset() {} -/// # is_asset::(); -/// ``` -/// -/// See the `assets/custom_asset.rs` example in the repository for more details. -/// -/// In order to load assets into your game you must either add them manually to an asset storage -/// with [`Assets::add`] or load them from the filesystem with [`AssetServer::load`]. -pub trait Asset: TypeUuid + TypePath + AssetDynamic {} - -/// An untyped version of the [`Asset`] trait. -pub trait AssetDynamic: Downcast + TypeUuidDynamic + Send + Sync + 'static {} -impl_downcast!(AssetDynamic); - -impl Asset for T where T: TypeUuid + TypePath + AssetDynamic + TypeUuidDynamic {} - -impl AssetDynamic for T where T: Send + Sync + 'static + TypeUuidDynamic {} - -/// A complete asset processed in an [`AssetLoader`]. -pub struct LoadedAsset { - pub(crate) value: Option, - pub(crate) dependencies: Vec>, +/// Provides type-erased access to an [`AssetLoader`]. +pub trait ErasedAssetLoader: Send + Sync + 'static { + /// Asynchronously loads the asset(s) from the bytes provided by [`Reader`]. + fn load<'a>( + &'a self, + reader: &'a mut Reader, + meta: Box, + load_context: LoadContext<'a>, + ) -> BoxedFuture<'a, Result>; + + /// Returns a list of extensions supported by this asset loader, without the preceding dot. + fn extensions(&self) -> &[&str]; + /// Deserializes metadata from the input `meta` bytes into the appropriate type (erased as [`Box`]). + fn deserialize_meta(&self, meta: &[u8]) -> Result, DeserializeMetaError>; + /// Returns the default meta value for the [`AssetLoader`] (erased as [`Box`]). + fn default_meta(&self) -> Box; + /// Returns the type name of the [`AssetLoader`]. + fn type_name(&self) -> &'static str; + /// Returns the [`TypeId`] of the [`AssetLoader`]. + fn type_id(&self) -> TypeId; + /// Returns the type name of the top-level [`Asset`] loaded by the [`AssetLoader`]. + fn asset_type_name(&self) -> &'static str; + /// Returns the [`TypeId`] of the top-level [`Asset`] loaded by the [`AssetLoader`]. + fn asset_type_id(&self) -> TypeId; } -impl LoadedAsset { - /// Creates a new loaded asset. - pub fn new(value: T) -> Self { - Self { - value: Some(value), - dependencies: Vec::new(), - } +/// An error encountered during [`AssetLoader::load`]. +#[derive(Error, Debug)] +pub enum AssetLoaderError { + /// Any error that occurs during load. + #[error(transparent)] + Load(#[from] anyhow::Error), + /// A failure to deserialize metadata during load. + #[error(transparent)] + DeserializeMeta(#[from] DeserializeMetaError), +} + +impl ErasedAssetLoader for L +where + L: AssetLoader + Send + Sync, +{ + /// Processes the asset in an asynchronous closure. + fn load<'a>( + &'a self, + reader: &'a mut Reader, + meta: Box, + mut load_context: LoadContext<'a>, + ) -> BoxedFuture<'a, Result> { + Box::pin(async move { + let settings = meta + .loader_settings() + .expect("Loader settings should exist") + .downcast_ref::() + .expect("AssetLoader settings should match the loader type"); + let asset = ::load(self, reader, settings, &mut load_context).await?; + Ok(load_context.finish(asset, Some(meta)).into()) + }) + } + + fn deserialize_meta(&self, meta: &[u8]) -> Result, DeserializeMetaError> { + let meta = AssetMeta::::deserialize(meta)?; + Ok(Box::new(meta)) + } + + fn default_meta(&self) -> Box { + Box::new(AssetMeta::::new(crate::meta::AssetAction::Load { + loader: self.type_name().to_string(), + settings: L::Settings::default(), + })) + } + + fn extensions(&self) -> &[&str] { + ::extensions(self) } - /// Adds a dependency on another asset at the provided path. - pub fn add_dependency(&mut self, asset_path: AssetPath) { - self.dependencies.push(asset_path.to_owned()); + fn type_name(&self) -> &'static str { + std::any::type_name::() } - /// Adds a dependency on another asset at the provided path. - #[must_use] - pub fn with_dependency(mut self, asset_path: AssetPath) -> Self { - self.add_dependency(asset_path); - self + fn type_id(&self) -> TypeId { + TypeId::of::() } - /// Adds dependencies on other assets at the provided paths. - #[must_use] - pub fn with_dependencies(mut self, asset_paths: Vec>) -> Self { - for asset_path in asset_paths { - self.add_dependency(asset_path); + fn asset_type_id(&self) -> TypeId { + TypeId::of::() + } + + fn asset_type_name(&self) -> &'static str { + std::any::type_name::() + } +} + +pub(crate) struct LabeledAsset { + pub(crate) asset: ErasedLoadedAsset, + pub(crate) handle: UntypedHandle, +} + +/// The successful result of an [`AssetLoader::load`] call. This contains the loaded "root" asset and any other "labeled" assets produced +/// by the loader. It also holds the input [`AssetMeta`] (if it exists) and tracks dependencies: +/// * normal dependencies: dependencies that must be loaded as part of this asset load (ex: assets a given asset has handles to). +/// * Loader dependencies: dependencies whose actual asset values are used during the load process +pub struct LoadedAsset { + pub(crate) value: A, + pub(crate) dependencies: HashSet, + pub(crate) loader_dependencies: HashMap, AssetHash>, + pub(crate) labeled_assets: HashMap, + pub(crate) meta: Option>, +} + +impl LoadedAsset { + /// Create a new loaded asset. This will use [`VisitAssetDependencies`](crate::VisitAssetDependencies) to populate `dependencies`. + pub fn new_with_dependencies(value: A, meta: Option>) -> Self { + let mut dependencies = HashSet::new(); + value.visit_dependencies(&mut |id| { + dependencies.insert(id); + }); + LoadedAsset { + value, + dependencies, + loader_dependencies: HashMap::default(), + labeled_assets: HashMap::default(), + meta, } - self } } -pub(crate) struct BoxedLoadedAsset { - pub(crate) value: Option>, - pub(crate) dependencies: Vec>, +impl From for LoadedAsset { + fn from(asset: A) -> Self { + LoadedAsset::new_with_dependencies(asset, None) + } } -impl From> for BoxedLoadedAsset { - fn from(asset: LoadedAsset) -> Self { - BoxedLoadedAsset { - value: asset - .value - .map(|value| Box::new(value) as Box), +/// A "type erased / boxed" counterpart to [`LoadedAsset`]. This is used in places where the loaded type is not statically known. +pub struct ErasedLoadedAsset { + pub(crate) value: Box, + pub(crate) dependencies: HashSet, + pub(crate) loader_dependencies: HashMap, AssetHash>, + pub(crate) labeled_assets: HashMap, + pub(crate) meta: Option>, +} + +impl From> for ErasedLoadedAsset { + fn from(asset: LoadedAsset) -> Self { + ErasedLoadedAsset { + value: Box::new(asset.value), dependencies: asset.dependencies, + loader_dependencies: asset.loader_dependencies, + labeled_assets: asset.labeled_assets, + meta: asset.meta, } } } -/// An asynchronous context where an [`Asset`] is processed. -/// -/// The load context is created by the [`AssetServer`] to process an asset source after loading its -/// contents into memory. It is then passed to the appropriate [`AssetLoader`] based on the file -/// extension of the asset's path. -/// -/// An asset source can define one or more assets from a single source path. The main asset is set -/// using [`LoadContext::set_default_asset`] and sub-assets are defined with -/// [`LoadContext::set_labeled_asset`]. +impl ErasedLoadedAsset { + /// Cast (and take ownership) of the [`Asset`] value of the given type. This will return [`Some`] if + /// the stored type matches `A` and [`None`] if it does not. + pub fn take(self) -> Option { + self.value.downcast::().map(|a| *a).ok() + } + + /// Retrieves a reference to the internal [`Asset`] type, if it matches the type `A`. Otherwise returns [`None`]. + pub fn get(&self) -> Option<&A> { + self.value.downcast_ref::() + } + + /// Retrieves the [`TypeId`] of the stored [`Asset`] type. + pub fn asset_type_id(&self) -> TypeId { + (*self.value).type_id() + } + + /// Retrieves the `type_name` of the stored [`Asset`] type. + pub fn asset_type_name(&self) -> &'static str { + self.value.asset_type_name() + } + + /// Returns the [`ErasedLoadedAsset`] for the given label, if it exists. + pub fn get_labeled(&self, label: &str) -> Option<&ErasedLoadedAsset> { + self.labeled_assets.get(label).map(|a| &a.asset) + } + + /// Iterate over all labels for "labeled assets" in the loaded asset + pub fn iter_labels(&self) -> impl Iterator { + self.labeled_assets.keys().map(|s| s.as_str()) + } +} + +/// A type erased container for an [`Asset`] value that is capable of inserting the [`Asset`] into a [`World`]'s [`Assets`] collection. +pub trait AssetContainer: Downcast + Any + Send + Sync + 'static { + fn insert(self: Box, id: UntypedAssetId, world: &mut World); + fn asset_type_name(&self) -> &'static str; +} + +impl_downcast!(AssetContainer); + +impl AssetContainer for A { + fn insert(self: Box, id: UntypedAssetId, world: &mut World) { + world.resource_mut::>().insert(id.typed(), *self); + } + + fn asset_type_name(&self) -> &'static str { + std::any::type_name::() + } +} + +/// An error that occurs when attempting to call [`LoadContext::load_direct`] +#[derive(Error, Debug)] +#[error("Failed to load dependency {dependency:?} {error}")] +pub struct LoadDirectError { + pub dependency: AssetPath<'static>, + pub error: AssetLoadError, +} + +/// An error that occurs while deserializing [`AssetMeta`]. +#[derive(Error, Debug)] +pub enum DeserializeMetaError { + #[error("Failed to deserialize asset meta: {0:?}")] + DeserializeSettings(#[from] SpannedError), + #[error("Failed to deserialize minimal asset meta: {0:?}")] + DeserializeMinimal(SpannedError), +} + +/// A context that provides access to assets in [`AssetLoader`]s, tracks dependencies, and collects asset load state. +/// Any asset state accessed by [`LoadContext`] will be tracked and stored for use in dependency events and asset preprocessing. pub struct LoadContext<'a> { - pub(crate) ref_change_channel: &'a RefChangeChannel, - pub(crate) asset_io: &'a dyn AssetIo, - pub(crate) labeled_assets: HashMap, BoxedLoadedAsset>, - pub(crate) path: &'a Path, - pub(crate) version: usize, + asset_server: &'a AssetServer, + should_load_dependencies: bool, + populate_hashes: bool, + asset_path: AssetPath<'static>, + dependencies: HashSet, + /// Direct dependencies used by this loader. + loader_dependencies: HashMap, AssetHash>, + labeled_assets: HashMap, } impl<'a> LoadContext<'a> { + /// Creates a new [`LoadContext`] instance. pub(crate) fn new( - path: &'a Path, - ref_change_channel: &'a RefChangeChannel, - asset_io: &'a dyn AssetIo, - version: usize, + asset_server: &'a AssetServer, + asset_path: AssetPath<'static>, + should_load_dependencies: bool, + populate_hashes: bool, ) -> Self { Self { - ref_change_channel, - asset_io, - labeled_assets: Default::default(), - version, - path, + asset_server, + asset_path, + populate_hashes, + should_load_dependencies, + dependencies: HashSet::default(), + loader_dependencies: HashMap::default(), + labeled_assets: HashMap::default(), } } - /// Gets the source path for this load context. - pub fn path(&self) -> &Path { - self.path + /// Begins a new labeled asset load. Use the returned [`LoadContext`] to load + /// dependencies for the new asset and call [`LoadContext::finish`] to finalize the asset load. + /// When finished, make sure you call [`LoadContext::add_labeled_asset`] to add the results back to the parent + /// context. + /// Prefer [`LoadContext::labeled_asset_scope`] when possible, which will automatically add + /// the labeled [`LoadContext`] back to the parent context. + /// [`LoadContext::begin_labeled_asset`] exists largely to enable parallel asset loading. + /// + /// See [`AssetPath`] for more on labeled assets. + /// + /// ```no_run + /// # use bevy_asset::{Asset, LoadContext}; + /// # use bevy_reflect::TypePath; + /// # #[derive(Asset, TypePath, Default)] + /// # struct Image; + /// # let load_context: LoadContext = panic!(); + /// let mut handles = Vec::new(); + /// for i in 0..2 { + /// let mut labeled = load_context.begin_labeled_asset(); + /// handles.push(std::thread::spawn(move || { + /// (i.to_string(), labeled.finish(Image::default(), None)) + /// })); + /// } + + /// for handle in handles { + /// let (label, loaded_asset) = handle.join().unwrap(); + /// load_context.add_loaded_labeled_asset(label, loaded_asset); + /// } + /// ``` + pub fn begin_labeled_asset(&self) -> LoadContext { + LoadContext::new( + self.asset_server, + self.asset_path.clone(), + self.should_load_dependencies, + self.populate_hashes, + ) } - /// Returns `true` if the load context contains an asset with the specified label. - pub fn has_labeled_asset(&self, label: &str) -> bool { - self.labeled_assets.contains_key(&Some(label.to_string())) + /// Creates a new [`LoadContext`] for the given `label`. The `load` function is responsible for loading an [`Asset`] of + /// type `A`. `load` will be called immediately and the result will be used to finalize the [`LoadContext`], resulting in a new + /// [`LoadedAsset`], which is registered under the `label` label. + /// + /// This exists to remove the need to manually call [`LoadContext::begin_labeled_asset`] and then manually register the + /// result with [`LoadContext::add_labeled_asset`]. + /// + /// See [`AssetPath`] for more on labeled assets. + pub fn labeled_asset_scope( + &mut self, + label: String, + load: impl FnOnce(&mut LoadContext) -> A, + ) -> Handle { + let mut context = self.begin_labeled_asset(); + let asset = load(&mut context); + let loaded_asset = context.finish(asset, None); + self.add_loaded_labeled_asset(label, loaded_asset) } - /// Sets the primary asset loaded from the asset source. - pub fn set_default_asset(&mut self, asset: LoadedAsset) { - self.labeled_assets.insert(None, asset.into()); + /// This will add the given `asset` as a "labeled [`Asset`]" with the `label` label. + /// + /// See [`AssetPath`] for more on labeled assets. + pub fn add_labeled_asset(&mut self, label: String, asset: A) -> Handle { + self.labeled_asset_scope(label, |_| asset) } - /// Sets a secondary asset loaded from the asset source. - pub fn set_labeled_asset(&mut self, label: &str, asset: LoadedAsset) -> Handle { - assert!(!label.is_empty()); - self.labeled_assets - .insert(Some(label.to_string()), asset.into()); - self.get_handle(AssetPath::new_ref(self.path(), Some(label))) + /// Add a [`LoadedAsset`] that is a "labeled sub asset" of the root path of this load context. + /// This can be used in combination with [`LoadContext::begin_labeled_asset`] to parallelize + /// sub asset loading. + /// + /// See [`AssetPath`] for more on labeled assets. + pub fn add_loaded_labeled_asset( + &mut self, + label: String, + loaded_asset: LoadedAsset, + ) -> Handle { + let loaded_asset: ErasedLoadedAsset = loaded_asset.into(); + let labeled_path = self.asset_path.with_label(label.clone()); + let handle = self + .asset_server + .get_or_create_path_handle(labeled_path, None); + self.labeled_assets.insert( + label, + LabeledAsset { + asset: loaded_asset, + handle: handle.clone().untyped(), + }, + ); + handle } - /// Gets a strong handle to an asset of type `T` from its id. - pub fn get_handle, T: Asset>(&self, id: I) -> Handle { - Handle::strong(id.into(), self.ref_change_channel.sender.clone()) + /// Returns `true` if an asset with the label `label` exists in this context. + /// + /// See [`AssetPath`] for more on labeled assets. + pub fn has_labeled_asset(&self, label: &str) -> bool { + let path = self.asset_path.with_label(label); + self.asset_server.get_handle_untyped(path).is_some() } - /// Gets an untyped strong handle for an asset with the provided id. - pub fn get_handle_untyped>(&self, id: I) -> HandleUntyped { - HandleUntyped::strong(id.into(), self.ref_change_channel.sender.clone()) + /// "Finishes" this context by populating the final [`Asset`] value (and the erased [`AssetMeta`] value, if it exists). + /// The relevant asset metadata collected in this context will be stored in the returned [`LoadedAsset`]. + pub fn finish(self, value: A, meta: Option>) -> LoadedAsset { + LoadedAsset { + value, + dependencies: self.dependencies, + loader_dependencies: self.loader_dependencies, + labeled_assets: self.labeled_assets, + meta, + } } - /// Reads the contents of the file at the specified path through the [`AssetIo`] associated - /// with this context. - pub async fn read_asset_bytes>(&self, path: P) -> Result, AssetIoError> { - self.asset_io - .watch_path_for_changes(path.as_ref(), Some(self.path.to_owned()))?; - self.asset_io.load_path(path.as_ref()).await + /// Gets the source path for this load context. + pub fn path(&self) -> &Path { + self.asset_path.path() } - /// Generates metadata for the assets managed by this load context. - pub fn get_asset_metas(&self) -> Vec { - let mut asset_metas = Vec::new(); - for (label, asset) in &self.labeled_assets { - asset_metas.push(AssetMeta { - dependencies: asset.dependencies.clone(), - label: label.clone(), - type_uuid: asset.value.as_ref().unwrap().type_uuid(), - }); - } - asset_metas + /// Gets the source asset path for this load context. + pub fn asset_path(&self) -> &AssetPath { + &self.asset_path } - /// Gets the asset I/O associated with this load context. - pub fn asset_io(&self) -> &dyn AssetIo { - self.asset_io + /// Gets the source asset path for this load context. + pub async fn read_asset_bytes<'b>( + &mut self, + path: &'b Path, + ) -> Result, ReadAssetBytesError> { + let mut reader = self.asset_server.reader().read(path).await?; + let hash = if self.populate_hashes { + // NOTE: ensure meta is read while the asset bytes reader is still active to ensure transactionality + // See `ProcessorGatedReader` for more info + let meta_bytes = self.asset_server.reader().read_meta_bytes(path).await?; + let minimal: ProcessedInfoMinimal = ron::de::from_bytes(&meta_bytes) + .map_err(DeserializeMetaError::DeserializeMinimal)?; + let processed_info = minimal + .processed_info + .ok_or(ReadAssetBytesError::MissingAssetHash)?; + processed_info.full_hash + } else { + Default::default() + }; + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; + self.loader_dependencies + .insert(AssetPath::new(path.to_owned(), None), hash); + Ok(bytes) } -} -/// The result of loading an asset of type `T`. -#[derive(Debug)] -pub struct AssetResult { - /// The asset itself. - pub asset: Box, - /// The unique id of the asset. - pub id: HandleId, - /// Change version. - pub version: usize, -} - -/// An event channel used by asset server to update the asset storage of a `T` asset. -#[derive(Debug)] -pub struct AssetLifecycleChannel { - /// The sender endpoint of the channel. - pub sender: Sender>, - /// The receiver endpoint of the channel. - pub receiver: Receiver>, -} - -/// Events for the [`AssetLifecycleChannel`]. -pub enum AssetLifecycleEvent { - /// An asset was created. - Create(AssetResult), - /// An asset was freed. - Free(HandleId), -} + /// Retrieves a handle for the asset at the given path and adds that path as a dependency of the asset. + /// If the current context is a normal [`AssetServer::load`], an actual asset load will be kicked off immediately, which ensures the load happens + /// as soon as possible. + /// "Normal loads" kicked from within a normal Bevy App will generally configure the context to kick off loads immediately. + /// If the current context is configured to not load dependencies automatically (ex: [`AssetProcessor`](crate::processor::AssetProcessor)), + /// a load will not be kicked off automatically. It is then the calling context's responsibility to begin a load if necessary. + pub fn load<'b, A: Asset>(&mut self, path: impl Into>) -> Handle { + let path = path.into().to_owned(); + let handle = if self.should_load_dependencies { + self.asset_server.load(path) + } else { + self.asset_server.get_or_create_path_handle(path, None) + }; + self.dependencies.insert(handle.id().untyped()); + handle + } -/// A trait for sending lifecycle notifications from assets in the asset server. -pub trait AssetLifecycle: Downcast + Send + Sync + 'static { - /// Notifies the asset server that a new asset was created. - fn create_asset(&self, id: HandleId, asset: Box, version: usize); - /// Notifies the asset server that an asset was freed. - fn free_asset(&self, id: HandleId); -} -impl_downcast!(AssetLifecycle); - -impl AssetLifecycle for AssetLifecycleChannel { - fn create_asset(&self, id: HandleId, asset: Box, version: usize) { - if let Ok(asset) = asset.downcast::() { - self.sender - .send(AssetLifecycleEvent::Create(AssetResult { - asset, - id, - version, - })) - .unwrap(); + /// Loads the [`Asset`] of type `A` at the given `path` with the given [`AssetLoader::Settings`] settings `S`. This is a "deferred" + /// load. If the settings type `S` does not match the settings expected by `A`'s asset loader, an error will be printed to the log + /// and the asset load will fail. + pub fn load_with_settings<'b, A: Asset, S: Settings + Default>( + &mut self, + path: impl Into>, + settings: impl Fn(&mut S) + Send + Sync + 'static, + ) -> Handle { + let path = path.into().to_owned(); + let handle = if self.should_load_dependencies { + self.asset_server.load_with_settings(path.clone(), settings) } else { - panic!( - "Failed to downcast asset to {}.", - std::any::type_name::() - ); - } + self.asset_server.get_or_create_path_handle( + path.clone(), + Some(loader_settings_meta_transform(settings)), + ) + }; + self.dependencies.insert(handle.id().untyped()); + handle } - fn free_asset(&self, id: HandleId) { - self.sender.send(AssetLifecycleEvent::Free(id)).unwrap(); + /// Returns a handle to an asset of type `A` with the label `label`. This [`LoadContext`] must produce an asset of the + /// given type and the given label or the dependencies of this asset will never be considered "fully loaded". However you + /// can call this method before _or_ after adding the labeled asset. + pub fn get_label_handle(&mut self, label: &str) -> Handle { + let path = self.asset_path.with_label(label).to_owned(); + let handle = self.asset_server.get_or_create_path_handle::(path, None); + self.dependencies.insert(handle.id().untyped()); + handle } -} -impl Default for AssetLifecycleChannel { - fn default() -> Self { - let (sender, receiver) = crossbeam_channel::unbounded(); - AssetLifecycleChannel { sender, receiver } + /// Loads the asset at the given `path` directly. This is an async function that will wait until the asset is fully loaded before + /// returning. Use this if you need the _value_ of another asset in order to load the current asset. For example, if you are + /// deriving a new asset from the referenced asset, or you are building a collection of assets. This will add the `path` as a + /// "load dependency". + /// + /// If the current loader is used in a [`Process`] "asset preprocessor", such as a [`LoadAndSave`] preprocessor, + /// changing a "load dependency" will result in re-processing of the asset. + /// + /// [`Process`]: crate::processor::Process + /// [`LoadAndSave`]: crate::processor::LoadAndSave + pub async fn load_direct<'b>( + &mut self, + path: impl Into>, + ) -> Result { + let path = path.into(); + let to_error = |e: AssetLoadError| -> LoadDirectError { + LoadDirectError { + dependency: path.to_owned(), + error: e, + } + }; + let (meta, loader, mut reader) = self + .asset_server + .get_meta_loader_and_reader(&path) + .await + .map_err(to_error)?; + let loaded_asset = self + .asset_server + .load_with_meta_loader_and_reader( + &path, + meta, + &*loader, + &mut *reader, + false, + self.populate_hashes, + ) + .await + .map_err(to_error)?; + let info = loaded_asset + .meta + .as_ref() + .and_then(|m| m.processed_info().as_ref()); + let hash = info.map(|i| i.full_hash).unwrap_or(Default::default()); + self.loader_dependencies.insert(path.to_owned(), hash); + Ok(loaded_asset) } } -/// Updates the [`Assets`] collection according to the changes queued up by [`AssetServer`]. -pub fn update_asset_storage_system( - asset_server: Res, - assets: ResMut>, -) { - asset_server.update_asset_storage(assets); +/// An error produced when calling [`LoadContext::read_asset_bytes`] +#[derive(Error, Debug)] +pub enum ReadAssetBytesError { + #[error(transparent)] + DeserializeMetaError(#[from] DeserializeMetaError), + #[error(transparent)] + AssetReaderError(#[from] AssetReaderError), + /// Encountered an I/O error while loading an asset. + #[error("Encountered an io error while loading asset: {0}")] + Io(#[from] std::io::Error), + #[error("The LoadContext for this read_asset_bytes call requires hash metadata, but it was not provided. This is likely an internal implementation error.")] + MissingAssetHash, } diff --git a/crates/bevy_asset/src/meta.rs b/crates/bevy_asset/src/meta.rs new file mode 100644 index 0000000000000..c2945e5887e99 --- /dev/null +++ b/crates/bevy_asset/src/meta.rs @@ -0,0 +1,250 @@ +use crate::{self as bevy_asset, DeserializeMetaError, VisitAssetDependencies}; +use crate::{loader::AssetLoader, processor::Process, Asset, AssetPath}; +use bevy_log::error; +use downcast_rs::{impl_downcast, Downcast}; +use ron::ser::PrettyConfig; +use serde::{Deserialize, Serialize}; + +pub const META_FORMAT_VERSION: &str = "1.0"; +pub type MetaTransform = Box; + +/// Asset metadata that informs how an [`Asset`] should be handled by the asset system. +/// +/// `L` is the [`AssetLoader`] (if one is configured) for the [`AssetAction`]. This can be `()` if it is not required. +/// `P` is the [`Process`] processor, if one is configured for the [`AssetAction`]. This can be `()` if it is not required. +#[derive(Serialize, Deserialize)] +pub struct AssetMeta { + /// The version of the meta format being used. This will change whenever a breaking change is made to + /// the meta format. + pub meta_format_version: String, + /// Information produced by the [`AssetProcessor`] _after_ processing this asset. + /// This will only exist alongside processed versions of assets. You should not manually set it in your asset source files. + /// + /// [`AssetProcessor`]: crate::processor::AssetProcessor + #[serde(skip_serializing_if = "Option::is_none")] + pub processed_info: Option, + /// How to handle this asset in the asset system. See [`AssetAction`]. + pub asset: AssetAction, +} + +impl AssetMeta { + pub fn new(asset: AssetAction) -> Self { + Self { + meta_format_version: META_FORMAT_VERSION.to_string(), + processed_info: None, + asset, + } + } + + /// Deserializes the given serialized byte representation of the asset meta. + pub fn deserialize(bytes: &[u8]) -> Result { + Ok(ron::de::from_bytes(bytes)?) + } +} + +/// Configures how an asset source file should be handled by the asset system. +#[derive(Serialize, Deserialize)] +pub enum AssetAction { + /// Load the asset with the given loader and settings + /// See [`AssetLoader`]. + Load { + loader: String, + settings: LoaderSettings, + }, + /// Process the asset with the given processor and settings. + /// See [`Process`] and [`AssetProcessor`]. + /// + /// [`AssetProcessor`]: crate::processor::AssetProcessor + Process { + processor: String, + settings: ProcessSettings, + }, + /// Do nothing with the asset + Ignore, +} + +/// Info produced by the [`AssetProcessor`] for a given processed asset. This is used to determine if an +/// asset source file (or its dependencies) has changed. +/// +/// [`AssetProcessor`]: crate::processor::AssetProcessor +#[derive(Serialize, Deserialize, Default, Debug, Clone)] +pub struct ProcessedInfo { + /// A hash of the asset bytes and the asset .meta data + pub hash: AssetHash, + /// A hash of the asset bytes, the asset .meta data, and the `full_hash` of every process_dependency + pub full_hash: AssetHash, + /// Information about the "process dependencies" used to process this asset. + pub process_dependencies: Vec, +} + +/// Information about a dependency used to process an asset. This is used to determine whether an asset's "process dependency" +/// has changed. +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct ProcessDependencyInfo { + pub full_hash: AssetHash, + pub path: AssetPath<'static>, +} + +/// This is a minimal counterpart to [`AssetMeta`] that exists to speed up (or enable) serialization in cases where the whole [`AssetMeta`] isn't +/// necessary. +// PERF: +// Currently, this is used when retrieving asset loader and processor information (when the actual type is not known yet). This could probably +// be replaced (and made more efficient) by a custom deserializer that reads the loader/processor information _first_, then deserializes the contents +// using a type registry. +#[derive(Serialize, Deserialize)] +pub struct AssetMetaMinimal { + pub asset: AssetActionMinimal, +} + +/// This is a minimal counterpart to [`AssetAction`] that exists to speed up (or enable) serialization in cases where the whole [`AssetActionMinimal`] +/// isn't necessary. +#[derive(Serialize, Deserialize)] +pub enum AssetActionMinimal { + Load { loader: String }, + Process { processor: String }, + Ignore, +} + +/// This is a minimal counterpart to [`ProcessedInfo`] that exists to speed up serialization in cases where the whole [`ProcessedInfo`] isn't +/// necessary. +#[derive(Serialize, Deserialize)] +pub struct ProcessedInfoMinimal { + pub processed_info: Option, +} + +/// A dynamic type-erased counterpart to [`AssetMeta`] that enables passing around and interacting with [`AssetMeta`] without knowing +/// its type. +pub trait AssetMetaDyn: Downcast + Send + Sync { + /// Returns a reference to the [`AssetLoader`] settings, if they exist. + fn loader_settings(&self) -> Option<&dyn Settings>; + /// Returns a mutable reference to the [`AssetLoader`] settings, if they exist. + fn loader_settings_mut(&mut self) -> Option<&mut dyn Settings>; + /// Serializes the internal [`AssetMeta`]. + fn serialize(&self) -> Vec; + /// Returns a reference to the [`ProcessedInfo`] if it exists. + fn processed_info(&self) -> &Option; + /// Returns a mutable reference to the [`ProcessedInfo`] if it exists. + fn processed_info_mut(&mut self) -> &mut Option; +} + +impl AssetMetaDyn for AssetMeta { + fn serialize(&self) -> Vec { + ron::ser::to_string_pretty(&self, PrettyConfig::default()) + .expect("type is convertible to ron") + .into_bytes() + } + fn loader_settings(&self) -> Option<&dyn Settings> { + if let AssetAction::Load { settings, .. } = &self.asset { + Some(settings) + } else { + None + } + } + fn loader_settings_mut(&mut self) -> Option<&mut dyn Settings> { + if let AssetAction::Load { settings, .. } = &mut self.asset { + Some(settings) + } else { + None + } + } + fn processed_info(&self) -> &Option { + &self.processed_info + } + fn processed_info_mut(&mut self) -> &mut Option { + &mut self.processed_info + } +} + +impl_downcast!(AssetMetaDyn); + +/// Settings used by the asset system, such as by [`AssetLoader`], [`Process`], and [`AssetSaver`] +/// +/// [`AssetSaver`]: crate::saver::AssetSaver +pub trait Settings: Downcast + Send + Sync + 'static {} + +impl Settings for T where T: Send + Sync {} + +impl_downcast!(Settings); + +/// The () processor should never be called. This implementation exists to make the meta format nicer to work with. +impl Process for () { + type Settings = (); + type OutputLoader = (); + + fn process<'a>( + &'a self, + _context: &'a mut bevy_asset::processor::ProcessContext, + _meta: AssetMeta<(), Self>, + _writer: &'a mut bevy_asset::io::Writer, + ) -> bevy_utils::BoxedFuture<'a, Result<(), bevy_asset::processor::ProcessError>> { + unreachable!() + } +} + +impl Asset for () {} + +impl VisitAssetDependencies for () { + fn visit_dependencies(&self, _visit: &mut impl FnMut(bevy_asset::UntypedAssetId)) { + unreachable!() + } +} + +/// The () loader should never be called. This implementation exists to make the meta format nicer to work with. +impl AssetLoader for () { + type Asset = (); + type Settings = (); + fn load<'a>( + &'a self, + _reader: &'a mut crate::io::Reader, + _settings: &'a Self::Settings, + _load_context: &'a mut crate::LoadContext, + ) -> bevy_utils::BoxedFuture<'a, Result> { + unreachable!(); + } + + fn extensions(&self) -> &[&str] { + unreachable!(); + } +} + +pub(crate) fn loader_settings_meta_transform( + settings: impl Fn(&mut S) + Send + Sync + 'static, +) -> MetaTransform { + Box::new(move |meta| { + if let Some(loader_settings) = meta.loader_settings_mut() { + if let Some(loader_settings) = loader_settings.downcast_mut::() { + settings(loader_settings); + } else { + error!( + "Configured settings type {} does not match AssetLoader settings type", + std::any::type_name::(), + ); + } + } + }) +} + +pub type AssetHash = [u8; 16]; + +/// NOTE: changing the hashing logic here is a _breaking change_ that requires a [`META_FORMAT_VERSION`] bump. +pub(crate) fn get_asset_hash(meta_bytes: &[u8], asset_bytes: &[u8]) -> AssetHash { + let mut context = md5::Context::new(); + context.consume(meta_bytes); + context.consume(asset_bytes); + let digest = context.compute(); + digest.0 +} + +/// NOTE: changing the hashing logic here is a _breaking change_ that requires a [`META_FORMAT_VERSION`] bump. +pub(crate) fn get_full_asset_hash( + asset_hash: AssetHash, + dependency_hashes: impl Iterator, +) -> AssetHash { + let mut context = md5::Context::new(); + context.consume(asset_hash); + for hash in dependency_hashes { + context.consume(hash); + } + let digest = context.compute(); + digest.0 +} diff --git a/crates/bevy_asset/src/path.rs b/crates/bevy_asset/src/path.rs index 85ca220a357da..108371e8589ca 100644 --- a/crates/bevy_asset/src/path.rs +++ b/crates/bevy_asset/src/path.rs @@ -1,18 +1,59 @@ use bevy_reflect::{Reflect, ReflectDeserialize, ReflectSerialize}; -use bevy_utils::{AHasher, RandomState}; use serde::{Deserialize, Serialize}; use std::{ borrow::Cow, - hash::{BuildHasher, Hash, Hasher}, + fmt::{Debug, Display}, + hash::Hash, path::{Path, PathBuf}, }; -/// Represents a path to an asset in the file system. -#[derive(Debug, Eq, PartialEq, Hash, Clone, Serialize, Deserialize, Reflect)] +/// Represents a path to an asset in a "virtual filesystem". +/// +/// Asset paths consist of two main parts: +/// * [`AssetPath::path`]: The "virtual filesystem path" pointing to an asset source file. +/// * [`AssetPath::label`]: An optional "named sub asset". When assets are loaded, they are +/// allowed to load "sub assets" of any type, which are identified by a named "label". +/// +/// Asset paths are generally constructed (and visualized) as strings: +/// +/// ```no_run +/// # use bevy_asset::{Asset, AssetServer, Handle}; +/// # use bevy_reflect::TypePath; +/// # +/// # #[derive(Asset, TypePath, Default)] +/// # struct Mesh; +/// # +/// # #[derive(Asset, TypePath, Default)] +/// # struct Scene; +/// # +/// # let asset_server: AssetServer = panic!(); +/// // This loads the `my_scene.scn` base asset. +/// let scene: Handle = asset_server.load("my_scene.scn"); +/// +/// // This loads the `PlayerMesh` labeled asset from the `my_scene.scn` base asset. +/// let mesh: Handle = asset_server.load("my_scene.scn#PlayerMesh"); +/// ``` +#[derive(Eq, PartialEq, Hash, Clone, Serialize, Deserialize, Reflect)] #[reflect(Debug, PartialEq, Hash, Serialize, Deserialize)] pub struct AssetPath<'a> { - path: Cow<'a, Path>, - label: Option>, + pub path: Cow<'a, Path>, + pub label: Option>, +} + +impl<'a> Debug for AssetPath<'a> { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + Display::fmt(self, f) + } +} + +impl<'a> Display for AssetPath<'a> { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "{}", self.path.display())?; + if let Some(label) = &self.label { + write!(f, "#{label}")?; + } + Ok(()) + } } impl<'a> AssetPath<'a> { @@ -34,24 +75,40 @@ impl<'a> AssetPath<'a> { } } - /// Constructs an identifier from this asset path. - #[inline] - pub fn get_id(&self) -> AssetPathId { - AssetPathId::from(self) - } - - /// Gets the sub-asset label. + /// Gets the "sub-asset label". #[inline] pub fn label(&self) -> Option<&str> { self.label.as_ref().map(|label| label.as_ref()) } - /// Gets the path to the asset in the filesystem. + /// Gets the path to the asset in the "virtual filesystem". #[inline] pub fn path(&self) -> &Path { &self.path } + /// Gets the path to the asset in the "virtual filesystem" without a label (if a label is currently set). + #[inline] + pub fn without_label(&self) -> AssetPath<'_> { + AssetPath::new_ref(&self.path, None) + } + + /// Removes a "sub-asset label" from this [`AssetPath`] and returns it, if one was set. + #[inline] + pub fn remove_label(&mut self) -> Option> { + self.label.take() + } + + /// Returns this asset path with the given label. This will replace the previous + /// label if it exists. + #[inline] + pub fn with_label(&self, label: impl Into>) -> AssetPath<'a> { + AssetPath { + path: self.path.clone(), + label: Some(label.into()), + } + } + /// Converts the borrowed path data to owned. #[inline] pub fn to_owned(&self) -> AssetPath<'static> { @@ -63,93 +120,24 @@ impl<'a> AssetPath<'a> { .map(|value| Cow::Owned(value.to_string())), } } -} - -/// An unique identifier to an asset path. -#[derive( - Debug, Clone, Copy, Eq, PartialEq, Hash, Ord, PartialOrd, Serialize, Deserialize, Reflect, -)] -#[reflect_value(PartialEq, Hash, Serialize, Deserialize)] -pub struct AssetPathId(SourcePathId, LabelId); - -/// An unique identifier to the source path of an asset. -#[derive( - Debug, Clone, Copy, Eq, PartialEq, Hash, Ord, PartialOrd, Serialize, Deserialize, Reflect, -)] -#[reflect_value(PartialEq, Hash, Serialize, Deserialize)] -pub struct SourcePathId(u64); - -/// An unique identifier to a sub-asset label. -#[derive( - Debug, Clone, Copy, Eq, PartialEq, Hash, Ord, PartialOrd, Serialize, Deserialize, Reflect, -)] -#[reflect_value(PartialEq, Hash, Serialize, Deserialize)] -pub struct LabelId(u64); - -impl<'a> From<&'a Path> for SourcePathId { - fn from(value: &'a Path) -> Self { - let mut hasher = get_hasher(); - value.hash(&mut hasher); - SourcePathId(hasher.finish()) - } -} - -impl From for SourcePathId { - fn from(id: AssetPathId) -> Self { - id.source_path_id() - } -} - -impl<'a> From> for SourcePathId { - fn from(path: AssetPath) -> Self { - AssetPathId::from(path).source_path_id() - } -} - -impl<'a> From> for LabelId { - fn from(value: Option<&'a str>) -> Self { - let mut hasher = get_hasher(); - value.hash(&mut hasher); - LabelId(hasher.finish()) - } -} -impl AssetPathId { - /// Gets the id of the source path. - pub fn source_path_id(&self) -> SourcePathId { - self.0 + /// Returns the full extension (including multiple '.' values). + /// Ex: Returns `"config.ron"` for `"my_asset.config.ron"` + pub fn get_full_extension(&self) -> Option { + let file_name = self.path.file_name()?.to_str()?; + let index = file_name.find('.')?; + let extension = file_name[index + 1..].to_lowercase(); + Some(extension) } - /// Gets the id of the sub-asset label. - pub fn label_id(&self) -> LabelId { - self.1 - } -} - -/// this hasher provides consistent results across runs -pub(crate) fn get_hasher() -> AHasher { - RandomState::with_seeds(42, 23, 13, 8).build_hasher() -} - -impl<'a, T> From for AssetPathId -where - T: Into>, -{ - fn from(value: T) -> Self { - let asset_path: AssetPath = value.into(); - AssetPathId( - SourcePathId::from(asset_path.path()), - LabelId::from(asset_path.label()), - ) - } -} - -impl<'a, 'b> From<&'a AssetPath<'b>> for AssetPathId { - fn from(asset_path: &'a AssetPath<'b>) -> Self { - AssetPathId( - SourcePathId::from(asset_path.path()), - LabelId::from(asset_path.label()), - ) + pub(crate) fn iter_secondary_extensions(full_extension: &str) -> impl Iterator { + full_extension.chars().enumerate().filter_map(|(i, c)| { + if c == '.' { + Some(&full_extension[i + 1..]) + } else { + None + } + }) } } @@ -189,14 +177,11 @@ impl<'a> From for AssetPath<'a> { } } -impl<'a> From for AssetPath<'a> { - fn from(asset_path: String) -> Self { - let mut parts = asset_path.splitn(2, '#'); - let path = PathBuf::from(parts.next().expect("Path must be set.")); - let label = parts.next().map(String::from); - AssetPath { - path: Cow::Owned(path), - label: label.map(Cow::Owned), +impl<'a> From> for PathBuf { + fn from(path: AssetPath<'a>) -> Self { + match path.path { + Cow::Borrowed(borrowed) => borrowed.to_owned(), + Cow::Owned(owned) => owned, } } } diff --git a/crates/bevy_asset/src/processor/log.rs b/crates/bevy_asset/src/processor/log.rs new file mode 100644 index 0000000000000..febe712ed2a5e --- /dev/null +++ b/crates/bevy_asset/src/processor/log.rs @@ -0,0 +1,194 @@ +use async_fs::File; +use bevy_log::error; +use bevy_utils::HashSet; +use futures_lite::{AsyncReadExt, AsyncWriteExt}; +use std::path::{Path, PathBuf}; +use thiserror::Error; + +/// An in-memory representation of a single [`ProcessorTransactionLog`] entry. +#[derive(Debug)] +pub(crate) enum LogEntry { + BeginProcessing(PathBuf), + EndProcessing(PathBuf), + UnrecoverableError, +} + +/// A "write ahead" logger that helps ensure asset importing is transactional. +/// Prior to processing an asset, we write to the log to indicate it has started +/// After processing an asset, we write to the log to indicate it has finished. +/// On startup, the log can be read to determine if any transactions were incomplete. +// TODO: this should be a trait +pub struct ProcessorTransactionLog { + log_file: File, +} + +/// An error that occurs when reading from the [`ProcessorTransactionLog`] fails. +#[derive(Error, Debug)] +pub enum ReadLogError { + #[error("Encountered an invalid log line: '{0}'")] + InvalidLine(String), + #[error("Failed to read log file: {0}")] + Io(#[from] futures_io::Error), +} + +/// An error that occurs when writing to the [`ProcessorTransactionLog`] fails. +#[derive(Error, Debug)] +#[error( + "Failed to write {log_entry:?} to the asset processor log. This is not recoverable. {error}" +)] +pub struct WriteLogError { + log_entry: LogEntry, + error: futures_io::Error, +} + +/// An error that occurs when validating the [`ProcessorTransactionLog`] fails. +#[derive(Error, Debug)] +pub enum ValidateLogError { + #[error("Encountered an unrecoverable error. All assets will be reprocessed.")] + UnrecoverableError, + #[error(transparent)] + ReadLogError(#[from] ReadLogError), + #[error("Encountered a duplicate process asset transaction: {0:?}")] + EntryErrors(Vec), +} + +/// An error that occurs when validating individual [`ProcessorTransactionLog`] entries. +#[derive(Error, Debug)] +pub enum LogEntryError { + #[error("Encountered a duplicate process asset transaction: {0:?}")] + DuplicateTransaction(PathBuf), + #[error("A transaction was ended that never started {0:?}")] + EndedMissingTransaction(PathBuf), + #[error("An asset started processing but never finished: {0:?}")] + UnfinishedTransaction(PathBuf), +} + +const LOG_PATH: &str = "imported_assets/log"; +const ENTRY_BEGIN: &str = "Begin "; +const ENTRY_END: &str = "End "; +const UNRECOVERABLE_ERROR: &str = "UnrecoverableError"; + +impl ProcessorTransactionLog { + fn full_log_path() -> PathBuf { + #[cfg(not(target_arch = "wasm32"))] + let base_path = crate::io::file::get_base_path(); + #[cfg(target_arch = "wasm32")] + let base_path = PathBuf::new(); + base_path.join(LOG_PATH) + } + /// Create a new, fresh log file. This will delete the previous log file if it exists. + pub(crate) async fn new() -> Result { + let path = Self::full_log_path(); + match async_fs::remove_file(&path).await { + Ok(_) => { /* successfully removed file */ } + Err(err) => { + // if the log file is not found, we assume we are starting in a fresh (or good) state + if err.kind() != futures_io::ErrorKind::NotFound { + error!("Failed to remove previous log file {}", err); + } + } + } + + Ok(Self { + log_file: File::create(path).await?, + }) + } + + pub(crate) async fn read() -> Result, ReadLogError> { + let mut log_lines = Vec::new(); + let mut file = match File::open(Self::full_log_path()).await { + Ok(file) => file, + Err(err) => { + if err.kind() == futures_io::ErrorKind::NotFound { + // if the log file doesn't exist, this is equivalent to an empty file + return Ok(log_lines); + } + return Err(err.into()); + } + }; + let mut string = String::new(); + file.read_to_string(&mut string).await?; + for line in string.lines() { + if let Some(path_str) = line.strip_prefix(ENTRY_BEGIN) { + log_lines.push(LogEntry::BeginProcessing(PathBuf::from(path_str))); + } else if let Some(path_str) = line.strip_prefix(ENTRY_END) { + log_lines.push(LogEntry::EndProcessing(PathBuf::from(path_str))); + } else if line.is_empty() { + continue; + } else { + return Err(ReadLogError::InvalidLine(line.to_string())); + } + } + Ok(log_lines) + } + + pub(crate) async fn validate() -> Result<(), ValidateLogError> { + let mut transactions: HashSet = Default::default(); + let mut errors: Vec = Vec::new(); + let entries = Self::read().await?; + for entry in entries { + match entry { + LogEntry::BeginProcessing(path) => { + // There should never be duplicate "start transactions" in a log + // Every start should be followed by: + // * nothing (if there was an abrupt stop) + // * an End (if the transaction was completed) + if !transactions.insert(path.clone()) { + errors.push(LogEntryError::DuplicateTransaction(path)); + } + } + LogEntry::EndProcessing(path) => { + if !transactions.remove(&path) { + errors.push(LogEntryError::EndedMissingTransaction(path)); + } + } + LogEntry::UnrecoverableError => return Err(ValidateLogError::UnrecoverableError), + } + } + for transaction in transactions { + errors.push(LogEntryError::UnfinishedTransaction(transaction)); + } + if !errors.is_empty() { + return Err(ValidateLogError::EntryErrors(errors)); + } + Ok(()) + } + + /// Logs the start of an asset being processed. If this is not followed at some point in the log by a closing [`ProcessorTransactionLog::end_processing`], + /// in the next run of the processor the asset processing will be considered "incomplete" and it will be reprocessed. + pub(crate) async fn begin_processing(&mut self, path: &Path) -> Result<(), WriteLogError> { + self.write(&format!("{ENTRY_BEGIN}{}\n", path.to_string_lossy())) + .await + .map_err(|e| WriteLogError { + log_entry: LogEntry::BeginProcessing(path.to_owned()), + error: e, + }) + } + + /// Logs the end of an asset being successfully processed. See [`ProcessorTransactionLog::begin_processing`]. + pub(crate) async fn end_processing(&mut self, path: &Path) -> Result<(), WriteLogError> { + self.write(&format!("{ENTRY_END}{}\n", path.to_string_lossy())) + .await + .map_err(|e| WriteLogError { + log_entry: LogEntry::EndProcessing(path.to_owned()), + error: e, + }) + } + + /// Logs an unrecoverable error. On the next run of the processor, all assets will be regenerated. This should only be used as a last resort. + /// Every call to this should be considered with scrutiny and ideally replaced with something more granular. + pub(crate) async fn unrecoverable(&mut self) -> Result<(), WriteLogError> { + self.write(UNRECOVERABLE_ERROR) + .await + .map_err(|e| WriteLogError { + log_entry: LogEntry::UnrecoverableError, + error: e, + }) + } + + async fn write(&mut self, line: &str) -> Result<(), futures_io::Error> { + self.log_file.write_all(line.as_bytes()).await?; + self.log_file.flush().await?; + Ok(()) + } +} diff --git a/crates/bevy_asset/src/processor/mod.rs b/crates/bevy_asset/src/processor/mod.rs new file mode 100644 index 0000000000000..b1a9f84ccdf63 --- /dev/null +++ b/crates/bevy_asset/src/processor/mod.rs @@ -0,0 +1,1269 @@ +mod log; +mod process; + +pub use log::*; +pub use process::*; + +use crate::{ + io::{ + processor_gated::ProcessorGatedReader, AssetProvider, AssetProviders, AssetReader, + AssetReaderError, AssetSourceEvent, AssetWatcher, AssetWriter, AssetWriterError, + }, + meta::{ + get_asset_hash, get_full_asset_hash, AssetAction, AssetActionMinimal, AssetHash, AssetMeta, + AssetMetaDyn, AssetMetaMinimal, ProcessedInfo, ProcessedInfoMinimal, + }, + AssetLoadError, AssetLoaderError, AssetPath, AssetServer, DeserializeMetaError, + LoadDirectError, MissingAssetLoaderForExtensionError, CANNOT_WATCH_ERROR_MESSAGE, +}; +use bevy_ecs::prelude::*; +use bevy_log::{debug, error, trace, warn}; +use bevy_tasks::IoTaskPool; +use bevy_utils::{BoxedFuture, HashMap, HashSet}; +use futures_io::ErrorKind; +use futures_lite::{AsyncReadExt, AsyncWriteExt, StreamExt}; +use parking_lot::RwLock; +use std::{ + collections::VecDeque, + path::{Path, PathBuf}, + sync::Arc, +}; +use thiserror::Error; + +/// A "background" asset processor that reads asset values from a source [`AssetProvider`] (which corresponds to an [`AssetReader`] / [`AssetWriter`] pair), +/// processes them in some way, and writes them to a destination [`AssetProvider`]. +/// +/// This will create .meta files (a human-editable serialized form of [`AssetMeta`]) in the source [`AssetProvider`] for assets that +/// that can be loaded and/or processed. This enables developers to configure how each asset should be loaded and/or processed. +/// +/// [`AssetProcessor`] can be run in the background while a Bevy App is running. Changes to assets will be automatically detected and hot-reloaded. +/// +/// Assets will only be re-processed if they have been changed. A hash of each asset source is stored in the metadata of the processed version of the +/// asset, which is used to determine if the asset source has actually changed. +/// +/// A [`ProcessorTransactionLog`] is produced, which uses "write-ahead logging" to make the [`AssetProcessor`] crash and failure resistant. If a failed/unfinished +/// transaction from a previous run is detected, the affected asset(s) will be re-processed. +/// +/// [`AssetProcessor`] can be cloned. It is backed by an [`Arc`] so clones will share state. Clones can be freely used in parallel. +#[derive(Resource, Clone)] +pub struct AssetProcessor { + server: AssetServer, + pub(crate) data: Arc, +} + +pub struct AssetProcessorData { + pub(crate) asset_infos: async_lock::RwLock, + log: async_lock::RwLock>, + processors: RwLock>>, + /// Default processors for file extensions + default_processors: RwLock>, + state: async_lock::RwLock, + source_reader: Box, + source_writer: Box, + destination_reader: Box, + destination_writer: Box, + initialized_sender: async_broadcast::Sender<()>, + initialized_receiver: async_broadcast::Receiver<()>, + finished_sender: async_broadcast::Sender<()>, + finished_receiver: async_broadcast::Receiver<()>, + source_event_receiver: crossbeam_channel::Receiver, + _source_watcher: Option>, +} + +impl AssetProcessor { + /// Creates a new [`AssetProcessor`] instance. + pub fn new( + providers: &mut AssetProviders, + source: &AssetProvider, + destination: &AssetProvider, + ) -> Self { + let data = Arc::new(AssetProcessorData::new( + providers.get_source_reader(source), + providers.get_source_writer(source), + providers.get_destination_reader(destination), + providers.get_destination_writer(destination), + )); + let destination_reader = providers.get_destination_reader(destination); + // The asset processor uses its own asset server with its own id space + let server = AssetServer::new( + Box::new(ProcessorGatedReader::new(destination_reader, data.clone())), + true, + ); + Self { server, data } + } + + /// The "internal" [`AssetServer`] used by the [`AssetProcessor`]. This is _separate_ from the asset processor used by + /// the main App. It has different processor-specific configuration and a different ID space. + pub fn server(&self) -> &AssetServer { + &self.server + } + + async fn set_state(&self, state: ProcessorState) { + let mut state_guard = self.data.state.write().await; + let last_state = *state_guard; + *state_guard = state; + if last_state != ProcessorState::Finished && state == ProcessorState::Finished { + self.data.finished_sender.broadcast(()).await.unwrap(); + } else if last_state != ProcessorState::Processing && state == ProcessorState::Processing { + self.data.initialized_sender.broadcast(()).await.unwrap(); + } + } + + /// Retrieves the current [`ProcessorState`] + pub async fn get_state(&self) -> ProcessorState { + *self.data.state.read().await + } + + /// Retrieves the "source" [`AssetReader`] (the place where user-provided unprocessed "asset sources" are stored) + pub fn source_reader(&self) -> &dyn AssetReader { + &*self.data.source_reader + } + + /// Retrieves the "source" [`AssetWriter`] (the place where user-provided unprocessed "asset sources" are stored) + pub fn source_writer(&self) -> &dyn AssetWriter { + &*self.data.source_writer + } + + /// Retrieves the "destination" [`AssetReader`] (the place where processed / [`AssetProcessor`]-managed assets are stored) + pub fn destination_reader(&self) -> &dyn AssetReader { + &*self.data.destination_reader + } + + /// Retrieves the "destination" [`AssetWriter`] (the place where processed / [`AssetProcessor`]-managed assets are stored) + pub fn destination_writer(&self) -> &dyn AssetWriter { + &*self.data.destination_writer + } + + /// Logs an unrecoverable error. On the next run of the processor, all assets will be regenerated. This should only be used as a last resort. + /// Every call to this should be considered with scrutiny and ideally replaced with something more granular. + async fn log_unrecoverable(&self) { + let mut log = self.data.log.write().await; + let log = log.as_mut().unwrap(); + log.unrecoverable().await.unwrap(); + } + + /// Logs the start of an asset being processed. If this is not followed at some point in the log by a closing [`AssetProcessor::log_end_processing`], + /// in the next run of the processor the asset processing will be considered "incomplete" and it will be reprocessed. + async fn log_begin_processing(&self, path: &Path) { + let mut log = self.data.log.write().await; + let log = log.as_mut().unwrap(); + log.begin_processing(path).await.unwrap(); + } + + /// Logs the end of an asset being successfully processed. See [`AssetProcessor::log_begin_processing`]. + async fn log_end_processing(&self, path: &Path) { + let mut log = self.data.log.write().await; + let log = log.as_mut().unwrap(); + log.end_processing(path).await.unwrap(); + } + + /// Starts the processor in a background thread. + pub fn start(_processor: Res) { + #[cfg(any(target_arch = "wasm32", not(feature = "multi-threaded")))] + error!("Cannot run AssetProcessor in single threaded mode (or WASM) yet."); + #[cfg(all(not(target_arch = "wasm32"), feature = "multi-threaded"))] + { + let processor = _processor.clone(); + std::thread::spawn(move || { + processor.process_assets(); + futures_lite::future::block_on(processor.listen_for_source_change_events()); + }); + } + } + + /// Processes all assets. This will: + /// * Scan the [`ProcessorTransactionLog`] and recover from any failures detected + /// * Scan the destination [`AssetProvider`] to build the current view of already processed assets. + /// * Scan the source [`AssetProvider`] and remove any processed "destination" assets that are invalid or no longer exist. + /// * For each asset in the `source` [`AssetProvider`], kick off a new "process job", which will process the asset + /// (if the latest version of the asset has not been processed). + #[cfg(all(not(target_arch = "wasm32"), feature = "multi-threaded"))] + pub fn process_assets(&self) { + let start_time = std::time::Instant::now(); + debug!("Processing Assets"); + IoTaskPool::get().scope(|scope| { + scope.spawn(async move { + self.initialize().await.unwrap(); + let path = PathBuf::from(""); + self.process_assets_internal(scope, path).await.unwrap(); + }); + }); + // This must happen _after_ the scope resolves or it will happen "too early" + // Don't move this into the async scope above! process_assets is a blocking/sync function this is fine + futures_lite::future::block_on(self.finish_processing_assets()); + let end_time = std::time::Instant::now(); + debug!("Processing finished in {:?}", end_time - start_time); + } + + /// Listens for changes to assets in the source [`AssetProvider`] and update state accordingly. + // PERF: parallelize change event processing + pub async fn listen_for_source_change_events(&self) { + debug!("Listening for changes to source assets"); + loop { + let mut started_processing = false; + + for event in self.data.source_event_receiver.try_iter() { + if !started_processing { + self.set_state(ProcessorState::Processing).await; + started_processing = true; + } + + self.handle_asset_source_event(event).await; + } + + if started_processing { + self.finish_processing_assets().await; + } + } + } + + async fn handle_asset_source_event(&self, event: AssetSourceEvent) { + trace!("{event:?}"); + match event { + AssetSourceEvent::AddedAsset(path) + | AssetSourceEvent::AddedMeta(path) + | AssetSourceEvent::ModifiedAsset(path) + | AssetSourceEvent::ModifiedMeta(path) => { + self.process_asset(&path).await; + } + AssetSourceEvent::RemovedAsset(path) => { + self.handle_removed_asset(path).await; + } + AssetSourceEvent::RemovedMeta(path) => { + self.handle_removed_meta(&path).await; + } + AssetSourceEvent::AddedFolder(path) => { + self.handle_added_folder(path).await; + } + // NOTE: As a heads up for future devs: this event shouldn't be run in parallel with other events that might + // touch this folder (ex: the folder might be re-created with new assets). Clean up the old state first. + // Currently this event handler is not parallel, but it could be (and likely should be) in the future. + AssetSourceEvent::RemovedFolder(path) => { + self.handle_removed_folder(&path).await; + } + AssetSourceEvent::RenamedAsset { old, new } => { + // If there was a rename event, but the path hasn't changed, this asset might need reprocessing. + // Sometimes this event is returned when an asset is moved "back" into the asset folder + if old == new { + self.process_asset(&new).await; + } else { + self.handle_renamed_asset(old, new).await; + } + } + AssetSourceEvent::RenamedMeta { old, new } => { + // If there was a rename event, but the path hasn't changed, this asset meta might need reprocessing. + // Sometimes this event is returned when an asset meta is moved "back" into the asset folder + if old == new { + self.process_asset(&new).await; + } else { + debug!("Meta renamed from {old:?} to {new:?}"); + let mut infos = self.data.asset_infos.write().await; + // Renaming meta should not assume that an asset has also been renamed. Check both old and new assets to see + // if they should be re-imported (and/or have new meta generated) + infos.check_reprocess_queue.push_back(old); + infos.check_reprocess_queue.push_back(new); + } + } + AssetSourceEvent::RenamedFolder { old, new } => { + // If there was a rename event, but the path hasn't changed, this asset folder might need reprocessing. + // Sometimes this event is returned when an asset meta is moved "back" into the asset folder + if old == new { + self.handle_added_folder(new).await; + } else { + // PERF: this reprocesses everything in the moved folder. this is not necessary in most cases, but + // requires some nuance when it comes to path handling. + self.handle_removed_folder(&old).await; + self.handle_added_folder(new).await; + } + } + AssetSourceEvent::RemovedUnknown { path, is_meta } => { + match self.destination_reader().is_directory(&path).await { + Ok(is_directory) => { + if is_directory { + self.handle_removed_folder(&path).await; + } else if is_meta { + self.handle_removed_meta(&path).await; + } else { + self.handle_removed_asset(path).await; + } + } + Err(err) => { + if let AssetReaderError::NotFound(_) = err { + // if the path is not found, a processed version does not exist + } else { + error!( + "Path '{path:?}' as removed, but the destination reader could not determine if it \ + was a folder or a file due to the following error: {err}" + ); + } + } + } + } + } + } + + async fn handle_added_folder(&self, path: PathBuf) { + debug!("Folder {:?} was added. Attempting to re-process", path); + #[cfg(any(target_arch = "wasm32", not(feature = "multi-threaded")))] + error!("AddFolder event cannot be handled in single threaded mode (or WASM) yet."); + #[cfg(all(not(target_arch = "wasm32"), feature = "multi-threaded"))] + IoTaskPool::get().scope(|scope| { + scope.spawn(async move { + self.process_assets_internal(scope, path).await.unwrap(); + }); + }); + } + + /// Responds to a removed meta event by reprocessing the asset at the given path. + async fn handle_removed_meta(&self, path: &Path) { + // If meta was removed, we might need to regenerate it. + // Likewise, the user might be manually re-adding the asset. + // Therefore, we shouldn't automatically delete the asset ... that is a + // user-initiated action. + debug!( + "Meta for asset {:?} was removed. Attempting to re-process", + path + ); + self.process_asset(path).await; + } + + /// Removes all processed assets stored at the given path (respecting transactionality), then removes the folder itself. + async fn handle_removed_folder(&self, path: &Path) { + debug!("Removing folder {:?} because source was removed", path); + match self.destination_reader().read_directory(path).await { + Ok(mut path_stream) => { + while let Some(child_path) = path_stream.next().await { + self.handle_removed_asset(child_path).await; + } + } + Err(err) => match err { + AssetReaderError::NotFound(_err) => { + // The processed folder does not exist. No need to update anything + } + AssetReaderError::Io(err) => { + self.log_unrecoverable().await; + error!( + "Unrecoverable Error: Failed to read the processed assets at {path:?} in order to remove assets that no longer exist \ + in the source directory. Restart the asset processor to fully reprocess assets. Error: {err}" + ); + } + }, + } + if let Err(AssetWriterError::Io(err)) = + self.destination_writer().remove_directory(path).await + { + // we can ignore NotFound because if the "final" file in a folder was removed + // then we automatically clean up this folder + if err.kind() != ErrorKind::NotFound { + error!("Failed to remove destination folder that no longer exists in asset source {path:?}: {err}"); + } + } + } + + /// Removes the processed version of an asset and associated in-memory metadata. This will block until all existing reads/writes to the + /// asset have finished, thanks to the `file_transaction_lock`. + async fn handle_removed_asset(&self, path: PathBuf) { + debug!("Removing processed {:?} because source was removed", path); + let asset_path = AssetPath::new(path, None); + let mut infos = self.data.asset_infos.write().await; + if let Some(info) = infos.get(&asset_path) { + // we must wait for uncontested write access to the asset source to ensure existing readers / writers + // can finish their operations + let _write_lock = info.file_transaction_lock.write(); + self.remove_processed_asset_and_meta(asset_path.path()) + .await; + } + infos.remove(&asset_path).await; + } + + /// Handles a renamed source asset by moving it's processed results to the new location and updating in-memory paths + metadata. + /// This will cause direct path dependencies to break. + async fn handle_renamed_asset(&self, old: PathBuf, new: PathBuf) { + let mut infos = self.data.asset_infos.write().await; + let old_asset_path = AssetPath::new(old, None); + if let Some(info) = infos.get(&old_asset_path) { + // we must wait for uncontested write access to the asset source to ensure existing readers / writers + // can finish their operations + let _write_lock = info.file_transaction_lock.write(); + let old = &old_asset_path.path; + self.destination_writer().rename(old, &new).await.unwrap(); + self.destination_writer() + .rename_meta(old, &new) + .await + .unwrap(); + } + let new_asset_path = AssetPath::new(new.clone(), None); + infos.rename(&old_asset_path, &new_asset_path).await; + } + + async fn finish_processing_assets(&self) { + self.try_reprocessing_queued().await; + // clean up metadata in asset server + self.server.data.infos.write().consume_handle_drop_events(); + self.set_state(ProcessorState::Finished).await; + } + + #[allow(unused)] + #[cfg(all(not(target_arch = "wasm32"), feature = "multi-threaded"))] + fn process_assets_internal<'scope>( + &'scope self, + scope: &'scope bevy_tasks::Scope<'scope, '_, ()>, + path: PathBuf, + ) -> bevy_utils::BoxedFuture<'scope, Result<(), AssetReaderError>> { + Box::pin(async move { + if self.source_reader().is_directory(&path).await? { + let mut path_stream = self.source_reader().read_directory(&path).await?; + while let Some(path) = path_stream.next().await { + self.process_assets_internal(scope, path).await?; + } + } else { + // Files without extensions are skipped + let processor = self.clone(); + scope.spawn(async move { + processor.process_asset(&path).await; + }); + } + Ok(()) + }) + } + + async fn try_reprocessing_queued(&self) { + loop { + let mut check_reprocess_queue = + std::mem::take(&mut self.data.asset_infos.write().await.check_reprocess_queue); + IoTaskPool::get().scope(|scope| { + for path in check_reprocess_queue.drain(..) { + let processor = self.clone(); + scope.spawn(async move { + processor.process_asset(&path).await; + }); + } + }); + let infos = self.data.asset_infos.read().await; + if infos.check_reprocess_queue.is_empty() { + break; + } + } + } + + /// Register a new asset processor. + pub fn register_processor(&self, processor: P) { + let mut process_plans = self.data.processors.write(); + process_plans.insert(std::any::type_name::

(), Arc::new(processor)); + } + + /// Set the default processor for the given `extension`. Make sure `P` is registered with [`AssetProcessor::register_processor`]. + pub fn set_default_processor(&self, extension: &str) { + let mut default_processors = self.data.default_processors.write(); + default_processors.insert(extension.to_string(), std::any::type_name::

()); + } + + /// Returns the default processor for the given `extension`, if it exists. + pub fn get_default_processor(&self, extension: &str) -> Option> { + let default_processors = self.data.default_processors.read(); + let key = default_processors.get(extension)?; + self.data.processors.read().get(key).cloned() + } + + /// Returns the processor with the given `processor_type_name`, if it exists. + pub fn get_processor(&self, processor_type_name: &str) -> Option> { + let processors = self.data.processors.read(); + processors.get(processor_type_name).cloned() + } + + /// Populates the initial view of each asset by scanning the source and destination folders. + /// This info will later be used to determine whether or not to re-process an asset + /// + /// This will validate transactions and recover failed transactions when necessary. + #[allow(unused)] + async fn initialize(&self) -> Result<(), InitializeError> { + self.validate_transaction_log_and_recover().await; + let mut asset_infos = self.data.asset_infos.write().await; + + /// Retrieves asset paths recursively. If `clean_empty_folders_writer` is Some, it will be used to clean up empty + /// folders when they are discovered. + fn get_asset_paths<'a>( + reader: &'a dyn AssetReader, + clean_empty_folders_writer: Option<&'a dyn AssetWriter>, + path: PathBuf, + paths: &'a mut Vec, + ) -> BoxedFuture<'a, Result> { + Box::pin(async move { + if reader.is_directory(&path).await? { + let mut path_stream = reader.read_directory(&path).await?; + let mut contains_files = false; + while let Some(child_path) = path_stream.next().await { + contains_files = + get_asset_paths(reader, clean_empty_folders_writer, child_path, paths) + .await? + && contains_files; + } + if !contains_files { + if let Some(writer) = clean_empty_folders_writer { + // it is ok for this to fail as it is just a cleanup job. + let _ = writer.remove_empty_directory(&path).await; + } + } + Ok(contains_files) + } else { + paths.push(path); + Ok(true) + } + }) + } + + let mut source_paths = Vec::new(); + let source_reader = self.source_reader(); + get_asset_paths(source_reader, None, PathBuf::from(""), &mut source_paths) + .await + .map_err(InitializeError::FailedToReadSourcePaths)?; + + let mut destination_paths = Vec::new(); + let destination_reader = self.destination_reader(); + let destination_writer = self.destination_writer(); + get_asset_paths( + destination_reader, + Some(destination_writer), + PathBuf::from(""), + &mut destination_paths, + ) + .await + .map_err(InitializeError::FailedToReadDestinationPaths)?; + + for path in &source_paths { + asset_infos.get_or_insert(AssetPath::new(path.to_owned(), None)); + } + + for path in &destination_paths { + let asset_path = AssetPath::new(path.to_owned(), None); + let mut dependencies = Vec::new(); + if let Some(info) = asset_infos.get_mut(&asset_path) { + match self.destination_reader().read_meta_bytes(path).await { + Ok(meta_bytes) => { + match ron::de::from_bytes::(&meta_bytes) { + Ok(minimal) => { + trace!( + "Populated processed info for asset {path:?} {:?}", + minimal.processed_info + ); + + if let Some(processed_info) = &minimal.processed_info { + for process_dependency_info in + &processed_info.process_dependencies + { + dependencies.push(process_dependency_info.path.to_owned()); + } + } + info.processed_info = minimal.processed_info; + } + Err(err) => { + trace!("Removing processed data for {path:?} because meta could not be parsed: {err}"); + self.remove_processed_asset_and_meta(path).await; + } + } + } + Err(err) => { + trace!("Removing processed data for {path:?} because meta failed to load: {err}"); + self.remove_processed_asset_and_meta(path).await; + } + } + } else { + trace!("Removing processed data for non-existent asset {path:?}"); + self.remove_processed_asset_and_meta(path).await; + } + + for dependency in dependencies { + asset_infos.add_dependant(&dependency, asset_path.to_owned()); + } + } + + self.set_state(ProcessorState::Processing).await; + + Ok(()) + } + + /// Removes the processed version of an asset and its metadata, if it exists. This _is not_ transactional like `remove_processed_asset_transactional`, nor + /// does it remove existing in-memory metadata. + async fn remove_processed_asset_and_meta(&self, path: &Path) { + if let Err(err) = self.destination_writer().remove(path).await { + warn!("Failed to remove non-existent asset {path:?}: {err}"); + } + + if let Err(err) = self.destination_writer().remove_meta(path).await { + warn!("Failed to remove non-existent meta {path:?}: {err}"); + } + + self.clean_empty_processed_ancestor_folders(path).await; + } + + async fn clean_empty_processed_ancestor_folders(&self, path: &Path) { + // As a safety precaution don't delete absolute paths to avoid deleting folders outside of the destination folder + if path.is_absolute() { + error!("Attempted to clean up ancestor folders of an absolute path. This is unsafe so the operation was skipped."); + return; + } + while let Some(parent) = path.parent() { + if parent == Path::new("") { + break; + } + if self + .destination_writer() + .remove_empty_directory(parent) + .await + .is_err() + { + // if we fail to delete a folder, stop walking up the tree + break; + } + } + } + + /// Processes the asset (if it has not already been processed or the asset source has changed). + /// If the asset has "process dependencies" (relies on the values of other assets), it will asynchronously await until + /// the dependencies have been processed (See [`ProcessorGatedReader`], which is used in the [`AssetProcessor`]'s [`AssetServer`] + /// to block reads until the asset is processed). + /// + /// [`LoadContext`]: crate::loader::LoadContext + async fn process_asset(&self, path: &Path) { + let result = self.process_asset_internal(path).await; + let mut infos = self.data.asset_infos.write().await; + let asset_path = AssetPath::new(path.to_owned(), None); + infos.finish_processing(asset_path, result).await; + } + + async fn process_asset_internal(&self, path: &Path) -> Result { + if path.extension().is_none() { + return Err(ProcessError::ExtensionRequired); + } + let asset_path = AssetPath::new(path.to_owned(), None); + // TODO: check if already processing to protect against duplicate hot-reload events + debug!("Processing {:?}", path); + let server = &self.server; + + // Note: we get the asset source reader first because we don't want to create meta files for assets that don't have source files + let mut reader = self.source_reader().read(path).await.map_err(|e| match e { + AssetReaderError::NotFound(_) => ProcessError::MissingAssetSource(path.to_owned()), + AssetReaderError::Io(err) => ProcessError::AssetSourceIoError(err), + })?; + + let (mut source_meta, meta_bytes, processor) = match self + .source_reader() + .read_meta_bytes(path) + .await + { + Ok(meta_bytes) => { + let minimal: AssetMetaMinimal = ron::de::from_bytes(&meta_bytes).map_err(|e| { + ProcessError::DeserializeMetaError(DeserializeMetaError::DeserializeMinimal(e)) + })?; + let (meta, processor) = match minimal.asset { + AssetActionMinimal::Load { loader } => { + let loader = server.get_asset_loader_with_type_name(&loader).await?; + let meta = loader.deserialize_meta(&meta_bytes)?; + (meta, None) + } + AssetActionMinimal::Process { processor } => { + let processor = self + .get_processor(&processor) + .ok_or_else(|| ProcessError::MissingProcessor(processor))?; + let meta = processor.deserialize_meta(&meta_bytes)?; + (meta, Some(processor)) + } + AssetActionMinimal::Ignore => { + let meta: Box = + Box::new(AssetMeta::<(), ()>::deserialize(&meta_bytes)?); + (meta, None) + } + }; + (meta, meta_bytes, processor) + } + Err(AssetReaderError::NotFound(_path)) => { + let (meta, processor) = if let Some(processor) = asset_path + .get_full_extension() + .and_then(|ext| self.get_default_processor(&ext)) + { + let meta = processor.default_meta(); + (meta, Some(processor)) + } else { + match server.get_path_asset_loader(&asset_path).await { + Ok(loader) => (loader.default_meta(), None), + Err(MissingAssetLoaderForExtensionError { .. }) => { + let meta: Box = + Box::new(AssetMeta::<(), ()>::new(AssetAction::Ignore)); + (meta, None) + } + } + }; + let meta_bytes = meta.serialize(); + // write meta to source location if it doesn't already exist + self.source_writer() + .write_meta_bytes(path, &meta_bytes) + .await?; + (meta, meta_bytes, processor) + } + Err(err) => return Err(ProcessError::ReadAssetMetaError(err)), + }; + + let mut asset_bytes = Vec::new(); + reader + .read_to_end(&mut asset_bytes) + .await + .map_err(ProcessError::AssetSourceIoError)?; + + // PERF: in theory these hashes could be streamed if we want to avoid allocating the whole asset. + // The downside is that reading assets would need to happen twice (once for the hash and once for the asset loader) + // Hard to say which is worse + let new_hash = get_asset_hash(&meta_bytes, &asset_bytes); + let mut new_processed_info = ProcessedInfo { + hash: new_hash, + full_hash: new_hash, + process_dependencies: Vec::new(), + }; + + { + let infos = self.data.asset_infos.read().await; + if let Some(current_processed_info) = infos + .get(&asset_path) + .and_then(|i| i.processed_info.as_ref()) + { + if current_processed_info.hash == new_hash { + let mut dependency_changed = false; + for current_dep_info in ¤t_processed_info.process_dependencies { + let live_hash = infos + .get(¤t_dep_info.path) + .and_then(|i| i.processed_info.as_ref()) + .map(|i| i.full_hash); + if live_hash != Some(current_dep_info.full_hash) { + dependency_changed = true; + break; + } + } + if !dependency_changed { + return Ok(ProcessResult::SkippedNotChanged); + } + } + } + } + // Note: this lock must remain alive until all processed asset asset and meta writes have finished (or failed) + // See ProcessedAssetInfo::file_transaction_lock docs for more info + let _transaction_lock = { + let mut infos = self.data.asset_infos.write().await; + let info = infos.get_or_insert(asset_path.clone()); + info.file_transaction_lock.write_arc().await + }; + + // NOTE: if processing the asset fails this will produce an "unfinished" log entry, forcing a rebuild on next run. + // Directly writing to the asset destination in the processor necessitates this behavior + // TODO: this class of failure can be recovered via re-processing + smarter log validation that allows for duplicate transactions in the event of failures + self.log_begin_processing(path).await; + if let Some(processor) = processor { + let mut writer = self.destination_writer().write(path).await?; + let mut processed_meta = { + let mut context = + ProcessContext::new(self, &asset_path, &asset_bytes, &mut new_processed_info); + processor + .process(&mut context, source_meta, &mut *writer) + .await? + }; + + writer.flush().await.map_err(AssetWriterError::Io)?; + + let full_hash = get_full_asset_hash( + new_hash, + new_processed_info + .process_dependencies + .iter() + .map(|i| i.full_hash), + ); + new_processed_info.full_hash = full_hash; + *processed_meta.processed_info_mut() = Some(new_processed_info.clone()); + let meta_bytes = processed_meta.serialize(); + self.destination_writer() + .write_meta_bytes(path, &meta_bytes) + .await?; + } else { + self.destination_writer() + .write_bytes(path, &asset_bytes) + .await?; + *source_meta.processed_info_mut() = Some(new_processed_info.clone()); + let meta_bytes = source_meta.serialize(); + self.destination_writer() + .write_meta_bytes(path, &meta_bytes) + .await?; + } + self.log_end_processing(path).await; + + Ok(ProcessResult::Processed(new_processed_info)) + } + + async fn validate_transaction_log_and_recover(&self) { + if let Err(err) = ProcessorTransactionLog::validate().await { + let state_is_valid = match err { + ValidateLogError::ReadLogError(err) => { + error!("Failed to read processor log file. Processed assets cannot be validated so they must be re-generated {err}"); + false + } + ValidateLogError::UnrecoverableError => { + error!("Encountered an unrecoverable error in the last run. Processed assets cannot be validated so they must be re-generated"); + false + } + ValidateLogError::EntryErrors(entry_errors) => { + let mut state_is_valid = true; + for entry_error in entry_errors { + match entry_error { + LogEntryError::DuplicateTransaction(_) + | LogEntryError::EndedMissingTransaction(_) => { + error!("{}", entry_error); + state_is_valid = false; + break; + } + LogEntryError::UnfinishedTransaction(path) => { + debug!("Asset {path:?} did not finish processing. Clearning state for that asset"); + if let Err(err) = self.destination_writer().remove(&path).await { + match err { + AssetWriterError::Io(err) => { + // any error but NotFound means we could be in a bad state + if err.kind() != ErrorKind::NotFound { + error!("Failed to remove asset {path:?}: {err}"); + state_is_valid = false; + } + } + } + } + if let Err(err) = self.destination_writer().remove_meta(&path).await + { + match err { + AssetWriterError::Io(err) => { + // any error but NotFound means we could be in a bad state + if err.kind() != ErrorKind::NotFound { + error!( + "Failed to remove asset meta {path:?}: {err}" + ); + state_is_valid = false; + } + } + } + } + } + } + } + state_is_valid + } + }; + + if !state_is_valid { + error!("Processed asset transaction log state was invalid and unrecoverable for some reason (see previous logs). Removing processed assets and starting fresh."); + if let Err(err) = self + .destination_writer() + .remove_assets_in_directory(Path::new("")) + .await + { + panic!("Processed assets were in a bad state. To correct this, the asset processor attempted to remove all processed assets and start from scratch. This failed. There is no way to continue. Try restarting, or deleting imported asset folder manually. {err}"); + } + } + } + let mut log = self.data.log.write().await; + *log = match ProcessorTransactionLog::new().await { + Ok(log) => Some(log), + Err(err) => panic!("Failed to initialize asset processor log. This cannot be recovered. Try restarting. If that doesn't work, try deleting processed asset folder. {}", err), + }; + } +} + +impl AssetProcessorData { + pub fn new( + source_reader: Box, + source_writer: Box, + destination_reader: Box, + destination_writer: Box, + ) -> Self { + let (mut finished_sender, finished_receiver) = async_broadcast::broadcast(1); + let (mut initialized_sender, initialized_receiver) = async_broadcast::broadcast(1); + // allow overflow on these "one slot" channels to allow receivers to retrieve the "latest" state, and to allow senders to + // not block if there was older state present. + finished_sender.set_overflow(true); + initialized_sender.set_overflow(true); + let (source_event_sender, source_event_receiver) = crossbeam_channel::unbounded(); + // TODO: watching for changes could probably be entirely optional / we could just warn here + let source_watcher = source_reader.watch_for_changes(source_event_sender); + if source_watcher.is_none() { + error!("{}", CANNOT_WATCH_ERROR_MESSAGE); + } + AssetProcessorData { + source_reader, + source_writer, + destination_reader, + destination_writer, + finished_sender, + finished_receiver, + initialized_sender, + initialized_receiver, + source_event_receiver, + _source_watcher: source_watcher, + state: async_lock::RwLock::new(ProcessorState::Initializing), + log: Default::default(), + processors: Default::default(), + asset_infos: Default::default(), + default_processors: Default::default(), + } + } + + /// Returns a future that will not finish until the path has been processed. + pub async fn wait_until_processed(&self, path: &Path) -> ProcessStatus { + self.wait_until_initialized().await; + let mut receiver = { + let infos = self.asset_infos.write().await; + let info = infos.get(&AssetPath::new(path.to_owned(), None)); + match info { + Some(info) => match info.status { + Some(result) => return result, + // This receiver must be created prior to losing the read lock to ensure this is transactional + None => info.status_receiver.clone(), + }, + None => return ProcessStatus::NonExistent, + } + }; + receiver.recv().await.unwrap() + } + + /// Returns a future that will not finish until the processor has been initialized. + pub async fn wait_until_initialized(&self) { + let receiver = { + let state = self.state.read().await; + match *state { + ProcessorState::Initializing => { + // This receiver must be created prior to losing the read lock to ensure this is transactional + Some(self.initialized_receiver.clone()) + } + _ => None, + } + }; + + if let Some(mut receiver) = receiver { + receiver.recv().await.unwrap(); + } + } + + /// Returns a future that will not finish until processing has finished. + pub async fn wait_until_finished(&self) { + let receiver = { + let state = self.state.read().await; + match *state { + ProcessorState::Initializing | ProcessorState::Processing => { + // This receiver must be created prior to losing the read lock to ensure this is transactional + Some(self.finished_receiver.clone()) + } + ProcessorState::Finished => None, + } + }; + + if let Some(mut receiver) = receiver { + receiver.recv().await.unwrap(); + } + } +} + +/// The (successful) result of processing an asset +#[derive(Debug, Clone)] +pub enum ProcessResult { + Processed(ProcessedInfo), + SkippedNotChanged, +} + +/// The final status of processing an asset +#[derive(Debug, PartialEq, Eq, Copy, Clone)] +pub enum ProcessStatus { + Processed, + Failed, + NonExistent, +} + +// NOTE: if you add new fields to this struct, make sure they are propagated (when relevant) in ProcessorAssetInfos::rename +#[derive(Debug)] +pub(crate) struct ProcessorAssetInfo { + processed_info: Option, + /// Paths of assets that depend on this asset when they are being processed. + dependants: HashSet>, + status: Option, + /// A lock that controls read/write access to processed asset files. The lock is shared for both the asset bytes and the meta bytes. + /// _This lock must be locked whenever a read or write to processed assets occurs_ + /// There are scenarios where processed assets (and their metadata) are being read and written in multiple places at once: + /// * when the processor is running in parallel with an app + /// * when processing assets in parallel, the processor might read an asset's process_dependencies when processing new versions of those dependencies + /// * this second scenario almost certainly isn't possible with the current implementation, but its worth protecting against + /// This lock defends against those scenarios by ensuring readers don't read while processed files are being written. And it ensures + /// Because this lock is shared across meta and asset bytes, readers can esure they don't read "old" versions of metadata with "new" asset data. + pub(crate) file_transaction_lock: Arc>, + status_sender: async_broadcast::Sender, + status_receiver: async_broadcast::Receiver, +} + +impl Default for ProcessorAssetInfo { + fn default() -> Self { + let (mut status_sender, status_receiver) = async_broadcast::broadcast(1); + // allow overflow on these "one slot" channels to allow receivers to retrieve the "latest" state, and to allow senders to + // not block if there was older state present. + status_sender.set_overflow(true); + Self { + processed_info: Default::default(), + dependants: Default::default(), + file_transaction_lock: Default::default(), + status: None, + status_sender, + status_receiver, + } + } +} + +impl ProcessorAssetInfo { + async fn update_status(&mut self, status: ProcessStatus) { + if self.status != Some(status) { + self.status = Some(status); + self.status_sender.broadcast(status).await.unwrap(); + } + } +} + +/// The "current" in memory view of the asset space. This is "eventually consistent". It does not directly +/// represent the state of assets in storage, but rather a valid historical view that will gradually become more +/// consistent as events are processed. +#[derive(Default, Debug)] +pub struct ProcessorAssetInfos { + /// The "current" in memory view of the asset space. During processing, if path does not exist in this, it should + /// be considered non-existent. + /// NOTE: YOU MUST USE `Self::get_or_insert` or `Self::insert` TO ADD ITEMS TO THIS COLLECTION TO ENSURE + /// non_existent_dependants DATA IS CONSUMED + infos: HashMap, ProcessorAssetInfo>, + /// Dependants for assets that don't exist. This exists to track "dangling" asset references due to deleted / missing files. + /// If the dependant asset is added, it can "resolve" these dependencies and re-compute those assets. + /// Therefore this _must_ always be consistent with the `infos` data. If a new asset is added to `infos`, it should + /// check this maps for dependencies and add them. If an asset is removed, it should update the dependants here. + non_existent_dependants: HashMap, HashSet>>, + check_reprocess_queue: VecDeque, +} + +impl ProcessorAssetInfos { + fn get_or_insert(&mut self, asset_path: AssetPath<'static>) -> &mut ProcessorAssetInfo { + self.infos.entry(asset_path.clone()).or_insert_with(|| { + let mut info = ProcessorAssetInfo::default(); + // track existing dependants by resolving existing "hanging" dependants. + if let Some(dependants) = self.non_existent_dependants.remove(&asset_path) { + info.dependants = dependants; + } + info + }) + } + + pub(crate) fn get(&self, asset_path: &AssetPath<'static>) -> Option<&ProcessorAssetInfo> { + self.infos.get(asset_path) + } + + fn get_mut(&mut self, asset_path: &AssetPath<'static>) -> Option<&mut ProcessorAssetInfo> { + self.infos.get_mut(asset_path) + } + + fn add_dependant(&mut self, asset_path: &AssetPath<'static>, dependant: AssetPath<'static>) { + if let Some(info) = self.get_mut(asset_path) { + info.dependants.insert(dependant); + } else { + let dependants = self + .non_existent_dependants + .entry(asset_path.to_owned()) + .or_default(); + dependants.insert(dependant); + } + } + + /// Finalize processing the asset, which will incorporate the result of the processed asset into the in-memory view the processed assets. + async fn finish_processing( + &mut self, + asset_path: AssetPath<'static>, + result: Result, + ) { + match result { + Ok(ProcessResult::Processed(processed_info)) => { + debug!("Finished processing \"{:?}\"", asset_path); + // clean up old dependants + let old_processed_info = self + .infos + .get_mut(&asset_path) + .and_then(|i| i.processed_info.take()); + if let Some(old_processed_info) = old_processed_info { + self.clear_dependencies(&asset_path, old_processed_info); + } + + // populate new dependants + for process_dependency_info in &processed_info.process_dependencies { + self.add_dependant(&process_dependency_info.path, asset_path.to_owned()); + } + let info = self.get_or_insert(asset_path); + info.processed_info = Some(processed_info); + info.update_status(ProcessStatus::Processed).await; + let dependants = info.dependants.iter().cloned().collect::>(); + for path in dependants { + self.check_reprocess_queue.push_back(path.path().to_owned()); + } + } + Ok(ProcessResult::SkippedNotChanged) => { + debug!("Skipping processing (unchanged) \"{:?}\"", asset_path); + let info = self.get_mut(&asset_path).expect("info should exist"); + // NOTE: skipping an asset on a given pass doesn't mean it won't change in the future as a result + // of a dependency being re-processed. This means apps might receive an "old" (but valid) asset first. + // This is in the interest of fast startup times that don't block for all assets being checked + reprocessed + // Therefore this relies on hot-reloading in the app to pickup the "latest" version of the asset + // If "block until latest state is reflected" is required, we can easily add a less granular + // "block until first pass finished" mode + info.update_status(ProcessStatus::Processed).await; + } + Err(ProcessError::ExtensionRequired) => { + // Skip assets without extensions + } + Err(ProcessError::MissingAssetLoaderForExtension(_)) => { + trace!("No loader found for {:?}", asset_path); + } + Err(ProcessError::MissingAssetSource(_)) => { + // if there is no asset source, no processing can be done + trace!( + "No need to process asset {:?} because it does not exist", + asset_path + ); + } + Err(err) => { + error!("Failed to process asset {:?}: {:?}", asset_path, err); + // if this failed because a dependency could not be loaded, make sure it is reprocessed if that dependency is reprocessed + if let ProcessError::AssetLoadError(AssetLoadError::AssetLoaderError { + error: AssetLoaderError::Load(loader_error), + .. + }) = err + { + if let Some(error) = loader_error.downcast_ref::() { + let info = self.get_mut(&asset_path).expect("info should exist"); + info.processed_info = Some(ProcessedInfo { + hash: AssetHash::default(), + full_hash: AssetHash::default(), + process_dependencies: vec![], + }); + self.add_dependant(&error.dependency, asset_path.to_owned()); + } + } + + let info = self.get_mut(&asset_path).expect("info should exist"); + info.update_status(ProcessStatus::Failed).await; + } + } + } + + /// Remove the info for the given path. This should only happen if an asset's source is removed / non-existent + async fn remove(&mut self, asset_path: &AssetPath<'static>) { + let info = self.infos.remove(asset_path); + if let Some(info) = info { + if let Some(processed_info) = info.processed_info { + self.clear_dependencies(asset_path, processed_info); + } + // Tell all listeners this asset does not exist + info.status_sender + .broadcast(ProcessStatus::NonExistent) + .await + .unwrap(); + if !info.dependants.is_empty() { + error!( + "The asset at {asset_path} was removed, but it had assets that depend on it to be processed. Consider updating the path in the following assets: {:?}", + info.dependants + ); + self.non_existent_dependants + .insert(asset_path.clone(), info.dependants); + } + } + } + + /// Remove the info for the given path. This should only happen if an asset's source is removed / non-existent + async fn rename(&mut self, old: &AssetPath<'static>, new: &AssetPath<'static>) { + let info = self.infos.remove(old); + if let Some(mut info) = info { + if !info.dependants.is_empty() { + // TODO: We can't currently ensure "moved" folders with relative paths aren't broken because AssetPath + // doesn't distinguish between absolute and relative paths. We have "erased" relativeness. In the short term, + // we could do "remove everything in a folder and re-add", but that requires full rebuilds / destroying the cache. + // If processors / loaders could enumerate dependencies, we could check if the new deps line up with a rename. + // If deps encoded "relativeness" as part of loading, that would also work (this seems like the right call). + // TODO: it would be nice to log an error here for dependants that aren't also being moved + fixed. + // (see the remove impl). + error!( + "The asset at {old} was removed, but it had assets that depend on it to be processed. Consider updating the path in the following assets: {:?}", + info.dependants + ); + self.non_existent_dependants + .insert(old.clone(), std::mem::take(&mut info.dependants)); + } + if let Some(processed_info) = &info.processed_info { + // Update "dependant" lists for this asset's "process dependencies" to use new path. + for dep in &processed_info.process_dependencies { + if let Some(info) = self.infos.get_mut(&dep.path) { + info.dependants.remove(old); + info.dependants.insert(new.clone()); + } else if let Some(dependants) = self.non_existent_dependants.get_mut(&dep.path) + { + dependants.remove(old); + dependants.insert(new.clone()); + } + } + } + // Tell all listeners this asset no longer exists + info.status_sender + .broadcast(ProcessStatus::NonExistent) + .await + .unwrap(); + let dependants: Vec> = { + let new_info = self.get_or_insert(new.clone()); + new_info.processed_info = info.processed_info; + new_info.status = info.status; + // Ensure things waiting on the new path are informed of the status of this asset + if let Some(status) = new_info.status { + new_info.status_sender.broadcast(status).await.unwrap(); + } + new_info.dependants.iter().cloned().collect() + }; + // Queue the asset for a reprocess check, in case it needs new meta. + self.check_reprocess_queue.push_back(new.path().to_owned()); + for dependant in dependants { + // Queue dependants for reprocessing because they might have been waiting for this asset. + self.check_reprocess_queue.push_back(dependant.into()); + } + } + } + + fn clear_dependencies(&mut self, asset_path: &AssetPath<'static>, removed_info: ProcessedInfo) { + for old_load_dep in removed_info.process_dependencies { + if let Some(info) = self.infos.get_mut(&old_load_dep.path) { + info.dependants.remove(asset_path); + } else if let Some(dependants) = + self.non_existent_dependants.get_mut(&old_load_dep.path) + { + dependants.remove(asset_path); + } + } + } +} + +/// The current state of the [`AssetProcessor`]. +#[derive(Copy, Clone, PartialEq, Eq)] +pub enum ProcessorState { + /// The processor is still initializing, which involves scanning the current asset folders, + /// constructing an in-memory view of the asset space, recovering from previous errors / crashes, + /// and cleaning up old / unused assets. + Initializing, + /// The processor is currently processing assets. + Processing, + /// The processor has finished processing all valid assets and reporting invalid assets. + Finished, +} + +/// An error that occurs when initializing the [`AssetProcessor`]. +#[derive(Error, Debug)] +pub enum InitializeError { + #[error(transparent)] + FailedToReadSourcePaths(AssetReaderError), + #[error(transparent)] + FailedToReadDestinationPaths(AssetReaderError), + #[error("Failed to validate asset log: {0}")] + ValidateLogError(ValidateLogError), +} diff --git a/crates/bevy_asset/src/processor/process.rs b/crates/bevy_asset/src/processor/process.rs new file mode 100644 index 0000000000000..d0e9964c05c63 --- /dev/null +++ b/crates/bevy_asset/src/processor/process.rs @@ -0,0 +1,260 @@ +use crate::{ + io::{AssetReaderError, AssetWriterError, Writer}, + meta::{AssetAction, AssetMeta, AssetMetaDyn, ProcessDependencyInfo, ProcessedInfo, Settings}, + processor::AssetProcessor, + saver::{AssetSaver, SavedAsset}, + AssetLoadError, AssetLoader, AssetPath, DeserializeMetaError, ErasedLoadedAsset, + MissingAssetLoaderForExtensionError, MissingAssetLoaderForTypeNameError, +}; +use bevy_utils::BoxedFuture; +use serde::{Deserialize, Serialize}; +use std::{marker::PhantomData, path::PathBuf}; +use thiserror::Error; + +/// Asset "processor" logic that reads input asset bytes (stored on [`ProcessContext`]), processes the value in some way, +/// and then writes the final processed bytes with [`Writer`]. The resulting bytes must be loadable with the given [`Process::OutputLoader`]. +/// +/// This is a "low level", maximally flexible interface. Most use cases are better served by the [`LoadAndSave`] implementation +/// of [`Process`]. +pub trait Process: Send + Sync + Sized + 'static { + /// The configuration / settings used to process the asset. This will be stored in the [`AssetMeta`] and is user-configurable per-asset. + type Settings: Settings + Default + Serialize + for<'a> Deserialize<'a>; + /// The [`AssetLoader`] that will be used to load the final processed asset. + type OutputLoader: AssetLoader; + /// Processes the asset stored on `context` in some way using the settings stored on `meta`. The results are written to `writer`. The + /// final written processed asset is loadable using [`Process::OutputLoader`]. This load will use the returned [`AssetLoader::Settings`]. + fn process<'a>( + &'a self, + context: &'a mut ProcessContext, + meta: AssetMeta<(), Self>, + writer: &'a mut Writer, + ) -> BoxedFuture<'a, Result<::Settings, ProcessError>>; +} + +/// A flexible [`Process`] implementation that loads the source [`Asset`] using the `L` [`AssetLoader`], then +/// saves that `L` asset using the `S` [`AssetSaver`]. +/// +/// When creating custom processors, it is generally recommended to use the [`LoadAndSave`] [`Process`] implementation, +/// as it encourages you to write both an [`AssetLoader`] capable of loading assets without processing enabled _and_ +/// an [`AssetSaver`] that allows you to efficiently process that asset type when that is desirable by users. However you can +/// also implement [`Process`] directly if [`LoadAndSave`] feels limiting or unnecessary. +/// +/// This uses [`LoadAndSaveSettings`] to configure the processor. +/// +/// [`Asset`]: crate::Asset +pub struct LoadAndSave> { + saver: S, + marker: PhantomData L>, +} + +impl> From for LoadAndSave { + fn from(value: S) -> Self { + LoadAndSave { + saver: value, + marker: PhantomData, + } + } +} + +/// Settings for the [`LoadAndSave`] [`Process::Settings`] implementation. +/// +/// `LoaderSettings` corresponds to [`AssetLoader::Settings`] and `SaverSettings` corresponds to [`AssetSaver::Settings`]. +#[derive(Serialize, Deserialize, Default)] +pub struct LoadAndSaveSettings { + /// The [`AssetLoader::Settings`] for [`LoadAndSave`]. + pub loader_settings: LoaderSettings, + /// The [`AssetSaver::Settings`] for [`LoadAndSave`]. + pub saver_settings: SaverSettings, +} + +/// An error that is encountered during [`Process::process`]. +#[derive(Error, Debug)] +pub enum ProcessError { + #[error("The asset source file for '{0}' does not exist")] + MissingAssetSource(PathBuf), + #[error(transparent)] + AssetSourceIoError(std::io::Error), + #[error(transparent)] + MissingAssetLoaderForExtension(#[from] MissingAssetLoaderForExtensionError), + #[error(transparent)] + MissingAssetLoaderForTypeName(#[from] MissingAssetLoaderForTypeNameError), + #[error("The processor '{0}' does not exist")] + MissingProcessor(String), + #[error(transparent)] + AssetWriterError(#[from] AssetWriterError), + #[error("Failed to read asset metadata {0:?}")] + ReadAssetMetaError(AssetReaderError), + #[error(transparent)] + DeserializeMetaError(#[from] DeserializeMetaError), + #[error(transparent)] + AssetLoadError(#[from] AssetLoadError), + #[error("The wrong meta type was passed into a processor. This is probably an internal implementation error.")] + WrongMetaType, + #[error("Encountered an error while saving the asset: {0}")] + AssetSaveError(anyhow::Error), + #[error("Assets without extensions are not supported.")] + ExtensionRequired, +} + +impl> Process + for LoadAndSave +{ + type Settings = LoadAndSaveSettings; + type OutputLoader = Saver::OutputLoader; + + fn process<'a>( + &'a self, + context: &'a mut ProcessContext, + meta: AssetMeta<(), Self>, + writer: &'a mut Writer, + ) -> BoxedFuture<'a, Result<::Settings, ProcessError>> { + Box::pin(async move { + let AssetAction::Process { settings, .. } = meta.asset else { + return Err(ProcessError::WrongMetaType); + }; + let loader_meta = AssetMeta::::new(AssetAction::Load { + loader: std::any::type_name::().to_string(), + settings: settings.loader_settings, + }); + let loaded_asset = context.load_source_asset(loader_meta).await?; + let saved_asset = SavedAsset::::from_loaded(&loaded_asset).unwrap(); + let output_settings = self + .saver + .save(writer, saved_asset, &settings.saver_settings) + .await + .map_err(ProcessError::AssetSaveError)?; + Ok(output_settings) + }) + } +} + +/// A type-erased variant of [`Process`] that enables interacting with processor implementations without knowing +/// their type. +pub trait ErasedProcessor: Send + Sync { + /// Type-erased variant of [`Process::process`]. + fn process<'a>( + &'a self, + context: &'a mut ProcessContext, + meta: Box, + writer: &'a mut Writer, + ) -> BoxedFuture<'a, Result, ProcessError>>; + /// Deserialized `meta` as type-erased [`AssetMeta`], operating under the assumption that it matches the meta + /// for the underlying [`Process`] impl. + fn deserialize_meta(&self, meta: &[u8]) -> Result, DeserializeMetaError>; + /// Returns the default type-erased [`AssetMeta`] for the underlying [`Process`] impl. + fn default_meta(&self) -> Box; +} + +impl ErasedProcessor for P { + fn process<'a>( + &'a self, + context: &'a mut ProcessContext, + meta: Box, + writer: &'a mut Writer, + ) -> BoxedFuture<'a, Result, ProcessError>> { + Box::pin(async move { + let meta = meta + .downcast::>() + .map_err(|_e| ProcessError::WrongMetaType)?; + let loader_settings =

::process(self, context, *meta, writer).await?; + let output_meta: Box = + Box::new(AssetMeta::::new(AssetAction::Load { + loader: std::any::type_name::().to_string(), + settings: loader_settings, + })); + Ok(output_meta) + }) + } + + fn deserialize_meta(&self, meta: &[u8]) -> Result, DeserializeMetaError> { + let meta: AssetMeta<(), P> = ron::de::from_bytes(meta)?; + Ok(Box::new(meta)) + } + + fn default_meta(&self) -> Box { + Box::new(AssetMeta::<(), P>::new(AssetAction::Process { + processor: std::any::type_name::

().to_string(), + settings: P::Settings::default(), + })) + } +} + +/// Provides scoped data access to the [`AssetProcessor`]. +/// This must only expose processor data that is represented in the asset's hash. +pub struct ProcessContext<'a> { + /// The "new" processed info for the final processed asset. It is [`ProcessContext`]'s + /// job to populate `process_dependencies` with any asset dependencies used to process + /// this asset (ex: loading an asset value from the [`AssetServer`] of the [`AssetProcessor`]) + /// + /// DO NOT CHANGE ANY VALUES HERE OTHER THAN APPENDING TO `process_dependencies` + /// + /// Do not expose this publicly as it would be too easily to invalidate state. + /// + /// [`AssetServer`]: crate::server::AssetServer + pub(crate) new_processed_info: &'a mut ProcessedInfo, + /// This exists to expose access to asset values (via the [`AssetServer`]). + /// + /// ANY ASSET VALUE THAT IS ACCESSED SHOULD BE ADDED TO `new_processed_info.process_dependencies` + /// + /// Do not expose this publicly as it would be too easily to invalidate state by forgetting to update + /// `process_dependencies`. + /// + /// [`AssetServer`]: crate::server::AssetServer + processor: &'a AssetProcessor, + path: &'a AssetPath<'static>, + asset_bytes: &'a [u8], +} + +impl<'a> ProcessContext<'a> { + pub(crate) fn new( + processor: &'a AssetProcessor, + path: &'a AssetPath<'static>, + asset_bytes: &'a [u8], + new_processed_info: &'a mut ProcessedInfo, + ) -> Self { + Self { + processor, + path, + asset_bytes, + new_processed_info, + } + } + + /// Load the source asset using the `L` [`AssetLoader`] and the passed in `meta` config. + /// This will take the "load dependencies" (asset values used when loading with `L`]) and + /// register them as "process dependencies" because they are asset values required to process the + /// current asset. + pub async fn load_source_asset( + &mut self, + meta: AssetMeta, + ) -> Result { + let server = &self.processor.server; + let loader_name = std::any::type_name::(); + let loader = server.get_asset_loader_with_type_name(loader_name).await?; + let loaded_asset = server + .load_with_meta_loader_and_reader( + self.path, + Box::new(meta), + &*loader, + &mut self.asset_bytes, + false, + true, + ) + .await?; + for (path, full_hash) in loaded_asset.loader_dependencies.iter() { + self.new_processed_info + .process_dependencies + .push(ProcessDependencyInfo { + full_hash: *full_hash, + path: path.to_owned(), + }); + } + Ok(loaded_asset) + } + + /// The source bytes of the asset being processed. + #[inline] + pub fn asset_bytes(&self) -> &[u8] { + self.asset_bytes + } +} diff --git a/crates/bevy_asset/src/reflect.rs b/crates/bevy_asset/src/reflect.rs index 98c54c69a8e9f..6e00323826de1 100644 --- a/crates/bevy_asset/src/reflect.rs +++ b/crates/bevy_asset/src/reflect.rs @@ -1,9 +1,9 @@ use std::any::{Any, TypeId}; use bevy_ecs::world::{unsafe_world_cell::UnsafeWorldCell, World}; -use bevy_reflect::{FromReflect, FromType, Reflect, Uuid}; +use bevy_reflect::{FromReflect, FromType, Reflect}; -use crate::{Asset, Assets, Handle, HandleId, HandleUntyped}; +use crate::{Asset, Assets, Handle, UntypedAssetId, UntypedHandle}; /// Type data for the [`TypeRegistry`](bevy_reflect::TypeRegistry) used to operate on reflected [`Asset`]s. /// @@ -11,31 +11,25 @@ use crate::{Asset, Assets, Handle, HandleId, HandleUntyped}; /// [`add`](ReflectAsset::add) and [`remove`](ReflectAsset::remove), but can be used in situations where you don't know which asset type `T` you want /// until runtime. /// -/// [`ReflectAsset`] can be obtained via [`TypeRegistration::data`](bevy_reflect::TypeRegistration::data) if the asset was registered using [`register_asset_reflect`](crate::AddAsset::register_asset_reflect). +/// [`ReflectAsset`] can be obtained via [`TypeRegistration::data`](bevy_reflect::TypeRegistration::data) if the asset was registered using [`register_asset_reflect`](crate::AssetApp::register_asset_reflect). #[derive(Clone)] pub struct ReflectAsset { - type_uuid: Uuid, handle_type_id: TypeId, assets_resource_type_id: TypeId, - get: fn(&World, HandleUntyped) -> Option<&dyn Reflect>, + get: fn(&World, UntypedHandle) -> Option<&dyn Reflect>, // SAFETY: // - may only be called with an [`UnsafeWorldCell`] which can be used to access the corresponding `Assets` resource mutably // - may only be used to access **at most one** access at once - get_unchecked_mut: unsafe fn(UnsafeWorldCell<'_>, HandleUntyped) -> Option<&mut dyn Reflect>, - add: fn(&mut World, &dyn Reflect) -> HandleUntyped, - set: fn(&mut World, HandleUntyped, &dyn Reflect) -> HandleUntyped, + get_unchecked_mut: unsafe fn(UnsafeWorldCell<'_>, UntypedHandle) -> Option<&mut dyn Reflect>, + add: fn(&mut World, &dyn Reflect) -> UntypedHandle, + insert: fn(&mut World, UntypedHandle, &dyn Reflect), len: fn(&World) -> usize, - ids: for<'w> fn(&'w World) -> Box + 'w>, - remove: fn(&mut World, HandleUntyped) -> Option>, + ids: for<'w> fn(&'w World) -> Box + 'w>, + remove: fn(&mut World, UntypedHandle) -> Option>, } impl ReflectAsset { - /// The [`bevy_reflect::TypeUuid`] of the asset - pub fn type_uuid(&self) -> Uuid { - self.type_uuid - } - /// The [`TypeId`] of the [`Handle`] for this asset pub fn handle_type_id(&self) -> TypeId { self.handle_type_id @@ -47,7 +41,7 @@ impl ReflectAsset { } /// Equivalent of [`Assets::get`] - pub fn get<'w>(&self, world: &'w World, handle: HandleUntyped) -> Option<&'w dyn Reflect> { + pub fn get<'w>(&self, world: &'w World, handle: UntypedHandle) -> Option<&'w dyn Reflect> { (self.get)(world, handle) } @@ -55,7 +49,7 @@ impl ReflectAsset { pub fn get_mut<'w>( &self, world: &'w mut World, - handle: HandleUntyped, + handle: UntypedHandle, ) -> Option<&'w mut dyn Reflect> { // SAFETY: unique world access unsafe { (self.get_unchecked_mut)(world.as_unsafe_world_cell(), handle) } @@ -68,12 +62,12 @@ impl ReflectAsset { /// you can only have at most one alive at the same time. /// This means that this is *not allowed*: /// ```rust,no_run - /// # use bevy_asset::{ReflectAsset, HandleUntyped}; + /// # use bevy_asset::{ReflectAsset, UntypedHandle}; /// # use bevy_ecs::prelude::World; /// # let reflect_asset: ReflectAsset = unimplemented!(); /// # let mut world: World = unimplemented!(); - /// # let handle_1: HandleUntyped = unimplemented!(); - /// # let handle_2: HandleUntyped = unimplemented!(); + /// # let handle_1: UntypedHandle = unimplemented!(); + /// # let handle_2: UntypedHandle = unimplemented!(); /// let unsafe_world_cell = world.as_unsafe_world_cell(); /// let a = unsafe { reflect_asset.get_unchecked_mut(unsafe_world_cell, handle_1).unwrap() }; /// let b = unsafe { reflect_asset.get_unchecked_mut(unsafe_world_cell, handle_2).unwrap() }; @@ -91,28 +85,23 @@ impl ReflectAsset { pub unsafe fn get_unchecked_mut<'w>( &self, world: UnsafeWorldCell<'w>, - handle: HandleUntyped, + handle: UntypedHandle, ) -> Option<&'w mut dyn Reflect> { // SAFETY: requirements are deferred to the caller (self.get_unchecked_mut)(world, handle) } /// Equivalent of [`Assets::add`] - pub fn add(&self, world: &mut World, value: &dyn Reflect) -> HandleUntyped { + pub fn add(&self, world: &mut World, value: &dyn Reflect) -> UntypedHandle { (self.add)(world, value) } - /// Equivalent of [`Assets::set`] - pub fn set( - &self, - world: &mut World, - handle: HandleUntyped, - value: &dyn Reflect, - ) -> HandleUntyped { - (self.set)(world, handle, value) + /// Equivalent of [`Assets::insert`] + pub fn insert(&self, world: &mut World, handle: UntypedHandle, value: &dyn Reflect) { + (self.insert)(world, handle, value); } /// Equivalent of [`Assets::remove`] - pub fn remove(&self, world: &mut World, handle: HandleUntyped) -> Option> { + pub fn remove(&self, world: &mut World, handle: UntypedHandle) -> Option> { (self.remove)(world, handle) } @@ -128,7 +117,7 @@ impl ReflectAsset { } /// Equivalent of [`Assets::ids`] - pub fn ids<'w>(&self, world: &'w World) -> impl Iterator + 'w { + pub fn ids<'w>(&self, world: &'w World) -> impl Iterator + 'w { (self.ids)(world) } } @@ -136,32 +125,31 @@ impl ReflectAsset { impl FromType for ReflectAsset { fn from_type() -> Self { ReflectAsset { - type_uuid: A::TYPE_UUID, handle_type_id: TypeId::of::>(), assets_resource_type_id: TypeId::of::>(), get: |world, handle| { let assets = world.resource::>(); - let asset = assets.get(&handle.typed()); + let asset = assets.get(&handle.typed_debug_checked()); asset.map(|asset| asset as &dyn Reflect) }, get_unchecked_mut: |world, handle| { // SAFETY: `get_unchecked_mut` must be called with `UnsafeWorldCell` having access to `Assets`, // and must ensure to only have at most one reference to it live at all times. let assets = unsafe { world.get_resource_mut::>().unwrap().into_inner() }; - let asset = assets.get_mut(&handle.typed()); + let asset = assets.get_mut(&handle.typed_debug_checked()); asset.map(|asset| asset as &mut dyn Reflect) }, add: |world, value| { let mut assets = world.resource_mut::>(); let value: A = FromReflect::from_reflect(value) .expect("could not call `FromReflect::from_reflect` in `ReflectAsset::add`"); - assets.add(value).into() + assets.add(value).untyped() }, - set: |world, handle, value| { + insert: |world, handle, value| { let mut assets = world.resource_mut::>(); let value: A = FromReflect::from_reflect(value) .expect("could not call `FromReflect::from_reflect` in `ReflectAsset::set`"); - assets.set(handle, value).into() + assets.insert(handle.typed_debug_checked(), value); }, len: |world| { let assets = world.resource::>(); @@ -169,11 +157,11 @@ impl FromType for ReflectAsset { }, ids: |world| { let assets = world.resource::>(); - Box::new(assets.ids()) + Box::new(assets.ids().map(|i| i.untyped())) }, remove: |world, handle| { let mut assets = world.resource_mut::>(); - let value = assets.remove(handle); + let value = assets.remove(handle.typed_debug_checked()); value.map(|value| Box::new(value) as Box) }, } @@ -207,29 +195,24 @@ impl FromType for ReflectAsset { /// ``` #[derive(Clone)] pub struct ReflectHandle { - type_uuid: Uuid, asset_type_id: TypeId, - downcast_handle_untyped: fn(&dyn Any) -> Option, - typed: fn(HandleUntyped) -> Box, + downcast_handle_untyped: fn(&dyn Any) -> Option, + typed: fn(UntypedHandle) -> Box, } impl ReflectHandle { - /// The [`bevy_reflect::TypeUuid`] of the asset - pub fn type_uuid(&self) -> Uuid { - self.type_uuid - } /// The [`TypeId`] of the asset pub fn asset_type_id(&self) -> TypeId { self.asset_type_id } - /// A way to go from a [`Handle`] in a `dyn Any` to a [`HandleUntyped`] - pub fn downcast_handle_untyped(&self, handle: &dyn Any) -> Option { + /// A way to go from a [`Handle`] in a `dyn Any` to a [`UntypedHandle`] + pub fn downcast_handle_untyped(&self, handle: &dyn Any) -> Option { (self.downcast_handle_untyped)(handle) } - /// A way to go from a [`HandleUntyped`] to a [`Handle`] in a `Box`. - /// Equivalent of [`HandleUntyped::typed`]. - pub fn typed(&self, handle: HandleUntyped) -> Box { + /// A way to go from a [`UntypedHandle`] to a [`Handle`] in a `Box`. + /// Equivalent of [`UntypedHandle::typed`]. + pub fn typed(&self, handle: UntypedHandle) -> Box { (self.typed)(handle) } } @@ -237,14 +220,13 @@ impl ReflectHandle { impl FromType> for ReflectHandle { fn from_type() -> Self { ReflectHandle { - type_uuid: A::TYPE_UUID, asset_type_id: TypeId::of::(), downcast_handle_untyped: |handle: &dyn Any| { handle .downcast_ref::>() - .map(|handle| handle.clone_untyped()) + .map(|h| h.clone().untyped()) }, - typed: |handle: HandleUntyped| Box::new(handle.typed::()), + typed: |handle: UntypedHandle| Box::new(handle.typed_debug_checked::()), } } } @@ -253,14 +235,13 @@ impl FromType> for ReflectHandle { mod tests { use std::any::TypeId; + use crate as bevy_asset; + use crate::{Asset, AssetApp, AssetPlugin, ReflectAsset, UntypedHandle}; use bevy_app::App; use bevy_ecs::reflect::AppTypeRegistry; - use bevy_reflect::{Reflect, ReflectMut, TypeUuid}; - - use crate::{AddAsset, AssetPlugin, HandleUntyped, ReflectAsset}; + use bevy_reflect::{Reflect, ReflectMut}; - #[derive(Reflect, TypeUuid)] - #[uuid = "09191350-1238-4736-9a89-46f04bda6966"] + #[derive(Asset, Reflect)] struct AssetType { field: String, } @@ -269,7 +250,7 @@ mod tests { fn test_reflect_asset_operations() { let mut app = App::new(); app.add_plugins(AssetPlugin::default()) - .add_asset::() + .init_asset::() .register_asset_reflect::(); let reflect_asset = { @@ -304,7 +285,7 @@ mod tests { let ids: Vec<_> = reflect_asset.ids(&app.world).collect(); assert_eq!(ids.len(), 1); - let fetched_handle = HandleUntyped::weak(ids[0]); + let fetched_handle = UntypedHandle::Weak(ids[0]); let asset = reflect_asset .get(&app.world, fetched_handle.clone_weak()) .unwrap(); diff --git a/crates/bevy_asset/src/saver.rs b/crates/bevy_asset/src/saver.rs new file mode 100644 index 0000000000000..62cfcc0c59abe --- /dev/null +++ b/crates/bevy_asset/src/saver.rs @@ -0,0 +1,110 @@ +use crate::{io::Writer, meta::Settings, Asset, ErasedLoadedAsset}; +use crate::{AssetLoader, LabeledAsset}; +use bevy_utils::{BoxedFuture, HashMap}; +use serde::{Deserialize, Serialize}; +use std::ops::Deref; + +/// Saves an [`Asset`] of a given [`AssetSaver::Asset`] type. [`AssetSaver::OutputLoader`] will then be used to load the saved asset +/// in the final deployed application. The saver should produce asset bytes in a format that [`AssetSaver::OutputLoader`] can read. +pub trait AssetSaver: Send + Sync + 'static { + type Asset: Asset; + type Settings: Settings + Default + Serialize + for<'a> Deserialize<'a>; + type OutputLoader: AssetLoader; + + /// Saves the given runtime [`Asset`] by writing it to a byte format using `writer`. The passed in `settings` can influence how the + /// `asset` is saved. + fn save<'a>( + &'a self, + writer: &'a mut Writer, + asset: SavedAsset<'a, Self::Asset>, + settings: &'a Self::Settings, + ) -> BoxedFuture<'a, Result<::Settings, anyhow::Error>>; +} + +/// A type-erased dynamic variant of [`AssetSaver`] that allows callers to save assets without knowing the actual type of the [`AssetSaver`]. +pub trait ErasedAssetSaver: Send + Sync + 'static { + /// Saves the given runtime [`ErasedLoadedAsset`] by writing it to a byte format using `writer`. The passed in `settings` can influence how the + /// `asset` is saved. + fn save<'a>( + &'a self, + writer: &'a mut Writer, + asset: &'a ErasedLoadedAsset, + settings: &'a dyn Settings, + ) -> BoxedFuture<'a, Result<(), anyhow::Error>>; + + /// The type name of the [`AssetSaver`]. + fn type_name(&self) -> &'static str; +} + +impl ErasedAssetSaver for S { + fn save<'a>( + &'a self, + writer: &'a mut Writer, + asset: &'a ErasedLoadedAsset, + settings: &'a dyn Settings, + ) -> BoxedFuture<'a, Result<(), anyhow::Error>> { + Box::pin(async move { + let settings = settings + .downcast_ref::() + .expect("AssetLoader settings should match the loader type"); + let saved_asset = SavedAsset::::from_loaded(asset).unwrap(); + self.save(writer, saved_asset, settings).await?; + Ok(()) + }) + } + fn type_name(&self) -> &'static str { + std::any::type_name::() + } +} + +/// An [`Asset`] (and any labeled "sub assets") intended to be saved. +pub struct SavedAsset<'a, A: Asset> { + value: &'a A, + labeled_assets: &'a HashMap, +} + +impl<'a, A: Asset> Deref for SavedAsset<'a, A> { + type Target = A; + + fn deref(&self) -> &Self::Target { + self.value + } +} + +impl<'a, A: Asset> SavedAsset<'a, A> { + /// Creates a new [`SavedAsset`] from `asset` if its internal value matches `A`. + pub fn from_loaded(asset: &'a ErasedLoadedAsset) -> Option { + let value = asset.value.downcast_ref::()?; + Some(SavedAsset { + value, + labeled_assets: &asset.labeled_assets, + }) + } + + /// Retrieves the value of this asset. + #[inline] + pub fn get(&self) -> &'a A { + self.value + } + + /// Returns the labeled asset, if it exists and matches this type. + pub fn get_labeled(&self, label: &str) -> Option> { + let labeled = self.labeled_assets.get(label)?; + let value = labeled.asset.value.downcast_ref::()?; + Some(SavedAsset { + value, + labeled_assets: &labeled.asset.labeled_assets, + }) + } + + /// Returns the type-erased labeled asset, if it exists and matches this type. + pub fn get_erased_labeled(&self, label: &str) -> Option<&ErasedLoadedAsset> { + let labeled = self.labeled_assets.get(label)?; + Some(&labeled.asset) + } + + /// Iterate over all labels for "labeled assets" in the loaded asset + pub fn iter_labels(&self) -> impl Iterator { + self.labeled_assets.keys().map(|s| s.as_str()) + } +} diff --git a/crates/bevy_asset/src/server/info.rs b/crates/bevy_asset/src/server/info.rs new file mode 100644 index 0000000000000..83147c63a0cbe --- /dev/null +++ b/crates/bevy_asset/src/server/info.rs @@ -0,0 +1,603 @@ +use crate::{ + meta::{AssetHash, MetaTransform}, + Asset, AssetHandleProvider, AssetPath, DependencyLoadState, ErasedLoadedAsset, Handle, + InternalAssetEvent, LoadState, RecursiveDependencyLoadState, StrongHandle, UntypedAssetId, + UntypedHandle, +}; +use bevy_ecs::world::World; +use bevy_log::warn; +use bevy_utils::{Entry, HashMap, HashSet}; +use crossbeam_channel::Sender; +use std::{ + any::TypeId, + sync::{Arc, Weak}, +}; +use thiserror::Error; + +#[derive(Debug)] +pub(crate) struct AssetInfo { + weak_handle: Weak, + pub(crate) path: Option>, + pub(crate) load_state: LoadState, + pub(crate) dep_load_state: DependencyLoadState, + pub(crate) rec_dep_load_state: RecursiveDependencyLoadState, + loading_dependencies: HashSet, + failed_dependencies: HashSet, + loading_rec_dependencies: HashSet, + failed_rec_dependencies: HashSet, + dependants_waiting_on_load: HashSet, + dependants_waiting_on_recursive_dep_load: HashSet, + /// The asset paths required to load this asset. Hashes will only be set for processed assets. + /// This is set using the value from [`LoadedAsset`]. + /// This will only be populated if [`AssetInfos::watching_for_changes`] is set to `true` to + /// save memory. + /// + /// [`LoadedAsset`]: crate::loader::LoadedAsset + loader_dependencies: HashMap, AssetHash>, + /// The number of handle drops to skip for this asset. + /// See usage (and comments) in get_or_create_path_handle for context. + handle_drops_to_skip: usize, +} + +impl AssetInfo { + fn new(weak_handle: Weak, path: Option>) -> Self { + Self { + weak_handle, + path, + load_state: LoadState::NotLoaded, + dep_load_state: DependencyLoadState::NotLoaded, + rec_dep_load_state: RecursiveDependencyLoadState::NotLoaded, + loading_dependencies: HashSet::default(), + failed_dependencies: HashSet::default(), + loading_rec_dependencies: HashSet::default(), + failed_rec_dependencies: HashSet::default(), + loader_dependencies: HashMap::default(), + dependants_waiting_on_load: HashSet::default(), + dependants_waiting_on_recursive_dep_load: HashSet::default(), + handle_drops_to_skip: 0, + } + } +} + +#[derive(Default)] +pub(crate) struct AssetInfos { + path_to_id: HashMap, UntypedAssetId>, + infos: HashMap, + /// If set to `true`, this informs [`AssetInfos`] to track data relevant to watching for changes (such as `load_dependants`) + /// This should only be set at startup. + pub(crate) watching_for_changes: bool, + /// Tracks assets that depend on the "key" asset path inside their asset loaders ("loader dependencies") + /// This should only be set when watching for changes to avoid unnecessary work. + pub(crate) loader_dependants: HashMap, HashSet>>, + pub(crate) handle_providers: HashMap, + pub(crate) dependency_loaded_event_sender: HashMap, +} + +impl std::fmt::Debug for AssetInfos { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("AssetInfos") + .field("path_to_id", &self.path_to_id) + .field("infos", &self.infos) + .finish() + } +} + +impl AssetInfos { + pub(crate) fn create_loading_handle(&mut self) -> Handle { + unwrap_with_context( + Self::create_handle_internal( + &mut self.infos, + &self.handle_providers, + TypeId::of::(), + None, + None, + true, + ), + std::any::type_name::(), + ) + .typed_debug_checked() + } + + pub(crate) fn create_loading_handle_untyped( + &mut self, + type_id: TypeId, + type_name: &'static str, + ) -> UntypedHandle { + unwrap_with_context( + Self::create_handle_internal( + &mut self.infos, + &self.handle_providers, + type_id, + None, + None, + true, + ), + type_name, + ) + } + + fn create_handle_internal( + infos: &mut HashMap, + handle_providers: &HashMap, + type_id: TypeId, + path: Option>, + meta_transform: Option, + loading: bool, + ) -> Result { + let provider = handle_providers + .get(&type_id) + .ok_or(MissingHandleProviderError(type_id))?; + + let handle = provider.reserve_handle_internal(true, path.clone(), meta_transform); + let mut info = AssetInfo::new(Arc::downgrade(&handle), path); + if loading { + info.load_state = LoadState::Loading; + info.dep_load_state = DependencyLoadState::Loading; + info.rec_dep_load_state = RecursiveDependencyLoadState::Loading; + } + infos.insert(handle.id, info); + Ok(UntypedHandle::Strong(handle)) + } + + pub(crate) fn get_or_create_path_handle( + &mut self, + path: AssetPath<'static>, + loading_mode: HandleLoadingMode, + meta_transform: Option, + ) -> (Handle, bool) { + let result = self.get_or_create_path_handle_internal( + path, + TypeId::of::(), + loading_mode, + meta_transform, + ); + let (handle, should_load) = unwrap_with_context(result, std::any::type_name::()); + (handle.typed_unchecked(), should_load) + } + + pub(crate) fn get_or_create_path_handle_untyped( + &mut self, + path: AssetPath<'static>, + type_id: TypeId, + type_name: &'static str, + loading_mode: HandleLoadingMode, + meta_transform: Option, + ) -> (UntypedHandle, bool) { + let result = + self.get_or_create_path_handle_internal(path, type_id, loading_mode, meta_transform); + unwrap_with_context(result, type_name) + } + + /// Retrieves asset tracking data, or creates it if it doesn't exist. + /// Returns true if an asset load should be kicked off + pub fn get_or_create_path_handle_internal( + &mut self, + path: AssetPath<'static>, + type_id: TypeId, + loading_mode: HandleLoadingMode, + meta_transform: Option, + ) -> Result<(UntypedHandle, bool), MissingHandleProviderError> { + match self.path_to_id.entry(path.clone()) { + Entry::Occupied(entry) => { + let id = *entry.get(); + // if there is a path_to_id entry, info always exists + let info = self.infos.get_mut(&id).unwrap(); + let mut should_load = false; + if loading_mode == HandleLoadingMode::Force + || (loading_mode == HandleLoadingMode::Request + && info.load_state == LoadState::NotLoaded) + { + info.load_state = LoadState::Loading; + info.dep_load_state = DependencyLoadState::Loading; + info.rec_dep_load_state = RecursiveDependencyLoadState::Loading; + should_load = true; + } + + if let Some(strong_handle) = info.weak_handle.upgrade() { + // If we can upgrade the handle, there is at least one live handle right now, + // The asset load has already kicked off (and maybe completed), so we can just + // return a strong handle + Ok((UntypedHandle::Strong(strong_handle), should_load)) + } else { + // Asset meta exists, but all live handles were dropped. This means the `track_assets` system + // hasn't been run yet to remove the current asset + // (note that this is guaranteed to be transactional with the `track_assets` system because + // because it locks the AssetInfos collection) + + // We must create a new strong handle for the existing id and ensure that the drop of the old + // strong handle doesn't remove the asset from the Assets collection + info.handle_drops_to_skip += 1; + let provider = self + .handle_providers + .get(&type_id) + .ok_or(MissingHandleProviderError(type_id))?; + let handle = + provider.get_handle(id.internal(), true, Some(path), meta_transform); + info.weak_handle = Arc::downgrade(&handle); + Ok((UntypedHandle::Strong(handle), should_load)) + } + } + // The entry does not exist, so this is a "fresh" asset load. We must create a new handle + Entry::Vacant(entry) => { + let should_load = match loading_mode { + HandleLoadingMode::NotLoading => false, + HandleLoadingMode::Request | HandleLoadingMode::Force => true, + }; + let handle = Self::create_handle_internal( + &mut self.infos, + &self.handle_providers, + type_id, + Some(path), + meta_transform, + should_load, + )?; + entry.insert(handle.id()); + Ok((handle, should_load)) + } + } + } + + pub(crate) fn get(&self, id: UntypedAssetId) -> Option<&AssetInfo> { + self.infos.get(&id) + } + + pub(crate) fn get_mut(&mut self, id: UntypedAssetId) -> Option<&mut AssetInfo> { + self.infos.get_mut(&id) + } + + pub(crate) fn get_path_handle(&self, path: AssetPath) -> Option { + let id = *self.path_to_id.get(&path)?; + self.get_id_handle(id) + } + + pub(crate) fn get_id_handle(&self, id: UntypedAssetId) -> Option { + let info = self.infos.get(&id)?; + let strong_handle = info.weak_handle.upgrade()?; + Some(UntypedHandle::Strong(strong_handle)) + } + + /// Returns `true` if this path has + pub(crate) fn is_path_alive(&self, path: &AssetPath) -> bool { + if let Some(id) = self.path_to_id.get(path) { + if let Some(info) = self.infos.get(id) { + return info.weak_handle.strong_count() > 0; + } + } + false + } + + // Returns `true` if the asset should be removed from the collection + pub(crate) fn process_handle_drop(&mut self, id: UntypedAssetId) -> bool { + Self::process_handle_drop_internal( + &mut self.infos, + &mut self.path_to_id, + &mut self.loader_dependants, + self.watching_for_changes, + id, + ) + } + + /// Updates [`AssetInfo`] / load state for an asset that has finished loading (and relevant dependencies / dependants). + pub(crate) fn process_asset_load( + &mut self, + loaded_asset_id: UntypedAssetId, + loaded_asset: ErasedLoadedAsset, + world: &mut World, + sender: &Sender, + ) { + loaded_asset.value.insert(loaded_asset_id, world); + let mut loading_deps = loaded_asset.dependencies; + let mut failed_deps = HashSet::new(); + let mut loading_rec_deps = loading_deps.clone(); + let mut failed_rec_deps = HashSet::new(); + loading_deps.retain(|dep_id| { + if let Some(dep_info) = self.get_mut(*dep_id) { + match dep_info.rec_dep_load_state { + RecursiveDependencyLoadState::Loading + | RecursiveDependencyLoadState::NotLoaded => { + // If dependency is loading, wait for it. + dep_info + .dependants_waiting_on_recursive_dep_load + .insert(loaded_asset_id); + } + RecursiveDependencyLoadState::Loaded => { + // If dependency is loaded, reduce our count by one + loading_rec_deps.remove(dep_id); + } + RecursiveDependencyLoadState::Failed => { + failed_rec_deps.insert(*dep_id); + loading_rec_deps.remove(dep_id); + } + } + match dep_info.load_state { + LoadState::NotLoaded | LoadState::Loading => { + // If dependency is loading, wait for it. + dep_info.dependants_waiting_on_load.insert(loaded_asset_id); + true + } + LoadState::Loaded => { + // If dependency is loaded, reduce our count by one + false + } + LoadState::Failed => { + failed_deps.insert(*dep_id); + false + } + } + } else { + // the dependency id does not exist, which implies it was manually removed or never existed in the first place + warn!( + "Dependency {:?} from asset {:?} is unknown. This asset's dependency load status will not switch to 'Loaded' until the unknown dependency is loaded.", + dep_id, loaded_asset_id + ); + true + } + }); + + let dep_load_state = match (loading_deps.len(), failed_deps.len()) { + (0, 0) => DependencyLoadState::Loaded, + (_loading, 0) => DependencyLoadState::Loading, + (_loading, _failed) => DependencyLoadState::Failed, + }; + + let rec_dep_load_state = match (loading_rec_deps.len(), failed_rec_deps.len()) { + (0, 0) => { + sender + .send(InternalAssetEvent::LoadedWithDependencies { + id: loaded_asset_id, + }) + .unwrap(); + RecursiveDependencyLoadState::Loaded + } + (_loading, 0) => RecursiveDependencyLoadState::Loading, + (_loading, _failed) => RecursiveDependencyLoadState::Failed, + }; + + let (dependants_waiting_on_load, dependants_waiting_on_rec_load) = { + let watching_for_changes = self.watching_for_changes; + // if watching for changes, track reverse loader dependencies for hot reloading + if watching_for_changes { + let info = self + .infos + .get(&loaded_asset_id) + .expect("Asset info should always exist at this point"); + if let Some(asset_path) = &info.path { + for loader_dependency in loaded_asset.loader_dependencies.keys() { + let dependants = self + .loader_dependants + .entry(loader_dependency.clone()) + .or_default(); + dependants.insert(asset_path.clone()); + } + } + } + let info = self + .get_mut(loaded_asset_id) + .expect("Asset info should always exist at this point"); + info.loading_dependencies = loading_deps; + info.failed_dependencies = failed_deps; + info.loading_rec_dependencies = loading_rec_deps; + info.failed_rec_dependencies = failed_rec_deps; + info.load_state = LoadState::Loaded; + info.dep_load_state = dep_load_state; + info.rec_dep_load_state = rec_dep_load_state; + if watching_for_changes { + info.loader_dependencies = loaded_asset.loader_dependencies; + } + + let dependants_waiting_on_rec_load = if matches!( + rec_dep_load_state, + RecursiveDependencyLoadState::Loaded | RecursiveDependencyLoadState::Failed + ) { + Some(std::mem::take( + &mut info.dependants_waiting_on_recursive_dep_load, + )) + } else { + None + }; + + ( + std::mem::take(&mut info.dependants_waiting_on_load), + dependants_waiting_on_rec_load, + ) + }; + + for id in dependants_waiting_on_load { + if let Some(info) = self.get_mut(id) { + info.loading_dependencies.remove(&loaded_asset_id); + if info.loading_dependencies.is_empty() { + // send dependencies loaded event + info.dep_load_state = DependencyLoadState::Loaded; + } + } + } + + if let Some(dependants_waiting_on_rec_load) = dependants_waiting_on_rec_load { + match rec_dep_load_state { + RecursiveDependencyLoadState::Loaded => { + for dep_id in dependants_waiting_on_rec_load { + Self::propagate_loaded_state(self, loaded_asset_id, dep_id, sender); + } + } + RecursiveDependencyLoadState::Failed => { + for dep_id in dependants_waiting_on_rec_load { + Self::propagate_failed_state(self, loaded_asset_id, dep_id); + } + } + RecursiveDependencyLoadState::Loading | RecursiveDependencyLoadState::NotLoaded => { + // dependants_waiting_on_rec_load should be None in this case + unreachable!("`Loading` and `NotLoaded` state should never be propagated.") + } + } + } + } + + /// Recursively propagates loaded state up the dependency tree. + fn propagate_loaded_state( + infos: &mut AssetInfos, + loaded_id: UntypedAssetId, + waiting_id: UntypedAssetId, + sender: &Sender, + ) { + let dependants_waiting_on_rec_load = if let Some(info) = infos.get_mut(waiting_id) { + info.loading_rec_dependencies.remove(&loaded_id); + if info.loading_rec_dependencies.is_empty() && info.failed_rec_dependencies.is_empty() { + info.rec_dep_load_state = RecursiveDependencyLoadState::Loaded; + if info.load_state == LoadState::Loaded { + sender + .send(InternalAssetEvent::LoadedWithDependencies { id: waiting_id }) + .unwrap(); + } + Some(std::mem::take( + &mut info.dependants_waiting_on_recursive_dep_load, + )) + } else { + None + } + } else { + None + }; + + if let Some(dependants_waiting_on_rec_load) = dependants_waiting_on_rec_load { + for dep_id in dependants_waiting_on_rec_load { + Self::propagate_loaded_state(infos, waiting_id, dep_id, sender); + } + } + } + + /// Recursively propagates failed state up the dependency tree + fn propagate_failed_state( + infos: &mut AssetInfos, + failed_id: UntypedAssetId, + waiting_id: UntypedAssetId, + ) { + let dependants_waiting_on_rec_load = if let Some(info) = infos.get_mut(waiting_id) { + info.loading_rec_dependencies.remove(&failed_id); + info.failed_rec_dependencies.insert(failed_id); + info.rec_dep_load_state = RecursiveDependencyLoadState::Failed; + Some(std::mem::take( + &mut info.dependants_waiting_on_recursive_dep_load, + )) + } else { + None + }; + + if let Some(dependants_waiting_on_rec_load) = dependants_waiting_on_rec_load { + for dep_id in dependants_waiting_on_rec_load { + Self::propagate_failed_state(infos, waiting_id, dep_id); + } + } + } + + pub(crate) fn process_asset_fail(&mut self, failed_id: UntypedAssetId) { + let (dependants_waiting_on_load, dependants_waiting_on_rec_load) = { + let info = self + .get_mut(failed_id) + .expect("Asset info should always exist at this point"); + info.load_state = LoadState::Failed; + info.dep_load_state = DependencyLoadState::Failed; + info.rec_dep_load_state = RecursiveDependencyLoadState::Failed; + ( + std::mem::take(&mut info.dependants_waiting_on_load), + std::mem::take(&mut info.dependants_waiting_on_recursive_dep_load), + ) + }; + + for waiting_id in dependants_waiting_on_load { + if let Some(info) = self.get_mut(waiting_id) { + info.loading_dependencies.remove(&failed_id); + info.failed_dependencies.insert(failed_id); + info.dep_load_state = DependencyLoadState::Failed; + } + } + + for waiting_id in dependants_waiting_on_rec_load { + Self::propagate_failed_state(self, failed_id, waiting_id); + } + } + + fn process_handle_drop_internal( + infos: &mut HashMap, + path_to_id: &mut HashMap, UntypedAssetId>, + loader_dependants: &mut HashMap, HashSet>>, + watching_for_changes: bool, + id: UntypedAssetId, + ) -> bool { + match infos.entry(id) { + Entry::Occupied(mut entry) => { + if entry.get_mut().handle_drops_to_skip > 0 { + entry.get_mut().handle_drops_to_skip -= 1; + false + } else { + let info = entry.remove(); + if let Some(path) = info.path { + if watching_for_changes { + for loader_dependency in info.loader_dependencies.keys() { + if let Some(dependants) = + loader_dependants.get_mut(loader_dependency) + { + dependants.remove(&path); + } + } + } + path_to_id.remove(&path); + } + true + } + } + // Either the asset was already dropped, it doesn't exist, or it isn't managed by the asset server + // None of these cases should result in a removal from the Assets collection + Entry::Vacant(_) => false, + } + } + + /// Consumes all current handle drop events. This will update information in [`AssetInfos`], but it + /// will not affect [`Assets`] storages. For normal use cases, prefer `Assets::track_assets()` + /// This should only be called if `Assets` storage isn't being used (such as in [`AssetProcessor`](crate::processor::AssetProcessor)) + /// + /// [`Assets`]: crate::Assets + pub(crate) fn consume_handle_drop_events(&mut self) { + for provider in self.handle_providers.values() { + while let Ok(drop_event) = provider.drop_receiver.try_recv() { + let id = drop_event.id; + if drop_event.asset_server_managed { + Self::process_handle_drop_internal( + &mut self.infos, + &mut self.path_to_id, + &mut self.loader_dependants, + self.watching_for_changes, + id.untyped(provider.type_id), + ); + } + } + } + } +} + +/// Determines how a handle should be initialized +#[derive(Copy, Clone, PartialEq, Eq)] +pub(crate) enum HandleLoadingMode { + /// The handle is for an asset that isn't loading/loaded yet. + NotLoading, + /// The handle is for an asset that is being _requested_ to load (if it isn't already loading) + Request, + /// The handle is for an asset that is being forced to load (even if it has already loaded) + Force, +} + +#[derive(Error, Debug)] +#[error("Cannot allocate a handle because no handle provider exists for asset type {0:?}")] +pub struct MissingHandleProviderError(TypeId); + +fn unwrap_with_context( + result: Result, + type_name: &'static str, +) -> T { + match result { + Ok(value) => value, + Err(_) => { + panic!("Cannot allocate an Asset Handle of type '{type_name}' because the asset type has not been initialized. \ + Make sure you have called app.init_asset::<{type_name}>()") + } + } +} diff --git a/crates/bevy_asset/src/server/mod.rs b/crates/bevy_asset/src/server/mod.rs new file mode 100644 index 0000000000000..6feda15955e6a --- /dev/null +++ b/crates/bevy_asset/src/server/mod.rs @@ -0,0 +1,913 @@ +mod info; + +use crate::{ + folder::LoadedFolder, + io::{AssetReader, AssetReaderError, AssetSourceEvent, AssetWatcher, Reader}, + loader::{AssetLoader, AssetLoaderError, ErasedAssetLoader, LoadContext, LoadedAsset}, + meta::{ + loader_settings_meta_transform, AssetActionMinimal, AssetMetaDyn, AssetMetaMinimal, + MetaTransform, Settings, + }, + path::AssetPath, + Asset, AssetEvent, AssetHandleProvider, AssetId, Assets, DeserializeMetaError, + ErasedLoadedAsset, Handle, UntypedAssetId, UntypedHandle, +}; +use bevy_ecs::prelude::*; +use bevy_log::{error, info, warn}; +use bevy_tasks::IoTaskPool; +use bevy_utils::{HashMap, HashSet}; +use crossbeam_channel::{Receiver, Sender}; +use futures_lite::StreamExt; +use info::*; +use parking_lot::RwLock; +use std::{any::TypeId, path::Path, sync::Arc}; +use thiserror::Error; + +/// Loads and tracks the state of [`Asset`] values from a configured [`AssetReader`]. This can be used to kick off new asset loads and +/// retrieve their current load states. +/// +/// The general process to load an asset is: +/// 1. Initialize a new [`Asset`] type with the [`AssetServer`] via [`AssetApp::init_asset`], which will internally call [`AssetServer::register_asset`] +/// and set up related ECS [`Assets`] storage and systems. +/// 2. Register one or more [`AssetLoader`]s for that asset with [`AssetApp::init_asset_loader`] +/// 3. Add the asset to your asset folder (defaults to `assets`). +/// 4. Call [`AssetServer::load`] with a path to your asset. +/// +/// [`AssetServer`] can be cloned. It is backed by an [`Arc`] so clones will share state. Clones can be freely used in parallel. +/// +/// [`AssetApp::init_asset`]: crate::AssetApp::init_asset +/// [`AssetApp::init_asset_loader`]: crate::AssetApp::init_asset_loader +#[derive(Resource, Clone)] +pub struct AssetServer { + pub(crate) data: Arc, +} + +/// Internal data used by [`AssetServer`]. This is intended to be used from within an [`Arc`]. +pub(crate) struct AssetServerData { + pub(crate) infos: RwLock, + pub(crate) loaders: Arc>, + asset_event_sender: Sender, + asset_event_receiver: Receiver, + source_event_receiver: Receiver, + reader: Box, + _watcher: Option>, +} + +impl AssetServer { + /// Create a new instance of [`AssetServer`]. If `watch_for_changes` is true, the [`AssetReader`] storage will watch for changes to + /// asset sources and hot-reload them. + pub fn new(reader: Box, watch_for_changes: bool) -> Self { + Self::new_with_loaders(reader, Default::default(), watch_for_changes) + } + + pub(crate) fn new_with_loaders( + reader: Box, + loaders: Arc>, + watch_for_changes: bool, + ) -> Self { + let (asset_event_sender, asset_event_receiver) = crossbeam_channel::unbounded(); + let (source_event_sender, source_event_receiver) = crossbeam_channel::unbounded(); + let mut infos = AssetInfos::default(); + let watcher = if watch_for_changes { + infos.watching_for_changes = true; + let watcher = reader.watch_for_changes(source_event_sender); + if watcher.is_none() { + error!("{}", CANNOT_WATCH_ERROR_MESSAGE); + } + watcher + } else { + None + }; + Self { + data: Arc::new(AssetServerData { + reader, + _watcher: watcher, + asset_event_sender, + asset_event_receiver, + source_event_receiver, + loaders, + infos: RwLock::new(infos), + }), + } + } + + /// Returns the primary [`AssetReader`]. + pub fn reader(&self) -> &dyn AssetReader { + &*self.data.reader + } + + /// Registers a new [`AssetLoader`]. [`AssetLoader`]s must be registered before they can be used. + pub fn register_loader(&self, loader: L) { + let mut loaders = self.data.loaders.write(); + let type_name = std::any::type_name::(); + let loader = Arc::new(loader); + let (loader_index, is_new) = + if let Some(index) = loaders.preregistered_loaders.remove(type_name) { + (index, false) + } else { + (loaders.values.len(), true) + }; + for extension in loader.extensions() { + loaders + .extension_to_index + .insert(extension.to_string(), loader_index); + } + + if is_new { + loaders.type_name_to_index.insert(type_name, loader_index); + loaders.values.push(MaybeAssetLoader::Ready(loader)); + } else { + let maybe_loader = std::mem::replace( + &mut loaders.values[loader_index], + MaybeAssetLoader::Ready(loader.clone()), + ); + match maybe_loader { + MaybeAssetLoader::Ready(_) => unreachable!(), + MaybeAssetLoader::Pending { sender, .. } => { + IoTaskPool::get() + .spawn(async move { + let _ = sender.broadcast(loader).await; + }) + .detach(); + } + } + } + } + + /// Registers a new [`Asset`] type. [`Asset`] types must be registered before assets of that type can be loaded. + pub fn register_asset(&self, assets: &Assets) { + self.register_handle_provider(assets.get_handle_provider()); + fn sender(world: &mut World, id: UntypedAssetId) { + world + .resource_mut::>>() + .send(AssetEvent::LoadedWithDependencies { id: id.typed() }); + } + self.data + .infos + .write() + .dependency_loaded_event_sender + .insert(TypeId::of::(), sender::); + } + + pub(crate) fn register_handle_provider(&self, handle_provider: AssetHandleProvider) { + let mut infos = self.data.infos.write(); + infos + .handle_providers + .insert(handle_provider.type_id, handle_provider); + } + + /// Returns the registered [`AssetLoader`] associated with the given extension, if it exists. + pub async fn get_asset_loader_with_extension( + &self, + extension: &str, + ) -> Result, MissingAssetLoaderForExtensionError> { + let loader = { + let loaders = self.data.loaders.read(); + let index = *loaders.extension_to_index.get(extension).ok_or_else(|| { + MissingAssetLoaderForExtensionError { + extensions: vec![extension.to_string()], + } + })?; + loaders.values[index].clone() + }; + + match loader { + MaybeAssetLoader::Ready(loader) => Ok(loader), + MaybeAssetLoader::Pending { mut receiver, .. } => Ok(receiver.recv().await.unwrap()), + } + } + + /// Returns the registered [`AssetLoader`] associated with the given [`std::any::type_name`], if it exists. + pub async fn get_asset_loader_with_type_name( + &self, + type_name: &str, + ) -> Result, MissingAssetLoaderForTypeNameError> { + let loader = { + let loaders = self.data.loaders.read(); + let index = *loaders.type_name_to_index.get(type_name).ok_or_else(|| { + MissingAssetLoaderForTypeNameError { + type_name: type_name.to_string(), + } + })?; + + loaders.values[index].clone() + }; + match loader { + MaybeAssetLoader::Ready(loader) => Ok(loader), + MaybeAssetLoader::Pending { mut receiver, .. } => Ok(receiver.recv().await.unwrap()), + } + } + + /// Retrieves the default [`AssetLoader`] for the given path, if one can be found. + pub async fn get_path_asset_loader<'a>( + &self, + path: &AssetPath<'a>, + ) -> Result, MissingAssetLoaderForExtensionError> { + let full_extension = + path.get_full_extension() + .ok_or(MissingAssetLoaderForExtensionError { + extensions: Vec::new(), + })?; + if let Ok(loader) = self.get_asset_loader_with_extension(&full_extension).await { + return Ok(loader); + } + for extension in AssetPath::iter_secondary_extensions(&full_extension) { + if let Ok(loader) = self.get_asset_loader_with_extension(extension).await { + return Ok(loader); + } + } + let mut extensions = vec![full_extension.clone()]; + extensions + .extend(AssetPath::iter_secondary_extensions(&full_extension).map(|e| e.to_string())); + Err(MissingAssetLoaderForExtensionError { extensions }) + } + + /// Begins loading an [`Asset`] of type `A` stored at `path`. This will not block on the asset load. Instead, + /// it returns a "strong" [`Handle`]. When the [`Asset`] is loaded (and enters [`LoadState::Loaded`]), it will be added to the + /// associated [`Assets`] resource. + /// + /// You can check the asset's load state by reading [`AssetEvent`] events, calling [`AssetServer::load_state`], or checking + /// the [`Assets`] storage to see if the [`Asset`] exists yet. + /// + /// The asset load will fail and an error will be printed to the logs if the asset stored at `path` is not of type `A`. + #[must_use = "not using the returned strong handle may result in the unexpected release of the asset"] + pub fn load<'a, A: Asset>(&self, path: impl Into>) -> Handle { + self.load_with_meta_transform(path, None) + } + + /// Begins loading an [`Asset`] of type `A` stored at `path`. The given `settings` function will override the asset's + /// [`AssetLoader`] settings. The type `S` _must_ match the configured [`AssetLoader::Settings`] or `settings` changes + /// will be ignored and an error will be printed to the log. + #[must_use = "not using the returned strong handle may result in the unexpected release of the asset"] + pub fn load_with_settings<'a, A: Asset, S: Settings>( + &self, + path: impl Into>, + settings: impl Fn(&mut S) + Send + Sync + 'static, + ) -> Handle { + self.load_with_meta_transform(path, Some(loader_settings_meta_transform(settings))) + } + + fn load_with_meta_transform<'a, A: Asset>( + &self, + path: impl Into>, + meta_transform: Option, + ) -> Handle { + let path: AssetPath = path.into(); + let (handle, should_load) = self.data.infos.write().get_or_create_path_handle::( + path.to_owned(), + HandleLoadingMode::Request, + meta_transform, + ); + + if should_load { + let mut owned_handle = Some(handle.clone().untyped()); + let mut owned_path = path.to_owned(); + let server = self.clone(); + IoTaskPool::get() + .spawn(async move { + if owned_path.label().is_some() { + owned_path.remove_label(); + owned_handle = None; + } + if let Err(err) = server + .load_internal(owned_handle, owned_path, false, None) + .await + { + error!("{}", err); + } + }) + .detach(); + } + + handle + } + + #[must_use = "not using the returned strong handle may result in the unexpected release of the asset"] + pub(crate) async fn load_untyped_async<'a>( + &self, + path: impl Into>, + ) -> Result { + self.load_internal(None, path.into(), false, None).await + } + + async fn load_internal<'a>( + &self, + input_handle: Option, + mut path: AssetPath<'a>, + force: bool, + meta_transform: Option, + ) -> Result { + let owned_path = path.to_owned(); + let (mut meta, loader, mut reader) = self + .get_meta_loader_and_reader(&owned_path) + .await + .map_err(|e| { + // if there was an input handle, a "load" operation has already started, so we must produce a "failure" event, if + // we cannot find the meta and loader + if let Some(handle) = &input_handle { + self.send_asset_event(InternalAssetEvent::Failed { id: handle.id() }); + } + e + })?; + + let has_label = path.label().is_some(); + + let (handle, should_load) = match input_handle { + Some(handle) => { + if !has_label && handle.type_id() != loader.asset_type_id() { + return Err(AssetLoadError::RequestedHandleTypeMismatch { + path: path.to_owned(), + requested: handle.type_id(), + actual_asset_name: loader.asset_type_name(), + loader_name: loader.type_name(), + }); + } + // if a handle was passed in, the "should load" check was already done + (handle, true) + } + None => { + let mut infos = self.data.infos.write(); + infos.get_or_create_path_handle_untyped( + path.to_owned(), + loader.asset_type_id(), + loader.asset_type_name(), + HandleLoadingMode::Request, + meta_transform, + ) + } + }; + + if !should_load && !force { + return Ok(handle); + } + let base_asset_id = if has_label { + path.remove_label(); + // If the path has a label, the current id does not match the asset root type. + // We need to get the actual asset id + let mut infos = self.data.infos.write(); + let (actual_handle, _) = infos.get_or_create_path_handle_untyped( + path.to_owned(), + loader.asset_type_id(), + loader.asset_type_name(), + // ignore current load state ... we kicked off this sub asset load because it needed to be loaded but + // does not currently exist + HandleLoadingMode::Force, + None, + ); + actual_handle.id() + } else { + handle.id() + }; + + if let Some(meta_transform) = handle.meta_transform() { + (*meta_transform)(&mut *meta); + } + + match self + .load_with_meta_loader_and_reader(&path, meta, &*loader, &mut *reader, true, false) + .await + { + Ok(mut loaded_asset) => { + for (_, labeled_asset) in loaded_asset.labeled_assets.drain() { + self.send_asset_event(InternalAssetEvent::Loaded { + id: labeled_asset.handle.id(), + loaded_asset: labeled_asset.asset, + }); + } + self.send_asset_event(InternalAssetEvent::Loaded { + id: base_asset_id, + loaded_asset, + }); + Ok(handle) + } + Err(err) => { + self.send_asset_event(InternalAssetEvent::Failed { id: base_asset_id }); + Err(err) + } + } + } + + /// Kicks off a reload of the asset stored at the given path. This will only reload the asset if it currently loaded. + pub fn reload<'a>(&self, path: impl Into>) { + let server = self.clone(); + let path = path.into(); + let owned_path = path.to_owned(); + IoTaskPool::get() + .spawn(async move { + if server.data.infos.read().is_path_alive(&owned_path) { + info!("Reloading {owned_path} because it has changed"); + if let Err(err) = server.load_internal(None, owned_path, true, None).await { + error!("{}", err); + } + } + }) + .detach(); + } + + /// Queues a new asset to be tracked by the [`AssetServer`] and returns a [`Handle`] to it. This can be used to track + /// dependencies of assets created at runtime. + /// + /// After the asset has been fully loaded by the [`AssetServer`], it will show up in the relevant [`Assets`] storage. + #[must_use = "not using the returned strong handle may result in the unexpected release of the asset"] + pub fn add(&self, asset: A) -> Handle { + self.load_asset(LoadedAsset::new_with_dependencies(asset, None)) + } + + pub(crate) fn load_asset(&self, asset: impl Into>) -> Handle { + let loaded_asset: LoadedAsset = asset.into(); + let erased_loaded_asset: ErasedLoadedAsset = loaded_asset.into(); + self.load_asset_untyped(None, erased_loaded_asset) + .typed_debug_checked() + } + + #[must_use = "not using the returned strong handle may result in the unexpected release of the asset"] + pub(crate) fn load_asset_untyped( + &self, + path: Option<&AssetPath<'static>>, + asset: impl Into, + ) -> UntypedHandle { + let loaded_asset = asset.into(); + let handle = if let Some(path) = path { + let (handle, _) = self.data.infos.write().get_or_create_path_handle_untyped( + path.clone(), + loaded_asset.asset_type_id(), + loaded_asset.asset_type_name(), + HandleLoadingMode::NotLoading, + None, + ); + handle + } else { + self.data.infos.write().create_loading_handle_untyped( + loaded_asset.asset_type_id(), + loaded_asset.asset_type_name(), + ) + }; + self.send_asset_event(InternalAssetEvent::Loaded { + id: handle.id(), + loaded_asset, + }); + handle + } + + /// Loads all assets from the specified folder recursively. The [`LoadedFolder`] asset (when it loads) will + /// contain handles to all assets in the folder. You can wait for all assets to load by checking the [`LoadedFolder`]'s + /// [`RecursiveDependencyLoadState`]. + #[must_use = "not using the returned strong handle may result in the unexpected release of the assets"] + pub fn load_folder(&self, path: impl AsRef) -> Handle { + let handle = { + let mut infos = self.data.infos.write(); + infos.create_loading_handle::() + }; + let id = handle.id().untyped(); + + fn load_folder<'a>( + path: &'a Path, + server: &'a AssetServer, + handles: &'a mut Vec, + ) -> bevy_utils::BoxedFuture<'a, Result<(), AssetLoadError>> { + Box::pin(async move { + let is_dir = server.reader().is_directory(path).await?; + if is_dir { + let mut path_stream = server.reader().read_directory(path.as_ref()).await?; + while let Some(child_path) = path_stream.next().await { + if server.reader().is_directory(&child_path).await? { + load_folder(&child_path, server, handles).await?; + } else { + let path = child_path.to_str().expect("Path should be a valid string."); + match server.load_untyped_async(path).await { + Ok(handle) => handles.push(handle), + // skip assets that cannot be loaded + Err( + AssetLoadError::MissingAssetLoaderForTypeName(_) + | AssetLoadError::MissingAssetLoaderForExtension(_), + ) => {} + Err(err) => return Err(err), + } + } + } + } + Ok(()) + }) + } + + let server = self.clone(); + let owned_path = path.as_ref().to_owned(); + IoTaskPool::get() + .spawn(async move { + let mut handles = Vec::new(); + match load_folder(&owned_path, &server, &mut handles).await { + Ok(_) => server.send_asset_event(InternalAssetEvent::Loaded { + id, + loaded_asset: LoadedAsset::new_with_dependencies( + LoadedFolder { handles }, + None, + ) + .into(), + }), + Err(_) => server.send_asset_event(InternalAssetEvent::Failed { id }), + } + }) + .detach(); + + handle + } + + fn send_asset_event(&self, event: InternalAssetEvent) { + self.data.asset_event_sender.send(event).unwrap(); + } + + /// Retrieves all loads states for the given asset id. + pub fn get_load_states( + &self, + id: impl Into, + ) -> Option<(LoadState, DependencyLoadState, RecursiveDependencyLoadState)> { + self.data + .infos + .read() + .get(id.into()) + .map(|i| (i.load_state, i.dep_load_state, i.rec_dep_load_state)) + } + + /// Retrieves the main [`LoadState`] of a given asset `id`. + pub fn get_load_state(&self, id: impl Into) -> Option { + self.data.infos.read().get(id.into()).map(|i| i.load_state) + } + + /// Retrieves the [`RecursiveDependencyLoadState`] of a given asset `id`. + pub fn get_recursive_dependency_load_state( + &self, + id: impl Into, + ) -> Option { + self.data + .infos + .read() + .get(id.into()) + .map(|i| i.rec_dep_load_state) + } + + /// Retrieves the main [`LoadState`] of a given asset `id`. + pub fn load_state(&self, id: impl Into) -> LoadState { + self.get_load_state(id).unwrap_or(LoadState::NotLoaded) + } + + /// Retrieves the [`RecursiveDependencyLoadState`] of a given asset `id`. + pub fn recursive_dependency_load_state( + &self, + id: impl Into, + ) -> RecursiveDependencyLoadState { + self.get_recursive_dependency_load_state(id) + .unwrap_or(RecursiveDependencyLoadState::NotLoaded) + } + + /// Returns an active handle for the given path, if the asset at the given path has already started loading, + /// or is still "alive". + pub fn get_handle<'a, A: Asset>(&self, path: impl Into>) -> Option> { + self.get_handle_untyped(path) + .map(|h| h.typed_debug_checked()) + } + + pub fn get_id_handle(&self, id: AssetId) -> Option> { + self.get_id_handle_untyped(id.untyped()).map(|h| h.typed()) + } + + pub fn get_id_handle_untyped(&self, id: UntypedAssetId) -> Option { + self.data.infos.read().get_id_handle(id) + } + + /// Returns an active untyped handle for the given path, if the asset at the given path has already started loading, + /// or is still "alive". + pub fn get_handle_untyped<'a>(&self, path: impl Into>) -> Option { + let infos = self.data.infos.read(); + let path = path.into(); + infos.get_path_handle(path) + } + + /// Returns the path for the given `id`, if it has one. + pub fn get_path(&self, id: impl Into) -> Option> { + let infos = self.data.infos.read(); + let info = infos.get(id.into())?; + Some(info.path.as_ref()?.to_owned()) + } + + /// Pre-register a loader that will later be added. + /// + /// Assets loaded with matching extensions will be blocked until the + /// real loader is added. + pub fn preregister_loader(&self, extensions: &[&str]) { + let mut loaders = self.data.loaders.write(); + let loader_index = loaders.values.len(); + let type_name = std::any::type_name::(); + loaders + .preregistered_loaders + .insert(type_name, loader_index); + loaders.type_name_to_index.insert(type_name, loader_index); + for extension in extensions { + if loaders + .extension_to_index + .insert(extension.to_string(), loader_index) + .is_some() + { + warn!("duplicate preregistration for `{extension}`, any assets loaded with the previous loader will never complete."); + } + } + let (mut sender, receiver) = async_broadcast::broadcast(1); + sender.set_overflow(true); + loaders + .values + .push(MaybeAssetLoader::Pending { sender, receiver }); + } + + /// Retrieve a handle for the given path. This will create a handle (and [`AssetInfo`]) if it does not exist + pub(crate) fn get_or_create_path_handle( + &self, + path: AssetPath<'static>, + meta_transform: Option, + ) -> Handle { + let mut infos = self.data.infos.write(); + infos + .get_or_create_path_handle::(path, HandleLoadingMode::NotLoading, meta_transform) + .0 + } + + pub(crate) async fn get_meta_loader_and_reader<'a>( + &'a self, + asset_path: &'a AssetPath<'_>, + ) -> Result< + ( + Box, + Arc, + Box>, + ), + AssetLoadError, + > { + // NOTE: We grab the asset byte reader first to ensure this is transactional for AssetReaders like ProcessorGatedReader + // The asset byte reader will "lock" the processed asset, preventing writes for the duration of the lock. + // Then the meta reader, if meta exists, will correspond to the meta for the current "version" of the asset. + // See ProcessedAssetInfo::file_transaction_lock for more context + let reader = self.data.reader.read(asset_path.path()).await?; + match self.data.reader.read_meta_bytes(asset_path.path()).await { + Ok(meta_bytes) => { + // TODO: this isn't fully minimal yet. we only need the loader + let minimal: AssetMetaMinimal = ron::de::from_bytes(&meta_bytes).map_err(|e| { + AssetLoadError::DeserializeMeta(DeserializeMetaError::DeserializeMinimal(e)) + })?; + let loader_name = match minimal.asset { + AssetActionMinimal::Load { loader } => loader, + AssetActionMinimal::Process { .. } => { + return Err(AssetLoadError::CannotLoadProcessedAsset { + path: asset_path.to_owned(), + }) + } + AssetActionMinimal::Ignore => { + return Err(AssetLoadError::CannotLoadIgnoredAsset { + path: asset_path.to_owned(), + }) + } + }; + let loader = self.get_asset_loader_with_type_name(&loader_name).await?; + let meta = loader.deserialize_meta(&meta_bytes).map_err(|e| { + AssetLoadError::AssetLoaderError { + path: asset_path.to_owned(), + loader: loader.type_name(), + error: AssetLoaderError::DeserializeMeta(e), + } + })?; + + Ok((meta, loader, reader)) + } + Err(AssetReaderError::NotFound(_)) => { + let loader = self.get_path_asset_loader(asset_path).await?; + let meta = loader.default_meta(); + Ok((meta, loader, reader)) + } + Err(err) => Err(err.into()), + } + } + + pub(crate) async fn load_with_meta_loader_and_reader( + &self, + asset_path: &AssetPath<'_>, + meta: Box, + loader: &dyn ErasedAssetLoader, + reader: &mut Reader<'_>, + load_dependencies: bool, + populate_hashes: bool, + ) -> Result { + let load_context = LoadContext::new( + self, + asset_path.to_owned(), + load_dependencies, + populate_hashes, + ); + loader.load(reader, meta, load_context).await.map_err(|e| { + AssetLoadError::AssetLoaderError { + loader: loader.type_name(), + path: asset_path.to_owned(), + error: e, + } + }) + } +} + +/// A system that manages internal [`AssetServer`] events, such as finalizing asset loads. +pub fn handle_internal_asset_events(world: &mut World) { + world.resource_scope(|world, server: Mut| { + let mut infos = server.data.infos.write(); + for event in server.data.asset_event_receiver.try_iter() { + match event { + InternalAssetEvent::Loaded { id, loaded_asset } => { + infos.process_asset_load( + id, + loaded_asset, + world, + &server.data.asset_event_sender, + ); + } + InternalAssetEvent::LoadedWithDependencies { id } => { + let sender = infos + .dependency_loaded_event_sender + .get(&id.type_id()) + .expect("Asset event sender should exist"); + sender(world, id); + } + InternalAssetEvent::Failed { id } => infos.process_asset_fail(id), + } + } + + fn queue_ancestors( + asset_path: &AssetPath, + infos: &AssetInfos, + paths_to_reload: &mut HashSet>, + ) { + if let Some(dependants) = infos.loader_dependants.get(asset_path) { + for dependant in dependants { + paths_to_reload.insert(dependant.to_owned()); + queue_ancestors(dependant, infos, paths_to_reload); + } + } + } + + let mut paths_to_reload = HashSet::new(); + for event in server.data.source_event_receiver.try_iter() { + match event { + // TODO: if the asset was processed and the processed file was changed, the first modified event + // should be skipped? + AssetSourceEvent::ModifiedAsset(path) | AssetSourceEvent::ModifiedMeta(path) => { + queue_ancestors( + &AssetPath::new_ref(&path, None), + &infos, + &mut paths_to_reload, + ); + paths_to_reload.insert(path.into()); + } + _ => {} + } + } + + for path in paths_to_reload { + server.reload(path); + } + }); +} + +#[derive(Default)] +pub(crate) struct AssetLoaders { + values: Vec, + extension_to_index: HashMap, + type_name_to_index: HashMap<&'static str, usize>, + preregistered_loaders: HashMap<&'static str, usize>, +} + +#[derive(Clone)] +enum MaybeAssetLoader { + Ready(Arc), + Pending { + sender: async_broadcast::Sender>, + receiver: async_broadcast::Receiver>, + }, +} + +/// Internal events for asset load results +#[allow(clippy::large_enum_variant)] +pub(crate) enum InternalAssetEvent { + Loaded { + id: UntypedAssetId, + loaded_asset: ErasedLoadedAsset, + }, + LoadedWithDependencies { + id: UntypedAssetId, + }, + Failed { + id: UntypedAssetId, + }, +} + +/// The load state of an asset. +#[derive(Component, Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd)] +pub enum LoadState { + /// The asset has not started loading yet + NotLoaded, + /// The asset is in the process of loading. + Loading, + /// The asset has been loaded and has been added to the [`World`] + Loaded, + /// The asset failed to load. + Failed, +} + +/// The load state of an asset's dependencies. +#[derive(Component, Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd)] +pub enum DependencyLoadState { + /// The asset has not started loading yet + NotLoaded, + /// Dependencies are still loading + Loading, + /// Dependencies have all loaded + Loaded, + /// One or more dependencies have failed to load + Failed, +} + +/// The recursive load state of an asset's dependencies. +#[derive(Component, Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd)] +pub enum RecursiveDependencyLoadState { + /// The asset has not started loading yet + NotLoaded, + /// Dependencies in this asset's dependency tree are still loading + Loading, + /// Dependencies in this asset's dependency tree have all loaded + Loaded, + /// One or more dependencies have failed to load in this asset's dependency tree + Failed, +} + +/// An error that occurs during an [`Asset`] load. +#[derive(Error, Debug)] +pub enum AssetLoadError { + #[error("Requested handle of type {requested:?} for asset '{path}' does not match actual asset type '{actual_asset_name}', which used loader '{loader_name}'")] + RequestedHandleTypeMismatch { + path: AssetPath<'static>, + requested: TypeId, + actual_asset_name: &'static str, + loader_name: &'static str, + }, + #[error(transparent)] + MissingAssetLoaderForExtension(#[from] MissingAssetLoaderForExtensionError), + #[error(transparent)] + MissingAssetLoaderForTypeName(#[from] MissingAssetLoaderForTypeNameError), + #[error(transparent)] + AssetReaderError(#[from] AssetReaderError), + #[error("Encountered an error while reading asset metadata bytes")] + AssetMetaReadError, + #[error(transparent)] + DeserializeMeta(DeserializeMetaError), + #[error("Asset '{path}' is configured to be processed. It cannot be loaded directly.")] + CannotLoadProcessedAsset { path: AssetPath<'static> }, + #[error("Asset '{path}' is configured to be ignored. It cannot be loaded.")] + CannotLoadIgnoredAsset { path: AssetPath<'static> }, + #[error("Asset '{path}' encountered an error in {loader}: {error}")] + AssetLoaderError { + path: AssetPath<'static>, + loader: &'static str, + error: AssetLoaderError, + }, +} + +/// An error that occurs when an [`AssetLoader`] is not registered for a given extension. +#[derive(Error, Debug)] +#[error("no `AssetLoader` found{}", format_missing_asset_ext(.extensions))] +pub struct MissingAssetLoaderForExtensionError { + extensions: Vec, +} + +/// An error that occurs when an [`AssetLoader`] is not registered for a given [`std::any::type_name`]. +#[derive(Error, Debug)] +#[error("no `AssetLoader` found with the name '{type_name}'")] +pub struct MissingAssetLoaderForTypeNameError { + type_name: String, +} + +fn format_missing_asset_ext(exts: &[String]) -> String { + if !exts.is_empty() { + format!( + " for the following extension{}: {}", + if exts.len() > 1 { "s" } else { "" }, + exts.join(", ") + ) + } else { + String::new() + } +} + +impl std::fmt::Debug for AssetServer { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("AssetServer") + .field("info", &self.data.infos.read()) + .finish() + } +} + +pub(crate) static CANNOT_WATCH_ERROR_MESSAGE: &str = + "Cannot watch for changes because the current `AssetReader` does not support it. If you are using \ + the FileAssetReader (the default on desktop platforms), enabling the filesystem_watcher feature will \ + add this functionality."; diff --git a/crates/bevy_audio/Cargo.toml b/crates/bevy_audio/Cargo.toml index 53f3764777bb0..b905ebf88ac2c 100644 --- a/crates/bevy_audio/Cargo.toml +++ b/crates/bevy_audio/Cargo.toml @@ -20,7 +20,6 @@ bevy_derive = { path = "../bevy_derive", version = "0.12.0-dev" } bevy_utils = { path = "../bevy_utils", version = "0.12.0-dev" } # other -anyhow = "1.0.4" rodio = { version = "0.17", default-features = false } parking_lot = "0.12.1" diff --git a/crates/bevy_audio/src/audio_source.rs b/crates/bevy_audio/src/audio_source.rs index 367e36ecc2d73..93c4093f462ad 100644 --- a/crates/bevy_audio/src/audio_source.rs +++ b/crates/bevy_audio/src/audio_source.rs @@ -1,12 +1,14 @@ -use anyhow::Result; -use bevy_asset::{Asset, AssetLoader, LoadContext, LoadedAsset}; -use bevy_reflect::{TypePath, TypeUuid}; +use bevy_asset::{ + anyhow::Error, + io::{AsyncReadExt, Reader}, + Asset, AssetLoader, LoadContext, +}; +use bevy_reflect::TypePath; use bevy_utils::BoxedFuture; use std::{io::Cursor, sync::Arc}; /// A source of audio data -#[derive(Debug, Clone, TypeUuid, TypePath)] -#[uuid = "7a14806a-672b-443b-8d16-4f18afefa463"] +#[derive(Asset, Debug, Clone, TypePath)] pub struct AudioSource { /// Raw data of the audio source. /// @@ -38,11 +40,22 @@ impl AsRef<[u8]> for AudioSource { pub struct AudioLoader; impl AssetLoader for AudioLoader { - fn load(&self, bytes: &[u8], load_context: &mut LoadContext) -> BoxedFuture> { - load_context.set_default_asset(LoadedAsset::new(AudioSource { - bytes: bytes.into(), - })); - Box::pin(async move { Ok(()) }) + type Asset = AudioSource; + type Settings = (); + + fn load<'a>( + &'a self, + reader: &'a mut Reader, + _settings: &'a Self::Settings, + _load_context: &'a mut LoadContext, + ) -> BoxedFuture<'a, Result> { + Box::pin(async move { + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; + Ok(AudioSource { + bytes: bytes.into(), + }) + }) } fn extensions(&self) -> &[&str] { @@ -64,8 +77,7 @@ impl AssetLoader for AudioLoader { } /// A type implementing this trait can be converted to a [`rodio::Source`] type. -/// It must be [`Send`] and [`Sync`], and usually implements [`Asset`] so needs to be [`TypeUuid`], -/// in order to be registered. +/// It must be [`Send`] and [`Sync`] in order to be registered. /// Types that implement this trait usually contain raw sound data that can be converted into an iterator of samples. /// This trait is implemented for [`AudioSource`]. /// Check the example [`decodable`](https://github.com/bevyengine/bevy/blob/latest/examples/audio/decodable.rs) for how to implement this trait on a custom type. diff --git a/crates/bevy_audio/src/lib.rs b/crates/bevy_audio/src/lib.rs index 071042235b7b5..a2003a3b5967f 100644 --- a/crates/bevy_audio/src/lib.rs +++ b/crates/bevy_audio/src/lib.rs @@ -50,7 +50,7 @@ pub use rodio::Sample; pub use sinks::*; use bevy_app::prelude::*; -use bevy_asset::{AddAsset, Asset}; +use bevy_asset::{Asset, AssetApp}; use bevy_ecs::prelude::*; use audio_output::*; @@ -90,7 +90,7 @@ impl AddAudioSource for App { T: Decodable + Asset, f32: rodio::cpal::FromSample, { - self.add_asset::().add_systems( + self.init_asset::().add_systems( PostUpdate, play_queued_audio_system::.in_set(AudioPlaySet), ); diff --git a/crates/bevy_audio/src/pitch.rs b/crates/bevy_audio/src/pitch.rs index 08423e282221f..300ea2d8a0dbb 100644 --- a/crates/bevy_audio/src/pitch.rs +++ b/crates/bevy_audio/src/pitch.rs @@ -1,10 +1,10 @@ use crate::{AudioSourceBundle, Decodable, SpatialAudioSourceBundle}; -use bevy_reflect::{TypePath, TypeUuid}; +use bevy_asset::Asset; +use bevy_reflect::TypePath; use rodio::{source::SineWave, source::TakeDuration, Source}; /// A source of sine wave sound -#[derive(Debug, Clone, TypeUuid, TypePath)] -#[uuid = "cbc63be3-b0b9-4d2c-a03c-88b58f1a19ef"] +#[derive(Asset, Debug, Clone, TypePath)] pub struct Pitch { /// Frequency at which sound will be played pub frequency: f32, diff --git a/crates/bevy_core/src/lib.rs b/crates/bevy_core/src/lib.rs index 977f01489f286..e5682d3a92262 100644 --- a/crates/bevy_core/src/lib.rs +++ b/crates/bevy_core/src/lib.rs @@ -23,7 +23,7 @@ pub mod prelude { use bevy_app::prelude::*; use bevy_ecs::prelude::*; use bevy_reflect::{ReflectDeserialize, ReflectSerialize}; -use bevy_utils::{Duration, HashSet, Instant}; +use bevy_utils::{Duration, HashSet, Instant, Uuid}; use std::borrow::Cow; use std::ffi::OsString; use std::marker::PhantomData; @@ -61,7 +61,8 @@ fn register_rust_types(app: &mut App) { .register_type::>() .register_type::>() .register_type::() - .register_type::(); + .register_type::() + .register_type::(); } fn register_math_types(app: &mut App) { diff --git a/crates/bevy_core_pipeline/src/blit/mod.rs b/crates/bevy_core_pipeline/src/blit/mod.rs index 98442953bd712..b6698838f4a54 100644 --- a/crates/bevy_core_pipeline/src/blit/mod.rs +++ b/crates/bevy_core_pipeline/src/blit/mod.rs @@ -1,13 +1,11 @@ use bevy_app::{App, Plugin}; -use bevy_asset::{load_internal_asset, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_ecs::prelude::*; -use bevy_reflect::TypeUuid; use bevy_render::{render_resource::*, renderer::RenderDevice, RenderApp}; use crate::fullscreen_vertex_shader::fullscreen_shader_vertex_state; -pub const BLIT_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2312396983770133547); +pub const BLIT_SHADER_HANDLE: Handle = Handle::weak_from_u128(2312396983770133547); /// Adds support for specialized "blit pipelines", which can be used to write one texture to another. pub struct BlitPlugin; @@ -86,7 +84,7 @@ impl SpecializedRenderPipeline for BlitPipeline { layout: vec![self.texture_bind_group.clone()], vertex: fullscreen_shader_vertex_state(), fragment: Some(FragmentState { - shader: BLIT_SHADER_HANDLE.typed(), + shader: BLIT_SHADER_HANDLE, shader_defs: vec![], entry_point: "fs_main".into(), targets: vec![Some(ColorTargetState { diff --git a/crates/bevy_core_pipeline/src/bloom/downsampling_pipeline.rs b/crates/bevy_core_pipeline/src/bloom/downsampling_pipeline.rs index 8c91065d5888e..747f4ee4c87d4 100644 --- a/crates/bevy_core_pipeline/src/bloom/downsampling_pipeline.rs +++ b/crates/bevy_core_pipeline/src/bloom/downsampling_pipeline.rs @@ -130,7 +130,7 @@ impl SpecializedRenderPipeline for BloomDownsamplingPipeline { layout, vertex: fullscreen_shader_vertex_state(), fragment: Some(FragmentState { - shader: BLOOM_SHADER_HANDLE.typed::(), + shader: BLOOM_SHADER_HANDLE, shader_defs, entry_point, targets: vec![Some(ColorTargetState { diff --git a/crates/bevy_core_pipeline/src/bloom/mod.rs b/crates/bevy_core_pipeline/src/bloom/mod.rs index 7ee2091031558..9f8d0aec2946f 100644 --- a/crates/bevy_core_pipeline/src/bloom/mod.rs +++ b/crates/bevy_core_pipeline/src/bloom/mod.rs @@ -9,10 +9,9 @@ use crate::{ core_3d::{self, CORE_3D}, }; use bevy_app::{App, Plugin}; -use bevy_asset::{load_internal_asset, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_ecs::{prelude::*, query::QueryItem}; use bevy_math::UVec2; -use bevy_reflect::TypeUuid; use bevy_render::{ camera::ExtractedCamera, extract_component::{ @@ -34,8 +33,7 @@ use upsampling_pipeline::{ prepare_upsampling_pipeline, BloomUpsamplingPipeline, UpsamplingPipelineIds, }; -const BLOOM_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 929599476923908); +const BLOOM_SHADER_HANDLE: Handle = Handle::weak_from_u128(929599476923908); const BLOOM_TEXTURE_FORMAT: TextureFormat = TextureFormat::Rg11b10Float; diff --git a/crates/bevy_core_pipeline/src/bloom/upsampling_pipeline.rs b/crates/bevy_core_pipeline/src/bloom/upsampling_pipeline.rs index eadd7e269d5a2..996ccce08343a 100644 --- a/crates/bevy_core_pipeline/src/bloom/upsampling_pipeline.rs +++ b/crates/bevy_core_pipeline/src/bloom/upsampling_pipeline.rs @@ -118,7 +118,7 @@ impl SpecializedRenderPipeline for BloomUpsamplingPipeline { layout: vec![self.bind_group_layout.clone()], vertex: fullscreen_shader_vertex_state(), fragment: Some(FragmentState { - shader: BLOOM_SHADER_HANDLE.typed::(), + shader: BLOOM_SHADER_HANDLE, shader_defs: vec![], entry_point: "upsample".into(), targets: vec![Some(ColorTargetState { diff --git a/crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/mod.rs b/crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/mod.rs index b26c02b54754b..89879938e8a8b 100644 --- a/crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/mod.rs +++ b/crates/bevy_core_pipeline/src/contrast_adaptive_sharpening/mod.rs @@ -4,9 +4,9 @@ use crate::{ fullscreen_vertex_shader::fullscreen_shader_vertex_state, }; use bevy_app::prelude::*; -use bevy_asset::{load_internal_asset, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_ecs::{prelude::*, query::QueryItem}; -use bevy_reflect::{Reflect, TypeUuid}; +use bevy_reflect::Reflect; use bevy_render::{ extract_component::{ExtractComponent, ExtractComponentPlugin, UniformComponentPlugin}, prelude::Camera, @@ -92,8 +92,8 @@ impl ExtractComponent for ContrastAdaptiveSharpeningSettings { } } -const CONTRAST_ADAPTIVE_SHARPENING_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 6925381244141981602); +const CONTRAST_ADAPTIVE_SHARPENING_SHADER_HANDLE: Handle = + Handle::weak_from_u128(6925381244141981602); /// Adds Support for Contrast Adaptive Sharpening (CAS). pub struct CASPlugin; @@ -231,7 +231,7 @@ impl SpecializedRenderPipeline for CASPipeline { layout: vec![self.texture_bind_group.clone()], vertex: fullscreen_shader_vertex_state(), fragment: Some(FragmentState { - shader: CONTRAST_ADAPTIVE_SHARPENING_SHADER_HANDLE.typed(), + shader: CONTRAST_ADAPTIVE_SHARPENING_SHADER_HANDLE, shader_defs, entry_point: "fragment".into(), targets: vec![Some(ColorTargetState { diff --git a/crates/bevy_core_pipeline/src/fullscreen_vertex_shader/mod.rs b/crates/bevy_core_pipeline/src/fullscreen_vertex_shader/mod.rs index 7227cfa3bc0cb..d01c34477503d 100644 --- a/crates/bevy_core_pipeline/src/fullscreen_vertex_shader/mod.rs +++ b/crates/bevy_core_pipeline/src/fullscreen_vertex_shader/mod.rs @@ -1,9 +1,7 @@ -use bevy_asset::HandleUntyped; -use bevy_reflect::TypeUuid; +use bevy_asset::Handle; use bevy_render::{prelude::Shader, render_resource::VertexState}; -pub const FULLSCREEN_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 7837534426033940724); +pub const FULLSCREEN_SHADER_HANDLE: Handle = Handle::weak_from_u128(7837534426033940724); /// uses the [`FULLSCREEN_SHADER_HANDLE`] to output a /// ```wgsl @@ -18,7 +16,7 @@ pub const FULLSCREEN_SHADER_HANDLE: HandleUntyped = /// The draw call should render one triangle: `render_pass.draw(0..3, 0..1);` pub fn fullscreen_shader_vertex_state() -> VertexState { VertexState { - shader: FULLSCREEN_SHADER_HANDLE.typed(), + shader: FULLSCREEN_SHADER_HANDLE, shader_defs: Vec::new(), entry_point: "fullscreen_vertex_shader".into(), buffers: Vec::new(), diff --git a/crates/bevy_core_pipeline/src/fxaa/mod.rs b/crates/bevy_core_pipeline/src/fxaa/mod.rs index 8b0c0433d96fb..f26e63a2d08c2 100644 --- a/crates/bevy_core_pipeline/src/fxaa/mod.rs +++ b/crates/bevy_core_pipeline/src/fxaa/mod.rs @@ -4,10 +4,10 @@ use crate::{ fullscreen_vertex_shader::fullscreen_shader_vertex_state, }; use bevy_app::prelude::*; -use bevy_asset::{load_internal_asset, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_derive::Deref; use bevy_ecs::prelude::*; -use bevy_reflect::{std_traits::ReflectDefault, Reflect, TypeUuid}; +use bevy_reflect::{std_traits::ReflectDefault, Reflect}; use bevy_render::{ extract_component::{ExtractComponent, ExtractComponentPlugin}, prelude::Camera, @@ -75,8 +75,7 @@ impl Default for Fxaa { } } -const FXAA_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 4182761465141723543); +const FXAA_SHADER_HANDLE: Handle = Handle::weak_from_u128(4182761465141723543); /// Adds support for Fast Approximate Anti-Aliasing (FXAA) pub struct FxaaPlugin; @@ -179,7 +178,7 @@ impl SpecializedRenderPipeline for FxaaPipeline { layout: vec![self.texture_bind_group.clone()], vertex: fullscreen_shader_vertex_state(), fragment: Some(FragmentState { - shader: FXAA_SHADER_HANDLE.typed(), + shader: FXAA_SHADER_HANDLE, shader_defs: vec![ format!("EDGE_THRESH_{}", key.edge_threshold.get_str()).into(), format!("EDGE_THRESH_MIN_{}", key.edge_threshold_min.get_str()).into(), diff --git a/crates/bevy_core_pipeline/src/skybox/mod.rs b/crates/bevy_core_pipeline/src/skybox/mod.rs index 86fa28aa8a530..bd2035d6b1b35 100644 --- a/crates/bevy_core_pipeline/src/skybox/mod.rs +++ b/crates/bevy_core_pipeline/src/skybox/mod.rs @@ -1,12 +1,11 @@ use bevy_app::{App, Plugin}; -use bevy_asset::{load_internal_asset, Handle, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_ecs::{ prelude::{Component, Entity}, query::With, schedule::IntoSystemConfigs, system::{Commands, Query, Res, ResMut, Resource}, }; -use bevy_reflect::TypeUuid; use bevy_render::{ extract_component::{ExtractComponent, ExtractComponentPlugin}, render_asset::RenderAssets, @@ -25,8 +24,7 @@ use bevy_render::{ Render, RenderApp, RenderSet, }; -const SKYBOX_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 55594763423201); +const SKYBOX_SHADER_HANDLE: Handle = Handle::weak_from_u128(55594763423201); pub struct SkyboxPlugin; @@ -134,7 +132,7 @@ impl SpecializedRenderPipeline for SkyboxPipeline { layout: vec![self.bind_group_layout.clone()], push_constant_ranges: Vec::new(), vertex: VertexState { - shader: SKYBOX_SHADER_HANDLE.typed(), + shader: SKYBOX_SHADER_HANDLE, shader_defs: Vec::new(), entry_point: "skybox_vertex".into(), buffers: Vec::new(), @@ -162,7 +160,7 @@ impl SpecializedRenderPipeline for SkyboxPipeline { alpha_to_coverage_enabled: false, }, fragment: Some(FragmentState { - shader: SKYBOX_SHADER_HANDLE.typed(), + shader: SKYBOX_SHADER_HANDLE, shader_defs: Vec::new(), entry_point: "skybox_fragment".into(), targets: vec![Some(ColorTargetState { diff --git a/crates/bevy_core_pipeline/src/taa/mod.rs b/crates/bevy_core_pipeline/src/taa/mod.rs index a674ab825bcde..e61dccc5d4bf5 100644 --- a/crates/bevy_core_pipeline/src/taa/mod.rs +++ b/crates/bevy_core_pipeline/src/taa/mod.rs @@ -5,7 +5,7 @@ use crate::{ prepass::{DepthPrepass, MotionVectorPrepass, ViewPrepassTextures}, }; use bevy_app::{App, Plugin}; -use bevy_asset::{load_internal_asset, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_core::FrameCount; use bevy_ecs::{ prelude::{Bundle, Component, Entity}, @@ -15,7 +15,7 @@ use bevy_ecs::{ world::{FromWorld, World}, }; use bevy_math::vec2; -use bevy_reflect::{Reflect, TypeUuid}; +use bevy_reflect::Reflect; use bevy_render::{ camera::{ExtractedCamera, MipBias, TemporalJitter}, prelude::{Camera, Projection}, @@ -42,8 +42,7 @@ mod draw_3d_graph { } } -const TAA_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 656865235226276); +const TAA_SHADER_HANDLE: Handle = Handle::weak_from_u128(656865235226276); /// Plugin for temporal anti-aliasing. Disables multisample anti-aliasing (MSAA). /// @@ -392,7 +391,7 @@ impl SpecializedRenderPipeline for TAAPipeline { layout: vec![self.taa_bind_group_layout.clone()], vertex: fullscreen_shader_vertex_state(), fragment: Some(FragmentState { - shader: TAA_SHADER_HANDLE.typed::(), + shader: TAA_SHADER_HANDLE, shader_defs, entry_point: "taa".into(), targets: vec![ diff --git a/crates/bevy_core_pipeline/src/tonemapping/mod.rs b/crates/bevy_core_pipeline/src/tonemapping/mod.rs index 5ff93f1fdc25d..2af1d48b6eed0 100644 --- a/crates/bevy_core_pipeline/src/tonemapping/mod.rs +++ b/crates/bevy_core_pipeline/src/tonemapping/mod.rs @@ -1,8 +1,8 @@ use crate::fullscreen_vertex_shader::fullscreen_shader_vertex_state; use bevy_app::prelude::*; -use bevy_asset::{load_internal_asset, Assets, Handle, HandleUntyped}; +use bevy_asset::{load_internal_asset, Assets, Handle}; use bevy_ecs::prelude::*; -use bevy_reflect::{Reflect, TypeUuid}; +use bevy_reflect::Reflect; use bevy_render::camera::Camera; use bevy_render::extract_component::{ExtractComponent, ExtractComponentPlugin}; use bevy_render::extract_resource::{ExtractResource, ExtractResourcePlugin}; @@ -17,11 +17,10 @@ mod node; use bevy_utils::default; pub use node::TonemappingNode; -const TONEMAPPING_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 17015368199668024512); +const TONEMAPPING_SHADER_HANDLE: Handle = Handle::weak_from_u128(17015368199668024512); -const TONEMAPPING_SHARED_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2499430578245347910); +const TONEMAPPING_SHARED_SHADER_HANDLE: Handle = + Handle::weak_from_u128(2499430578245347910); /// 3D LUT (look up table) textures used for tonemapping #[derive(Resource, Clone, ExtractResource)] @@ -207,7 +206,7 @@ impl SpecializedRenderPipeline for TonemappingPipeline { layout: vec![self.texture_bind_group.clone()], vertex: fullscreen_shader_vertex_state(), fragment: Some(FragmentState { - shader: TONEMAPPING_SHADER_HANDLE.typed(), + shader: TONEMAPPING_SHADER_HANDLE, shader_defs, entry_point: "fragment".into(), targets: vec![Some(ColorTargetState { diff --git a/crates/bevy_gizmos/src/lib.rs b/crates/bevy_gizmos/src/lib.rs index 631cb058fa4ac..79f696e1f063e 100644 --- a/crates/bevy_gizmos/src/lib.rs +++ b/crates/bevy_gizmos/src/lib.rs @@ -16,10 +16,21 @@ //! //! See the documentation on [`Gizmos`](crate::gizmos::Gizmos) for more examples. -use std::mem; +pub mod gizmos; + +#[cfg(feature = "bevy_sprite")] +mod pipeline_2d; +#[cfg(feature = "bevy_pbr")] +mod pipeline_3d; + +/// The `bevy_gizmos` prelude. +pub mod prelude { + #[doc(hidden)] + pub use crate::{gizmos::Gizmos, AabbGizmo, AabbGizmoConfig, GizmoConfig}; +} use bevy_app::{Last, Plugin, PostUpdate}; -use bevy_asset::{load_internal_asset, AddAsset, Assets, Handle, HandleUntyped}; +use bevy_asset::{load_internal_asset, Asset, AssetApp, Assets, Handle}; use bevy_core::cast_slice; use bevy_ecs::{ change_detection::DetectChanges, @@ -33,7 +44,7 @@ use bevy_ecs::{ Commands, Query, Res, ResMut, Resource, SystemParamItem, }, }; -use bevy_reflect::{std_traits::ReflectDefault, Reflect, TypePath, TypeUuid}; +use bevy_reflect::{std_traits::ReflectDefault, Reflect, TypePath}; use bevy_render::{ color::Color, extract_component::{ComponentUniforms, DynamicUniformIndex, UniformComponentPlugin}, @@ -54,24 +65,10 @@ use bevy_transform::{ components::{GlobalTransform, Transform}, TransformSystem, }; - -pub mod gizmos; - -#[cfg(feature = "bevy_sprite")] -mod pipeline_2d; -#[cfg(feature = "bevy_pbr")] -mod pipeline_3d; - use gizmos::{GizmoStorage, Gizmos}; +use std::mem; -/// The `bevy_gizmos` prelude. -pub mod prelude { - #[doc(hidden)] - pub use crate::{gizmos::Gizmos, AabbGizmo, AabbGizmoConfig, GizmoConfig}; -} - -const LINE_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 7414812689238026784); +const LINE_SHADER_HANDLE: Handle = Handle::weak_from_u128(7414812689238026784); /// A [`Plugin`] that provides an immediate mode drawing api for visual debugging. pub struct GizmoPlugin; @@ -81,7 +78,7 @@ impl Plugin for GizmoPlugin { load_internal_asset!(app, LINE_SHADER_HANDLE, "lines.wgsl", Shader::from_wgsl); app.add_plugins(UniformComponentPlugin::::default()) - .add_asset::() + .init_asset::() .add_plugins(RenderAssetPlugin::::default()) .init_resource::() .init_resource::() @@ -351,8 +348,7 @@ struct LineGizmoUniform { _padding: bevy_math::Vec2, } -#[derive(Debug, Default, Clone, TypeUuid, TypePath)] -#[uuid = "02b99cbf-bb26-4713-829a-aee8e08dedc0"] +#[derive(Asset, Debug, Default, Clone, TypePath)] struct LineGizmo { positions: Vec<[f32; 3]>, colors: Vec<[f32; 4]>, diff --git a/crates/bevy_gizmos/src/pipeline_2d.rs b/crates/bevy_gizmos/src/pipeline_2d.rs index fa345f2bf05e1..2ec777e8ca52b 100644 --- a/crates/bevy_gizmos/src/pipeline_2d.rs +++ b/crates/bevy_gizmos/src/pipeline_2d.rs @@ -97,13 +97,13 @@ impl SpecializedRenderPipeline for LineGizmoPipeline { RenderPipelineDescriptor { vertex: VertexState { - shader: LINE_SHADER_HANDLE.typed(), + shader: LINE_SHADER_HANDLE, entry_point: "vertex".into(), shader_defs: shader_defs.clone(), buffers: line_gizmo_vertex_buffer_layouts(key.strip), }, fragment: Some(FragmentState { - shader: LINE_SHADER_HANDLE.typed(), + shader: LINE_SHADER_HANDLE, shader_defs, entry_point: "fragment".into(), targets: vec![Some(ColorTargetState { diff --git a/crates/bevy_gizmos/src/pipeline_3d.rs b/crates/bevy_gizmos/src/pipeline_3d.rs index 33712fa020557..acb827dff3346 100644 --- a/crates/bevy_gizmos/src/pipeline_3d.rs +++ b/crates/bevy_gizmos/src/pipeline_3d.rs @@ -103,13 +103,13 @@ impl SpecializedRenderPipeline for LineGizmoPipeline { RenderPipelineDescriptor { vertex: VertexState { - shader: LINE_SHADER_HANDLE.typed(), + shader: LINE_SHADER_HANDLE, entry_point: "vertex".into(), shader_defs: shader_defs.clone(), buffers: line_gizmo_vertex_buffer_layouts(key.strip), }, fragment: Some(FragmentState { - shader: LINE_SHADER_HANDLE.typed(), + shader: LINE_SHADER_HANDLE, shader_defs, entry_point: "fragment".into(), targets: vec![Some(ColorTargetState { diff --git a/crates/bevy_gltf/Cargo.toml b/crates/bevy_gltf/Cargo.toml index 9a86255e7521e..eac57697e66aa 100644 --- a/crates/bevy_gltf/Cargo.toml +++ b/crates/bevy_gltf/Cargo.toml @@ -37,7 +37,6 @@ gltf = { version = "1.3.0", default-features = false, features = [ "utils", ] } thiserror = "1.0" -anyhow = "1.0.4" base64 = "0.13.0" percent-encoding = "2.1" serde = { version = "1.0", features = ["derive"] } diff --git a/crates/bevy_gltf/src/lib.rs b/crates/bevy_gltf/src/lib.rs index bd77ed2deba02..8474ab66e77c7 100644 --- a/crates/bevy_gltf/src/lib.rs +++ b/crates/bevy_gltf/src/lib.rs @@ -9,10 +9,10 @@ mod vertex_attributes; pub use loader::*; use bevy_app::prelude::*; -use bevy_asset::{AddAsset, Handle}; +use bevy_asset::{Asset, AssetApp, Handle}; use bevy_ecs::{prelude::Component, reflect::ReflectComponent}; use bevy_pbr::StandardMaterial; -use bevy_reflect::{Reflect, TypePath, TypeUuid}; +use bevy_reflect::{Reflect, TypePath}; use bevy_render::{ mesh::{Mesh, MeshVertexAttribute}, renderer::RenderDevice, @@ -41,11 +41,11 @@ impl GltfPlugin { impl Plugin for GltfPlugin { fn build(&self, app: &mut App) { app.register_type::() - .add_asset::() - .add_asset::() - .add_asset::() - .add_asset::() - .preregister_asset_loader(&["gltf", "glb"]); + .init_asset::() + .init_asset::() + .init_asset::() + .init_asset::() + .preregister_asset_loader::(&["gltf", "glb"]); } fn finish(&self, app: &mut App) { @@ -54,17 +54,15 @@ impl Plugin for GltfPlugin { None => CompressedImageFormats::NONE, }; - app.add_asset_loader::(GltfLoader { + app.register_asset_loader(GltfLoader { supported_compressed_formats, custom_vertex_attributes: self.custom_vertex_attributes.clone(), }); } } -/// Representation of a loaded glTF file -/// (file loaded via the `AssetServer` with the extension `.glb` or `.gltf`). -#[derive(Debug, TypeUuid, TypePath)] -#[uuid = "5c7d5f8a-f7b0-4e45-a09e-406c0372fea2"] +/// Representation of a loaded glTF file. +#[derive(Asset, Debug, TypePath)] pub struct Gltf { pub scenes: Vec>, pub named_scenes: HashMap>, @@ -83,8 +81,7 @@ pub struct Gltf { /// A glTF node with all of its child nodes, its [`GltfMesh`], /// [`Transform`](bevy_transform::prelude::Transform) and an optional [`GltfExtras`]. -#[derive(Debug, Clone, TypeUuid, TypePath)] -#[uuid = "dad74750-1fd6-460f-ac51-0a7937563865"] +#[derive(Asset, Debug, Clone, TypePath)] pub struct GltfNode { pub children: Vec, pub mesh: Option>, @@ -94,16 +91,14 @@ pub struct GltfNode { /// A glTF mesh, which may consist of multiple [`GltfPrimitives`](GltfPrimitive) /// and an optional [`GltfExtras`]. -#[derive(Debug, Clone, TypeUuid, TypePath)] -#[uuid = "8ceaec9a-926a-4f29-8ee3-578a69f42315"] +#[derive(Asset, Debug, Clone, TypePath)] pub struct GltfMesh { pub primitives: Vec, pub extras: Option, } /// Part of a [`GltfMesh`] that consists of a [`Mesh`], an optional [`StandardMaterial`] and [`GltfExtras`]. -#[derive(Debug, Clone, TypeUuid, TypePath)] -#[uuid = "cbfca302-82fd-41cb-af77-cab6b3d50af1"] +#[derive(Asset, Debug, Clone, TypePath)] pub struct GltfPrimitive { pub mesh: Handle, pub material: Option>, diff --git a/crates/bevy_gltf/src/loader.rs b/crates/bevy_gltf/src/loader.rs index b5c1fae3c66a6..64d17305b3e04 100644 --- a/crates/bevy_gltf/src/loader.rs +++ b/crates/bevy_gltf/src/loader.rs @@ -1,7 +1,7 @@ -use crate::{vertex_attributes::*, Gltf, GltfExtras, GltfNode}; -use anyhow::Result; +use crate::{vertex_attributes::convert_attribute, Gltf, GltfExtras, GltfNode}; use bevy_asset::{ - AssetIoError, AssetLoader, AssetPath, BoxedFuture, Handle, HandleId, LoadContext, LoadedAsset, + anyhow, io::Reader, AssetLoadError, AssetLoader, AsyncReadExt, Handle, LoadContext, + ReadAssetBytesError, }; use bevy_core::Name; use bevy_core_pipeline::prelude::Camera3dBundle; @@ -24,7 +24,9 @@ use bevy_render::{ prelude::SpatialBundle, primitives::Aabb, render_resource::{AddressMode, Face, FilterMode, PrimitiveTopology, SamplerDescriptor}, - texture::{CompressedImageFormats, Image, ImageSampler, ImageType, TextureError}, + texture::{ + CompressedImageFormats, Image, ImageLoaderSettings, ImageSampler, ImageType, TextureError, + }, }; use bevy_scene::Scene; #[cfg(not(target_arch = "wasm32"))] @@ -38,7 +40,10 @@ use gltf::{ Material, Node, Primitive, }; use serde::Deserialize; -use std::{collections::VecDeque, path::Path}; +use std::{ + collections::VecDeque, + path::{Path, PathBuf}, +}; use thiserror::Error; /// An error that occurs when loading a glTF file. @@ -58,8 +63,10 @@ pub enum GltfError { InvalidImageMimeType(String), #[error("You may need to add the feature for the file format: {0}")] ImageError(#[from] TextureError), - #[error("failed to load an asset path: {0}")] - AssetIoError(#[from] AssetIoError), + #[error("failed to read bytes from an asset path: {0}")] + ReadAssetBytesError(#[from] ReadAssetBytesError), + #[error("failed to load asset from an asset path: {0}")] + AssetLoadError(#[from] AssetLoadError), #[error("Missing sampler for animation {0}")] MissingAnimationSampler(usize), #[error("failed to generate tangents: {0}")] @@ -75,12 +82,19 @@ pub struct GltfLoader { } impl AssetLoader for GltfLoader { + type Asset = Gltf; + type Settings = (); fn load<'a>( &'a self, - bytes: &'a [u8], + reader: &'a mut Reader, + _settings: &'a (), load_context: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<()>> { - Box::pin(async move { Ok(load_gltf(bytes, load_context, self).await?) }) + ) -> bevy_utils::BoxedFuture<'a, Result> { + Box::pin(async move { + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; + Ok(load_gltf(self, &bytes, load_context).await?) + }) } fn extensions(&self) -> &[&str] { @@ -89,23 +103,17 @@ impl AssetLoader for GltfLoader { } /// Loads an entire glTF file. -async fn load_gltf<'a, 'b>( - bytes: &'a [u8], - load_context: &'a mut LoadContext<'b>, +async fn load_gltf<'a, 'b, 'c>( loader: &GltfLoader, -) -> Result<(), GltfError> { + bytes: &'a [u8], + load_context: &'b mut LoadContext<'c>, +) -> Result { let gltf = gltf::Gltf::from_slice(bytes)?; - let buffer_data = load_buffers(&gltf, load_context, load_context.path()).await?; + let buffer_data = load_buffers(&gltf, load_context).await?; - let mut materials = vec![]; - let mut named_materials = HashMap::default(); let mut linear_textures = HashSet::default(); + for material in gltf.materials() { - let handle = load_material(&material, load_context); - if let Some(name) = material.name() { - named_materials.insert(name.to_string(), handle.clone()); - } - materials.push(handle); if let Some(texture) = material.normal_texture() { linear_textures.insert(texture.texture().index()); } @@ -202,10 +210,8 @@ async fn load_gltf<'a, 'b>( ); } } - let handle = load_context.set_labeled_asset( - &format!("Animation{}", animation.index()), - LoadedAsset::new(animation_clip), - ); + let handle = load_context + .add_labeled_asset(format!("Animation{}", animation.index()), animation_clip); if let Some(name) = animation.name() { named_animations.insert(name.to_string(), handle.clone()); } @@ -214,6 +220,89 @@ async fn load_gltf<'a, 'b>( (animations, named_animations, animation_roots) }; + // TODO: use the threaded impl on wasm once wasm thread pool doesn't deadlock on it + // See https://github.com/bevyengine/bevy/issues/1924 for more details + // The taskpool use is also avoided when there is only one texture for performance reasons and + // to avoid https://github.com/bevyengine/bevy/pull/2725 + // PERF: could this be a Vec instead? Are gltf texture indices dense? + fn process_loaded_texture( + load_context: &mut LoadContext, + handles: &mut Vec>, + texture: ImageOrPath, + ) { + let handle = match texture { + ImageOrPath::Image { label, image } => load_context.add_labeled_asset(label, image), + ImageOrPath::Path { path, is_srgb } => { + load_context.load_with_settings(path, move |settings: &mut ImageLoaderSettings| { + settings.is_srgb = is_srgb; + }) + } + }; + handles.push(handle); + } + + // We collect handles to ensure loaded images from paths are not unloaded before they are used elsewhere + // in the loader. This prevents "reloads", but it also prevents dropping the is_srgb context on reload. + // + // In theory we could store a mapping between texture.index() and handle to use + // later in the loader when looking up handles for materials. However this would mean + // that the material's load context would no longer track those images as dependencies. + let mut _texture_handles = Vec::new(); + if gltf.textures().len() == 1 || cfg!(target_arch = "wasm32") { + for texture in gltf.textures() { + let parent_path = load_context.path().parent().unwrap(); + let image = load_image( + texture, + &buffer_data, + &linear_textures, + parent_path, + loader.supported_compressed_formats, + ) + .await?; + process_loaded_texture(load_context, &mut _texture_handles, image); + } + } else { + #[cfg(not(target_arch = "wasm32"))] + IoTaskPool::get() + .scope(|scope| { + gltf.textures().for_each(|gltf_texture| { + let parent_path = load_context.path().parent().unwrap(); + let linear_textures = &linear_textures; + let buffer_data = &buffer_data; + scope.spawn(async move { + load_image( + gltf_texture, + buffer_data, + linear_textures, + parent_path, + loader.supported_compressed_formats, + ) + .await + }); + }); + }) + .into_iter() + .for_each(|result| match result { + Ok(image) => { + process_loaded_texture(load_context, &mut _texture_handles, image); + } + Err(err) => { + warn!("Error loading glTF texture: {}", err); + } + }); + } + + let mut materials = vec![]; + let mut named_materials = HashMap::default(); + // NOTE: materials must be loaded after textures because image load() calls will happen before load_with_settings, preventing is_srgb from being set properly + for material in gltf.materials() { + let handle = load_material(&material, load_context); + if let Some(name) = material.name() { + named_materials.insert(name.to_string(), handle.clone()); + } + materials.push(handle); + } + let mut meshes = vec![]; let mut named_meshes = HashMap::default(); for gltf_mesh in gltf.meshes() { @@ -255,10 +344,8 @@ async fn load_gltf<'a, 'b>( morph_target_reader.map(PrimitiveMorphAttributesIter), mesh.count_vertices(), )?; - let handle = load_context.set_labeled_asset( - &morph_targets_label, - LoadedAsset::new(morph_target_image.0), - ); + let handle = + load_context.add_labeled_asset(morph_targets_label, morph_target_image.0); mesh.set_morph_targets(handle); let extras = gltf_mesh.extras().as_ref(); @@ -306,7 +393,7 @@ async fn load_gltf<'a, 'b>( } } - let mesh = load_context.set_labeled_asset(&primitive_label, LoadedAsset::new(mesh)); + let mesh = load_context.add_labeled_asset(primitive_label, mesh); primitives.push(super::GltfPrimitive { mesh, material: primitive @@ -318,12 +405,12 @@ async fn load_gltf<'a, 'b>( }); } - let handle = load_context.set_labeled_asset( - &mesh_label(&gltf_mesh), - LoadedAsset::new(super::GltfMesh { + let handle = load_context.add_labeled_asset( + mesh_label(&gltf_mesh), + super::GltfMesh { primitives, extras: get_gltf_extras(gltf_mesh.extras()), - }), + }, ); if let Some(name) = gltf_mesh.name() { named_meshes.insert(name.to_string(), handle.clone()); @@ -369,7 +456,7 @@ async fn load_gltf<'a, 'b>( } let nodes = resolve_node_hierarchy(nodes_intermediate, load_context.path()) .into_iter() - .map(|(label, node)| load_context.set_labeled_asset(&label, LoadedAsset::new(node))) + .map(|(label, node)| load_context.add_labeled_asset(label, node)) .collect::>>(); let named_nodes = named_nodes_intermediate .into_iter() @@ -380,54 +467,6 @@ async fn load_gltf<'a, 'b>( }) .collect(); - // TODO: use the threaded impl on wasm once wasm thread pool doesn't deadlock on it - // See https://github.com/bevyengine/bevy/issues/1924 for more details - // The taskpool use is also avoided when there is only one texture for performance reasons and - // to avoid https://github.com/bevyengine/bevy/pull/2725 - if gltf.textures().len() == 1 || cfg!(target_arch = "wasm32") { - for gltf_texture in gltf.textures() { - let (texture, label) = load_texture( - gltf_texture, - &buffer_data, - &linear_textures, - load_context, - loader.supported_compressed_formats, - ) - .await?; - load_context.set_labeled_asset(&label, LoadedAsset::new(texture)); - } - } else { - #[cfg(not(target_arch = "wasm32"))] - IoTaskPool::get() - .scope(|scope| { - gltf.textures().for_each(|gltf_texture| { - let linear_textures = &linear_textures; - let load_context: &LoadContext = load_context; - let buffer_data = &buffer_data; - scope.spawn(async move { - load_texture( - gltf_texture, - buffer_data, - linear_textures, - load_context, - loader.supported_compressed_formats, - ) - .await - }); - }); - }) - .into_iter() - .filter_map(|res| { - if let Err(err) = res.as_ref() { - warn!("Error loading glTF texture: {}", err); - } - res.ok() - }) - .for_each(|(texture, label)| { - load_context.set_labeled_asset(&label, LoadedAsset::new(texture)); - }); - } - let skinned_mesh_inverse_bindposes: Vec<_> = gltf .skins() .map(|gltf_skin| { @@ -438,9 +477,9 @@ async fn load_gltf<'a, 'b>( .map(|mat| Mat4::from_cols_array_2d(&mat)) .collect(); - load_context.set_labeled_asset( - &skin_label(&gltf_skin), - LoadedAsset::new(SkinnedMeshInverseBindposes::from(inverse_bindposes)), + load_context.add_labeled_asset( + skin_label(&gltf_skin), + SkinnedMeshInverseBindposes::from(inverse_bindposes), ) }) .collect(); @@ -514,8 +553,7 @@ async fn load_gltf<'a, 'b>( }); } - let scene_handle = load_context - .set_labeled_asset(&scene_label(&scene), LoadedAsset::new(Scene::new(world))); + let scene_handle = load_context.add_labeled_asset(scene_label(&scene), Scene::new(world)); if let Some(name) = scene.name() { named_scenes.insert(name.to_string(), scene_handle.clone()); @@ -523,7 +561,7 @@ async fn load_gltf<'a, 'b>( scenes.push(scene_handle); } - load_context.set_default_asset(LoadedAsset::new(Gltf { + Ok(Gltf { default_scene: gltf .default_scene() .and_then(|scene| scenes.get(scene.index())) @@ -540,9 +578,7 @@ async fn load_gltf<'a, 'b>( animations, #[cfg(feature = "bevy_animation")] named_animations, - })); - - Ok(()) + }) } fn get_gltf_extras(extras: &gltf::json::Extras) -> Option { @@ -575,107 +611,98 @@ fn paths_recur( } /// Loads a glTF texture as a bevy [`Image`] and returns it together with its label. -async fn load_texture<'a>( +async fn load_image<'a, 'b>( gltf_texture: gltf::Texture<'a>, buffer_data: &[Vec], linear_textures: &HashSet, - load_context: &LoadContext<'a>, + parent_path: &'b Path, supported_compressed_formats: CompressedImageFormats, -) -> Result<(Image, String), GltfError> { +) -> Result { let is_srgb = !linear_textures.contains(&gltf_texture.index()); - let mut texture = match gltf_texture.source().source() { + match gltf_texture.source().source() { gltf::image::Source::View { view, mime_type } => { let start = view.offset(); let end = view.offset() + view.length(); let buffer = &buffer_data[view.buffer().index()][start..end]; - Image::from_buffer( + let mut image = Image::from_buffer( buffer, ImageType::MimeType(mime_type), supported_compressed_formats, is_srgb, - )? + )?; + image.sampler_descriptor = ImageSampler::Descriptor(texture_sampler(&gltf_texture)); + Ok(ImageOrPath::Image { + image, + label: texture_label(&gltf_texture), + }) } gltf::image::Source::Uri { uri, mime_type } => { let uri = percent_encoding::percent_decode_str(uri) .decode_utf8() .unwrap(); let uri = uri.as_ref(); - let (bytes, image_type) = if let Ok(data_uri) = DataUri::parse(uri) { - (data_uri.decode()?, ImageType::MimeType(data_uri.mime_type)) + if let Ok(data_uri) = DataUri::parse(uri) { + let bytes = data_uri.decode()?; + let image_type = ImageType::MimeType(data_uri.mime_type); + Ok(ImageOrPath::Image { + image: Image::from_buffer( + &bytes, + mime_type.map(ImageType::MimeType).unwrap_or(image_type), + supported_compressed_formats, + is_srgb, + )?, + label: texture_label(&gltf_texture), + }) } else { - let parent = load_context.path().parent().unwrap(); - let image_path = parent.join(uri); - let bytes = load_context.read_asset_bytes(image_path.clone()).await?; - - let extension = Path::new(uri).extension().unwrap().to_str().unwrap(); - let image_type = ImageType::Extension(extension); - - (bytes, image_type) - }; - - Image::from_buffer( - &bytes, - mime_type.map(ImageType::MimeType).unwrap_or(image_type), - supported_compressed_formats, - is_srgb, - )? + let image_path = parent_path.join(uri); + Ok(ImageOrPath::Path { + path: image_path, + is_srgb, + }) + } } - }; - texture.sampler_descriptor = ImageSampler::Descriptor(texture_sampler(&gltf_texture)); - - Ok((texture, texture_label(&gltf_texture))) + } } /// Loads a glTF material as a bevy [`StandardMaterial`] and returns it. fn load_material(material: &Material, load_context: &mut LoadContext) -> Handle { let material_label = material_label(material); + load_context.labeled_asset_scope(material_label, |load_context| { + let pbr = material.pbr_metallic_roughness(); + + // TODO: handle missing label handle errors here? + let color = pbr.base_color_factor(); + let base_color_texture = pbr.base_color_texture().map(|info| { + // TODO: handle info.tex_coord() (the *set* index for the right texcoords) + texture_handle(load_context, &info.texture()) + }); - let pbr = material.pbr_metallic_roughness(); - - let color = pbr.base_color_factor(); - let base_color_texture = pbr.base_color_texture().map(|info| { - // TODO: handle info.tex_coord() (the *set* index for the right texcoords) - let label = texture_label(&info.texture()); - let path = AssetPath::new_ref(load_context.path(), Some(&label)); - load_context.get_handle(path) - }); + let normal_map_texture: Option> = + material.normal_texture().map(|normal_texture| { + // TODO: handle normal_texture.scale + // TODO: handle normal_texture.tex_coord() (the *set* index for the right texcoords) + texture_handle(load_context, &normal_texture.texture()) + }); - let normal_map_texture: Option> = - material.normal_texture().map(|normal_texture| { - // TODO: handle normal_texture.scale - // TODO: handle normal_texture.tex_coord() (the *set* index for the right texcoords) - let label = texture_label(&normal_texture.texture()); - let path = AssetPath::new_ref(load_context.path(), Some(&label)); - load_context.get_handle(path) + let metallic_roughness_texture = pbr.metallic_roughness_texture().map(|info| { + // TODO: handle info.tex_coord() (the *set* index for the right texcoords) + texture_handle(load_context, &info.texture()) }); - let metallic_roughness_texture = pbr.metallic_roughness_texture().map(|info| { - // TODO: handle info.tex_coord() (the *set* index for the right texcoords) - let label = texture_label(&info.texture()); - let path = AssetPath::new_ref(load_context.path(), Some(&label)); - load_context.get_handle(path) - }); - - let occlusion_texture = material.occlusion_texture().map(|occlusion_texture| { - // TODO: handle occlusion_texture.tex_coord() (the *set* index for the right texcoords) - // TODO: handle occlusion_texture.strength() (a scalar multiplier for occlusion strength) - let label = texture_label(&occlusion_texture.texture()); - let path = AssetPath::new_ref(load_context.path(), Some(&label)); - load_context.get_handle(path) - }); + let occlusion_texture = material.occlusion_texture().map(|occlusion_texture| { + // TODO: handle occlusion_texture.tex_coord() (the *set* index for the right texcoords) + // TODO: handle occlusion_texture.strength() (a scalar multiplier for occlusion strength) + texture_handle(load_context, &occlusion_texture.texture()) + }); - let emissive = material.emissive_factor(); - let emissive_texture = material.emissive_texture().map(|info| { - // TODO: handle occlusion_texture.tex_coord() (the *set* index for the right texcoords) - // TODO: handle occlusion_texture.strength() (a scalar multiplier for occlusion strength) - let label = texture_label(&info.texture()); - let path = AssetPath::new_ref(load_context.path(), Some(&label)); - load_context.get_handle(path) - }); + let emissive = material.emissive_factor(); + let emissive_texture = material.emissive_texture().map(|info| { + // TODO: handle occlusion_texture.tex_coord() (the *set* index for the right texcoords) + // TODO: handle occlusion_texture.strength() (a scalar multiplier for occlusion strength) + texture_handle(load_context, &info.texture()) + }); - load_context.set_labeled_asset( - &material_label, - LoadedAsset::new(StandardMaterial { + StandardMaterial { base_color: Color::rgba_linear(color[0], color[1], color[2], color[3]), base_color_texture, perceptual_roughness: pbr.roughness_factor(), @@ -695,8 +722,8 @@ fn load_material(material: &Material, load_context: &mut LoadContext) -> Handle< unlit: material.unlit(), alpha_mode: alpha_mode(material), ..Default::default() - }), - ) + } + }) } /// Loads a glTF node. @@ -771,8 +798,8 @@ fn load_node( if let Some(weights) = mesh.weights() { let first_mesh = if let Some(primitive) = mesh.primitives().next() { let primitive_label = primitive_label(&mesh, &primitive); - let path = AssetPath::new_ref(load_context.path(), Some(&primitive_label)); - Some(Handle::weak(HandleId::from(path))) + let handle: Handle = load_context.get_label_handle(&primitive_label); + Some(handle) } else { None }; @@ -796,14 +823,11 @@ fn load_node( let primitive_label = primitive_label(&mesh, &primitive); let bounds = primitive.bounding_box(); - let mesh_asset_path = - AssetPath::new_ref(load_context.path(), Some(&primitive_label)); - let material_asset_path = - AssetPath::new_ref(load_context.path(), Some(&material_label)); - - let mut primitive_entity = parent.spawn(PbrBundle { - mesh: load_context.get_handle(mesh_asset_path), - material: load_context.get_handle(material_asset_path), + + let mut mesh_entity = parent.spawn(PbrBundle { + // TODO: handle missing label handle errors here? + mesh: load_context.get_label_handle(&primitive_label), + material: load_context.get_label_handle(&material_label), ..Default::default() }); let target_count = primitive.morph_targets().len(); @@ -818,23 +842,23 @@ fn load_node( // they should all have the same length. // > All morph target accessors MUST have the same count as // > the accessors of the original primitive. - primitive_entity.insert(MeshMorphWeights::new(weights).unwrap()); + mesh_entity.insert(MeshMorphWeights::new(weights).unwrap()); } - primitive_entity.insert(Aabb::from_min_max( + mesh_entity.insert(Aabb::from_min_max( Vec3::from_slice(&bounds.min), Vec3::from_slice(&bounds.max), )); if let Some(extras) = primitive.extras() { - primitive_entity.insert(super::GltfExtras { + mesh_entity.insert(super::GltfExtras { value: extras.get().to_string(), }); } - primitive_entity.insert(Name::new(primitive_name(&mesh, &primitive))); + mesh_entity.insert(Name::new(primitive_name(&mesh, &primitive))); // Mark for adding skinned mesh if let Some(skin) = gltf_node.skin() { - entity_to_skin_index_map.insert(primitive_entity.id(), skin.index()); + entity_to_skin_index_map.insert(mesh_entity.id(), skin.index()); } } } @@ -979,6 +1003,29 @@ fn texture_label(texture: &gltf::Texture) -> String { format!("Texture{}", texture.index()) } +fn texture_handle(load_context: &mut LoadContext, texture: &gltf::Texture) -> Handle { + match texture.source().source() { + gltf::image::Source::View { .. } => { + let label = texture_label(texture); + load_context.get_label_handle(&label) + } + gltf::image::Source::Uri { uri, .. } => { + let uri = percent_encoding::percent_decode_str(uri) + .decode_utf8() + .unwrap(); + let uri = uri.as_ref(); + if let Ok(_data_uri) = DataUri::parse(uri) { + let label = texture_label(texture); + load_context.get_label_handle(&label) + } else { + let parent = load_context.path().parent().unwrap(); + let image_path = parent.join(uri); + load_context.load(image_path) + } + } + } +} + /// Returns the label for the `node`. fn node_label(node: &gltf::Node) -> String { format!("Node{}", node.index()) @@ -1070,8 +1117,7 @@ fn alpha_mode(material: &Material) -> AlphaMode { /// Loads the raw glTF buffer data for a specific glTF file. async fn load_buffers( gltf: &gltf::Gltf, - load_context: &LoadContext<'_>, - asset_path: &Path, + load_context: &mut LoadContext<'_>, ) -> Result>, GltfError> { const VALID_MIME_TYPES: &[&str] = &["application/octet-stream", "application/gltf-buffer"]; @@ -1090,8 +1136,8 @@ async fn load_buffers( Ok(_) => return Err(GltfError::BufferFormatUnsupported), Err(()) => { // TODO: Remove this and add dep - let buffer_path = asset_path.parent().unwrap().join(uri); - load_context.read_asset_bytes(buffer_path).await? + let buffer_path = load_context.path().parent().unwrap().join(uri); + load_context.read_asset_bytes(&buffer_path).await? } }; buffer_data.push(buffer_bytes); @@ -1164,6 +1210,11 @@ fn resolve_node_hierarchy( .collect() } +enum ImageOrPath { + Image { image: Image, label: String }, + Path { path: PathBuf, is_srgb: bool }, +} + struct DataUri<'a> { mime_type: &'a str, base64: bool, diff --git a/crates/bevy_internal/Cargo.toml b/crates/bevy_internal/Cargo.toml index 28252e43d37c1..95f2d888b54ec 100644 --- a/crates/bevy_internal/Cargo.toml +++ b/crates/bevy_internal/Cargo.toml @@ -23,7 +23,6 @@ trace_chrome = [ "bevy_log/tracing-chrome" ] trace_tracy = ["bevy_render?/tracing-tracy", "bevy_log/tracing-tracy" ] trace_tracy_memory = ["bevy_log/trace_tracy_memory"] wgpu_trace = ["bevy_render/wgpu_trace"] -debug_asset_server = ["bevy_asset/debug_asset_server"] detailed_trace = ["bevy_utils/detailed_trace"] # Image format support for texture loading (PNG and HDR are enabled by default) @@ -62,11 +61,8 @@ symphonia-wav = ["bevy_audio/symphonia-wav"] shader_format_glsl = ["bevy_render/shader_format_glsl"] shader_format_spirv = ["bevy_render/shader_format_spirv"] -# Enable watching file system for asset hot reload -filesystem_watcher = ["bevy_asset/filesystem_watcher"] - serialize = ["bevy_core/serialize", "bevy_input/serialize", "bevy_time/serialize", "bevy_window/serialize", "bevy_transform/serialize", "bevy_math/serialize", "bevy_scene/serialize"] -multi-threaded = ["bevy_ecs/multi-threaded", "bevy_tasks/multi-threaded"] +multi-threaded = ["bevy_asset/multi-threaded", "bevy_ecs/multi-threaded", "bevy_tasks/multi-threaded"] # Display server protocol support (X11 is enabled by default) wayland = ["bevy_winit/wayland"] @@ -105,6 +101,9 @@ glam_assert = ["bevy_math/glam_assert"] default_font = ["bevy_text?/default_font"] +# Enables watching the filesystem for Bevy Asset hot-reloading +filesystem_watcher = ["bevy_asset?/filesystem_watcher"] + [dependencies] # bevy bevy_a11y = { path = "../bevy_a11y", version = "0.12.0-dev" } diff --git a/crates/bevy_internal/src/default_plugins.rs b/crates/bevy_internal/src/default_plugins.rs index fb5e3da69173f..c5eac80e7cae3 100644 --- a/crates/bevy_internal/src/default_plugins.rs +++ b/crates/bevy_internal/src/default_plugins.rs @@ -12,7 +12,6 @@ use bevy_app::{PluginGroup, PluginGroupBuilder}; /// * [`InputPlugin`](crate::input::InputPlugin) /// * [`WindowPlugin`](crate::window::WindowPlugin) /// * [`AssetPlugin`](crate::asset::AssetPlugin) - with feature `bevy_asset` -/// * [`DebugAssetPlugin`](crate::asset::debug_asset_server::DebugAssetServerPlugin) - with feature `debug_asset_server` /// * [`ScenePlugin`](crate::scene::ScenePlugin) - with feature `bevy_scene` /// * [`WinitPlugin`](crate::winit::WinitPlugin) - with feature `bevy_winit` /// * [`RenderPlugin`](crate::render::RenderPlugin) - with feature `bevy_render` @@ -58,11 +57,6 @@ impl PluginGroup for DefaultPlugins { group = group.add(bevy_asset::AssetPlugin::default()); } - #[cfg(feature = "debug_asset_server")] - { - group = group.add(bevy_asset::debug_asset_server::DebugAssetServerPlugin); - } - #[cfg(feature = "bevy_scene")] { group = group.add(bevy_scene::ScenePlugin); diff --git a/crates/bevy_pbr/src/environment_map/mod.rs b/crates/bevy_pbr/src/environment_map/mod.rs index 7e6e8732028eb..bc3bda5a86c87 100644 --- a/crates/bevy_pbr/src/environment_map/mod.rs +++ b/crates/bevy_pbr/src/environment_map/mod.rs @@ -1,8 +1,8 @@ use bevy_app::{App, Plugin}; -use bevy_asset::{load_internal_asset, Handle, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_core_pipeline::prelude::Camera3d; use bevy_ecs::{prelude::Component, query::With}; -use bevy_reflect::{Reflect, TypeUuid}; +use bevy_reflect::Reflect; use bevy_render::{ extract_component::{ExtractComponent, ExtractComponentPlugin}, render_asset::RenderAssets, @@ -13,8 +13,8 @@ use bevy_render::{ texture::{FallbackImageCubemap, Image}, }; -pub const ENVIRONMENT_MAP_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 154476556247605696); +pub const ENVIRONMENT_MAP_SHADER_HANDLE: Handle = + Handle::weak_from_u128(154476556247605696); pub struct EnvironmentMapPlugin; diff --git a/crates/bevy_pbr/src/lib.rs b/crates/bevy_pbr/src/lib.rs index 33021e8413f93..228de223caa98 100644 --- a/crates/bevy_pbr/src/lib.rs +++ b/crates/bevy_pbr/src/lib.rs @@ -52,9 +52,8 @@ pub mod draw_3d_graph { } use bevy_app::prelude::*; -use bevy_asset::{load_internal_asset, AddAsset, Assets, Handle, HandleUntyped}; +use bevy_asset::{load_internal_asset, AssetApp, Assets, Handle}; use bevy_ecs::prelude::*; -use bevy_reflect::TypeUuid; use bevy_render::{ camera::CameraUpdateSystem, extract_resource::ExtractResourcePlugin, prelude::Color, render_asset::prepare_assets, render_graph::RenderGraph, render_phase::sort_phase_system, @@ -64,28 +63,18 @@ use bevy_render::{ use bevy_transform::TransformSystem; use environment_map::EnvironmentMapPlugin; -pub const PBR_TYPES_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 1708015359337029744); -pub const PBR_BINDINGS_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 5635987986427308186); -pub const UTILS_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 1900548483293416725); -pub const CLUSTERED_FORWARD_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 166852093121196815); -pub const PBR_LIGHTING_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 14170772752254856967); -pub const SHADOWS_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 11350275143789590502); -pub const PBR_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 4805239651767701046); -pub const PBR_PREPASS_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 9407115064344201137); -pub const PBR_FUNCTIONS_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 16550102964439850292); -pub const PBR_AMBIENT_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2441520459096337034); -pub const PARALLAX_MAPPING_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 17035894873630133905); +pub const PBR_TYPES_SHADER_HANDLE: Handle = Handle::weak_from_u128(1708015359337029744); +pub const PBR_BINDINGS_SHADER_HANDLE: Handle = Handle::weak_from_u128(5635987986427308186); +pub const UTILS_HANDLE: Handle = Handle::weak_from_u128(1900548483293416725); +pub const CLUSTERED_FORWARD_HANDLE: Handle = Handle::weak_from_u128(166852093121196815); +pub const PBR_LIGHTING_HANDLE: Handle = Handle::weak_from_u128(14170772752254856967); +pub const SHADOWS_HANDLE: Handle = Handle::weak_from_u128(11350275143789590502); +pub const PBR_SHADER_HANDLE: Handle = Handle::weak_from_u128(4805239651767701046); +pub const PBR_PREPASS_SHADER_HANDLE: Handle = Handle::weak_from_u128(9407115064344201137); +pub const PBR_FUNCTIONS_HANDLE: Handle = Handle::weak_from_u128(16550102964439850292); +pub const PBR_AMBIENT_HANDLE: Handle = Handle::weak_from_u128(2441520459096337034); +pub const PARALLAX_MAPPING_SHADER_HANDLE: Handle = + Handle::weak_from_u128(17035894873630133905); /// Sets up the entire PBR infrastructure of bevy. pub struct PbrPlugin { @@ -247,16 +236,14 @@ impl Plugin for PbrPlugin { ), ); - app.world - .resource_mut::>() - .set_untracked( - Handle::::default(), - StandardMaterial { - base_color: Color::rgb(1.0, 0.0, 0.5), - unlit: true, - ..Default::default() - }, - ); + app.world.resource_mut::>().insert( + Handle::::default(), + StandardMaterial { + base_color: Color::rgb(1.0, 0.0, 0.5), + unlit: true, + ..Default::default() + }, + ); let render_app = match app.get_sub_app_mut(RenderApp) { Ok(render_app) => render_app, diff --git a/crates/bevy_pbr/src/material.rs b/crates/bevy_pbr/src/material.rs index 68361938dec16..8f6c59ce0f7c3 100644 --- a/crates/bevy_pbr/src/material.rs +++ b/crates/bevy_pbr/src/material.rs @@ -4,7 +4,7 @@ use crate::{ SetMeshBindGroup, SetMeshViewBindGroup, Shadow, }; use bevy_app::{App, Plugin}; -use bevy_asset::{AddAsset, AssetEvent, AssetServer, Assets, Handle}; +use bevy_asset::{Asset, AssetApp, AssetEvent, AssetId, AssetServer, Assets, Handle}; use bevy_core_pipeline::{ core_3d::{AlphaMask3d, Opaque3d, Transparent3d}, experimental::taa::TemporalAntiAliasSettings, @@ -19,7 +19,6 @@ use bevy_ecs::{ SystemParamItem, }, }; -use bevy_reflect::{TypePath, TypeUuid}; use bevy_render::{ extract_component::ExtractComponentPlugin, mesh::{Mesh, MeshVertexBufferLayout}, @@ -50,8 +49,6 @@ use std::marker::PhantomData; /// Materials must implement [`AsBindGroup`] to define how data will be transferred to the GPU and bound in shaders. /// [`AsBindGroup`] can be derived, which makes generating bindings straightforward. See the [`AsBindGroup`] docs for details. /// -/// Materials must also implement [`TypeUuid`] so they can be treated as an [`Asset`](bevy_asset::Asset). -/// /// # Example /// /// Here is a simple Material implementation. The [`AsBindGroup`] derive has many features. To see what else is available, @@ -61,10 +58,9 @@ use std::marker::PhantomData; /// # use bevy_ecs::prelude::*; /// # use bevy_reflect::{TypeUuid, TypePath}; /// # use bevy_render::{render_resource::{AsBindGroup, ShaderRef}, texture::Image, color::Color}; -/// # use bevy_asset::{Handle, AssetServer, Assets}; +/// # use bevy_asset::{Handle, AssetServer, Assets, Asset}; /// -/// #[derive(AsBindGroup, TypeUuid, TypePath, Debug, Clone)] -/// #[uuid = "f690fdae-d598-45ab-8225-97e2a3f056e0"] +/// #[derive(AsBindGroup, Debug, Clone, Asset, TypePath)] /// pub struct CustomMaterial { /// // Uniform bindings must implement `ShaderType`, which will be used to convert the value to /// // its shader-compatible equivalent. Most core math types already implement `ShaderType`. @@ -106,7 +102,7 @@ use std::marker::PhantomData; /// @group(1) @binding(2) /// var color_sampler: sampler; /// ``` -pub trait Material: AsBindGroup + Send + Sync + Clone + TypeUuid + TypePath + Sized { +pub trait Material: Asset + AsBindGroup + Clone + Sized { /// Returns this material's vertex shader. If [`ShaderRef::Default`] is returned, the default mesh vertex shader /// will be used. fn vertex_shader() -> ShaderRef { @@ -187,7 +183,7 @@ where M::Data: PartialEq + Eq + Hash + Clone, { fn build(&self, app: &mut App) { - app.add_asset::() + app.init_asset::() .add_plugins(ExtractComponentPlugin::>::extract_visible()); if let Ok(render_app) = app.get_sub_app_mut(RenderApp) { @@ -366,7 +362,7 @@ impl RenderCommand

for SetMaterial materials: SystemParamItem<'w, '_, Self::Param>, pass: &mut TrackedRenderPass<'w>, ) -> RenderCommandResult { - let material = materials.into_inner().get(material_handle).unwrap(); + let material = materials.into_inner().get(&material_handle.id()).unwrap(); pass.set_bind_group(I, &material.bind_group, &[]); RenderCommandResult::Success } @@ -472,7 +468,7 @@ pub fn queue_material_meshes( { if let (Some(mesh), Some(material)) = ( render_meshes.get(mesh_handle), - render_materials.get(material_handle), + render_materials.get(&material_handle.id()), ) { let mut mesh_key = MeshPipelineKey::from_primitive_topology(mesh.primitive_topology) @@ -576,8 +572,8 @@ pub struct PreparedMaterial { #[derive(Resource)] pub struct ExtractedMaterials { - extracted: Vec<(Handle, M)>, - removed: Vec>, + extracted: Vec<(AssetId, M)>, + removed: Vec>, } impl Default for ExtractedMaterials { @@ -591,7 +587,7 @@ impl Default for ExtractedMaterials { /// Stores all prepared representations of [`Material`] assets for as long as they exist. #[derive(Resource, Deref, DerefMut)] -pub struct RenderMaterials(pub HashMap, PreparedMaterial>); +pub struct RenderMaterials(pub HashMap, PreparedMaterial>); impl Default for RenderMaterials { fn default() -> Self { @@ -610,20 +606,23 @@ pub fn extract_materials( let mut removed = Vec::new(); for event in events.read() { match event { - AssetEvent::Created { handle } | AssetEvent::Modified { handle } => { - changed_assets.insert(handle.clone_weak()); + AssetEvent::Added { id } | AssetEvent::Modified { id } => { + changed_assets.insert(*id); + } + AssetEvent::Removed { id } => { + changed_assets.remove(id); + removed.push(*id); } - AssetEvent::Removed { handle } => { - changed_assets.remove(handle); - removed.push(handle.clone_weak()); + AssetEvent::LoadedWithDependencies { .. } => { + // TODO: handle this } } } let mut extracted_assets = Vec::new(); - for handle in changed_assets.drain() { - if let Some(asset) = assets.get(&handle) { - extracted_assets.push((handle, asset.clone())); + for id in changed_assets.drain() { + if let Some(asset) = assets.get(id) { + extracted_assets.push((id, asset.clone())); } } @@ -635,7 +634,7 @@ pub fn extract_materials( /// All [`Material`] values of a given type that should be prepared next frame. pub struct PrepareNextFrameMaterials { - assets: Vec<(Handle, M)>, + assets: Vec<(AssetId, M)>, } impl Default for PrepareNextFrameMaterials { @@ -658,7 +657,7 @@ pub fn prepare_materials( pipeline: Res>, ) { let queued_assets = std::mem::take(&mut prepare_next_frame.assets); - for (handle, material) in queued_assets.into_iter() { + for (id, material) in queued_assets.into_iter() { match prepare_material( &material, &render_device, @@ -667,10 +666,10 @@ pub fn prepare_materials( &pipeline, ) { Ok(prepared_asset) => { - render_materials.insert(handle, prepared_asset); + render_materials.insert(id, prepared_asset); } Err(AsBindGroupError::RetryNextUpdate) => { - prepare_next_frame.assets.push((handle, material)); + prepare_next_frame.assets.push((id, material)); } } } @@ -679,7 +678,7 @@ pub fn prepare_materials( render_materials.remove(&removed); } - for (handle, material) in std::mem::take(&mut extracted_assets.extracted) { + for (id, material) in std::mem::take(&mut extracted_assets.extracted) { match prepare_material( &material, &render_device, @@ -688,10 +687,10 @@ pub fn prepare_materials( &pipeline, ) { Ok(prepared_asset) => { - render_materials.insert(handle, prepared_asset); + render_materials.insert(id, prepared_asset); } Err(AsBindGroupError::RetryNextUpdate) => { - prepare_next_frame.assets.push((handle, material)); + prepare_next_frame.assets.push((id, material)); } } } diff --git a/crates/bevy_pbr/src/pbr_material.rs b/crates/bevy_pbr/src/pbr_material.rs index 8e3520160d1f1..f979765c5d471 100644 --- a/crates/bevy_pbr/src/pbr_material.rs +++ b/crates/bevy_pbr/src/pbr_material.rs @@ -2,9 +2,9 @@ use crate::{ AlphaMode, Material, MaterialPipeline, MaterialPipelineKey, ParallaxMappingMethod, PBR_PREPASS_SHADER_HANDLE, PBR_SHADER_HANDLE, }; -use bevy_asset::Handle; +use bevy_asset::{Asset, Handle}; use bevy_math::Vec4; -use bevy_reflect::{std_traits::ReflectDefault, Reflect, TypeUuid}; +use bevy_reflect::{std_traits::ReflectDefault, Reflect}; use bevy_render::{ color::Color, mesh::MeshVertexBufferLayout, render_asset::RenderAssets, render_resource::*, texture::Image, @@ -15,8 +15,7 @@ use bevy_render::{ /// . /// /// May be created directly from a [`Color`] or an [`Image`]. -#[derive(AsBindGroup, Reflect, Debug, Clone, TypeUuid)] -#[uuid = "7494888b-c082-457b-aacf-517228cc0c22"] +#[derive(Asset, AsBindGroup, Reflect, Debug, Clone)] #[bind_group_data(StandardMaterialKey)] #[uniform(0, StandardMaterialUniform)] #[reflect(Default, Debug)] @@ -45,6 +44,7 @@ pub struct StandardMaterial { /// [`base_color`]: StandardMaterial::base_color #[texture(1)] #[sampler(2)] + #[dependency] pub base_color_texture: Option>, // Use a color for user friendliness even though we technically don't use the alpha channel @@ -74,6 +74,7 @@ pub struct StandardMaterial { /// [`emissive`]: StandardMaterial::emissive #[texture(3)] #[sampler(4)] + #[dependency] pub emissive_texture: Option>, /// Linear perceptual roughness, clamped to `[0.089, 1.0]` in the shader. @@ -122,6 +123,7 @@ pub struct StandardMaterial { /// [`perceptual_roughness`]: StandardMaterial::perceptual_roughness #[texture(5)] #[sampler(6)] + #[dependency] pub metallic_roughness_texture: Option>, /// Specular intensity for non-metals on a linear scale of `[0.0, 1.0]`. @@ -157,6 +159,7 @@ pub struct StandardMaterial { /// [`Mesh::generate_tangents`]: bevy_render::mesh::Mesh::generate_tangents #[texture(9)] #[sampler(10)] + #[dependency] pub normal_map_texture: Option>, /// Normal map textures authored for DirectX have their y-component flipped. Set this to flip @@ -175,6 +178,7 @@ pub struct StandardMaterial { /// This is similar to ambient occlusion, but built into the model. #[texture(7)] #[sampler(8)] + #[dependency] pub occlusion_texture: Option>, /// Support two-sided lighting by automatically flipping the normals for "back" faces @@ -278,6 +282,7 @@ pub struct StandardMaterial { /// [`max_parallax_layer_count`]: StandardMaterial::max_parallax_layer_count #[texture(11)] #[sampler(12)] + #[dependency] pub depth_map: Option>, /// How deep the offset introduced by the depth map should be. @@ -465,7 +470,8 @@ impl AsBindGroupShaderType for StandardMaterial { } let has_normal_map = self.normal_map_texture.is_some(); if has_normal_map { - if let Some(texture) = images.get(self.normal_map_texture.as_ref().unwrap()) { + let normal_map_id = self.normal_map_texture.as_ref().map(|h| h.id()).unwrap(); + if let Some(texture) = images.get(normal_map_id) { match texture.texture_format { // All 2-component unorm formats TextureFormat::Rg8Unorm @@ -561,11 +567,11 @@ impl Material for StandardMaterial { } fn prepass_fragment_shader() -> ShaderRef { - PBR_PREPASS_SHADER_HANDLE.typed().into() + PBR_PREPASS_SHADER_HANDLE.into() } fn fragment_shader() -> ShaderRef { - PBR_SHADER_HANDLE.typed().into() + PBR_SHADER_HANDLE.into() } #[inline] diff --git a/crates/bevy_pbr/src/prepass/mod.rs b/crates/bevy_pbr/src/prepass/mod.rs index 21ea58e57f8f6..3c9bdd7ddb4ec 100644 --- a/crates/bevy_pbr/src/prepass/mod.rs +++ b/crates/bevy_pbr/src/prepass/mod.rs @@ -1,5 +1,5 @@ use bevy_app::{Plugin, PreUpdate}; -use bevy_asset::{load_internal_asset, AssetServer, Handle, HandleUntyped}; +use bevy_asset::{load_internal_asset, AssetServer, Handle}; use bevy_core_pipeline::{ prelude::Camera3d, prepass::{ @@ -16,7 +16,6 @@ use bevy_ecs::{ }, }; use bevy_math::{Affine3A, Mat4}; -use bevy_reflect::TypeUuid; use bevy_render::{ globals::{GlobalsBuffer, GlobalsUniform}, mesh::MeshVertexBufferLayout, @@ -52,14 +51,12 @@ use crate::{ use std::{hash::Hash, marker::PhantomData}; -pub const PREPASS_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 921124473254008983); +pub const PREPASS_SHADER_HANDLE: Handle = Handle::weak_from_u128(921124473254008983); -pub const PREPASS_BINDINGS_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 5533152893177403494); +pub const PREPASS_BINDINGS_SHADER_HANDLE: Handle = + Handle::weak_from_u128(5533152893177403494); -pub const PREPASS_UTILS_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 4603948296044544); +pub const PREPASS_UTILS_SHADER_HANDLE: Handle = Handle::weak_from_u128(4603948296044544); /// Sets up everything required to use the prepass pipeline. /// @@ -463,7 +460,7 @@ where // Use the fragment shader from the material let frag_shader_handle = match self.material_fragment_shader.clone() { Some(frag_shader_handle) => frag_shader_handle, - _ => PREPASS_SHADER_HANDLE.typed::(), + _ => PREPASS_SHADER_HANDLE, }; FragmentState { @@ -478,7 +475,7 @@ where let vert_shader_handle = if let Some(handle) = &self.material_vertex_shader { handle.clone() } else { - PREPASS_SHADER_HANDLE.typed::() + PREPASS_SHADER_HANDLE }; let mut push_constant_ranges = Vec::with_capacity(1); @@ -805,7 +802,7 @@ pub fn queue_prepass_material_meshes( }; let (Some(material), Some(mesh)) = ( - render_materials.get(material_handle), + render_materials.get(&material_handle.id()), render_meshes.get(mesh_handle), ) else { continue; diff --git a/crates/bevy_pbr/src/render/fog.rs b/crates/bevy_pbr/src/render/fog.rs index 3cf9d4528af12..10fb61ff2bcc9 100644 --- a/crates/bevy_pbr/src/render/fog.rs +++ b/crates/bevy_pbr/src/render/fog.rs @@ -1,8 +1,7 @@ use bevy_app::{App, Plugin}; -use bevy_asset::{load_internal_asset, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_ecs::prelude::*; use bevy_math::{Vec3, Vec4}; -use bevy_reflect::TypeUuid; use bevy_render::{ extract_component::ExtractComponentPlugin, render_resource::{DynamicUniformBuffer, Shader, ShaderType}, @@ -121,8 +120,7 @@ pub struct ViewFogUniformOffset { } /// Handle for the fog WGSL Shader internal asset -pub const FOG_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 4913569193382610166); +pub const FOG_SHADER_HANDLE: Handle = Handle::weak_from_u128(4913569193382610166); /// A plugin that consolidates fog extraction, preparation and related resources/assets pub struct FogPlugin; diff --git a/crates/bevy_pbr/src/render/light.rs b/crates/bevy_pbr/src/render/light.rs index e18e9d4acae4a..4e708e454a912 100644 --- a/crates/bevy_pbr/src/render/light.rs +++ b/crates/bevy_pbr/src/render/light.rs @@ -1597,7 +1597,7 @@ pub fn queue_shadows( if let Ok((mesh_handle, material_handle)) = casting_meshes.get(entity) { if let (Some(mesh), Some(material)) = ( render_meshes.get(mesh_handle), - render_materials.get(material_handle), + render_materials.get(&material_handle.id()), ) { let mut mesh_key = MeshPipelineKey::from_primitive_topology(mesh.primitive_topology) diff --git a/crates/bevy_pbr/src/render/mesh.rs b/crates/bevy_pbr/src/render/mesh.rs index e9abd14609851..a8127d0ce59ea 100644 --- a/crates/bevy_pbr/src/render/mesh.rs +++ b/crates/bevy_pbr/src/render/mesh.rs @@ -6,7 +6,7 @@ use crate::{ CLUSTERED_FORWARD_STORAGE_BUFFER_COUNT, MAX_CASCADES_PER_LIGHT, MAX_DIRECTIONAL_LIGHTS, }; use bevy_app::Plugin; -use bevy_asset::{load_internal_asset, Assets, Handle, HandleId, HandleUntyped}; +use bevy_asset::{load_internal_asset, AssetId, Assets, Handle}; use bevy_core_pipeline::{ core_3d::{AlphaMask3d, Opaque3d, Transparent3d}, prepass::ViewPrepassTextures, @@ -20,7 +20,6 @@ use bevy_ecs::{ system::{lifetimeless::*, SystemParamItem, SystemState}, }; use bevy_math::{Affine3, Affine3A, Mat4, Vec2, Vec3Swizzles, Vec4}; -use bevy_reflect::TypeUuid; use bevy_render::{ globals::{GlobalsBuffer, GlobalsUniform}, mesh::{ @@ -57,24 +56,15 @@ pub const MAX_JOINTS: usize = 256; const JOINT_SIZE: usize = std::mem::size_of::(); pub(crate) const JOINT_BUFFER_SIZE: usize = MAX_JOINTS * JOINT_SIZE; -pub const MESH_VERTEX_OUTPUT: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2645551199423808407); -pub const MESH_VIEW_TYPES_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 8140454348013264787); -pub const MESH_VIEW_BINDINGS_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 9076678235888822571); -pub const MESH_TYPES_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2506024101911992377); -pub const MESH_BINDINGS_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 16831548636314682308); -pub const MESH_FUNCTIONS_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 6300874327833745635); -pub const MESH_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 3252377289100772450); -pub const SKINNING_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 13215291596265391738); -pub const MORPH_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 970982813587607345); +pub const MESH_VERTEX_OUTPUT: Handle = Handle::weak_from_u128(2645551199423808407); +pub const MESH_VIEW_TYPES_HANDLE: Handle = Handle::weak_from_u128(8140454348013264787); +pub const MESH_VIEW_BINDINGS_HANDLE: Handle = Handle::weak_from_u128(9076678235888822571); +pub const MESH_TYPES_HANDLE: Handle = Handle::weak_from_u128(2506024101911992377); +pub const MESH_BINDINGS_HANDLE: Handle = Handle::weak_from_u128(16831548636314682308); +pub const MESH_FUNCTIONS_HANDLE: Handle = Handle::weak_from_u128(6300874327833745635); +pub const MESH_SHADER_HANDLE: Handle = Handle::weak_from_u128(3252377289100772450); +pub const SKINNING_HANDLE: Handle = Handle::weak_from_u128(13215291596265391738); +pub const MORPH_HANDLE: Handle = Handle::weak_from_u128(970982813587607345); impl Plugin for MeshRenderPlugin { fn build(&self, app: &mut bevy_app::App) { @@ -1025,13 +1015,13 @@ impl SpecializedMeshPipeline for MeshPipeline { Ok(RenderPipelineDescriptor { vertex: VertexState { - shader: MESH_SHADER_HANDLE.typed::(), + shader: MESH_SHADER_HANDLE, entry_point: "vertex".into(), shader_defs: shader_defs.clone(), buffers: vec![vertex_buffer_layout], }, fragment: Some(FragmentState { - shader: MESH_SHADER_HANDLE.typed::(), + shader: MESH_SHADER_HANDLE, shader_defs, entry_point: "fragment".into(), targets: vec![Some(ColorTargetState { @@ -1082,7 +1072,7 @@ impl SpecializedMeshPipeline for MeshPipeline { pub struct MeshBindGroups { model_only: Option, skinned: Option, - morph_targets: HashMap, + morph_targets: HashMap, BindGroup>, } impl MeshBindGroups { pub fn reset(&mut self) { @@ -1091,7 +1081,12 @@ impl MeshBindGroups { self.morph_targets.clear(); } /// Get the `BindGroup` for `GpuMesh` with given `handle_id`. - pub fn get(&self, handle_id: HandleId, is_skinned: bool, morph: bool) -> Option<&BindGroup> { + pub fn get( + &self, + handle_id: AssetId, + is_skinned: bool, + morph: bool, + ) -> Option<&BindGroup> { match (is_skinned, morph) { (_, true) => self.morph_targets.get(&handle_id), (true, false) => self.skinned.as_ref(), @@ -1129,7 +1124,7 @@ pub fn prepare_mesh_bind_group( } else { layouts.morphed(&render_device, &model, weights, targets) }; - groups.morph_targets.insert(id.id(), group); + groups.morph_targets.insert(id, group); } } } diff --git a/crates/bevy_pbr/src/ssao/mod.rs b/crates/bevy_pbr/src/ssao/mod.rs index 30d8c07141dbd..3a31c0afbe620 100644 --- a/crates/bevy_pbr/src/ssao/mod.rs +++ b/crates/bevy_pbr/src/ssao/mod.rs @@ -1,5 +1,5 @@ use bevy_app::{App, Plugin}; -use bevy_asset::{load_internal_asset, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_core_pipeline::{ core_3d::CORE_3D, prelude::Camera3d, @@ -13,7 +13,7 @@ use bevy_ecs::{ system::{Commands, Query, Res, ResMut, Resource}, world::{FromWorld, World}, }; -use bevy_reflect::{Reflect, TypeUuid}; +use bevy_reflect::Reflect; use bevy_render::{ camera::{ExtractedCamera, TemporalJitter}, extract_component::ExtractComponent, @@ -48,14 +48,10 @@ pub mod draw_3d_graph { } } -const PREPROCESS_DEPTH_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 102258915420479); -const GTAO_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 253938746510568); -const SPATIAL_DENOISE_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 466162052558226); -const GTAO_UTILS_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 366465052568786); +const PREPROCESS_DEPTH_SHADER_HANDLE: Handle = Handle::weak_from_u128(102258915420479); +const GTAO_SHADER_HANDLE: Handle = Handle::weak_from_u128(253938746510568); +const SPATIAL_DENOISE_SHADER_HANDLE: Handle = Handle::weak_from_u128(466162052558226); +const GTAO_UTILS_SHADER_HANDLE: Handle = Handle::weak_from_u128(366465052568786); /// Plugin for screen space ambient occlusion. pub struct ScreenSpaceAmbientOcclusionPlugin; @@ -537,7 +533,7 @@ impl FromWorld for SsaoPipelines { common_bind_group_layout.clone(), ], push_constant_ranges: vec![], - shader: PREPROCESS_DEPTH_SHADER_HANDLE.typed(), + shader: PREPROCESS_DEPTH_SHADER_HANDLE, shader_defs: Vec::new(), entry_point: "preprocess_depth".into(), }); @@ -550,7 +546,7 @@ impl FromWorld for SsaoPipelines { common_bind_group_layout.clone(), ], push_constant_ranges: vec![], - shader: SPATIAL_DENOISE_SHADER_HANDLE.typed(), + shader: SPATIAL_DENOISE_SHADER_HANDLE, shader_defs: Vec::new(), entry_point: "spatial_denoise".into(), }); @@ -601,7 +597,7 @@ impl SpecializedComputePipeline for SsaoPipelines { self.common_bind_group_layout.clone(), ], push_constant_ranges: vec![], - shader: GTAO_SHADER_HANDLE.typed(), + shader: GTAO_SHADER_HANDLE, shader_defs, entry_point: "gtao".into(), } diff --git a/crates/bevy_pbr/src/wireframe.rs b/crates/bevy_pbr/src/wireframe.rs index e227bfdb7d4d6..58dd5ada64251 100644 --- a/crates/bevy_pbr/src/wireframe.rs +++ b/crates/bevy_pbr/src/wireframe.rs @@ -1,11 +1,11 @@ use crate::{DrawMesh, MeshPipelineKey, SetMeshBindGroup, SetMeshViewBindGroup}; use crate::{MeshPipeline, MeshTransforms}; use bevy_app::Plugin; -use bevy_asset::{load_internal_asset, Handle, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_core_pipeline::core_3d::Opaque3d; use bevy_ecs::{prelude::*, reflect::ReflectComponent}; use bevy_reflect::std_traits::ReflectDefault; -use bevy_reflect::{Reflect, TypeUuid}; +use bevy_reflect::Reflect; use bevy_render::extract_component::{ExtractComponent, ExtractComponentPlugin}; use bevy_render::Render; use bevy_render::{ @@ -22,8 +22,7 @@ use bevy_render::{ }; use bevy_utils::tracing::error; -pub const WIREFRAME_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 192598014480025766); +pub const WIREFRAME_SHADER_HANDLE: Handle = Handle::weak_from_u128(192598014480025766); #[derive(Debug, Default)] pub struct WireframePlugin; @@ -81,7 +80,7 @@ impl FromWorld for WireframePipeline { fn from_world(render_world: &mut World) -> Self { WireframePipeline { mesh_pipeline: render_world.resource::().clone(), - shader: WIREFRAME_SHADER_HANDLE.typed(), + shader: WIREFRAME_SHADER_HANDLE, } } } diff --git a/crates/bevy_reflect/src/impls/std.rs b/crates/bevy_reflect/src/impls/std.rs index 72c79bb76511c..0f594a7f81235 100644 --- a/crates/bevy_reflect/src/impls/std.rs +++ b/crates/bevy_reflect/src/impls/std.rs @@ -199,6 +199,7 @@ impl_reflect_value!(::core::num::NonZeroI8( Serialize, Deserialize )); +impl_reflect_value!(::std::sync::Arc); // `Serialize` and `Deserialize` only for platforms supported by serde: // https://github.com/serde-rs/serde/blob/3ffb86fc70efd3d329519e2dddfa306cc04f167c/serde/src/de/impls.rs#L1732 diff --git a/crates/bevy_render/Cargo.toml b/crates/bevy_render/Cargo.toml index dd2f90aef4683..32573be7b9a11 100644 --- a/crates/bevy_render/Cargo.toml +++ b/crates/bevy_render/Cargo.toml @@ -68,7 +68,6 @@ downcast-rs = "1.2.0" thread_local = "1.1" thiserror = "1.0" futures-lite = "1.4.0" -anyhow = "1.0" hexasphere = "9.0" parking_lot = "0.12.1" ddsfile = { version = "0.5.0", optional = true } diff --git a/crates/bevy_render/src/camera/camera.rs b/crates/bevy_render/src/camera/camera.rs index 5fbf037d1fe1d..4403eb76308f9 100644 --- a/crates/bevy_render/src/camera/camera.rs +++ b/crates/bevy_render/src/camera/camera.rs @@ -7,7 +7,7 @@ use crate::{ view::{ColorGrading, ExtractedView, ExtractedWindows, RenderLayers, VisibleEntities}, Extract, }; -use bevy_asset::{AssetEvent, Assets, Handle}; +use bevy_asset::{AssetEvent, AssetId, Assets, Handle}; use bevy_derive::{Deref, DerefMut}; use bevy_ecs::{ change_detection::DetectChanges, @@ -528,14 +528,14 @@ impl NormalizedRenderTarget { fn is_changed( &self, changed_window_ids: &HashSet, - changed_image_handles: &HashSet<&Handle>, + changed_image_handles: &HashSet<&AssetId>, ) -> bool { match self { NormalizedRenderTarget::Window(window_ref) => { changed_window_ids.contains(&window_ref.entity()) } NormalizedRenderTarget::Image(image_handle) => { - changed_image_handles.contains(&image_handle) + changed_image_handles.contains(&image_handle.id()) } NormalizedRenderTarget::TextureView(_) => true, } @@ -578,11 +578,11 @@ pub fn camera_system( changed_window_ids.extend(window_created_events.read().map(|event| event.window)); changed_window_ids.extend(window_resized_events.read().map(|event| event.window)); - let changed_image_handles: HashSet<&Handle> = image_asset_events + let changed_image_handles: HashSet<&AssetId> = image_asset_events .read() .filter_map(|event| { - if let AssetEvent::Modified { handle } = event { - Some(handle) + if let AssetEvent::Modified { id } = event { + Some(id) } else { None } diff --git a/crates/bevy_render/src/globals.rs b/crates/bevy_render/src/globals.rs index 6d49d09373688..3ceed24476c75 100644 --- a/crates/bevy_render/src/globals.rs +++ b/crates/bevy_render/src/globals.rs @@ -6,14 +6,13 @@ use crate::{ Extract, ExtractSchedule, Render, RenderApp, RenderSet, }; use bevy_app::{App, Plugin}; -use bevy_asset::{load_internal_asset, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_core::FrameCount; use bevy_ecs::prelude::*; -use bevy_reflect::{Reflect, TypeUuid}; +use bevy_reflect::Reflect; use bevy_time::Time; -pub const GLOBALS_TYPE_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 17924628719070609599); +pub const GLOBALS_TYPE_HANDLE: Handle = Handle::weak_from_u128(17924628719070609599); pub struct GlobalsPlugin; diff --git a/crates/bevy_render/src/lib.rs b/crates/bevy_render/src/lib.rs index 8c8251a95ca02..2f36d3a1cf8a6 100644 --- a/crates/bevy_render/src/lib.rs +++ b/crates/bevy_render/src/lib.rs @@ -24,11 +24,6 @@ pub mod settings; mod spatial_bundle; pub mod texture; pub mod view; - -use bevy_hierarchy::ValidParentCheckPlugin; -use bevy_reflect::TypeUuid; -pub use extract_param::Extract; - pub mod prelude { #[doc(hidden)] pub use crate::{ @@ -43,6 +38,9 @@ pub mod prelude { }; } +pub use extract_param::Extract; + +use bevy_hierarchy::ValidParentCheckPlugin; use bevy_window::{PrimaryWindow, RawHandleWrapper}; use globals::GlobalsPlugin; use renderer::{RenderAdapter, RenderAdapterInfo, RenderDevice, RenderQueue}; @@ -58,7 +56,7 @@ use crate::{ view::{ViewPlugin, WindowRenderPlugin}, }; use bevy_app::{App, AppLabel, Plugin, SubApp}; -use bevy_asset::{load_internal_asset, AddAsset, AssetServer, HandleUntyped}; +use bevy_asset::{load_internal_asset, AssetApp, AssetServer, Handle}; use bevy_ecs::{prelude::*, schedule::ScheduleLabel, system::SystemState}; use bevy_utils::tracing::debug; use std::{ @@ -232,18 +230,15 @@ struct FutureRendererResources( #[derive(Debug, Clone, Copy, Hash, PartialEq, Eq, AppLabel)] pub struct RenderApp; -pub const INSTANCE_INDEX_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 10313207077636615845); -pub const MATHS_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 10665356303104593376); +pub const INSTANCE_INDEX_SHADER_HANDLE: Handle = + Handle::weak_from_u128(10313207077636615845); +pub const MATHS_SHADER_HANDLE: Handle = Handle::weak_from_u128(10665356303104593376); impl Plugin for RenderPlugin { /// Initializes the renderer, sets up the [`RenderSet`](RenderSet) and creates the rendering sub-app. fn build(&self, app: &mut App) { - app.add_asset::() - .add_debug_asset::() - .init_asset_loader::() - .init_debug_asset_loader::(); + app.init_asset::() + .init_asset_loader::(); if let Some(backends) = self.wgpu_settings.backends { let future_renderer_resources_wrapper = Arc::new(Mutex::new(None)); diff --git a/crates/bevy_render/src/mesh/mesh/mod.rs b/crates/bevy_render/src/mesh/mesh/mod.rs index 50e75c6893d24..4e711cf29b250 100644 --- a/crates/bevy_render/src/mesh/mesh/mod.rs +++ b/crates/bevy_render/src/mesh/mesh/mod.rs @@ -1,6 +1,5 @@ mod conversions; pub mod skinning; -use bevy_log::warn; pub use wgpu::PrimitiveTopology; use crate::{ @@ -10,12 +9,13 @@ use crate::{ render_resource::{Buffer, TextureView, VertexBufferLayout}, renderer::RenderDevice, }; -use bevy_asset::Handle; +use bevy_asset::{Asset, Handle}; use bevy_core::cast_slice; use bevy_derive::EnumVariantMeta; use bevy_ecs::system::{lifetimeless::SRes, SystemParamItem}; +use bevy_log::warn; use bevy_math::*; -use bevy_reflect::{TypePath, TypeUuid}; +use bevy_reflect::TypePath; use bevy_utils::{tracing::error, Hashed}; use std::{collections::BTreeMap, hash::Hash, iter::FusedIterator}; use thiserror::Error; @@ -110,8 +110,7 @@ pub const VERTEX_ATTRIBUTE_BUFFER_ID: u64 = 10; /// is the side of the triangle from where the vertices appear in a *counter-clockwise* order. /// // TODO: allow values to be unloaded after been submitting to the GPU to conserve memory -#[derive(Debug, TypeUuid, TypePath, Clone)] -#[uuid = "8ecbac0f-f545-4473-ad43-e1f4243af51e"] +#[derive(Asset, Debug, TypePath, Clone)] pub struct Mesh { primitive_topology: PrimitiveTopology, /// `std::collections::BTreeMap` with all defined vertex attributes (Positions, Normals, ...) diff --git a/crates/bevy_render/src/mesh/mesh/skinning.rs b/crates/bevy_render/src/mesh/mesh/skinning.rs index 20df623c5a899..e5c4cd9fcc925 100644 --- a/crates/bevy_render/src/mesh/mesh/skinning.rs +++ b/crates/bevy_render/src/mesh/mesh/skinning.rs @@ -1,4 +1,4 @@ -use bevy_asset::Handle; +use bevy_asset::{Asset, Handle}; use bevy_ecs::{ component::Component, entity::{Entity, EntityMapper, MapEntities}, @@ -6,7 +6,7 @@ use bevy_ecs::{ reflect::ReflectMapEntities, }; use bevy_math::Mat4; -use bevy_reflect::{Reflect, TypePath, TypeUuid}; +use bevy_reflect::{Reflect, TypePath}; use std::ops::Deref; #[derive(Component, Debug, Default, Clone, Reflect)] @@ -24,8 +24,7 @@ impl MapEntities for SkinnedMesh { } } -#[derive(Debug, TypeUuid, TypePath)] -#[uuid = "b9f155a9-54ec-4026-988f-e0a03e99a76f"] +#[derive(Asset, TypePath, Debug)] pub struct SkinnedMeshInverseBindposes(Box<[Mat4]>); impl From> for SkinnedMeshInverseBindposes { diff --git a/crates/bevy_render/src/mesh/mod.rs b/crates/bevy_render/src/mesh/mod.rs index 0d9efccf029d6..841328403233a 100644 --- a/crates/bevy_render/src/mesh/mod.rs +++ b/crates/bevy_render/src/mesh/mod.rs @@ -8,7 +8,7 @@ pub use mesh::*; use crate::{prelude::Image, render_asset::RenderAssetPlugin}; use bevy_app::{App, Plugin}; -use bevy_asset::AddAsset; +use bevy_asset::AssetApp; use bevy_ecs::entity::Entity; /// Adds the [`Mesh`] as an asset and makes sure that they are extracted and prepared for the GPU. @@ -16,8 +16,8 @@ pub struct MeshPlugin; impl Plugin for MeshPlugin { fn build(&self, app: &mut App) { - app.add_asset::() - .add_asset::() + app.init_asset::() + .init_asset::() .register_type::() .register_type::>() // 'Mesh' must be prepared after 'Image' as meshes rely on the morph target image being ready diff --git a/crates/bevy_render/src/render_asset.rs b/crates/bevy_render/src/render_asset.rs index 0b6f17048fe36..266a3e148c2d4 100644 --- a/crates/bevy_render/src/render_asset.rs +++ b/crates/bevy_render/src/render_asset.rs @@ -1,7 +1,6 @@ use crate::{Extract, ExtractSchedule, Render, RenderApp, RenderSet}; use bevy_app::{App, Plugin}; -use bevy_asset::{Asset, AssetEvent, Assets, Handle}; -use bevy_derive::{Deref, DerefMut}; +use bevy_asset::{Asset, AssetEvent, AssetId, Assets}; use bevy_ecs::{ prelude::*, schedule::SystemConfigs, @@ -103,8 +102,8 @@ impl RenderAssetDependency for A { /// Temporarily stores the extracted and removed assets of the current frame. #[derive(Resource)] pub struct ExtractedAssets { - extracted: Vec<(Handle, A::ExtractedAsset)>, - removed: Vec>, + extracted: Vec<(AssetId, A::ExtractedAsset)>, + removed: Vec>, } impl Default for ExtractedAssets { @@ -118,8 +117,8 @@ impl Default for ExtractedAssets { /// Stores all GPU representations ([`RenderAsset::PreparedAssets`](RenderAsset::PreparedAsset)) /// of [`RenderAssets`](RenderAsset) as long as they exist. -#[derive(Resource, Deref, DerefMut)] -pub struct RenderAssets(HashMap, A::PreparedAsset>); +#[derive(Resource)] +pub struct RenderAssets(HashMap, A::PreparedAsset>); impl Default for RenderAssets { fn default() -> Self { @@ -127,6 +126,36 @@ impl Default for RenderAssets { } } +impl RenderAssets { + pub fn get(&self, id: impl Into>) -> Option<&A::PreparedAsset> { + self.0.get(&id.into()) + } + + pub fn get_mut(&mut self, id: impl Into>) -> Option<&mut A::PreparedAsset> { + self.0.get_mut(&id.into()) + } + + pub fn insert( + &mut self, + id: impl Into>, + value: A::PreparedAsset, + ) -> Option { + self.0.insert(id.into(), value) + } + + pub fn remove(&mut self, id: impl Into>) -> Option { + self.0.remove(&id.into()) + } + + pub fn iter(&self) -> impl Iterator, &A::PreparedAsset)> { + self.0.iter().map(|(k, v)| (*k, v)) + } + + pub fn iter_mut(&mut self) -> impl Iterator, &mut A::PreparedAsset)> { + self.0.iter_mut().map(|(k, v)| (*k, v)) + } +} + /// This system extracts all crated or modified assets of the corresponding [`RenderAsset`] type /// into the "render world". fn extract_render_asset( @@ -138,20 +167,23 @@ fn extract_render_asset( let mut removed = Vec::new(); for event in events.read() { match event { - AssetEvent::Created { handle } | AssetEvent::Modified { handle } => { - changed_assets.insert(handle.clone_weak()); + AssetEvent::Added { id } | AssetEvent::Modified { id } => { + changed_assets.insert(*id); + } + AssetEvent::Removed { id } => { + changed_assets.remove(id); + removed.push(*id); } - AssetEvent::Removed { handle } => { - changed_assets.remove(handle); - removed.push(handle.clone_weak()); + AssetEvent::LoadedWithDependencies { .. } => { + // TODO: handle this } } } let mut extracted_assets = Vec::new(); - for handle in changed_assets.drain() { - if let Some(asset) = assets.get(&handle) { - extracted_assets.push((handle, asset.extract_asset())); + for id in changed_assets.drain() { + if let Some(asset) = assets.get(id) { + extracted_assets.push((id, asset.extract_asset())); } } @@ -165,7 +197,7 @@ fn extract_render_asset( /// All assets that should be prepared next frame. #[derive(Resource)] pub struct PrepareNextFrameAssets { - assets: Vec<(Handle, A::ExtractedAsset)>, + assets: Vec<(AssetId, A::ExtractedAsset)>, } impl Default for PrepareNextFrameAssets { @@ -186,28 +218,28 @@ pub fn prepare_assets( ) { let mut param = param.into_inner(); let queued_assets = std::mem::take(&mut prepare_next_frame.assets); - for (handle, extracted_asset) in queued_assets { + for (id, extracted_asset) in queued_assets { match R::prepare_asset(extracted_asset, &mut param) { Ok(prepared_asset) => { - render_assets.insert(handle, prepared_asset); + render_assets.insert(id, prepared_asset); } Err(PrepareAssetError::RetryNextUpdate(extracted_asset)) => { - prepare_next_frame.assets.push((handle, extracted_asset)); + prepare_next_frame.assets.push((id, extracted_asset)); } } } for removed in std::mem::take(&mut extracted_assets.removed) { - render_assets.remove(&removed); + render_assets.remove(removed); } - for (handle, extracted_asset) in std::mem::take(&mut extracted_assets.extracted) { + for (id, extracted_asset) in std::mem::take(&mut extracted_assets.extracted) { match R::prepare_asset(extracted_asset, &mut param) { Ok(prepared_asset) => { - render_assets.insert(handle, prepared_asset); + render_assets.insert(id, prepared_asset); } Err(PrepareAssetError::RetryNextUpdate(extracted_asset)) => { - prepare_next_frame.assets.push((handle, extracted_asset)); + prepare_next_frame.assets.push((id, extracted_asset)); } } } diff --git a/crates/bevy_render/src/render_resource/pipeline_cache.rs b/crates/bevy_render/src/render_resource/pipeline_cache.rs index 2ab51da14ee77..c9d643fb7ff6c 100644 --- a/crates/bevy_render/src/render_resource/pipeline_cache.rs +++ b/crates/bevy_render/src/render_resource/pipeline_cache.rs @@ -7,7 +7,7 @@ use crate::{ renderer::RenderDevice, Extract, }; -use bevy_asset::{AssetEvent, Assets, Handle}; +use bevy_asset::{AssetEvent, AssetId, Assets}; use bevy_ecs::system::{Res, ResMut}; use bevy_ecs::{event::EventReader, system::Resource}; use bevy_utils::{ @@ -121,15 +121,15 @@ impl CachedPipelineState { struct ShaderData { pipelines: HashSet, processed_shaders: HashMap, ErasedShaderModule>, - resolved_imports: HashMap>, - dependents: HashSet>, + resolved_imports: HashMap>, + dependents: HashSet>, } struct ShaderCache { - data: HashMap, ShaderData>, - shaders: HashMap, Shader>, - import_path_shaders: HashMap>, - waiting_on_import: HashMap>>, + data: HashMap, ShaderData>, + shaders: HashMap, Shader>, + import_path_shaders: HashMap>, + waiting_on_import: HashMap>>, composer: naga_oil::compose::Composer, } @@ -210,8 +210,8 @@ impl ShaderCache { fn add_import_to_composer( composer: &mut naga_oil::compose::Composer, - import_path_shaders: &HashMap>, - shaders: &HashMap, Shader>, + import_path_shaders: &HashMap>, + shaders: &HashMap, Shader>, import: &ShaderImport, ) -> Result<(), PipelineCacheError> { if !composer.contains_module(&import.module_name()) { @@ -240,14 +240,14 @@ impl ShaderCache { &mut self, render_device: &RenderDevice, pipeline: CachedPipelineId, - handle: &Handle, + id: AssetId, shader_defs: &[ShaderDefVal], ) -> Result { let shader = self .shaders - .get(handle) - .ok_or_else(|| PipelineCacheError::ShaderNotLoaded(handle.clone_weak()))?; - let data = self.data.entry(handle.clone_weak()).or_default(); + .get(&id) + .ok_or(PipelineCacheError::ShaderNotLoaded(id))?; + let data = self.data.entry(id).or_default(); let n_asset_imports = shader .imports() .filter(|import| matches!(import, ShaderImport::AssetPath(_))) @@ -281,7 +281,7 @@ impl ShaderCache { debug!( "processing shader {:?}, with shader defs {:?}", - handle, shader_defs + id, shader_defs ); let shader_source = match &shader.source { #[cfg(feature = "shader_format_spirv")] @@ -357,14 +357,14 @@ impl ShaderCache { Ok(module.clone()) } - fn clear(&mut self, handle: &Handle) -> Vec { - let mut shaders_to_clear = vec![handle.clone_weak()]; + fn clear(&mut self, id: AssetId) -> Vec { + let mut shaders_to_clear = vec![id]; let mut pipelines_to_queue = Vec::new(); while let Some(handle) = shaders_to_clear.pop() { if let Some(data) = self.data.get_mut(&handle) { data.processed_shaders.clear(); - pipelines_to_queue.extend(data.pipelines.iter().cloned()); - shaders_to_clear.extend(data.dependents.iter().map(|h| h.clone_weak())); + pipelines_to_queue.extend(data.pipelines.iter().copied()); + shaders_to_clear.extend(data.dependents.iter().copied()); if let Some(Shader { import_path, .. }) = self.shaders.get(&handle) { self.composer @@ -376,45 +376,42 @@ impl ShaderCache { pipelines_to_queue } - fn set_shader(&mut self, handle: &Handle, shader: Shader) -> Vec { - let pipelines_to_queue = self.clear(handle); + fn set_shader(&mut self, id: AssetId, shader: Shader) -> Vec { + let pipelines_to_queue = self.clear(id); let path = shader.import_path(); - self.import_path_shaders - .insert(path.clone(), handle.clone_weak()); + self.import_path_shaders.insert(path.clone(), id); if let Some(waiting_shaders) = self.waiting_on_import.get_mut(path) { for waiting_shader in waiting_shaders.drain(..) { // resolve waiting shader import - let data = self.data.entry(waiting_shader.clone_weak()).or_default(); - data.resolved_imports - .insert(path.clone(), handle.clone_weak()); + let data = self.data.entry(waiting_shader).or_default(); + data.resolved_imports.insert(path.clone(), id); // add waiting shader as dependent of this shader - let data = self.data.entry(handle.clone_weak()).or_default(); - data.dependents.insert(waiting_shader.clone_weak()); + let data = self.data.entry(id).or_default(); + data.dependents.insert(waiting_shader); } } for import in shader.imports() { - if let Some(import_handle) = self.import_path_shaders.get(import) { + if let Some(import_id) = self.import_path_shaders.get(import).copied() { // resolve import because it is currently available - let data = self.data.entry(handle.clone_weak()).or_default(); - data.resolved_imports - .insert(import.clone(), import_handle.clone_weak()); + let data = self.data.entry(id).or_default(); + data.resolved_imports.insert(import.clone(), import_id); // add this shader as a dependent of the import - let data = self.data.entry(import_handle.clone_weak()).or_default(); - data.dependents.insert(handle.clone_weak()); + let data = self.data.entry(import_id).or_default(); + data.dependents.insert(id); } else { let waiting = self.waiting_on_import.entry(import.clone()).or_default(); - waiting.push(handle.clone_weak()); + waiting.push(id); } } - self.shaders.insert(handle.clone_weak(), shader); + self.shaders.insert(id, shader); pipelines_to_queue } - fn remove(&mut self, handle: &Handle) -> Vec { - let pipelines_to_queue = self.clear(handle); - if let Some(shader) = self.shaders.remove(handle) { + fn remove(&mut self, id: AssetId) -> Vec { + let pipelines_to_queue = self.clear(id); + if let Some(shader) = self.shaders.remove(&id) { self.import_path_shaders.remove(shader.import_path()); } @@ -625,15 +622,15 @@ impl PipelineCache { id } - fn set_shader(&mut self, handle: &Handle, shader: &Shader) { - let pipelines_to_queue = self.shader_cache.set_shader(handle, shader.clone()); + fn set_shader(&mut self, id: AssetId, shader: &Shader) { + let pipelines_to_queue = self.shader_cache.set_shader(id, shader.clone()); for cached_pipeline in pipelines_to_queue { self.pipelines[cached_pipeline].state = CachedPipelineState::Queued; self.waiting_pipelines.insert(cached_pipeline); } } - fn remove_shader(&mut self, shader: &Handle) { + fn remove_shader(&mut self, shader: AssetId) { let pipelines_to_queue = self.shader_cache.remove(shader); for cached_pipeline in pipelines_to_queue { self.pipelines[cached_pipeline].state = CachedPipelineState::Queued; @@ -649,7 +646,7 @@ impl PipelineCache { let vertex_module = match self.shader_cache.get( &self.device, id, - &descriptor.vertex.shader, + descriptor.vertex.shader.id(), &descriptor.vertex.shader_defs, ) { Ok(module) => module, @@ -662,7 +659,7 @@ impl PipelineCache { let fragment_module = match self.shader_cache.get( &self.device, id, - &fragment.shader, + fragment.shader.id(), &fragment.shader_defs, ) { Ok(module) => module, @@ -734,7 +731,7 @@ impl PipelineCache { let compute_module = match self.shader_cache.get( &self.device, id, - &descriptor.shader, + descriptor.shader.id(), &descriptor.shader_defs, ) { Ok(module) => module, @@ -834,12 +831,15 @@ impl PipelineCache { ) { for event in events.read() { match event { - AssetEvent::Created { handle } | AssetEvent::Modified { handle } => { - if let Some(shader) = shaders.get(handle) { - cache.set_shader(handle, shader); + AssetEvent::Added { id } | AssetEvent::Modified { id } => { + if let Some(shader) = shaders.get(*id) { + cache.set_shader(*id, shader); } } - AssetEvent::Removed { handle } => cache.remove_shader(handle), + AssetEvent::Removed { id } => cache.remove_shader(*id), + AssetEvent::LoadedWithDependencies { .. } => { + // TODO: handle this + } } } } @@ -851,7 +851,7 @@ pub enum PipelineCacheError { #[error( "Pipeline could not be compiled because the following shader is not loaded yet: {0:?}" )] - ShaderNotLoaded(Handle), + ShaderNotLoaded(AssetId), #[error(transparent)] ProcessShaderError(#[from] naga_oil::compose::ComposerError), #[error("Shader import not yet available.")] diff --git a/crates/bevy_render/src/render_resource/shader.rs b/crates/bevy_render/src/render_resource/shader.rs index 6dbb5af54dcd4..3993e0693a44b 100644 --- a/crates/bevy_render/src/render_resource/shader.rs +++ b/crates/bevy_render/src/render_resource/shader.rs @@ -1,9 +1,9 @@ use super::ShaderDefVal; use crate::define_atomic_id; -use bevy_asset::{AssetLoader, AssetPath, Handle, LoadContext, LoadedAsset}; -use bevy_reflect::{TypePath, TypeUuid}; +use bevy_asset::{anyhow, io::Reader, Asset, AssetLoader, AssetPath, Handle, LoadContext}; +use bevy_reflect::TypePath; use bevy_utils::{tracing::error, BoxedFuture}; - +use futures_lite::AsyncReadExt; use std::{borrow::Cow, marker::Copy}; use thiserror::Error; @@ -24,8 +24,7 @@ pub enum ShaderReflectError { } /// A shader, as defined by its [`ShaderSource`](wgpu::ShaderSource) and [`ShaderStage`](naga::ShaderStage) /// This is an "unprocessed" shader. It can contain preprocessor directives. -#[derive(Debug, Clone, TypeUuid, TypePath)] -#[uuid = "d95bc916-6c55-4de3-9622-37e7b6969fda"] +#[derive(Asset, TypePath, Debug, Clone)] pub struct Shader { pub path: String, pub source: Source, @@ -234,34 +233,37 @@ impl From<&Source> for naga_oil::compose::ShaderType { pub struct ShaderLoader; impl AssetLoader for ShaderLoader { + type Asset = Shader; + type Settings = (); fn load<'a>( &'a self, - bytes: &'a [u8], + reader: &'a mut Reader, + _settings: &'a Self::Settings, load_context: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<(), anyhow::Error>> { + ) -> BoxedFuture<'a, Result> { Box::pin(async move { let ext = load_context.path().extension().unwrap().to_str().unwrap(); + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; let shader = match ext { - "spv" => { - Shader::from_spirv(Vec::from(bytes), load_context.path().to_string_lossy()) - } + "spv" => Shader::from_spirv(bytes, load_context.path().to_string_lossy()), "wgsl" => Shader::from_wgsl( - String::from_utf8(Vec::from(bytes))?, + String::from_utf8(bytes)?, load_context.path().to_string_lossy(), ), "vert" => Shader::from_glsl( - String::from_utf8(Vec::from(bytes))?, + String::from_utf8(bytes)?, naga::ShaderStage::Vertex, load_context.path().to_string_lossy(), ), "frag" => Shader::from_glsl( - String::from_utf8(Vec::from(bytes))?, + String::from_utf8(bytes)?, naga::ShaderStage::Fragment, load_context.path().to_string_lossy(), ), "comp" => Shader::from_glsl( - String::from_utf8(Vec::from(bytes))?, + String::from_utf8(bytes)?, naga::ShaderStage::Compute, load_context.path().to_string_lossy(), ), @@ -269,25 +271,13 @@ impl AssetLoader for ShaderLoader { }; // collect file dependencies - let dependencies = shader - .imports - .iter() - .flat_map(|import| { - if let ShaderImport::AssetPath(asset_path) = import { - Some(asset_path.clone()) - } else { - None - } - }) - .collect::>(); - - let mut asset = LoadedAsset::new(shader); - for dependency in dependencies { - asset.add_dependency(dependency.into()); + for import in shader.imports.iter() { + if let ShaderImport::AssetPath(asset_path) = import { + // TODO: should we just allow this handle to be dropped? + let _handle: Handle = load_context.load(asset_path); + } } - - load_context.set_default_asset(asset); - Ok(()) + Ok(shader) }) } diff --git a/crates/bevy_render/src/texture/basis.rs b/crates/bevy_render/src/texture/basis.rs index 886a0cb9bb7df..e91c40e840d47 100644 --- a/crates/bevy_render/src/texture/basis.rs +++ b/crates/bevy_render/src/texture/basis.rs @@ -101,7 +101,8 @@ pub fn basis_buffer_to_image( width: image0_info.m_orig_width, height: image0_info.m_orig_height, depth_or_array_layers: image_count, - }; + } + .physical_size(texture_format); image.texture_descriptor.mip_level_count = image0_mip_level_count; image.texture_descriptor.format = texture_format; image.texture_descriptor.dimension = match texture_type { diff --git a/crates/bevy_render/src/texture/compressed_image_saver.rs b/crates/bevy_render/src/texture/compressed_image_saver.rs new file mode 100644 index 0000000000000..d3e6f88f00721 --- /dev/null +++ b/crates/bevy_render/src/texture/compressed_image_saver.rs @@ -0,0 +1,56 @@ +use crate::texture::{Image, ImageFormat, ImageFormatSetting, ImageLoader, ImageLoaderSettings}; +use bevy_asset::{ + anyhow::Error, + saver::{AssetSaver, SavedAsset}, +}; +use futures_lite::{AsyncWriteExt, FutureExt}; + +pub struct CompressedImageSaver; + +impl AssetSaver for CompressedImageSaver { + type Asset = Image; + + type Settings = (); + type OutputLoader = ImageLoader; + + fn save<'a>( + &'a self, + writer: &'a mut bevy_asset::io::Writer, + image: SavedAsset<'a, Self::Asset>, + _settings: &'a Self::Settings, + ) -> bevy_utils::BoxedFuture<'a, std::result::Result> { + // PERF: this should live inside the future, but CompressorParams and Compressor are not Send / can't be owned by the BoxedFuture (which _is_ Send) + let mut compressor_params = basis_universal::CompressorParams::new(); + compressor_params.set_basis_format(basis_universal::BasisTextureFormat::UASTC4x4); + compressor_params.set_generate_mipmaps(true); + let is_srgb = image.texture_descriptor.format.is_srgb(); + let color_space = if is_srgb { + basis_universal::ColorSpace::Srgb + } else { + basis_universal::ColorSpace::Linear + }; + compressor_params.set_color_space(color_space); + compressor_params.set_uastc_quality_level(basis_universal::UASTC_QUALITY_DEFAULT); + + let mut source_image = compressor_params.source_image_mut(0); + let size = image.size(); + source_image.init(&image.data, size.x as u32, size.y as u32, 4); + + let mut compressor = basis_universal::Compressor::new(4); + // SAFETY: the CompressorParams are "valid" to the best of our knowledge. The basis-universal + // library bindings note that invalid params might produce undefined behavior. + unsafe { + compressor.init(&compressor_params); + compressor.process().unwrap(); + } + let compressed_basis_data = compressor.basis_file().to_vec(); + async move { + writer.write_all(&compressed_basis_data).await?; + Ok(ImageLoaderSettings { + format: ImageFormatSetting::Format(ImageFormat::Basis), + is_srgb, + }) + } + .boxed() + } +} diff --git a/crates/bevy_render/src/texture/exr_texture_loader.rs b/crates/bevy_render/src/texture/exr_texture_loader.rs index 6e1c18a805d0a..6f548c49d88e8 100644 --- a/crates/bevy_render/src/texture/exr_texture_loader.rs +++ b/crates/bevy_render/src/texture/exr_texture_loader.rs @@ -1,6 +1,9 @@ use crate::texture::{Image, TextureFormatPixelInfo}; -use anyhow::Result; -use bevy_asset::{AssetLoader, LoadContext, LoadedAsset}; +use bevy_asset::{ + anyhow::Error, + io::{AsyncReadExt, Reader}, + AssetLoader, LoadContext, +}; use bevy_utils::BoxedFuture; use image::ImageDecoder; use wgpu::{Extent3d, TextureDimension, TextureFormat}; @@ -10,11 +13,15 @@ use wgpu::{Extent3d, TextureDimension, TextureFormat}; pub struct ExrTextureLoader; impl AssetLoader for ExrTextureLoader { + type Asset = Image; + type Settings = (); + fn load<'a>( &'a self, - bytes: &'a [u8], - load_context: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<()>> { + reader: &'a mut Reader, + _settings: &'a Self::Settings, + _load_context: &'a mut LoadContext, + ) -> BoxedFuture<'a, Result> { Box::pin(async move { let format = TextureFormat::Rgba32Float; debug_assert_eq!( @@ -23,6 +30,8 @@ impl AssetLoader for ExrTextureLoader { "Format should have 32bit x 4 size" ); + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; let decoder = image::codecs::openexr::OpenExrDecoder::with_alpha_preference( std::io::Cursor::new(bytes), Some(true), @@ -34,7 +43,7 @@ impl AssetLoader for ExrTextureLoader { let mut buf = vec![0u8; total_bytes]; decoder.read_image(buf.as_mut_slice())?; - let texture = Image::new( + Ok(Image::new( Extent3d { width, height, @@ -43,10 +52,7 @@ impl AssetLoader for ExrTextureLoader { TextureDimension::D2, buf, format, - ); - - load_context.set_default_asset(LoadedAsset::new(texture)); - Ok(()) + )) }) } diff --git a/crates/bevy_render/src/texture/hdr_texture_loader.rs b/crates/bevy_render/src/texture/hdr_texture_loader.rs index 81c539a061664..339429b319400 100644 --- a/crates/bevy_render/src/texture/hdr_texture_loader.rs +++ b/crates/bevy_render/src/texture/hdr_texture_loader.rs @@ -1,7 +1,5 @@ use crate::texture::{Image, TextureFormatPixelInfo}; -use anyhow::Result; -use bevy_asset::{AssetLoader, LoadContext, LoadedAsset}; -use bevy_utils::BoxedFuture; +use bevy_asset::{anyhow::Error, io::Reader, AssetLoader, AsyncReadExt, LoadContext}; use wgpu::{Extent3d, TextureDimension, TextureFormat}; /// Loads HDR textures as Texture assets @@ -9,11 +7,14 @@ use wgpu::{Extent3d, TextureDimension, TextureFormat}; pub struct HdrTextureLoader; impl AssetLoader for HdrTextureLoader { + type Asset = Image; + type Settings = (); fn load<'a>( &'a self, - bytes: &'a [u8], - load_context: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<()>> { + reader: &'a mut Reader, + _settings: &'a (), + _load_context: &'a mut LoadContext, + ) -> bevy_utils::BoxedFuture<'a, Result> { Box::pin(async move { let format = TextureFormat::Rgba32Float; debug_assert_eq!( @@ -22,7 +23,9 @@ impl AssetLoader for HdrTextureLoader { "Format should have 32bit x 4 size" ); - let decoder = image::codecs::hdr::HdrDecoder::new(bytes)?; + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; + let decoder = image::codecs::hdr::HdrDecoder::new(bytes.as_slice())?; let info = decoder.metadata(); let rgb_data = decoder.read_image_hdr()?; let mut rgba_data = Vec::with_capacity(rgb_data.len() * format.pixel_size()); @@ -36,7 +39,7 @@ impl AssetLoader for HdrTextureLoader { rgba_data.extend_from_slice(&alpha.to_ne_bytes()); } - let texture = Image::new( + Ok(Image::new( Extent3d { width: info.width, height: info.height, @@ -45,10 +48,7 @@ impl AssetLoader for HdrTextureLoader { TextureDimension::D2, rgba_data, format, - ); - - load_context.set_default_asset(LoadedAsset::new(texture)); - Ok(()) + )) }) } diff --git a/crates/bevy_render/src/texture/image.rs b/crates/bevy_render/src/texture/image.rs index 488f80dfdc4bd..d14388d87504c 100644 --- a/crates/bevy_render/src/texture/image.rs +++ b/crates/bevy_render/src/texture/image.rs @@ -11,22 +11,20 @@ use crate::{ renderer::{RenderDevice, RenderQueue}, texture::BevyDefault, }; -use bevy_asset::HandleUntyped; +use bevy_asset::Asset; use bevy_derive::{Deref, DerefMut}; use bevy_ecs::system::{lifetimeless::SRes, Resource, SystemParamItem}; use bevy_math::Vec2; -use bevy_reflect::{Reflect, TypeUuid}; - +use bevy_reflect::Reflect; +use serde::{Deserialize, Serialize}; use std::hash::Hash; use thiserror::Error; use wgpu::{Extent3d, TextureDimension, TextureFormat, TextureViewDescriptor}; pub const TEXTURE_ASSET_INDEX: u64 = 0; pub const SAMPLER_ASSET_INDEX: u64 = 1; -pub const DEFAULT_IMAGE_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Image::TYPE_UUID, 13148262314052771789); -#[derive(Debug)] +#[derive(Debug, Serialize, Deserialize, Copy, Clone)] pub enum ImageFormat { Avif, Basis, @@ -103,8 +101,7 @@ impl ImageFormat { } } -#[derive(Reflect, Debug, Clone, TypeUuid)] -#[uuid = "6ea26da6-6cf8-4ea2-9986-1d7bf6c17d6f"] +#[derive(Asset, Reflect, Debug, Clone)] #[reflect_value] pub struct Image { pub data: Vec, @@ -444,11 +441,14 @@ pub enum TextureError { } /// The type of a raw image buffer. +#[derive(Debug)] pub enum ImageType<'a> { /// The mime type of an image, for example `"image/png"`. MimeType(&'a str), /// The extension of an image file, for example `"png"`. Extension(&'a str), + /// The direct format of the image + Format(ImageFormat), } impl<'a> ImageType<'a> { @@ -458,6 +458,7 @@ impl<'a> ImageType<'a> { .ok_or_else(|| TextureError::InvalidImageMimeType(mime_type.to_string())), ImageType::Extension(extension) => ImageFormat::from_extension(extension) .ok_or_else(|| TextureError::InvalidImageExtension(extension.to_string())), + ImageType::Format(format) => Ok(*format), } } } diff --git a/crates/bevy_render/src/texture/image_texture_loader.rs b/crates/bevy_render/src/texture/image_loader.rs similarity index 62% rename from crates/bevy_render/src/texture/image_texture_loader.rs rename to crates/bevy_render/src/texture/image_loader.rs index da5845159b09a..d21847ec05082 100644 --- a/crates/bevy_render/src/texture/image_texture_loader.rs +++ b/crates/bevy_render/src/texture/image_loader.rs @@ -1,19 +1,19 @@ use anyhow::Result; -use bevy_asset::{AssetLoader, LoadContext, LoadedAsset}; +use bevy_asset::{anyhow, io::Reader, AssetLoader, AsyncReadExt, LoadContext}; use bevy_ecs::prelude::{FromWorld, World}; -use bevy_utils::BoxedFuture; use thiserror::Error; use crate::{ renderer::RenderDevice, - texture::{Image, ImageType, TextureError}, + texture::{Image, ImageFormat, ImageType, TextureError}, }; use super::CompressedImageFormats; +use serde::{Deserialize, Serialize}; /// Loader for images that can be read by the `image` crate. #[derive(Clone)] -pub struct ImageTextureLoader { +pub struct ImageLoader { supported_compressed_formats: CompressedImageFormats, } @@ -46,29 +46,57 @@ pub(crate) const IMG_FILE_EXTENSIONS: &[&str] = &[ "ppm", ]; -impl AssetLoader for ImageTextureLoader { +#[derive(Serialize, Deserialize, Default)] +pub enum ImageFormatSetting { + #[default] + FromExtension, + Format(ImageFormat), +} + +#[derive(Serialize, Deserialize)] +pub struct ImageLoaderSettings { + pub format: ImageFormatSetting, + pub is_srgb: bool, +} + +impl Default for ImageLoaderSettings { + fn default() -> Self { + Self { + format: ImageFormatSetting::default(), + is_srgb: true, + } + } +} + +impl AssetLoader for ImageLoader { + type Asset = Image; + type Settings = ImageLoaderSettings; fn load<'a>( &'a self, - bytes: &'a [u8], + reader: &'a mut Reader, + settings: &'a ImageLoaderSettings, load_context: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<()>> { + ) -> bevy_utils::BoxedFuture<'a, Result> { Box::pin(async move { // use the file extension for the image type let ext = load_context.path().extension().unwrap().to_str().unwrap(); - let dyn_img = Image::from_buffer( - bytes, - ImageType::Extension(ext), + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; + let image_type = match settings.format { + ImageFormatSetting::FromExtension => ImageType::Extension(ext), + ImageFormatSetting::Format(format) => ImageType::Format(format), + }; + Ok(Image::from_buffer( + &bytes, + image_type, self.supported_compressed_formats, - true, + settings.is_srgb, ) .map_err(|err| FileTextureError { error: err, path: format!("{}", load_context.path().display()), - })?; - - load_context.set_default_asset(LoadedAsset::new(dyn_img)); - Ok(()) + })?) }) } @@ -77,7 +105,7 @@ impl AssetLoader for ImageTextureLoader { } } -impl FromWorld for ImageTextureLoader { +impl FromWorld for ImageLoader { fn from_world(world: &mut World) -> Self { let supported_compressed_formats = match world.get_resource::() { Some(render_device) => CompressedImageFormats::from_features(render_device.features()), diff --git a/crates/bevy_render/src/texture/image_texture_conversion.rs b/crates/bevy_render/src/texture/image_texture_conversion.rs index 6cbdff876925e..82a931f6ca308 100644 --- a/crates/bevy_render/src/texture/image_texture_conversion.rs +++ b/crates/bevy_render/src/texture/image_texture_conversion.rs @@ -1,5 +1,5 @@ use crate::texture::{Image, TextureFormatPixelInfo}; -use anyhow::anyhow; +use bevy_asset::anyhow; use image::{DynamicImage, ImageBuffer}; use wgpu::{Extent3d, TextureDimension, TextureFormat}; @@ -163,7 +163,7 @@ impl Image { /// - `TextureFormat::Bgra8UnormSrgb` /// /// To convert [`Image`] to a different format see: [`Image::convert`]. - pub fn try_into_dynamic(self) -> anyhow::Result { + pub fn try_into_dynamic(self) -> Result { match self.texture_descriptor.format { TextureFormat::R8Unorm => ImageBuffer::from_raw( self.texture_descriptor.size.width, @@ -199,14 +199,14 @@ impl Image { .map(DynamicImage::ImageRgba8), // Throw and error if conversion isn't supported texture_format => { - return Err(anyhow!( + return Err(anyhow::anyhow!( "Conversion into dynamic image not supported for {:?}.", texture_format )) } } .ok_or_else(|| { - anyhow!( + anyhow::anyhow!( "Failed to convert into {:?}.", self.texture_descriptor.format ) diff --git a/crates/bevy_render/src/texture/mod.rs b/crates/bevy_render/src/texture/mod.rs index c086fff99262a..7f08ec46c93a3 100644 --- a/crates/bevy_render/src/texture/mod.rs +++ b/crates/bevy_render/src/texture/mod.rs @@ -1,5 +1,7 @@ #[cfg(feature = "basis-universal")] mod basis; +#[cfg(feature = "basis-universal")] +mod compressed_image_saver; #[cfg(feature = "dds")] mod dds; #[cfg(feature = "exr")] @@ -9,7 +11,7 @@ mod fallback_image; mod hdr_texture_loader; #[allow(clippy::module_inception)] mod image; -mod image_texture_loader; +mod image_loader; #[cfg(feature = "ktx2")] mod ktx2; mod texture_cache; @@ -26,15 +28,17 @@ pub use exr_texture_loader::*; #[cfg(feature = "hdr")] pub use hdr_texture_loader::*; +#[cfg(feature = "basis-universal")] +pub use compressed_image_saver::*; pub use fallback_image::*; -pub use image_texture_loader::*; +pub use image_loader::*; pub use texture_cache::*; use crate::{ render_asset::RenderAssetPlugin, renderer::RenderDevice, Render, RenderApp, RenderSet, }; use bevy_app::{App, Plugin}; -use bevy_asset::{AddAsset, Assets}; +use bevy_asset::{AssetApp, Assets, Handle}; use bevy_ecs::prelude::*; // TODO: replace Texture names with Image names? @@ -80,11 +84,22 @@ impl Plugin for ImagePlugin { app.add_plugins(RenderAssetPlugin::::default()) .register_type::() - .add_asset::() + .init_asset::() .register_asset_reflect::(); app.world .resource_mut::>() - .set_untracked(DEFAULT_IMAGE_HANDLE, Image::default()); + .insert(Handle::default(), Image::default()); + #[cfg(feature = "basis-universal")] + if let Some(processor) = app + .world + .get_resource::() + { + processor.register_processor::>( + CompressedImageSaver.into(), + ); + processor + .set_default_processor::>("png"); + } if let Ok(render_app) = app.get_sub_app_mut(RenderApp) { render_app.init_resource::().add_systems( @@ -102,7 +117,7 @@ impl Plugin for ImagePlugin { feature = "basis-universal", feature = "ktx2", ))] - app.preregister_asset_loader(IMG_FILE_EXTENSIONS); + app.preregister_asset_loader::(IMG_FILE_EXTENSIONS); } fn finish(&self, app: &mut App) { @@ -116,7 +131,7 @@ impl Plugin for ImagePlugin { feature = "ktx2", ))] { - app.init_asset_loader::(); + app.init_asset_loader::(); } if let Ok(render_app) = app.get_sub_app_mut(RenderApp) { diff --git a/crates/bevy_render/src/view/mod.rs b/crates/bevy_render/src/view/mod.rs index 322aaf35fdd3b..012a62f76d80c 100644 --- a/crates/bevy_render/src/view/mod.rs +++ b/crates/bevy_render/src/view/mod.rs @@ -1,7 +1,7 @@ pub mod visibility; pub mod window; -use bevy_asset::{load_internal_asset, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; pub use visibility::*; pub use window::*; @@ -19,7 +19,7 @@ use crate::{ use bevy_app::{App, Plugin}; use bevy_ecs::prelude::*; use bevy_math::{Mat4, UVec4, Vec3, Vec4, Vec4Swizzles}; -use bevy_reflect::{Reflect, TypeUuid}; +use bevy_reflect::Reflect; use bevy_transform::components::GlobalTransform; use bevy_utils::HashMap; use std::sync::{ @@ -31,8 +31,7 @@ use wgpu::{ TextureFormat, TextureUsages, }; -pub const VIEW_TYPE_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 15421373904451797197); +pub const VIEW_TYPE_HANDLE: Handle = Handle::weak_from_u128(15421373904451797197); pub struct ViewPlugin; diff --git a/crates/bevy_render/src/view/window/screenshot.rs b/crates/bevy_render/src/view/window/screenshot.rs index 440634c274e1d..6907cd690ca92 100644 --- a/crates/bevy_render/src/view/window/screenshot.rs +++ b/crates/bevy_render/src/view/window/screenshot.rs @@ -1,10 +1,9 @@ use std::{borrow::Cow, path::Path}; use bevy_app::Plugin; -use bevy_asset::{load_internal_asset, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_ecs::prelude::*; use bevy_log::{error, info, info_span}; -use bevy_reflect::TypeUuid; use bevy_tasks::AsyncComputeTaskPool; use bevy_utils::HashMap; use parking_lot::Mutex; @@ -122,8 +121,7 @@ impl ScreenshotManager { pub struct ScreenshotPlugin; -const SCREENSHOT_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 11918575842344596158); +const SCREENSHOT_SHADER_HANDLE: Handle = Handle::weak_from_u128(11918575842344596158); impl Plugin for ScreenshotPlugin { fn build(&self, app: &mut bevy_app::App) { @@ -231,7 +229,7 @@ impl SpecializedRenderPipeline for ScreenshotToScreenPipeline { buffers: vec![], shader_defs: vec![], entry_point: Cow::Borrowed("vs_main"), - shader: SCREENSHOT_SHADER_HANDLE.typed(), + shader: SCREENSHOT_SHADER_HANDLE, }, primitive: wgpu::PrimitiveState { cull_mode: Some(wgpu::Face::Back), @@ -240,7 +238,7 @@ impl SpecializedRenderPipeline for ScreenshotToScreenPipeline { depth_stencil: None, multisample: Default::default(), fragment: Some(FragmentState { - shader: SCREENSHOT_SHADER_HANDLE.typed(), + shader: SCREENSHOT_SHADER_HANDLE, entry_point: Cow::Borrowed("fs_main"), shader_defs: vec![], targets: vec![Some(wgpu::ColorTargetState { diff --git a/crates/bevy_scene/Cargo.toml b/crates/bevy_scene/Cargo.toml index b408a45ea762b..4f02e5db0f8ad 100644 --- a/crates/bevy_scene/Cargo.toml +++ b/crates/bevy_scene/Cargo.toml @@ -28,7 +28,6 @@ bevy_render = { path = "../bevy_render", version = "0.12.0-dev", optional = true serde = { version = "1.0", features = ["derive"], optional = true } ron = "0.8.0" uuid = { version = "1.1", features = ["v4"] } -anyhow = "1.0.4" thiserror = "1.0" [dev-dependencies] diff --git a/crates/bevy_scene/src/dynamic_scene.rs b/crates/bevy_scene/src/dynamic_scene.rs index e063f034d694c..8630d8e1dc243 100644 --- a/crates/bevy_scene/src/dynamic_scene.rs +++ b/crates/bevy_scene/src/dynamic_scene.rs @@ -1,17 +1,16 @@ -use std::any::TypeId; - use crate::{DynamicSceneBuilder, Scene, SceneSpawnError}; -use anyhow::Result; use bevy_ecs::{ entity::Entity, reflect::{AppTypeRegistry, ReflectComponent, ReflectMapEntities}, world::World, }; -use bevy_reflect::{Reflect, TypePath, TypeRegistryArc, TypeUuid}; +use bevy_reflect::{Reflect, TypePath, TypeRegistryArc}; use bevy_utils::HashMap; +use std::any::TypeId; #[cfg(feature = "serialize")] use crate::serde::SceneSerializer; +use bevy_asset::Asset; use bevy_ecs::reflect::ReflectResource; #[cfg(feature = "serialize")] use serde::Serialize; @@ -25,8 +24,7 @@ use serde::Serialize; /// * adding the [`Handle`](bevy_asset::Handle) to an entity (the scene will only be /// visible if the entity already has [`Transform`](bevy_transform::components::Transform) and /// [`GlobalTransform`](bevy_transform::components::GlobalTransform) components) -#[derive(Default, TypeUuid, TypePath)] -#[uuid = "749479b1-fb8c-4ff8-a775-623aa76014f5"] +#[derive(Asset, TypePath, Default)] pub struct DynamicScene { pub resources: Vec>, pub entities: Vec, diff --git a/crates/bevy_scene/src/lib.rs b/crates/bevy_scene/src/lib.rs index 7725627891e76..66d9d02043c0e 100644 --- a/crates/bevy_scene/src/lib.rs +++ b/crates/bevy_scene/src/lib.rs @@ -28,8 +28,8 @@ pub mod prelude { }; } -use bevy_app::{prelude::*, SpawnScene}; -use bevy_asset::AddAsset; +use bevy_app::prelude::*; +use bevy_asset::AssetApp; #[derive(Default)] pub struct ScenePlugin; @@ -37,8 +37,8 @@ pub struct ScenePlugin; #[cfg(feature = "serialize")] impl Plugin for ScenePlugin { fn build(&self, app: &mut App) { - app.add_asset::() - .add_asset::() + app.init_asset::() + .init_asset::() .init_asset_loader::() .add_event::() .init_resource::() diff --git a/crates/bevy_scene/src/scene.rs b/crates/bevy_scene/src/scene.rs index c47db7e749988..65b41ed2be93e 100644 --- a/crates/bevy_scene/src/scene.rs +++ b/crates/bevy_scene/src/scene.rs @@ -1,20 +1,19 @@ +use crate::{DynamicScene, InstanceInfo, SceneSpawnError}; +use bevy_asset::Asset; use bevy_ecs::{ reflect::{AppTypeRegistry, ReflectComponent, ReflectMapEntities, ReflectResource}, world::World, }; -use bevy_reflect::{TypePath, TypeUuid}; +use bevy_reflect::TypePath; use bevy_utils::HashMap; -use crate::{DynamicScene, InstanceInfo, SceneSpawnError}; - /// To spawn a scene, you can use either: /// * [`SceneSpawner::spawn`](crate::SceneSpawner::spawn) /// * adding the [`SceneBundle`](crate::SceneBundle) to an entity /// * adding the [`Handle`](bevy_asset::Handle) to an entity (the scene will only be /// visible if the entity already has [`Transform`](bevy_transform::components::Transform) and /// [`GlobalTransform`](bevy_transform::components::GlobalTransform) components) -#[derive(Debug, TypeUuid, TypePath)] -#[uuid = "c156503c-edd9-4ec7-8d33-dab392df03cd"] +#[derive(Asset, TypePath, Debug)] pub struct Scene { pub world: World, } diff --git a/crates/bevy_scene/src/scene_loader.rs b/crates/bevy_scene/src/scene_loader.rs index f89141e4ac9dd..3856601631ae8 100644 --- a/crates/bevy_scene/src/scene_loader.rs +++ b/crates/bevy_scene/src/scene_loader.rs @@ -1,12 +1,11 @@ #[cfg(feature = "serialize")] use crate::serde::SceneDeserializer; -use anyhow::{anyhow, Result}; -use bevy_asset::{AssetLoader, LoadContext, LoadedAsset}; +use crate::DynamicScene; +use bevy_asset::{anyhow, io::Reader, AssetLoader, AsyncReadExt, LoadContext}; use bevy_ecs::reflect::AppTypeRegistry; use bevy_ecs::world::{FromWorld, World}; use bevy_reflect::TypeRegistryArc; use bevy_utils::BoxedFuture; - #[cfg(feature = "serialize")] use serde::de::DeserializeSeed; @@ -26,29 +25,33 @@ impl FromWorld for SceneLoader { #[cfg(feature = "serialize")] impl AssetLoader for SceneLoader { + type Asset = DynamicScene; + type Settings = (); + fn load<'a>( &'a self, - bytes: &'a [u8], + reader: &'a mut Reader, + _settings: &'a (), load_context: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<()>> { + ) -> BoxedFuture<'a, Result> { Box::pin(async move { - let mut deserializer = ron::de::Deserializer::from_bytes(bytes)?; + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; + let mut deserializer = ron::de::Deserializer::from_bytes(&bytes)?; let scene_deserializer = SceneDeserializer { type_registry: &self.type_registry.read(), }; - let scene = scene_deserializer + scene_deserializer .deserialize(&mut deserializer) .map_err(|e| { let span_error = deserializer.span_error(e); - anyhow!( + anyhow::anyhow!( "{} at {}:{}", span_error.code, load_context.path().to_string_lossy(), span_error.position, ) - })?; - load_context.set_default_asset(LoadedAsset::new(scene)); - Ok(()) + }) }) } diff --git a/crates/bevy_scene/src/scene_spawner.rs b/crates/bevy_scene/src/scene_spawner.rs index a23e345ebb358..ebb037afdc764 100644 --- a/crates/bevy_scene/src/scene_spawner.rs +++ b/crates/bevy_scene/src/scene_spawner.rs @@ -1,5 +1,5 @@ use crate::{DynamicScene, Scene}; -use bevy_asset::{AssetEvent, Assets, Handle}; +use bevy_asset::{AssetEvent, AssetId, Assets}; use bevy_ecs::{ entity::Entity, event::{Event, Events, ManualEventReader}, @@ -39,13 +39,13 @@ impl InstanceId { #[derive(Default, Resource)] pub struct SceneSpawner { - spawned_scenes: HashMap, Vec>, - spawned_dynamic_scenes: HashMap, Vec>, + spawned_scenes: HashMap, Vec>, + spawned_dynamic_scenes: HashMap, Vec>, spawned_instances: HashMap, scene_asset_event_reader: ManualEventReader>, - dynamic_scenes_to_spawn: Vec<(Handle, InstanceId)>, - scenes_to_spawn: Vec<(Handle, InstanceId)>, - scenes_to_despawn: Vec>, + dynamic_scenes_to_spawn: Vec<(AssetId, InstanceId)>, + scenes_to_spawn: Vec<(AssetId, InstanceId)>, + scenes_to_despawn: Vec>, instances_to_despawn: Vec, scenes_with_parent: Vec<(InstanceId, Entity)>, } @@ -59,46 +59,44 @@ pub enum SceneSpawnError { #[error("scene contains the unregistered type `{type_name}`. consider registering the type using `app.register_type::()`")] UnregisteredType { type_name: String }, #[error("scene does not exist")] - NonExistentScene { handle: Handle }, + NonExistentScene { id: AssetId }, #[error("scene does not exist")] - NonExistentRealScene { handle: Handle }, + NonExistentRealScene { id: AssetId }, } impl SceneSpawner { - pub fn spawn_dynamic(&mut self, scene_handle: Handle) -> InstanceId { + pub fn spawn_dynamic(&mut self, id: impl Into>) -> InstanceId { let instance_id = InstanceId::new(); - self.dynamic_scenes_to_spawn - .push((scene_handle, instance_id)); + self.dynamic_scenes_to_spawn.push((id.into(), instance_id)); instance_id } pub fn spawn_dynamic_as_child( &mut self, - scene_handle: Handle, + id: impl Into>, parent: Entity, ) -> InstanceId { let instance_id = InstanceId::new(); - self.dynamic_scenes_to_spawn - .push((scene_handle, instance_id)); + self.dynamic_scenes_to_spawn.push((id.into(), instance_id)); self.scenes_with_parent.push((instance_id, parent)); instance_id } - pub fn spawn(&mut self, scene_handle: Handle) -> InstanceId { + pub fn spawn(&mut self, id: impl Into>) -> InstanceId { let instance_id = InstanceId::new(); - self.scenes_to_spawn.push((scene_handle, instance_id)); + self.scenes_to_spawn.push((id.into(), instance_id)); instance_id } - pub fn spawn_as_child(&mut self, scene_handle: Handle, parent: Entity) -> InstanceId { + pub fn spawn_as_child(&mut self, id: impl Into>, parent: Entity) -> InstanceId { let instance_id = InstanceId::new(); - self.scenes_to_spawn.push((scene_handle, instance_id)); + self.scenes_to_spawn.push((id.into(), instance_id)); self.scenes_with_parent.push((instance_id, parent)); instance_id } - pub fn despawn(&mut self, scene_handle: Handle) { - self.scenes_to_despawn.push(scene_handle); + pub fn despawn(&mut self, id: impl Into>) { + self.scenes_to_despawn.push(id.into()); } pub fn despawn_instance(&mut self, instance_id: InstanceId) { @@ -108,9 +106,9 @@ impl SceneSpawner { pub fn despawn_sync( &mut self, world: &mut World, - scene_handle: Handle, + id: impl Into>, ) -> Result<(), SceneSpawnError> { - if let Some(instance_ids) = self.spawned_dynamic_scenes.remove(&scene_handle) { + if let Some(instance_ids) = self.spawned_dynamic_scenes.remove(&id.into()) { for instance_id in instance_ids { self.despawn_instance_sync(world, &instance_id); } @@ -129,33 +127,28 @@ impl SceneSpawner { pub fn spawn_dynamic_sync( &mut self, world: &mut World, - scene_handle: &Handle, + id: impl Into>, ) -> Result<(), SceneSpawnError> { let mut entity_map = HashMap::default(); - Self::spawn_dynamic_internal(world, scene_handle, &mut entity_map)?; + let id = id.into(); + Self::spawn_dynamic_internal(world, id, &mut entity_map)?; let instance_id = InstanceId::new(); self.spawned_instances .insert(instance_id, InstanceInfo { entity_map }); - let spawned = self - .spawned_dynamic_scenes - .entry(scene_handle.clone()) - .or_default(); + let spawned = self.spawned_dynamic_scenes.entry(id).or_default(); spawned.push(instance_id); Ok(()) } fn spawn_dynamic_internal( world: &mut World, - scene_handle: &Handle, + id: AssetId, entity_map: &mut HashMap, ) -> Result<(), SceneSpawnError> { world.resource_scope(|world, scenes: Mut>| { - let scene = - scenes - .get(scene_handle) - .ok_or_else(|| SceneSpawnError::NonExistentScene { - handle: scene_handle.clone_weak(), - })?; + let scene = scenes + .get(id) + .ok_or(SceneSpawnError::NonExistentScene { id })?; scene.write_to_world(world, entity_map) }) } @@ -163,30 +156,27 @@ impl SceneSpawner { pub fn spawn_sync( &mut self, world: &mut World, - scene_handle: Handle, + id: AssetId, ) -> Result { - self.spawn_sync_internal(world, scene_handle, InstanceId::new()) + self.spawn_sync_internal(world, id, InstanceId::new()) } fn spawn_sync_internal( &mut self, world: &mut World, - scene_handle: Handle, + id: AssetId, instance_id: InstanceId, ) -> Result { world.resource_scope(|world, scenes: Mut>| { - let scene = - scenes - .get(&scene_handle) - .ok_or_else(|| SceneSpawnError::NonExistentRealScene { - handle: scene_handle.clone(), - })?; + let scene = scenes + .get(id) + .ok_or(SceneSpawnError::NonExistentRealScene { id })?; let instance_info = scene.write_to_world_with(world, &world.resource::().clone())?; self.spawned_instances.insert(instance_id, instance_info); - let spawned = self.spawned_scenes.entry(scene_handle).or_default(); + let spawned = self.spawned_scenes.entry(id).or_default(); spawned.push(instance_id); Ok(instance_id) }) @@ -195,17 +185,13 @@ impl SceneSpawner { pub fn update_spawned_scenes( &mut self, world: &mut World, - scene_handles: &[Handle], + scene_ids: &[AssetId], ) -> Result<(), SceneSpawnError> { - for scene_handle in scene_handles { - if let Some(spawned_instances) = self.spawned_dynamic_scenes.get(scene_handle) { + for id in scene_ids { + if let Some(spawned_instances) = self.spawned_dynamic_scenes.get(id) { for instance_id in spawned_instances { if let Some(instance_info) = self.spawned_instances.get_mut(instance_id) { - Self::spawn_dynamic_internal( - world, - scene_handle, - &mut instance_info.entity_map, - )?; + Self::spawn_dynamic_internal(world, *id, &mut instance_info.entity_map)?; } } } @@ -233,22 +219,21 @@ impl SceneSpawner { pub fn spawn_queued_scenes(&mut self, world: &mut World) -> Result<(), SceneSpawnError> { let scenes_to_spawn = std::mem::take(&mut self.dynamic_scenes_to_spawn); - for (scene_handle, instance_id) in scenes_to_spawn { + for (id, instance_id) in scenes_to_spawn { let mut entity_map = HashMap::default(); - match Self::spawn_dynamic_internal(world, &scene_handle, &mut entity_map) { + match Self::spawn_dynamic_internal(world, id, &mut entity_map) { Ok(_) => { self.spawned_instances .insert(instance_id, InstanceInfo { entity_map }); let spawned = self .spawned_dynamic_scenes - .entry(scene_handle.clone()) + .entry(id) .or_insert_with(Vec::new); spawned.push(instance_id); } Err(SceneSpawnError::NonExistentScene { .. }) => { - self.dynamic_scenes_to_spawn - .push((scene_handle, instance_id)); + self.dynamic_scenes_to_spawn.push((id, instance_id)); } Err(err) => return Err(err), } @@ -259,7 +244,7 @@ impl SceneSpawner { for (scene_handle, instance_id) in scenes_to_spawn { match self.spawn_sync_internal(world, scene_handle, instance_id) { Ok(_) => {} - Err(SceneSpawnError::NonExistentRealScene { handle }) => { + Err(SceneSpawnError::NonExistentRealScene { id: handle }) => { self.scenes_to_spawn.push((handle, instance_id)); } Err(err) => return Err(err), @@ -353,9 +338,9 @@ pub fn scene_spawner_system(world: &mut World) { .scene_asset_event_reader .read(scene_asset_events) { - if let AssetEvent::Modified { handle } = event { - if scene_spawner.spawned_dynamic_scenes.contains_key(handle) { - updated_spawned_scenes.push(handle.clone_weak()); + if let AssetEvent::Modified { id } = event { + if scene_spawner.spawned_dynamic_scenes.contains_key(id) { + updated_spawned_scenes.push(*id); } } } diff --git a/crates/bevy_scene/src/serde.rs b/crates/bevy_scene/src/serde.rs index b23f4b8510065..e8d2d6a2841b1 100644 --- a/crates/bevy_scene/src/serde.rs +++ b/crates/bevy_scene/src/serde.rs @@ -1,5 +1,4 @@ use crate::{DynamicEntity, DynamicScene}; -use anyhow::Result; use bevy_ecs::entity::Entity; use bevy_reflect::serde::{TypedReflectDeserializer, TypedReflectSerializer}; use bevy_reflect::{ diff --git a/crates/bevy_sprite/src/bundle.rs b/crates/bevy_sprite/src/bundle.rs index 1e88c342266d7..7013993482347 100644 --- a/crates/bevy_sprite/src/bundle.rs +++ b/crates/bevy_sprite/src/bundle.rs @@ -5,12 +5,12 @@ use crate::{ use bevy_asset::Handle; use bevy_ecs::bundle::Bundle; use bevy_render::{ - texture::{Image, DEFAULT_IMAGE_HANDLE}, + texture::Image, view::{InheritedVisibility, ViewVisibility, Visibility}, }; use bevy_transform::components::{GlobalTransform, Transform}; -#[derive(Bundle, Clone)] +#[derive(Bundle, Clone, Default)] pub struct SpriteBundle { pub sprite: Sprite, pub transform: Transform, @@ -24,19 +24,6 @@ pub struct SpriteBundle { pub view_visibility: ViewVisibility, } -impl Default for SpriteBundle { - fn default() -> Self { - Self { - sprite: Default::default(), - transform: Default::default(), - global_transform: Default::default(), - texture: DEFAULT_IMAGE_HANDLE.typed(), - visibility: Default::default(), - inherited_visibility: Default::default(), - view_visibility: Default::default(), - } - } -} /// A Bundle of components for drawing a single sprite from a sprite sheet (also referred /// to as a `TextureAtlas`) #[derive(Bundle, Clone, Default)] diff --git a/crates/bevy_sprite/src/lib.rs b/crates/bevy_sprite/src/lib.rs index b337df92d9687..1eb3b1a5cc64c 100644 --- a/crates/bevy_sprite/src/lib.rs +++ b/crates/bevy_sprite/src/lib.rs @@ -29,10 +29,9 @@ pub use texture_atlas::*; pub use texture_atlas_builder::*; use bevy_app::prelude::*; -use bevy_asset::{load_internal_asset, AddAsset, Assets, Handle, HandleUntyped}; +use bevy_asset::{load_internal_asset, AssetApp, Assets, Handle}; use bevy_core_pipeline::core_2d::Transparent2d; use bevy_ecs::prelude::*; -use bevy_reflect::TypeUuid; use bevy_render::{ mesh::Mesh, primitives::Aabb, @@ -46,8 +45,7 @@ use bevy_render::{ #[derive(Default)] pub struct SpritePlugin; -pub const SPRITE_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2763343953151597127); +pub const SPRITE_SHADER_HANDLE: Handle = Handle::weak_from_u128(2763343953151597127); #[derive(Debug, Hash, PartialEq, Eq, Clone, SystemSet)] pub enum SpriteSystem { @@ -62,7 +60,7 @@ impl Plugin for SpritePlugin { "render/sprite.wgsl", Shader::from_wgsl ); - app.add_asset::() + app.init_asset::() .register_asset_reflect::() .register_type::() .register_type::() diff --git a/crates/bevy_sprite/src/mesh2d/color_material.rs b/crates/bevy_sprite/src/mesh2d/color_material.rs index 6258d437d31b4..8e5e39b006a00 100644 --- a/crates/bevy_sprite/src/mesh2d/color_material.rs +++ b/crates/bevy_sprite/src/mesh2d/color_material.rs @@ -1,15 +1,14 @@ +use crate::{Material2d, Material2dPlugin, MaterialMesh2dBundle}; use bevy_app::{App, Plugin}; -use bevy_asset::{load_internal_asset, AddAsset, Assets, Handle, HandleUntyped}; +use bevy_asset::{load_internal_asset, Asset, AssetApp, Assets, Handle}; use bevy_math::Vec4; -use bevy_reflect::{prelude::*, TypeUuid}; +use bevy_reflect::prelude::*; use bevy_render::{ color::Color, prelude::Shader, render_asset::RenderAssets, render_resource::*, texture::Image, }; -use crate::{Material2d, Material2dPlugin, MaterialMesh2dBundle}; - -pub const COLOR_MATERIAL_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 3253086872234592509); +pub const COLOR_MATERIAL_SHADER_HANDLE: Handle = + Handle::weak_from_u128(3253086872234592509); #[derive(Default)] pub struct ColorMaterialPlugin; @@ -26,22 +25,19 @@ impl Plugin for ColorMaterialPlugin { app.add_plugins(Material2dPlugin::::default()) .register_asset_reflect::(); - app.world - .resource_mut::>() - .set_untracked( - Handle::::default(), - ColorMaterial { - color: Color::rgb(1.0, 0.0, 1.0), - ..Default::default() - }, - ); + app.world.resource_mut::>().insert( + Handle::::default(), + ColorMaterial { + color: Color::rgb(1.0, 0.0, 1.0), + ..Default::default() + }, + ); } } /// A [2d material](Material2d) that renders [2d meshes](crate::Mesh2dHandle) with a texture tinted by a uniform color -#[derive(AsBindGroup, Reflect, Debug, Clone, TypeUuid)] +#[derive(Asset, AsBindGroup, Reflect, Debug, Clone)] #[reflect(Default, Debug)] -#[uuid = "e228a544-e3ca-4e1e-bb9d-4d8bc1ad8c19"] #[uniform(0, ColorMaterialUniform)] pub struct ColorMaterial { pub color: Color, @@ -110,7 +106,7 @@ impl AsBindGroupShaderType for ColorMaterial { impl Material2d for ColorMaterial { fn fragment_shader() -> ShaderRef { - COLOR_MATERIAL_SHADER_HANDLE.typed().into() + COLOR_MATERIAL_SHADER_HANDLE.into() } } diff --git a/crates/bevy_sprite/src/mesh2d/material.rs b/crates/bevy_sprite/src/mesh2d/material.rs index 98fd7791935df..220931e6453c3 100644 --- a/crates/bevy_sprite/src/mesh2d/material.rs +++ b/crates/bevy_sprite/src/mesh2d/material.rs @@ -1,5 +1,5 @@ use bevy_app::{App, Plugin}; -use bevy_asset::{AddAsset, AssetEvent, AssetServer, Assets, Handle}; +use bevy_asset::{Asset, AssetApp, AssetEvent, AssetId, AssetServer, Assets, Handle}; use bevy_core_pipeline::{ core_2d::Transparent2d, tonemapping::{DebandDither, Tonemapping}, @@ -14,7 +14,6 @@ use bevy_ecs::{ }, }; use bevy_log::error; -use bevy_reflect::{TypePath, TypeUuid}; use bevy_render::{ extract_component::ExtractComponentPlugin, mesh::{Mesh, MeshVertexBufferLayout}, @@ -51,8 +50,6 @@ use crate::{ /// Material2ds must implement [`AsBindGroup`] to define how data will be transferred to the GPU and bound in shaders. /// [`AsBindGroup`] can be derived, which makes generating bindings straightforward. See the [`AsBindGroup`] docs for details. /// -/// Materials must also implement [`TypeUuid`] so they can be treated as an [`Asset`](bevy_asset::Asset). -/// /// # Example /// /// Here is a simple Material2d implementation. The [`AsBindGroup`] derive has many features. To see what else is available, @@ -60,12 +57,11 @@ use crate::{ /// ``` /// # use bevy_sprite::{Material2d, MaterialMesh2dBundle}; /// # use bevy_ecs::prelude::*; -/// # use bevy_reflect::{TypeUuid, TypePath}; +/// # use bevy_reflect::TypePath; /// # use bevy_render::{render_resource::{AsBindGroup, ShaderRef}, texture::Image, color::Color}; -/// # use bevy_asset::{Handle, AssetServer, Assets}; +/// # use bevy_asset::{Handle, AssetServer, Assets, Asset}; /// -/// #[derive(AsBindGroup, TypeUuid, TypePath, Debug, Clone)] -/// #[uuid = "f690fdae-d598-45ab-8225-97e2a3f056e0"] +/// #[derive(AsBindGroup, Debug, Clone, Asset, TypePath)] /// pub struct CustomMaterial { /// // Uniform bindings must implement `ShaderType`, which will be used to convert the value to /// // its shader-compatible equivalent. Most core math types already implement `ShaderType`. @@ -111,7 +107,7 @@ use crate::{ /// @group(1) @binding(2) /// var color_sampler: sampler; /// ``` -pub trait Material2d: AsBindGroup + Send + Sync + Clone + TypeUuid + TypePath + Sized { +pub trait Material2d: AsBindGroup + Asset + Clone + Sized { /// Returns this material's vertex shader. If [`ShaderRef::Default`] is returned, the default mesh vertex shader /// will be used. fn vertex_shader() -> ShaderRef { @@ -151,7 +147,7 @@ where M::Data: PartialEq + Eq + Hash + Clone, { fn build(&self, app: &mut App) { - app.add_asset::() + app.init_asset::() .add_plugins(ExtractComponentPlugin::>::extract_visible()); if let Ok(render_app) = app.get_sub_app_mut(RenderApp) { @@ -320,7 +316,7 @@ impl RenderCommand

materials: SystemParamItem<'w, '_, Self::Param>, pass: &mut TrackedRenderPass<'w>, ) -> RenderCommandResult { - let material2d = materials.into_inner().get(material2d_handle).unwrap(); + let material2d = materials.into_inner().get(&material2d_handle.id()).unwrap(); pass.set_bind_group(I, &material2d.bind_group, &[]); RenderCommandResult::Success } @@ -383,7 +379,7 @@ pub fn queue_material2d_meshes( if let Ok((material2d_handle, mesh2d_handle, mesh2d_uniform)) = material2d_meshes.get(*visible_entity) { - if let Some(material2d) = render_materials.get(material2d_handle) { + if let Some(material2d) = render_materials.get(&material2d_handle.id()) { if let Some(mesh) = render_meshes.get(&mesh2d_handle.0) { let mesh_key = view_key | Mesh2dPipelineKey::from_primitive_topology(mesh.primitive_topology); @@ -435,8 +431,8 @@ pub struct PreparedMaterial2d { #[derive(Resource)] pub struct ExtractedMaterials2d { - extracted: Vec<(Handle, M)>, - removed: Vec>, + extracted: Vec<(AssetId, M)>, + removed: Vec>, } impl Default for ExtractedMaterials2d { @@ -450,7 +446,7 @@ impl Default for ExtractedMaterials2d { /// Stores all prepared representations of [`Material2d`] assets for as long as they exist. #[derive(Resource, Deref, DerefMut)] -pub struct RenderMaterials2d(HashMap, PreparedMaterial2d>); +pub struct RenderMaterials2d(HashMap, PreparedMaterial2d>); impl Default for RenderMaterials2d { fn default() -> Self { @@ -469,20 +465,24 @@ pub fn extract_materials_2d( let mut removed = Vec::new(); for event in events.read() { match event { - AssetEvent::Created { handle } | AssetEvent::Modified { handle } => { - changed_assets.insert(handle.clone_weak()); + AssetEvent::Added { id } | AssetEvent::Modified { id } => { + changed_assets.insert(*id); + } + AssetEvent::Removed { id } => { + changed_assets.remove(id); + removed.push(*id); } - AssetEvent::Removed { handle } => { - changed_assets.remove(handle); - removed.push(handle.clone_weak()); + + AssetEvent::LoadedWithDependencies { .. } => { + // TODO: handle this } } } let mut extracted_assets = Vec::new(); - for handle in changed_assets.drain() { - if let Some(asset) = assets.get(&handle) { - extracted_assets.push((handle, asset.clone())); + for id in changed_assets.drain() { + if let Some(asset) = assets.get(id) { + extracted_assets.push((id, asset.clone())); } } @@ -494,7 +494,7 @@ pub fn extract_materials_2d( /// All [`Material2d`] values of a given type that should be prepared next frame. pub struct PrepareNextFrameMaterials { - assets: Vec<(Handle, M)>, + assets: Vec<(AssetId, M)>, } impl Default for PrepareNextFrameMaterials { @@ -517,7 +517,7 @@ pub fn prepare_materials_2d( pipeline: Res>, ) { let queued_assets = std::mem::take(&mut prepare_next_frame.assets); - for (handle, material) in queued_assets { + for (id, material) in queued_assets { match prepare_material2d( &material, &render_device, @@ -526,10 +526,10 @@ pub fn prepare_materials_2d( &pipeline, ) { Ok(prepared_asset) => { - render_materials.insert(handle, prepared_asset); + render_materials.insert(id, prepared_asset); } Err(AsBindGroupError::RetryNextUpdate) => { - prepare_next_frame.assets.push((handle, material)); + prepare_next_frame.assets.push((id, material)); } } } diff --git a/crates/bevy_sprite/src/mesh2d/mesh.rs b/crates/bevy_sprite/src/mesh2d/mesh.rs index 8489d152fa427..09b69b296a664 100644 --- a/crates/bevy_sprite/src/mesh2d/mesh.rs +++ b/crates/bevy_sprite/src/mesh2d/mesh.rs @@ -1,5 +1,5 @@ use bevy_app::Plugin; -use bevy_asset::{load_internal_asset, Handle, HandleUntyped}; +use bevy_asset::{load_internal_asset, Handle}; use bevy_ecs::{ prelude::*, @@ -7,7 +7,7 @@ use bevy_ecs::{ system::{lifetimeless::*, SystemParamItem, SystemState}, }; use bevy_math::{Mat4, Vec2}; -use bevy_reflect::{Reflect, TypeUuid}; +use bevy_reflect::Reflect; use bevy_render::{ extract_component::{ComponentUniforms, DynamicUniformIndex, UniformComponentPlugin}, globals::{GlobalsBuffer, GlobalsUniform}, @@ -42,20 +42,13 @@ impl From> for Mesh2dHandle { #[derive(Default)] pub struct Mesh2dRenderPlugin; -pub const MESH2D_VERTEX_OUTPUT: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 7646632476603252194); -pub const MESH2D_VIEW_TYPES_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 12677582416765805110); -pub const MESH2D_VIEW_BINDINGS_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 6901431444735842434); -pub const MESH2D_TYPES_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 8994673400261890424); -pub const MESH2D_BINDINGS_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 8983617858458862856); -pub const MESH2D_FUNCTIONS_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 4976379308250389413); -pub const MESH2D_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2971387252468633715); +pub const MESH2D_VERTEX_OUTPUT: Handle = Handle::weak_from_u128(7646632476603252194); +pub const MESH2D_VIEW_TYPES_HANDLE: Handle = Handle::weak_from_u128(12677582416765805110); +pub const MESH2D_VIEW_BINDINGS_HANDLE: Handle = Handle::weak_from_u128(6901431444735842434); +pub const MESH2D_TYPES_HANDLE: Handle = Handle::weak_from_u128(8994673400261890424); +pub const MESH2D_BINDINGS_HANDLE: Handle = Handle::weak_from_u128(8983617858458862856); +pub const MESH2D_FUNCTIONS_HANDLE: Handle = Handle::weak_from_u128(4976379308250389413); +pub const MESH2D_SHADER_HANDLE: Handle = Handle::weak_from_u128(2971387252468633715); impl Plugin for Mesh2dRenderPlugin { fn build(&self, app: &mut bevy_app::App) { @@ -438,13 +431,13 @@ impl SpecializedMeshPipeline for Mesh2dPipeline { Ok(RenderPipelineDescriptor { vertex: VertexState { - shader: MESH2D_SHADER_HANDLE.typed::(), + shader: MESH2D_SHADER_HANDLE, entry_point: "vertex".into(), shader_defs: shader_defs.clone(), buffers: vec![vertex_buffer_layout], }, fragment: Some(FragmentState { - shader: MESH2D_SHADER_HANDLE.typed::(), + shader: MESH2D_SHADER_HANDLE, shader_defs, entry_point: "fragment".into(), targets: vec![Some(ColorTargetState { diff --git a/crates/bevy_sprite/src/render/mod.rs b/crates/bevy_sprite/src/render/mod.rs index 3949fbe10d733..a0a16ea612796 100644 --- a/crates/bevy_sprite/src/render/mod.rs +++ b/crates/bevy_sprite/src/render/mod.rs @@ -4,7 +4,7 @@ use crate::{ texture_atlas::{TextureAtlas, TextureAtlasSprite}, Sprite, SPRITE_SHADER_HANDLE, }; -use bevy_asset::{AssetEvent, Assets, Handle, HandleId}; +use bevy_asset::{AssetEvent, AssetId, Assets, Handle}; use bevy_core_pipeline::{ core_2d::Transparent2d, tonemapping::{DebandDither, Tonemapping}, @@ -34,7 +34,7 @@ use bevy_render::{ Extract, }; use bevy_transform::components::GlobalTransform; -use bevy_utils::{FloatOrd, HashMap, Uuid}; +use bevy_utils::{FloatOrd, HashMap}; use bytemuck::{Pod, Zeroable}; use fixedbitset::FixedBitSet; @@ -276,13 +276,13 @@ impl SpecializedRenderPipeline for SpritePipeline { RenderPipelineDescriptor { vertex: VertexState { - shader: SPRITE_SHADER_HANDLE.typed::(), + shader: SPRITE_SHADER_HANDLE, entry_point: "vertex".into(), shader_defs: shader_defs.clone(), buffers: vec![instance_rate_vertex_buffer_layout], }, fragment: Some(FragmentState { - shader: SPRITE_SHADER_HANDLE.typed::(), + shader: SPRITE_SHADER_HANDLE, shader_defs, entry_point: "fragment".into(), targets: vec![Some(ColorTargetState { @@ -320,9 +320,9 @@ pub struct ExtractedSprite { pub rect: Option, /// Change the on-screen size of the sprite pub custom_size: Option, - /// Handle to the [`Image`] of this sprite - /// PERF: storing a `HandleId` instead of `Handle` enables some optimizations (`ExtractedSprite` becomes `Copy` and doesn't need to be dropped) - pub image_handle_id: HandleId, + /// Asset ID of the [`Image`] of this sprite + /// PERF: storing an `AssetId` instead of `Handle` enables some optimizations (`ExtractedSprite` becomes `Copy` and doesn't need to be dropped) + pub image_handle_id: AssetId, pub flip_x: bool, pub flip_y: bool, pub anchor: Vec2, @@ -345,19 +345,8 @@ pub fn extract_sprite_events( let SpriteAssetEvents { ref mut images } = *events; images.clear(); - for image in image_events.read() { - // AssetEvent: !Clone - images.push(match image { - AssetEvent::Created { handle } => AssetEvent::Created { - handle: handle.clone_weak(), - }, - AssetEvent::Modified { handle } => AssetEvent::Modified { - handle: handle.clone_weak(), - }, - AssetEvent::Removed { handle } => AssetEvent::Removed { - handle: handle.clone_weak(), - }, - }); + for event in image_events.read() { + images.push(*event); } } @@ -487,13 +476,13 @@ impl Default for SpriteMeta { #[derive(Component, PartialEq, Eq, Clone)] pub struct SpriteBatch { - image_handle_id: HandleId, + image_handle_id: AssetId, range: Range, } #[derive(Resource, Default)] pub struct ImageBindGroups { - values: HashMap, BindGroup>, + values: HashMap, BindGroup>, } #[allow(clippy::too_many_arguments)] @@ -611,9 +600,11 @@ pub fn prepare_sprites( // If an image has changed, the GpuImage has (probably) changed for event in &events.images { match event { - AssetEvent::Created { .. } => None, - AssetEvent::Modified { handle } | AssetEvent::Removed { handle } => { - image_bind_groups.values.remove(handle) + AssetEvent::Added {..} | + // images don't have dependencies + AssetEvent::LoadedWithDependencies { .. } => {} + AssetEvent::Modified { id } | AssetEvent::Removed { id } => { + image_bind_groups.values.remove(id); } }; } @@ -641,7 +632,7 @@ pub fn prepare_sprites( for mut transparent_phase in &mut phases { let mut batch_item_index = 0; let mut batch_image_size = Vec2::ZERO; - let mut batch_image_handle = HandleId::Id(Uuid::nil(), u64::MAX); + let mut batch_image_handle = AssetId::invalid(); // Iterate through the phase items and detect when successive sprites that can be batched. // Spawn an entity with a `SpriteBatch` component for each possible batch. @@ -652,15 +643,13 @@ pub fn prepare_sprites( // If there is a phase item that is not a sprite, then we must start a new // batch to draw the other phase item(s) and to respect draw order. This can be // done by invalidating the batch_image_handle - batch_image_handle = HandleId::Id(Uuid::nil(), u64::MAX); + batch_image_handle = AssetId::invalid(); continue; }; let batch_image_changed = batch_image_handle != extracted_sprite.image_handle_id; if batch_image_changed { - let Some(gpu_image) = - gpu_images.get(&Handle::weak(extracted_sprite.image_handle_id)) - else { + let Some(gpu_image) = gpu_images.get(extracted_sprite.image_handle_id) else { continue; }; @@ -668,7 +657,7 @@ pub fn prepare_sprites( batch_image_handle = extracted_sprite.image_handle_id; image_bind_groups .values - .entry(Handle::weak(batch_image_handle)) + .entry(batch_image_handle) .or_insert_with(|| { render_device.create_bind_group(&BindGroupDescriptor { entries: &[ @@ -835,7 +824,7 @@ impl RenderCommand

for SetSpriteTextureBindGrou I, image_bind_groups .values - .get(&Handle::weak(batch.image_handle_id)) + .get(&batch.image_handle_id) .unwrap(), &[], ); diff --git a/crates/bevy_sprite/src/texture_atlas.rs b/crates/bevy_sprite/src/texture_atlas.rs index ddd2bab648385..82c2e217862b0 100644 --- a/crates/bevy_sprite/src/texture_atlas.rs +++ b/crates/bevy_sprite/src/texture_atlas.rs @@ -1,16 +1,15 @@ use crate::Anchor; -use bevy_asset::Handle; +use bevy_asset::{Asset, AssetId, Handle}; use bevy_ecs::{component::Component, reflect::ReflectComponent}; use bevy_math::{Rect, Vec2}; -use bevy_reflect::{Reflect, TypeUuid}; +use bevy_reflect::Reflect; use bevy_render::{color::Color, texture::Image}; use bevy_utils::HashMap; /// An atlas containing multiple textures (like a spritesheet or a tilemap). /// [Example usage animating sprite.](https://github.com/bevyengine/bevy/blob/latest/examples/2d/sprite_sheet.rs) /// [Example usage loading sprite sheet.](https://github.com/bevyengine/bevy/blob/latest/examples/2d/texture_atlas.rs) -#[derive(Reflect, Debug, Clone, TypeUuid)] -#[uuid = "7233c597-ccfa-411f-bd59-9af349432ada"] +#[derive(Asset, Reflect, Debug, Clone)] #[reflect(Debug)] pub struct TextureAtlas { /// The handle to the texture in which the sprites are stored @@ -20,7 +19,7 @@ pub struct TextureAtlas { /// The specific areas of the atlas where each texture can be found pub textures: Vec, /// Mapping from texture handle to index - pub texture_handles: Option, usize>>, + pub(crate) texture_handles: Option, usize>>, } #[derive(Component, Debug, Clone, Reflect)] @@ -148,9 +147,10 @@ impl TextureAtlas { } /// Returns the index of the texture corresponding to the given image handle in the [`TextureAtlas`] - pub fn get_texture_index(&self, texture: &Handle) -> Option { + pub fn get_texture_index(&self, texture: impl Into>) -> Option { + let id = texture.into(); self.texture_handles .as_ref() - .and_then(|texture_handles| texture_handles.get(texture).cloned()) + .and_then(|texture_handles| texture_handles.get(&id).cloned()) } } diff --git a/crates/bevy_sprite/src/texture_atlas_builder.rs b/crates/bevy_sprite/src/texture_atlas_builder.rs index f77e0cce90c31..c57e1f11d208e 100644 --- a/crates/bevy_sprite/src/texture_atlas_builder.rs +++ b/crates/bevy_sprite/src/texture_atlas_builder.rs @@ -1,4 +1,4 @@ -use bevy_asset::{Assets, Handle}; +use bevy_asset::{AssetId, Assets}; use bevy_log::{debug, error, warn}; use bevy_math::{Rect, Vec2}; use bevy_render::{ @@ -29,7 +29,7 @@ pub enum TextureAtlasBuilderError { pub struct TextureAtlasBuilder { /// The grouped rects which must be placed with a key value pair of a /// texture handle to an index. - rects_to_place: GroupedRectsToPlace>, + rects_to_place: GroupedRectsToPlace>, /// The initial atlas size in pixels. initial_size: Vec2, /// The absolute maximum size of the texture atlas in pixels. @@ -80,9 +80,9 @@ impl TextureAtlasBuilder { } /// Adds a texture to be copied to the texture atlas. - pub fn add_texture(&mut self, texture_handle: Handle, texture: &Image) { + pub fn add_texture(&mut self, image_id: AssetId, texture: &Image) { self.rects_to_place.push_rect( - texture_handle, + image_id, None, RectToInsert::new( texture.texture_descriptor.size.width, @@ -207,16 +207,16 @@ impl TextureAtlasBuilder { let rect_placements = rect_placements.ok_or(TextureAtlasBuilderError::NotEnoughSpace)?; let mut texture_rects = Vec::with_capacity(rect_placements.packed_locations().len()); - let mut texture_handles = HashMap::default(); - for (texture_handle, (_, packed_location)) in rect_placements.packed_locations().iter() { - let texture = textures.get(texture_handle).unwrap(); + let mut texture_ids = HashMap::default(); + for (image_id, (_, packed_location)) in rect_placements.packed_locations().iter() { + let texture = textures.get(*image_id).unwrap(); let min = Vec2::new(packed_location.x() as f32, packed_location.y() as f32); let max = min + Vec2::new( packed_location.width() as f32, packed_location.height() as f32, ); - texture_handles.insert(texture_handle.clone_weak(), texture_rects.len()); + texture_ids.insert(*image_id, texture_rects.len()); texture_rects.push(Rect { min, max }); if texture.texture_descriptor.format != self.format && !self.auto_format_conversion { warn!( @@ -234,7 +234,7 @@ impl TextureAtlasBuilder { ), texture: textures.add(atlas_texture), textures: texture_rects, - texture_handles: Some(texture_handles), + texture_handles: Some(texture_ids), }) } } diff --git a/crates/bevy_text/Cargo.toml b/crates/bevy_text/Cargo.toml index 578526a1e3a0e..9055e3a97dac3 100644 --- a/crates/bevy_text/Cargo.toml +++ b/crates/bevy_text/Cargo.toml @@ -26,7 +26,6 @@ bevy_window = { path = "../bevy_window", version = "0.12.0-dev" } bevy_utils = { path = "../bevy_utils", version = "0.12.0-dev" } # other -anyhow = "1.0.4" ab_glyph = "0.2.6" glyph_brush_layout = "0.2.1" thiserror = "1.0" diff --git a/crates/bevy_text/src/font.rs b/crates/bevy_text/src/font.rs index 1d8a465a76ea2..e968e37febb85 100644 --- a/crates/bevy_text/src/font.rs +++ b/crates/bevy_text/src/font.rs @@ -1,12 +1,12 @@ use ab_glyph::{FontArc, FontVec, InvalidFont, OutlinedGlyph}; -use bevy_reflect::{TypePath, TypeUuid}; +use bevy_asset::Asset; +use bevy_reflect::TypePath; use bevy_render::{ render_resource::{Extent3d, TextureDimension, TextureFormat}, texture::Image, }; -#[derive(Debug, TypeUuid, TypePath, Clone)] -#[uuid = "97059ac6-c9ba-4da9-95b6-bed82c3ce198"] +#[derive(Asset, TypePath, Debug, Clone)] pub struct Font { pub font: FontArc, } diff --git a/crates/bevy_text/src/font_atlas_set.rs b/crates/bevy_text/src/font_atlas_set.rs index f28b4138ad8d0..d4c2c9f5073ad 100644 --- a/crates/bevy_text/src/font_atlas_set.rs +++ b/crates/bevy_text/src/font_atlas_set.rs @@ -1,9 +1,9 @@ use crate::{error::TextError, Font, FontAtlas}; use ab_glyph::{GlyphId, OutlinedGlyph, Point}; +use bevy_asset::{AssetEvent, AssetId}; use bevy_asset::{Assets, Handle}; +use bevy_ecs::prelude::*; use bevy_math::Vec2; -use bevy_reflect::TypePath; -use bevy_reflect::TypeUuid; use bevy_render::texture::Image; use bevy_sprite::TextureAtlas; use bevy_utils::FloatOrd; @@ -11,8 +11,31 @@ use bevy_utils::HashMap; type FontSizeKey = FloatOrd; -#[derive(TypeUuid, TypePath)] -#[uuid = "73ba778b-b6b5-4f45-982d-d21b6b86ace2"] +#[derive(Default, Resource)] +pub struct FontAtlasSets { + // PERF: in theory this could be optimized with Assets storage ... consider making some fast "simple" AssetMap + pub(crate) sets: HashMap, FontAtlasSet>, +} + +impl FontAtlasSets { + pub fn get(&self, id: impl Into>) -> Option<&FontAtlasSet> { + let id: AssetId = id.into(); + self.sets.get(&id) + } +} + +pub fn remove_dropped_font_atlas_sets( + mut font_atlas_sets: ResMut, + mut font_events: EventReader>, +) { + // Clean up font atlas sets for removed fonts + for event in font_events.read() { + if let AssetEvent::Removed { id } = event { + font_atlas_sets.sets.remove(id); + } + } +} + pub struct FontAtlasSet { font_atlases: HashMap>, } diff --git a/crates/bevy_text/src/font_loader.rs b/crates/bevy_text/src/font_loader.rs index e179ec9ccf82e..ec784fb84e40a 100644 --- a/crates/bevy_text/src/font_loader.rs +++ b/crates/bevy_text/src/font_loader.rs @@ -1,21 +1,23 @@ use crate::Font; -use anyhow::Result; -use bevy_asset::{AssetLoader, LoadContext, LoadedAsset}; -use bevy_utils::BoxedFuture; +use bevy_asset::{anyhow::Error, io::Reader, AssetLoader, AsyncReadExt, LoadContext}; #[derive(Default)] pub struct FontLoader; impl AssetLoader for FontLoader { + type Asset = Font; + type Settings = (); fn load<'a>( &'a self, - bytes: &'a [u8], - load_context: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<()>> { + reader: &'a mut Reader, + _settings: &'a (), + _load_context: &'a mut LoadContext, + ) -> bevy_utils::BoxedFuture<'a, Result> { Box::pin(async move { - let font = Font::try_from_bytes(bytes.into())?; - load_context.set_default_asset(LoadedAsset::new(font)); - Ok(()) + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; + let font = Font::try_from_bytes(bytes)?; + Ok(font) }) } diff --git a/crates/bevy_text/src/glyph_brush.rs b/crates/bevy_text/src/glyph_brush.rs index d9d04a1ba125e..61a837303a26c 100644 --- a/crates/bevy_text/src/glyph_brush.rs +++ b/crates/bevy_text/src/glyph_brush.rs @@ -1,5 +1,5 @@ use ab_glyph::{Font as _, FontArc, Glyph, PxScaleFont, ScaleFont as _}; -use bevy_asset::{Assets, Handle}; +use bevy_asset::{AssetId, Assets}; use bevy_math::{Rect, Vec2}; use bevy_render::texture::Image; use bevy_sprite::TextureAtlas; @@ -10,13 +10,13 @@ use glyph_brush_layout::{ }; use crate::{ - error::TextError, BreakLineOn, Font, FontAtlasSet, FontAtlasWarning, GlyphAtlasInfo, - TextAlignment, TextSettings, YAxisOrientation, + error::TextError, BreakLineOn, Font, FontAtlasSet, FontAtlasSets, FontAtlasWarning, + GlyphAtlasInfo, TextAlignment, TextSettings, YAxisOrientation, }; pub struct GlyphBrush { fonts: Vec, - handles: Vec>, + asset_ids: Vec>, latest_font_id: FontId, } @@ -24,7 +24,7 @@ impl Default for GlyphBrush { fn default() -> Self { GlyphBrush { fonts: Vec::new(), - handles: Vec::new(), + asset_ids: Vec::new(), latest_font_id: FontId(0), } } @@ -57,7 +57,7 @@ impl GlyphBrush { &self, glyphs: Vec, sections: &[SectionText], - font_atlas_set_storage: &mut Assets, + font_atlas_sets: &mut FontAtlasSets, fonts: &Assets, texture_atlases: &mut Assets, textures: &mut Assets, @@ -72,11 +72,11 @@ impl GlyphBrush { let sections_data = sections .iter() .map(|section| { - let handle = &self.handles[section.font_id.0]; - let font = fonts.get(handle).ok_or(TextError::NoSuchFont)?; + let asset_id = &self.asset_ids[section.font_id.0]; + let font = fonts.get(*asset_id).ok_or(TextError::NoSuchFont)?; let font_size = section.scale.y; Ok(( - handle, + asset_id, font, font_size, ab_glyph::Font::as_scaled(&font.font, font_size), @@ -100,9 +100,10 @@ impl GlyphBrush { let section_data = sections_data[sg.section_index]; if let Some(outlined_glyph) = section_data.1.font.outline_glyph(glyph) { let bounds = outlined_glyph.px_bounds(); - let handle_font_atlas: Handle = section_data.0.cast_weak(); - let font_atlas_set = font_atlas_set_storage - .get_or_insert_with(handle_font_atlas, FontAtlasSet::default); + let font_atlas_set = font_atlas_sets + .sets + .entry(*section_data.0) + .or_insert_with(FontAtlasSet::default); let atlas_info = font_atlas_set .get_glyph_atlas_info(section_data.2, glyph_id, glyph_position) @@ -148,9 +149,9 @@ impl GlyphBrush { Ok(positioned_glyphs) } - pub fn add_font(&mut self, handle: Handle, font: FontArc) -> FontId { + pub fn add_font(&mut self, asset_id: AssetId, font: FontArc) -> FontId { self.fonts.push(font); - self.handles.push(handle); + self.asset_ids.push(asset_id); let font_id = self.latest_font_id; self.latest_font_id = FontId(font_id.0 + 1); font_id diff --git a/crates/bevy_text/src/lib.rs b/crates/bevy_text/src/lib.rs index 410dbb2cfac1f..f5498d26ef1fb 100644 --- a/crates/bevy_text/src/lib.rs +++ b/crates/bevy_text/src/lib.rs @@ -28,9 +28,8 @@ pub mod prelude { use bevy_app::prelude::*; #[cfg(feature = "default_font")] use bevy_asset::load_internal_binary_asset; -use bevy_asset::{AddAsset, HandleUntyped}; +use bevy_asset::{AssetApp, Handle}; use bevy_ecs::prelude::*; -use bevy_reflect::TypeUuid; use bevy_render::{camera::CameraUpdateSystem, ExtractSchedule, RenderApp}; use bevy_sprite::SpriteSystem; use std::num::NonZeroUsize; @@ -70,14 +69,9 @@ pub enum YAxisOrientation { BottomToTop, } -pub const DEFAULT_FONT_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Font::TYPE_UUID, 1491772431825224042); - impl Plugin for TextPlugin { fn build(&self, app: &mut App) { - app.add_asset::() - .add_debug_asset::() - .add_asset::() + app.init_asset::() .register_type::() .register_type::() .register_type::() @@ -88,15 +82,19 @@ impl Plugin for TextPlugin { .init_asset_loader::() .init_resource::() .init_resource::() + .init_resource::() .insert_resource(TextPipeline::default()) .add_systems( PostUpdate, - update_text2d_layout - // Potential conflict: `Assets` - // In practice, they run independently since `bevy_render::camera_update_system` - // will only ever observe its own render target, and `update_text2d_layout` - // will never modify a pre-existing `Image` asset. - .ambiguous_with(CameraUpdateSystem), + ( + update_text2d_layout + // Potential conflict: `Assets` + // In practice, they run independently since `bevy_render::camera_update_system` + // will only ever observe its own render target, and `update_text2d_layout` + // will never modify a pre-existing `Image` asset. + .ambiguous_with(CameraUpdateSystem), + font_atlas_set::remove_dropped_font_atlas_sets, + ), ); if let Ok(render_app) = app.get_sub_app_mut(RenderApp) { @@ -109,7 +107,7 @@ impl Plugin for TextPlugin { #[cfg(feature = "default_font")] load_internal_binary_asset!( app, - DEFAULT_FONT_HANDLE, + Handle::default(), "FiraMono-subset.ttf", |bytes: &[u8], _path: String| { Font::try_from_bytes(bytes.to_vec()).unwrap() } ); diff --git a/crates/bevy_text/src/pipeline.rs b/crates/bevy_text/src/pipeline.rs index acde2c23ce612..fa616e499944b 100644 --- a/crates/bevy_text/src/pipeline.rs +++ b/crates/bevy_text/src/pipeline.rs @@ -1,24 +1,22 @@ -use ab_glyph::{Font as AbglyphFont, PxScale}; -use bevy_asset::{Assets, Handle, HandleId}; +use crate::{ + compute_text_bounds, error::TextError, glyph_brush::GlyphBrush, scale_value, BreakLineOn, Font, + FontAtlasSets, FontAtlasWarning, PositionedGlyph, Text, TextAlignment, TextSection, + TextSettings, YAxisOrientation, +}; +use ab_glyph::PxScale; +use bevy_asset::{AssetId, Assets, Handle}; use bevy_ecs::component::Component; use bevy_ecs::system::Resource; use bevy_math::Vec2; use bevy_render::texture::Image; use bevy_sprite::TextureAtlas; use bevy_utils::HashMap; - use glyph_brush_layout::{FontId, GlyphPositioner, SectionGeometry, SectionText, ToSectionText}; -use crate::{ - compute_text_bounds, error::TextError, glyph_brush::GlyphBrush, scale_value, BreakLineOn, Font, - FontAtlasSet, FontAtlasWarning, PositionedGlyph, Text, TextAlignment, TextSection, - TextSettings, YAxisOrientation, -}; - #[derive(Default, Resource)] pub struct TextPipeline { brush: GlyphBrush, - map_font_id: HashMap, + map_font_id: HashMap, FontId>, } /// Render information for a corresponding [`Text`](crate::Text) component. @@ -36,7 +34,7 @@ impl TextPipeline { *self .map_font_id .entry(handle.id()) - .or_insert_with(|| brush.add_font(handle.clone(), font.font.clone())) + .or_insert_with(|| brush.add_font(handle.id(), font.font.clone())) } #[allow(clippy::too_many_arguments)] @@ -48,7 +46,7 @@ impl TextPipeline { text_alignment: TextAlignment, linebreak_behavior: BreakLineOn, bounds: Vec2, - font_atlas_set_storage: &mut Assets, + font_atlas_sets: &mut FontAtlasSets, texture_atlases: &mut Assets, textures: &mut Assets, text_settings: &TextSettings, @@ -90,7 +88,7 @@ impl TextPipeline { let glyphs = self.brush.process_glyphs( section_glyphs, §ions, - font_atlas_set_storage, + font_atlas_sets, fonts, texture_atlases, textures, @@ -193,7 +191,7 @@ impl TextMeasureInfo { compute_text_bounds(§ion_glyphs, |index| { let font = &self.fonts[index]; let font_size = self.sections[index].scale; - font.into_scaled(font_size) + ab_glyph::Font::into_scaled(font, font_size) }) .size() } diff --git a/crates/bevy_text/src/text.rs b/crates/bevy_text/src/text.rs index 92e2bd0cce213..894289445ae89 100644 --- a/crates/bevy_text/src/text.rs +++ b/crates/bevy_text/src/text.rs @@ -5,7 +5,7 @@ use bevy_render::color::Color; use bevy_utils::default; use serde::{Deserialize, Serialize}; -use crate::{Font, DEFAULT_FONT_HANDLE}; +use crate::Font; #[derive(Component, Debug, Clone, Reflect)] #[reflect(Component, Default)] @@ -182,7 +182,7 @@ pub struct TextStyle { impl Default for TextStyle { fn default() -> Self { Self { - font: DEFAULT_FONT_HANDLE.typed(), + font: Default::default(), font_size: 12.0, color: Color::WHITE, } diff --git a/crates/bevy_text/src/text2d.rs b/crates/bevy_text/src/text2d.rs index 79fcc40094fc0..1f26e47d21367 100644 --- a/crates/bevy_text/src/text2d.rs +++ b/crates/bevy_text/src/text2d.rs @@ -1,3 +1,7 @@ +use crate::{ + BreakLineOn, Font, FontAtlasSets, FontAtlasWarning, PositionedGlyph, Text, TextError, + TextLayoutInfo, TextPipeline, TextSettings, YAxisOrientation, +}; use bevy_asset::Assets; use bevy_ecs::{ bundle::Bundle, @@ -22,11 +26,6 @@ use bevy_transform::prelude::{GlobalTransform, Transform}; use bevy_utils::HashSet; use bevy_window::{PrimaryWindow, Window, WindowScaleFactorChanged}; -use crate::{ - BreakLineOn, Font, FontAtlasSet, FontAtlasWarning, PositionedGlyph, Text, TextError, - TextLayoutInfo, TextPipeline, TextSettings, YAxisOrientation, -}; - /// The maximum width and height of text. The text will wrap according to the specified size. /// Characters out of the bounds after wrapping will be truncated. Text is aligned according to the /// specified [`TextAlignment`](crate::text::TextAlignment). @@ -160,7 +159,7 @@ pub fn update_text2d_layout( windows: Query<&Window, With>, mut scale_factor_changed: EventReader, mut texture_atlases: ResMut>, - mut font_atlas_set_storage: ResMut>, + mut font_atlas_sets: ResMut, mut text_pipeline: ResMut, mut text_query: Query<(Entity, Ref, Ref, &mut TextLayoutInfo)>, ) { @@ -191,7 +190,7 @@ pub fn update_text2d_layout( text.alignment, text.linebreak_behavior, text_bounds, - &mut font_atlas_set_storage, + &mut font_atlas_sets, &mut texture_atlases, &mut textures, text_settings.as_ref(), diff --git a/crates/bevy_ui/src/lib.rs b/crates/bevy_ui/src/lib.rs index fdd4dd432bf06..ae9f9f41885d9 100644 --- a/crates/bevy_ui/src/lib.rs +++ b/crates/bevy_ui/src/lib.rs @@ -4,15 +4,7 @@ //! # Basic usage //! Spawn UI elements with [`node_bundles::ButtonBundle`], [`node_bundles::ImageBundle`], [`node_bundles::TextBundle`] and [`node_bundles::NodeBundle`] //! This UI is laid out with the Flexbox and CSS Grid layout models (see ) -mod focus; -mod geometry; -mod layout; -mod render; -mod stack; -mod ui_node; -#[cfg(feature = "bevy_text")] -mod accessibility; pub mod camera_config; pub mod measurement; pub mod node_bundles; @@ -22,8 +14,14 @@ pub mod widget; use bevy_derive::{Deref, DerefMut}; use bevy_reflect::Reflect; #[cfg(feature = "bevy_text")] -use bevy_render::camera::CameraUpdateSystem; -use bevy_render::{extract_component::ExtractComponentPlugin, RenderApp}; +mod accessibility; +mod focus; +mod geometry; +mod layout; +mod render; +mod stack; +mod ui_node; + pub use focus::*; pub use geometry::*; pub use layout::*; @@ -43,8 +41,10 @@ pub mod prelude { use crate::prelude::UiCameraConfig; use bevy_app::prelude::*; +use bevy_asset::Assets; use bevy_ecs::prelude::*; use bevy_input::InputSystem; +use bevy_render::{extract_component::ExtractComponentPlugin, texture::Image, RenderApp}; use bevy_transform::TransformSystem; use stack::ui_stack_system; pub use stack::UiStack; @@ -137,12 +137,14 @@ impl Plugin for UiPlugin { // In practice, they run independently since `bevy_render::camera_update_system` // will only ever observe its own render target, and `widget::measure_text_system` // will never modify a pre-existing `Image` asset. - .ambiguous_with(CameraUpdateSystem) + .ambiguous_with(bevy_render::camera::CameraUpdateSystem) // Potential conflict: `Assets` // Since both systems will only ever insert new [`Image`] assets, // they will never observe each other's effects. .ambiguous_with(bevy_text::update_text2d_layout), - widget::text_system.after(UiSystem::Layout), + widget::text_system + .after(UiSystem::Layout) + .before(Assets::::track_assets), ), ); #[cfg(feature = "bevy_text")] diff --git a/crates/bevy_ui/src/render/mod.rs b/crates/bevy_ui/src/render/mod.rs index 2f4229b5b83f5..c4195a6f41200 100644 --- a/crates/bevy_ui/src/render/mod.rs +++ b/crates/bevy_ui/src/render/mod.rs @@ -16,11 +16,9 @@ use crate::{ }; use bevy_app::prelude::*; -use bevy_asset::{load_internal_asset, AssetEvent, Assets, Handle, HandleId, HandleUntyped}; +use bevy_asset::{load_internal_asset, AssetEvent, AssetId, Assets, Handle}; use bevy_ecs::prelude::*; use bevy_math::{Mat4, Rect, URect, UVec4, Vec2, Vec3, Vec4Swizzles}; -use bevy_reflect::TypeUuid; -use bevy_render::texture::DEFAULT_IMAGE_HANDLE; use bevy_render::{ camera::Camera, color::Color, @@ -33,13 +31,11 @@ use bevy_render::{ view::{ExtractedView, ViewUniforms}, Extract, RenderApp, RenderSet, }; -use bevy_sprite::SpriteAssetEvents; -use bevy_sprite::TextureAtlas; +use bevy_sprite::{SpriteAssetEvents, TextureAtlas}; #[cfg(feature = "bevy_text")] use bevy_text::{PositionedGlyph, Text, TextLayoutInfo}; use bevy_transform::components::GlobalTransform; -use bevy_utils::HashMap; -use bevy_utils::{FloatOrd, Uuid}; +use bevy_utils::{FloatOrd, HashMap}; use bytemuck::{Pod, Zeroable}; use std::ops::Range; @@ -54,8 +50,7 @@ pub mod draw_ui_graph { } } -pub const UI_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 13012847047162779583); +pub const UI_SHADER_HANDLE: Handle = Handle::weak_from_u128(13012847047162779583); #[derive(Debug, Hash, PartialEq, Eq, Clone, SystemSet)] pub enum RenderUiSystem { @@ -159,7 +154,7 @@ pub struct ExtractedUiNode { pub transform: Mat4, pub color: Color, pub rect: Rect, - pub image: Handle, + pub image: AssetId, pub atlas_size: Option, pub clip: Option, pub flip_x: bool, @@ -249,7 +244,7 @@ pub fn extract_atlas_uinodes( color: color.0, rect: atlas_rect, clip: clip.map(|clip| clip.clip), - image, + image: image.id(), atlas_size: Some(atlas_size), flip_x: atlas_image.flip_x, flip_y: atlas_image.flip_y, @@ -293,7 +288,7 @@ pub fn extract_uinode_borders( >, node_query: Extract>, ) { - let image = bevy_render::texture::DEFAULT_IMAGE_HANDLE.typed(); + let image = AssetId::::default(); let ui_logical_viewport_size = windows .get_single() @@ -381,7 +376,7 @@ pub fn extract_uinode_borders( max: edge.size(), ..Default::default() }, - image: image.clone_weak(), + image, atlas_size: None, clip: clip.map(|clip| clip.clip), flip_x: false, @@ -427,9 +422,9 @@ pub fn extract_uinodes( if !images.contains(&image.texture) { continue; } - (image.texture.clone_weak(), image.flip_x, image.flip_y) + (image.texture.id(), image.flip_x, image.flip_y) } else { - (DEFAULT_IMAGE_HANDLE.typed(), false, false) + (AssetId::default(), false, false) }; extracted_uinodes.uinodes.insert( @@ -591,7 +586,7 @@ pub fn extract_text_uinodes( * Mat4::from_translation(position.extend(0.) * inverse_scale_factor), color, rect, - image: atlas.texture.clone_weak(), + image: atlas.texture.id(), atlas_size: Some(atlas.size * inverse_scale_factor), clip: clip.map(|clip| clip.clip), flip_x: false, @@ -639,7 +634,7 @@ const QUAD_INDICES: [usize; 6] = [0, 2, 3, 0, 1, 2]; #[derive(Component)] pub struct UiBatch { pub range: Range, - pub image_handle_id: HandleId, + pub image: AssetId, } const TEXTURED_QUAD: u32 = 0; @@ -679,7 +674,7 @@ pub fn queue_uinodes( #[derive(Resource, Default)] pub struct UiImageBindGroups { - pub values: HashMap, BindGroup>, + pub values: HashMap, BindGroup>, } #[allow(clippy::too_many_arguments)] @@ -700,16 +695,18 @@ pub fn prepare_uinodes( // If an image has changed, the GpuImage has (probably) changed for event in &events.images { match event { - AssetEvent::Created { .. } => None, - AssetEvent::Modified { handle } | AssetEvent::Removed { handle } => { - image_bind_groups.values.remove(handle) + AssetEvent::Added { .. } | + // Images don't have dependencies + AssetEvent::LoadedWithDependencies { .. } => {} + AssetEvent::Modified { id } | AssetEvent::Removed { id } => { + image_bind_groups.values.remove(id); } }; } #[inline] - fn is_textured(image: &Handle) -> bool { - image.id() != DEFAULT_IMAGE_HANDLE.id() + fn is_textured(image: AssetId) -> bool { + image != AssetId::default() } if let Some(view_binding) = view_uniforms.uniforms.binding() { @@ -730,30 +727,30 @@ pub fn prepare_uinodes( for mut ui_phase in &mut phases { let mut batch_item_index = 0; - let mut batch_image_handle = HandleId::Id(Uuid::nil(), u64::MAX); + let mut batch_image_handle = AssetId::invalid(); for item_index in 0..ui_phase.items.len() { let item = &mut ui_phase.items[item_index]; if let Some(extracted_uinode) = extracted_uinodes.uinodes.get(item.entity) { let mut existing_batch = batches .last_mut() - .filter(|_| batch_image_handle == extracted_uinode.image.id()); + .filter(|_| batch_image_handle == extracted_uinode.image); if existing_batch.is_none() { - if let Some(gpu_image) = gpu_images.get(&extracted_uinode.image) { + if let Some(gpu_image) = gpu_images.get(extracted_uinode.image) { batch_item_index = item_index; - batch_image_handle = extracted_uinode.image.id(); + batch_image_handle = extracted_uinode.image; let new_batch = UiBatch { range: index..index, - image_handle_id: extracted_uinode.image.id(), + image: extracted_uinode.image, }; batches.push((item.entity, new_batch)); image_bind_groups .values - .entry(Handle::weak(batch_image_handle)) + .entry(batch_image_handle) .or_insert_with(|| { render_device.create_bind_group(&BindGroupDescriptor { entries: &[ @@ -781,7 +778,7 @@ pub fn prepare_uinodes( } } - let mode = if is_textured(&extracted_uinode.image) { + let mode = if is_textured(extracted_uinode.image) { TEXTURED_QUAD } else { UNTEXTURED_QUAD @@ -897,7 +894,7 @@ pub fn prepare_uinodes( existing_batch.unwrap().1.range.end = index; ui_phase.items[batch_item_index].batch_size += 1; } else { - batch_image_handle = HandleId::Id(Uuid::nil(), u64::MAX); + batch_image_handle = AssetId::invalid(); } } } diff --git a/crates/bevy_ui/src/render/pipeline.rs b/crates/bevy_ui/src/render/pipeline.rs index f6b4b0cc3c1ea..eaef41dbf4e90 100644 --- a/crates/bevy_ui/src/render/pipeline.rs +++ b/crates/bevy_ui/src/render/pipeline.rs @@ -85,13 +85,13 @@ impl SpecializedRenderPipeline for UiPipeline { RenderPipelineDescriptor { vertex: VertexState { - shader: super::UI_SHADER_HANDLE.typed::(), + shader: super::UI_SHADER_HANDLE, entry_point: "vertex".into(), shader_defs: shader_defs.clone(), buffers: vec![vertex_layout], }, fragment: Some(FragmentState { - shader: super::UI_SHADER_HANDLE.typed::(), + shader: super::UI_SHADER_HANDLE, shader_defs, entry_point: "fragment".into(), targets: vec![Some(ColorTargetState { diff --git a/crates/bevy_ui/src/render/render_pass.rs b/crates/bevy_ui/src/render/render_pass.rs index 697aa11104c7e..1eb3836b8c634 100644 --- a/crates/bevy_ui/src/render/render_pass.rs +++ b/crates/bevy_ui/src/render/render_pass.rs @@ -1,6 +1,5 @@ use super::{UiBatch, UiImageBindGroups, UiMeta}; use crate::{prelude::UiCameraConfig, DefaultCameraView}; -use bevy_asset::Handle; use bevy_ecs::{ prelude::*, system::{lifetimeless::*, SystemParamItem}, @@ -173,14 +172,7 @@ impl RenderCommand

for SetUiTextureBindGroup pass: &mut TrackedRenderPass<'w>, ) -> RenderCommandResult { let image_bind_groups = image_bind_groups.into_inner(); - pass.set_bind_group( - I, - image_bind_groups - .values - .get(&Handle::weak(batch.image_handle_id)) - .unwrap(), - &[], - ); + pass.set_bind_group(I, image_bind_groups.values.get(&batch.image).unwrap(), &[]); RenderCommandResult::Success } } diff --git a/crates/bevy_ui/src/ui_node.rs b/crates/bevy_ui/src/ui_node.rs index 1dfbbcd0204a4..cc83a6fbe43c0 100644 --- a/crates/bevy_ui/src/ui_node.rs +++ b/crates/bevy_ui/src/ui_node.rs @@ -3,10 +3,7 @@ use bevy_asset::Handle; use bevy_ecs::{prelude::Component, reflect::ReflectComponent}; use bevy_math::{Rect, Vec2}; use bevy_reflect::prelude::*; -use bevy_render::{ - color::Color, - texture::{Image, DEFAULT_IMAGE_HANDLE}, -}; +use bevy_render::{color::Color, texture::Image}; use bevy_transform::prelude::GlobalTransform; use serde::{Deserialize, Serialize}; use smallvec::SmallVec; @@ -1628,7 +1625,7 @@ impl Default for BorderColor { } /// The 2D texture displayed for this UI node -#[derive(Component, Clone, Debug, Reflect)] +#[derive(Component, Clone, Debug, Reflect, Default)] #[reflect(Component, Default)] pub struct UiImage { /// Handle to the texture @@ -1639,16 +1636,6 @@ pub struct UiImage { pub flip_y: bool, } -impl Default for UiImage { - fn default() -> UiImage { - UiImage { - texture: DEFAULT_IMAGE_HANDLE.typed(), - flip_x: false, - flip_y: false, - } - } -} - impl UiImage { pub fn new(texture: Handle) -> Self { Self { diff --git a/crates/bevy_ui/src/widget/text.rs b/crates/bevy_ui/src/widget/text.rs index 22c7285a806d6..ff042093c5dfa 100644 --- a/crates/bevy_ui/src/widget/text.rs +++ b/crates/bevy_ui/src/widget/text.rs @@ -12,7 +12,7 @@ use bevy_reflect::{std_traits::ReflectDefault, Reflect}; use bevy_render::texture::Image; use bevy_sprite::TextureAtlas; use bevy_text::{ - BreakLineOn, Font, FontAtlasSet, FontAtlasWarning, Text, TextError, TextLayoutInfo, + BreakLineOn, Font, FontAtlasSets, FontAtlasWarning, Text, TextError, TextLayoutInfo, TextMeasureInfo, TextPipeline, TextSettings, YAxisOrientation, }; use bevy_window::{PrimaryWindow, Window}; @@ -148,7 +148,7 @@ fn queue_text( fonts: &Assets, text_pipeline: &mut TextPipeline, font_atlas_warning: &mut FontAtlasWarning, - font_atlas_set_storage: &mut Assets, + font_atlas_sets: &mut FontAtlasSets, texture_atlases: &mut Assets, textures: &mut Assets, text_settings: &TextSettings, @@ -175,7 +175,7 @@ fn queue_text( text.alignment, text.linebreak_behavior, physical_node_size, - font_atlas_set_storage, + font_atlas_sets, texture_atlases, textures, text_settings, @@ -215,7 +215,7 @@ pub fn text_system( mut font_atlas_warning: ResMut, ui_scale: Res, mut texture_atlases: ResMut>, - mut font_atlas_set_storage: ResMut>, + mut font_atlas_sets: ResMut, mut text_pipeline: ResMut, mut text_query: Query<(Ref, &Text, &mut TextLayoutInfo, &mut TextFlags)>, ) { @@ -235,7 +235,7 @@ pub fn text_system( &fonts, &mut text_pipeline, &mut font_atlas_warning, - &mut font_atlas_set_storage, + &mut font_atlas_sets, &mut texture_atlases, &mut textures, &text_settings, @@ -256,7 +256,7 @@ pub fn text_system( &fonts, &mut text_pipeline, &mut font_atlas_warning, - &mut font_atlas_set_storage, + &mut font_atlas_sets, &mut texture_atlases, &mut textures, &text_settings, diff --git a/docs/cargo_features.md b/docs/cargo_features.md index 497fa403a332c..9c7ee1701fa5d 100644 --- a/docs/cargo_features.md +++ b/docs/cargo_features.md @@ -28,7 +28,6 @@ The default feature set enables most of the expected features of a game engine, |bevy_ui|A custom ECS-driven UI framework| |bevy_winit|winit window and input backend| |default_font|Include a default font, containing only ASCII characters, at the cost of a 20kB binary size increase| -|filesystem_watcher|Enable watching file system for asset hot reload| |hdr|HDR image format support| |ktx2|KTX2 compressed texture support| |multi-threaded|Enables multithreaded parallelism in the engine. Disabling it forces all engine tasks to run on a single thread.| @@ -49,10 +48,10 @@ The default feature set enables most of the expected features of a game engine, |bevy_dynamic_plugin|Plugin for dynamic loading (using [libloading](https://crates.io/crates/libloading))| |bmp|BMP image format support| |dds|DDS compressed texture support| -|debug_asset_server|Enable the "debug asset server" for hot reloading internal assets| |detailed_trace|Enable detailed trace event logging. These trace events are expensive even when off, thus they require compile time opt-in| |dynamic_linking|Force dynamic linking, which improves iterative compile times| |exr|EXR image format support| +|filesystem_watcher|Enables watching the filesystem for Bevy Asset hot-reloading| |flac|FLAC audio format support| |glam_assert|Enable assertions to check the validity of parameters passed to glam| |jpeg|JPEG image format support| diff --git a/examples/2d/custom_gltf_vertex_attribute.rs b/examples/2d/custom_gltf_vertex_attribute.rs index c0dcd5d5784c5..2f602d92b64fa 100644 --- a/examples/2d/custom_gltf_vertex_attribute.rs +++ b/examples/2d/custom_gltf_vertex_attribute.rs @@ -1,12 +1,14 @@ //! Renders a glTF mesh in 2D with a custom vertex attribute. -use bevy::gltf::GltfPlugin; -use bevy::prelude::*; -use bevy::reflect::{TypePath, TypeUuid}; -use bevy::render::mesh::{MeshVertexAttribute, MeshVertexBufferLayout}; -use bevy::render::render_resource::*; -use bevy::sprite::{ - Material2d, Material2dKey, Material2dPlugin, MaterialMesh2dBundle, Mesh2dHandle, +use bevy::{ + gltf::GltfPlugin, + prelude::*, + reflect::TypePath, + render::{ + mesh::{MeshVertexAttribute, MeshVertexBufferLayout}, + render_resource::*, + }, + sprite::{Material2d, Material2dKey, Material2dPlugin, MaterialMesh2dBundle, Mesh2dHandle}, }; /// This vertex attribute supplies barycentric coordinates for each triangle. @@ -55,8 +57,7 @@ fn setup( /// This custom material uses barycentric coordinates from /// `ATTRIBUTE_BARYCENTRIC` to shade a white border around each triangle. The /// thickness of the border is animated using the global time shader uniform. -#[derive(AsBindGroup, TypeUuid, TypePath, Debug, Clone)] -#[uuid = "50ffce9e-1582-42e9-87cb-2233724426c0"] +#[derive(Asset, TypePath, AsBindGroup, Debug, Clone)] struct CustomMaterial {} impl Material2d for CustomMaterial { diff --git a/examples/2d/mesh2d_manual.rs b/examples/2d/mesh2d_manual.rs index 7a3ee4a9d68d8..5a9e0547751b8 100644 --- a/examples/2d/mesh2d_manual.rs +++ b/examples/2d/mesh2d_manual.rs @@ -3,12 +3,9 @@ //! It doesn't use the [`Material2d`] abstraction, but changes the vertex buffer to include vertex color. //! Check out the "mesh2d" example for simpler / higher level 2d meshes. -use std::f32::consts::PI; - use bevy::{ core_pipeline::core_2d::Transparent2d, prelude::*, - reflect::TypeUuid, render::{ mesh::{Indices, MeshVertexAttribute}, render_asset::RenderAssets, @@ -29,6 +26,7 @@ use bevy::{ }, utils::FloatOrd, }; +use std::f32::consts::PI; fn main() { App::new() @@ -153,7 +151,7 @@ impl SpecializedRenderPipeline for ColoredMesh2dPipeline { RenderPipelineDescriptor { vertex: VertexState { // Use our custom shader - shader: COLORED_MESH2D_SHADER_HANDLE.typed::(), + shader: COLORED_MESH2D_SHADER_HANDLE, entry_point: "vertex".into(), shader_defs: Vec::new(), // Use our custom vertex buffer @@ -161,7 +159,7 @@ impl SpecializedRenderPipeline for ColoredMesh2dPipeline { }, fragment: Some(FragmentState { // Use our custom shader - shader: COLORED_MESH2D_SHADER_HANDLE.typed::(), + shader: COLORED_MESH2D_SHADER_HANDLE, shader_defs: Vec::new(), entry_point: "fragment".into(), targets: vec![Some(ColorTargetState { @@ -261,14 +259,14 @@ fn fragment(in: FragmentInput) -> @location(0) vec4 { pub struct ColoredMesh2dPlugin; /// Handle to the custom shader with a unique random ID -pub const COLORED_MESH2D_SHADER_HANDLE: HandleUntyped = - HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 13828845428412094821); +pub const COLORED_MESH2D_SHADER_HANDLE: Handle = + Handle::weak_from_u128(13828845428412094821); impl Plugin for ColoredMesh2dPlugin { fn build(&self, app: &mut App) { // Load our custom shader let mut shaders = app.world.resource_mut::>(); - shaders.set_untracked( + shaders.insert( COLORED_MESH2D_SHADER_HANDLE, Shader::from_wgsl(COLORED_MESH2D_SHADER, file!()), ); diff --git a/examples/2d/texture_atlas.rs b/examples/2d/texture_atlas.rs index 14fda578398d4..ce62f25e2fbb2 100644 --- a/examples/2d/texture_atlas.rs +++ b/examples/2d/texture_atlas.rs @@ -1,11 +1,10 @@ //! In this example we generate a new texture atlas (sprite sheet) from a folder containing //! individual sprites. -use bevy::{asset::LoadState, prelude::*}; +use bevy::{asset::LoadedFolder, prelude::*}; fn main() { App::new() - .init_resource::() .add_plugins(DefaultPlugins.set(ImagePlugin::default_nearest())) // prevents blurry sprites .add_state::() .add_systems(OnEnter(AppState::Setup), load_textures) @@ -22,53 +21,55 @@ enum AppState { } #[derive(Resource, Default)] -struct RpgSpriteHandles { - handles: Vec, -} +struct RpgSpriteFolder(Handle); -fn load_textures(mut rpg_sprite_handles: ResMut, asset_server: Res) { +fn load_textures(mut commands: Commands, asset_server: Res) { // load multiple, individual sprites from a folder - rpg_sprite_handles.handles = asset_server.load_folder("textures/rpg").unwrap(); + commands.insert_resource(RpgSpriteFolder(asset_server.load_folder("textures/rpg"))); } fn check_textures( mut next_state: ResMut>, - rpg_sprite_handles: ResMut, - asset_server: Res, + rpg_sprite_folder: ResMut, + mut events: EventReader>, ) { // Advance the `AppState` once all sprite handles have been loaded by the `AssetServer` - if let LoadState::Loaded = asset_server - .get_group_load_state(rpg_sprite_handles.handles.iter().map(|handle| handle.id())) - { - next_state.set(AppState::Finished); + for event in events.read() { + if event.is_loaded_with_dependencies(&rpg_sprite_folder.0) { + next_state.set(AppState::Finished); + } } } fn setup( mut commands: Commands, - rpg_sprite_handles: Res, + rpg_sprite_handles: Res, asset_server: Res, + loaded_folders: Res>, mut texture_atlases: ResMut>, mut textures: ResMut>, ) { // Build a `TextureAtlas` using the individual sprites let mut texture_atlas_builder = TextureAtlasBuilder::default(); - for handle in &rpg_sprite_handles.handles { - let handle = handle.typed_weak(); - let Some(texture) = textures.get(&handle) else { + let loaded_folder = loaded_folders.get(&rpg_sprite_handles.0).unwrap(); + for handle in loaded_folder.handles.iter() { + let id = handle.id().typed_unchecked::(); + let Some(texture) = textures.get(id) else { warn!( "{:?} did not resolve to an `Image` asset.", - asset_server.get_handle_path(handle) + handle.path().unwrap() ); continue; }; - texture_atlas_builder.add_texture(handle, texture); + texture_atlas_builder.add_texture(id, texture); } let texture_atlas = texture_atlas_builder.finish(&mut textures).unwrap(); let texture_atlas_texture = texture_atlas.texture.clone(); - let vendor_handle = asset_server.get_handle("textures/rpg/chars/vendor/generic-rpg-vendor.png"); + let vendor_handle = asset_server + .get_handle("textures/rpg/chars/vendor/generic-rpg-vendor.png") + .unwrap(); let vendor_index = texture_atlas.get_texture_index(&vendor_handle).unwrap(); let atlas_handle = texture_atlases.add(texture_atlas); diff --git a/examples/3d/lines.rs b/examples/3d/lines.rs index d94f0794b29c8..9dc3caba87b82 100644 --- a/examples/3d/lines.rs +++ b/examples/3d/lines.rs @@ -3,7 +3,7 @@ use bevy::{ pbr::{MaterialPipeline, MaterialPipelineKey}, prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, render::{ mesh::{MeshVertexBufferLayout, PrimitiveTopology}, render_resource::{ @@ -61,8 +61,7 @@ fn setup( }); } -#[derive(Default, AsBindGroup, TypeUuid, TypePath, Debug, Clone)] -#[uuid = "050ce6ac-080a-4d8c-b6b5-b5bab7560d8f"] +#[derive(Asset, TypePath, Default, AsBindGroup, Debug, Clone)] struct LineMaterial { #[uniform(0)] color: Color, diff --git a/examples/3d/load_gltf.rs b/examples/3d/load_gltf.rs index 33b1559907f0f..ccf7f7b5dc7b9 100644 --- a/examples/3d/load_gltf.rs +++ b/examples/3d/load_gltf.rs @@ -1,11 +1,10 @@ //! Loads and renders a glTF file as a scene. -use std::f32::consts::*; - use bevy::{ pbr::{CascadeShadowConfigBuilder, DirectionalLightShadowMap}, prelude::*, }; +use std::f32::consts::*; fn main() { App::new() diff --git a/examples/3d/pbr.rs b/examples/3d/pbr.rs index b35e4bd8e4682..305d073a953fb 100644 --- a/examples/3d/pbr.rs +++ b/examples/3d/pbr.rs @@ -151,8 +151,8 @@ fn environment_map_load_finish( label_query: Query>, ) { if let Ok(environment_map) = environment_maps.get_single() { - if asset_server.get_load_state(&environment_map.diffuse_map) == LoadState::Loaded - && asset_server.get_load_state(&environment_map.specular_map) == LoadState::Loaded + if asset_server.load_state(&environment_map.diffuse_map) == LoadState::Loaded + && asset_server.load_state(&environment_map.specular_map) == LoadState::Loaded { if let Ok(label_entity) = label_query.get_single() { commands.entity(label_entity).despawn(); diff --git a/examples/3d/skybox.rs b/examples/3d/skybox.rs index 619550f56b4d5..cad96aeb6c711 100644 --- a/examples/3d/skybox.rs +++ b/examples/3d/skybox.rs @@ -1,7 +1,5 @@ //! Load a cubemap texture onto a cube like a skybox and cycle through different compressed texture formats -use std::f32::consts::PI; - use bevy::{ asset::LoadState, core_pipeline::Skybox, @@ -13,6 +11,7 @@ use bevy::{ texture::CompressedImageFormats, }, }; +use std::f32::consts::PI; const CUBEMAPS: &[(&str, CompressedImageFormats)] = &[ ( @@ -141,9 +140,7 @@ fn asset_loaded( mut cubemap: ResMut, mut skyboxes: Query<&mut Skybox>, ) { - if !cubemap.is_loaded - && asset_server.get_load_state(cubemap.image_handle.clone_weak()) == LoadState::Loaded - { + if !cubemap.is_loaded && asset_server.load_state(&cubemap.image_handle) == LoadState::Loaded { info!("Swapping to {}...", CUBEMAPS[cubemap.index].0); let image = images.get_mut(&cubemap.image_handle).unwrap(); // NOTE: PNGs do not have any metadata that could indicate they contain a cubemap texture, diff --git a/examples/3d/tonemapping.rs b/examples/3d/tonemapping.rs index 687efaaef5845..595f3b0b4ecac 100644 --- a/examples/3d/tonemapping.rs +++ b/examples/3d/tonemapping.rs @@ -5,7 +5,7 @@ use bevy::{ math::vec2, pbr::CascadeShadowConfigBuilder, prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, render::{ render_resource::{ AsBindGroup, Extent3d, SamplerDescriptor, ShaderRef, TextureDimension, TextureFormat, @@ -310,7 +310,7 @@ fn update_image_viewer( for event in drop_events.read() { match event { FileDragAndDrop::DroppedFile { path_buf, .. } => { - new_image = Some(asset_server.load(path_buf.to_string_lossy().to_string())); + new_image = Some(asset_server.load(&path_buf.to_string_lossy().to_string())); *drop_hovered = false; } FileDragAndDrop::HoveredFile { .. } => *drop_hovered = true, @@ -329,17 +329,17 @@ fn update_image_viewer( } for event in image_events.read() { - let image_changed_h = match event { - AssetEvent::Created { handle } | AssetEvent::Modified { handle } => handle, + let image_changed_id = *match event { + AssetEvent::Added { id } | AssetEvent::Modified { id } => id, _ => continue, }; if let Some(base_color_texture) = mat.base_color_texture.clone() { - if image_changed_h == &base_color_texture { - if let Some(image_changed) = images.get(image_changed_h) { + if image_changed_id == base_color_texture.id() { + if let Some(image_changed) = images.get(image_changed_id) { let size = image_changed.size().normalize_or_zero() * 1.4; // Resize Mesh let quad = Mesh::from(shape::Quad::new(size)); - let _ = meshes.set(mesh_h, quad); + meshes.insert(mesh_h, quad); } } } @@ -691,8 +691,7 @@ impl Material for ColorGradientMaterial { } } -#[derive(AsBindGroup, Debug, Clone, TypeUuid, TypePath)] -#[uuid = "117f64fe-6844-1822-8926-e3ed372291c8"] +#[derive(Asset, TypePath, AsBindGroup, Debug, Clone)] pub struct ColorGradientMaterial {} #[derive(Resource)] diff --git a/examples/README.md b/examples/README.md index 6f53c34f7048c..d48c6646ae6e5 100644 --- a/examples/README.md +++ b/examples/README.md @@ -177,8 +177,9 @@ Example | Description Example | Description --- | --- [Asset Loading](../examples/asset/asset_loading.rs) | Demonstrates various methods to load assets +[Asset Processing](../examples/asset/processing/processing.rs) | Demonstrates how to process and load custom assets [Custom Asset](../examples/asset/custom_asset.rs) | Implements a custom asset loader -[Custom Asset IO](../examples/asset/custom_asset_io.rs) | Implements a custom asset io loader +[Custom Asset IO](../examples/asset/custom_asset_reader.rs) | Implements a custom AssetReader [Hot Reloading of Assets](../examples/asset/hot_asset_reloading.rs) | Demonstrates automatic reloading of assets when modified on disk ## Async Tasks diff --git a/examples/asset/asset_loading.rs b/examples/asset/asset_loading.rs index bdcbe2819cb6d..42d830ed7f2d5 100644 --- a/examples/asset/asset_loading.rs +++ b/examples/asset/asset_loading.rs @@ -1,6 +1,6 @@ //! This example illustrates various ways to load assets. -use bevy::prelude::*; +use bevy::{asset::LoadedFolder, prelude::*}; fn main() { App::new() @@ -37,11 +37,19 @@ fn setup( } // You can load all assets in a folder like this. They will be loaded in parallel without - // blocking - let _scenes: Vec = asset_server.load_folder("models/monkey").unwrap(); + // blocking. The LoadedFolder asset holds handles to each asset in the folder. These are all + // dependencies of the LoadedFolder asset, meaning you can wait for the LoadedFolder asset to + // fire AssetEvent::LoadedWithDependencies if you want to wait for all assets in the folder + // to load. + // If you want to keep the assets in the folder alive, make sure you store the returned handle + // somewhere. + let _loaded_folder: Handle = asset_server.load_folder("models/monkey"); - // Then any asset in the folder can be accessed like this: - let monkey_handle = asset_server.get_handle("models/monkey/Monkey.gltf#Mesh0/Primitive0"); + // If you want a handle to a specific asset in a loaded folder, the easiest way to get one is to call load. + // It will _not_ be loaded a second time. + // The LoadedFolder asset will ultimately also hold handles to the assets, but waiting for it to load + // and finding the right handle is more work! + let monkey_handle = asset_server.load("models/monkey/Monkey.gltf#Mesh0/Primitive0"); // You can also add assets directly to their Assets storage: let material_handle = materials.add(StandardMaterial { diff --git a/examples/asset/custom_asset.rs b/examples/asset/custom_asset.rs index 3fe033a3f25b8..600f4087f56ae 100644 --- a/examples/asset/custom_asset.rs +++ b/examples/asset/custom_asset.rs @@ -1,15 +1,15 @@ //! Implements loader for a custom asset type. use bevy::{ - asset::{AssetLoader, LoadContext, LoadedAsset}, + asset::{anyhow::Error, io::Reader, AssetLoader, LoadContext}, prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, utils::BoxedFuture, }; +use futures_lite::AsyncReadExt; use serde::Deserialize; -#[derive(Debug, Deserialize, TypeUuid, TypePath)] -#[uuid = "39cadc56-aa9c-4543-8640-a018b74b5052"] +#[derive(Asset, TypePath, Debug, Deserialize)] pub struct CustomAsset { pub value: i32, } @@ -18,15 +18,19 @@ pub struct CustomAsset { pub struct CustomAssetLoader; impl AssetLoader for CustomAssetLoader { + type Asset = CustomAsset; + type Settings = (); fn load<'a>( &'a self, - bytes: &'a [u8], - load_context: &'a mut LoadContext, - ) -> BoxedFuture<'a, Result<(), bevy::asset::Error>> { + reader: &'a mut Reader, + _settings: &'a (), + _load_context: &'a mut LoadContext, + ) -> BoxedFuture<'a, Result> { Box::pin(async move { - let custom_asset = ron::de::from_bytes::(bytes)?; - load_context.set_default_asset(LoadedAsset::new(custom_asset)); - Ok(()) + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; + let custom_asset = ron::de::from_bytes::(&bytes)?; + Ok(custom_asset) }) } @@ -39,7 +43,7 @@ fn main() { App::new() .add_plugins(DefaultPlugins) .init_resource::() - .add_asset::() + .init_asset::() .init_asset_loader::() .add_systems(Startup, setup) .add_systems(Update, print_on_load) diff --git a/examples/asset/custom_asset_io.rs b/examples/asset/custom_asset_io.rs deleted file mode 100644 index df73709633e60..0000000000000 --- a/examples/asset/custom_asset_io.rs +++ /dev/null @@ -1,92 +0,0 @@ -//! Implements a custom asset io loader. -//! An [`AssetIo`] is what the asset server uses to read the raw bytes of assets. -//! It does not know anything about the asset formats, only how to talk to the underlying storage. - -use bevy::{ - asset::{AssetIo, AssetIoError, ChangeWatcher, Metadata}, - prelude::*, - utils::BoxedFuture, -}; -use std::path::{Path, PathBuf}; - -/// A custom asset io implementation that simply defers to the platform default -/// implementation. -/// -/// This can be used as a starting point for developing a useful implementation -/// that can defer to the default when needed. -struct CustomAssetIo(Box); - -impl AssetIo for CustomAssetIo { - fn load_path<'a>(&'a self, path: &'a Path) -> BoxedFuture<'a, Result, AssetIoError>> { - info!("load_path({path:?})"); - self.0.load_path(path) - } - - fn read_directory( - &self, - path: &Path, - ) -> Result>, AssetIoError> { - info!("read_directory({path:?})"); - self.0.read_directory(path) - } - - fn watch_path_for_changes( - &self, - to_watch: &Path, - to_reload: Option, - ) -> Result<(), AssetIoError> { - info!("watch_path_for_changes({to_watch:?}, {to_reload:?})"); - self.0.watch_path_for_changes(to_watch, to_reload) - } - - fn watch_for_changes(&self, configuration: &ChangeWatcher) -> Result<(), AssetIoError> { - info!("watch_for_changes()"); - self.0.watch_for_changes(configuration) - } - - fn get_metadata(&self, path: &Path) -> Result { - info!("get_metadata({path:?})"); - self.0.get_metadata(path) - } -} - -/// A plugin used to execute the override of the asset io -struct CustomAssetIoPlugin; - -impl Plugin for CustomAssetIoPlugin { - fn build(&self, app: &mut App) { - let default_io = AssetPlugin::default().create_platform_default_asset_io(); - - // create the custom asset io instance - let asset_io = CustomAssetIo(default_io); - - // the asset server is constructed and added the resource manager - app.insert_resource(AssetServer::new(asset_io)); - } -} - -fn main() { - App::new() - .add_plugins( - DefaultPlugins - .build() - // the custom asset io plugin must be inserted in-between the - // `CorePlugin' and `AssetPlugin`. It needs to be after the - // CorePlugin, so that the IO task pool has already been constructed. - // And it must be before the `AssetPlugin` so that the asset plugin - // doesn't create another instance of an asset server. In general, - // the AssetPlugin should still run so that other aspects of the - // asset system are initialized correctly. - .add_before::(CustomAssetIoPlugin), - ) - .add_systems(Startup, setup) - .run(); -} - -fn setup(mut commands: Commands, asset_server: Res) { - commands.spawn(Camera2dBundle::default()); - commands.spawn(SpriteBundle { - texture: asset_server.load("branding/icon.png"), - ..default() - }); -} diff --git a/examples/asset/custom_asset_reader.rs b/examples/asset/custom_asset_reader.rs new file mode 100644 index 0000000000000..99ad8fe07e973 --- /dev/null +++ b/examples/asset/custom_asset_reader.rs @@ -0,0 +1,88 @@ +//! Implements a custom asset io loader. +//! An [`AssetReader`] is what the asset server uses to read the raw bytes of assets. +//! It does not know anything about the asset formats, only how to talk to the underlying storage. + +use bevy::{ + asset::io::{ + file::FileAssetReader, AssetProvider, AssetProviders, AssetReader, AssetReaderError, + PathStream, Reader, + }, + prelude::*, + utils::BoxedFuture, +}; +use std::path::Path; + +/// A custom asset reader implementation that wraps a given asset reader implementation +struct CustomAssetReader(T); + +impl AssetReader for CustomAssetReader { + fn read<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + info!("Reading {:?}", path); + self.0.read(path) + } + fn read_meta<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result>, AssetReaderError>> { + self.0.read_meta(path) + } + + fn read_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result, AssetReaderError>> { + self.0.read_directory(path) + } + + fn is_directory<'a>( + &'a self, + path: &'a Path, + ) -> BoxedFuture<'a, Result> { + self.0.is_directory(path) + } + + fn watch_for_changes( + &self, + event_sender: crossbeam_channel::Sender, + ) -> Option> { + self.0.watch_for_changes(event_sender) + } +} + +/// A plugins that registers our new asset reader +struct CustomAssetReaderPlugin; + +impl Plugin for CustomAssetReaderPlugin { + fn build(&self, app: &mut App) { + let mut asset_providers = app + .world + .get_resource_or_insert_with::(Default::default); + asset_providers.insert_reader("CustomAssetReader", || { + Box::new(CustomAssetReader(FileAssetReader::new("assets"))) + }); + } +} + +fn main() { + App::new() + .add_plugins(( + CustomAssetReaderPlugin, + DefaultPlugins.set(AssetPlugin::Unprocessed { + source: AssetProvider::Custom("CustomAssetReader".to_string()), + watch_for_changes: false, + }), + )) + .add_systems(Startup, setup) + .run(); +} + +fn setup(mut commands: Commands, asset_server: Res) { + commands.spawn(Camera2dBundle::default()); + commands.spawn(SpriteBundle { + texture: asset_server.load("branding/icon.png"), + ..default() + }); +} diff --git a/examples/asset/hot_asset_reloading.rs b/examples/asset/hot_asset_reloading.rs index 8c31d8d94d0c6..559087b704c4f 100644 --- a/examples/asset/hot_asset_reloading.rs +++ b/examples/asset/hot_asset_reloading.rs @@ -2,15 +2,11 @@ //! running. This lets you immediately see the results of your changes without restarting the game. //! This example illustrates hot reloading mesh changes. -use bevy::{asset::ChangeWatcher, prelude::*, utils::Duration}; +use bevy::prelude::*; fn main() { App::new() - .add_plugins(DefaultPlugins.set(AssetPlugin { - // Tell the asset server to watch for asset changes on disk: - watch_for_changes: ChangeWatcher::with_delay(Duration::from_millis(200)), - ..default() - })) + .add_plugins(DefaultPlugins.set(AssetPlugin::default().watch_for_changes())) .add_systems(Startup, setup) .run(); } diff --git a/examples/asset/processing/assets/a.cool.ron b/examples/asset/processing/assets/a.cool.ron new file mode 100644 index 0000000000000..6c6051f1e29de --- /dev/null +++ b/examples/asset/processing/assets/a.cool.ron @@ -0,0 +1,8 @@ +( + text: "a", + dependencies: [ + "foo/b.cool.ron", + "foo/c.cool.ron", + ], + embedded_dependencies: [], +) \ No newline at end of file diff --git a/examples/asset/processing/assets/a.cool.ron.meta b/examples/asset/processing/assets/a.cool.ron.meta new file mode 100644 index 0000000000000..7feb4d3a7bf3c --- /dev/null +++ b/examples/asset/processing/assets/a.cool.ron.meta @@ -0,0 +1,12 @@ +( + meta_format_version: "1.0", + asset: Process( + processor: "bevy_asset::processor::process::LoadAndSave", + settings: ( + loader_settings: (), + saver_settings: ( + appended: "X", + ), + ), + ), +) \ No newline at end of file diff --git a/examples/asset/processing/assets/d.cool.ron b/examples/asset/processing/assets/d.cool.ron new file mode 100644 index 0000000000000..cfe835b25888d --- /dev/null +++ b/examples/asset/processing/assets/d.cool.ron @@ -0,0 +1,8 @@ +( + text: "d", + dependencies: [ + ], + embedded_dependencies: [ + "foo/c.cool.ron" + ], +) \ No newline at end of file diff --git a/examples/asset/processing/assets/d.cool.ron.meta b/examples/asset/processing/assets/d.cool.ron.meta new file mode 100644 index 0000000000000..c79e622562868 --- /dev/null +++ b/examples/asset/processing/assets/d.cool.ron.meta @@ -0,0 +1,12 @@ +( + meta_format_version: "1.0", + asset: Process( + processor: "bevy_asset::processor::process::LoadAndSave", + settings: ( + loader_settings: (), + saver_settings: ( + appended: "", + ), + ), + ), +) \ No newline at end of file diff --git a/examples/asset/processing/assets/foo/b.cool.ron b/examples/asset/processing/assets/foo/b.cool.ron new file mode 100644 index 0000000000000..f72581c1db245 --- /dev/null +++ b/examples/asset/processing/assets/foo/b.cool.ron @@ -0,0 +1,5 @@ +( + text: "b", + dependencies: [], + embedded_dependencies: [], +) \ No newline at end of file diff --git a/examples/asset/processing/assets/foo/b.cool.ron.meta b/examples/asset/processing/assets/foo/b.cool.ron.meta new file mode 100644 index 0000000000000..c79e622562868 --- /dev/null +++ b/examples/asset/processing/assets/foo/b.cool.ron.meta @@ -0,0 +1,12 @@ +( + meta_format_version: "1.0", + asset: Process( + processor: "bevy_asset::processor::process::LoadAndSave", + settings: ( + loader_settings: (), + saver_settings: ( + appended: "", + ), + ), + ), +) \ No newline at end of file diff --git a/examples/asset/processing/assets/foo/c.cool.ron b/examples/asset/processing/assets/foo/c.cool.ron new file mode 100644 index 0000000000000..c145493f15109 --- /dev/null +++ b/examples/asset/processing/assets/foo/c.cool.ron @@ -0,0 +1,5 @@ +( + text: "c", + dependencies: [], + embedded_dependencies: ["a.cool.ron", "foo/b.cool.ron"], +) \ No newline at end of file diff --git a/examples/asset/processing/assets/foo/c.cool.ron.meta b/examples/asset/processing/assets/foo/c.cool.ron.meta new file mode 100644 index 0000000000000..c79e622562868 --- /dev/null +++ b/examples/asset/processing/assets/foo/c.cool.ron.meta @@ -0,0 +1,12 @@ +( + meta_format_version: "1.0", + asset: Process( + processor: "bevy_asset::processor::process::LoadAndSave", + settings: ( + loader_settings: (), + saver_settings: ( + appended: "", + ), + ), + ), +) \ No newline at end of file diff --git a/examples/asset/processing/processing.rs b/examples/asset/processing/processing.rs new file mode 100644 index 0000000000000..400508c02cfca --- /dev/null +++ b/examples/asset/processing/processing.rs @@ -0,0 +1,210 @@ +//! This example illustrates how to define custom `AssetLoader`s and `AssetSaver`s, how to configure them, and how to register asset processors. + +use bevy::{ + asset::{ + io::{AssetProviders, Reader, Writer}, + processor::LoadAndSave, + saver::{AssetSaver, SavedAsset}, + AssetLoader, AsyncReadExt, AsyncWriteExt, LoadContext, + }, + prelude::*, + reflect::TypePath, + utils::BoxedFuture, +}; +use serde::{Deserialize, Serialize}; + +fn main() { + App::new() + .insert_resource( + // This is just overriding the default paths to scope this to the correct example folder + // You can generally skip this in your own projects + AssetProviders::default() + .with_default_file_source("examples/asset/processing/assets".to_string()) + .with_default_file_destination( + "examples/asset/processing/imported_assets".to_string(), + ), + ) + // Enabling `processed_dev` will configure the AssetPlugin to use asset processing. + // This will run the AssetProcessor in the background, which will listen for changes to + // the `assets` folder, run them through configured asset processors, and write the results + // to the `imported_assets` folder. + // + // The AssetProcessor will create `.meta` files automatically for assets in the `assets` folder, + // which can then be used to configure how the asset will be processed. + .add_plugins((DefaultPlugins.set(AssetPlugin::processed_dev()), TextPlugin)) + // This is what a deployed app should use + // .add_plugins((DefaultPlugins.set(AssetPlugin::processed()), TextPlugin)) + .add_systems(Startup, setup) + .add_systems(Update, print_text) + .run(); +} + +/// This [`TextPlugin`] defines two assets types: +/// * [`CoolText`]: a custom RON text format that supports dependencies and embedded dependencies +/// * [`Text`]: a "normal" plain text file +/// +/// It also defines an asset processor that will load [`CoolText`], resolve embedded dependencies, and write the resulting +/// output to a "normal" plain text file. When the processed asset is loaded, it is loaded as a Text (plaintext) asset. +/// This illustrates that when you process an asset, you can change its type! However you don't _need_ to change the type. +pub struct TextPlugin; + +impl Plugin for TextPlugin { + fn build(&self, app: &mut App) { + app.init_asset::() + .init_asset::() + .register_asset_loader(CoolTextLoader) + .register_asset_loader(TextLoader) + .register_asset_processor::>( + LoadAndSave::from(CoolTextSaver), + ) + .set_default_asset_processor::>("cool.ron"); + } +} + +#[derive(Asset, TypePath, Debug)] +struct Text(String); + +#[derive(Default)] +struct TextLoader; + +#[derive(Default, Serialize, Deserialize)] +struct TextSettings { + text_override: Option, +} + +impl AssetLoader for TextLoader { + type Asset = Text; + type Settings = TextSettings; + fn load<'a>( + &'a self, + reader: &'a mut Reader, + settings: &'a TextSettings, + _load_context: &'a mut LoadContext, + ) -> BoxedFuture<'a, Result> { + Box::pin(async move { + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; + let value = if let Some(ref text) = settings.text_override { + text.clone() + } else { + String::from_utf8(bytes).unwrap() + }; + Ok(Text(value)) + }) + } + + fn extensions(&self) -> &[&str] { + &["txt"] + } +} + +#[derive(Serialize, Deserialize)] +pub struct CoolTextRon { + text: String, + dependencies: Vec, + embedded_dependencies: Vec, +} + +#[derive(Asset, TypePath, Debug)] +pub struct CoolText { + text: String, + #[allow(unused)] + dependencies: Vec>, +} + +#[derive(Default)] +struct CoolTextLoader; + +impl AssetLoader for CoolTextLoader { + type Asset = CoolText; + + type Settings = (); + + fn load<'a>( + &'a self, + reader: &'a mut Reader, + _settings: &'a Self::Settings, + load_context: &'a mut LoadContext, + ) -> BoxedFuture<'a, Result> { + Box::pin(async move { + let mut bytes = Vec::new(); + reader.read_to_end(&mut bytes).await?; + let ron: CoolTextRon = ron::de::from_bytes(&bytes)?; + let mut base_text = ron.text; + for embedded in ron.embedded_dependencies { + let loaded = load_context.load_direct(&embedded).await?; + let text = loaded.get::().unwrap(); + base_text.push_str(&text.0); + } + Ok(CoolText { + text: base_text, + dependencies: ron + .dependencies + .iter() + .map(|p| load_context.load(p)) + .collect(), + }) + }) + } + + fn extensions(&self) -> &[&str] { + &["cool.ron"] + } +} + +struct CoolTextSaver; + +#[derive(Default, Serialize, Deserialize)] +pub struct CoolTextSaverSettings { + appended: String, +} + +impl AssetSaver for CoolTextSaver { + type Asset = CoolText; + type Settings = CoolTextSaverSettings; + type OutputLoader = TextLoader; + + fn save<'a>( + &'a self, + writer: &'a mut Writer, + asset: SavedAsset<'a, Self::Asset>, + settings: &'a Self::Settings, + ) -> BoxedFuture<'a, Result> { + Box::pin(async move { + let text = format!("{}{}", asset.text.clone(), settings.appended); + writer.write_all(text.as_bytes()).await?; + Ok(TextSettings::default()) + }) + } +} + +#[derive(Resource)] +struct TextAssets { + a: Handle, + b: Handle, + c: Handle, + d: Handle, +} + +fn setup(mut commands: Commands, assets: Res) { + // This the final processed versions of `assets/a.cool.ron` and `assets/foo.c.cool.ron` + // Check out their counterparts in `imported_assets` to see what the outputs look like. + commands.insert_resource(TextAssets { + a: assets.load("a.cool.ron"), + b: assets.load("foo/b.cool.ron"), + c: assets.load("foo/c.cool.ron"), + d: assets.load("d.cool.ron"), + }); +} + +fn print_text(handles: Res, texts: Res>) { + // This prints the current values of the assets + // Hot-reloading is supported, so try modifying the source assets (and their meta files)! + println!("Current Values:"); + println!(" a: {:?}", texts.get(&handles.a)); + println!(" b: {:?}", texts.get(&handles.b)); + println!(" c: {:?}", texts.get(&handles.c)); + println!(" d: {:?}", texts.get(&handles.d)); + println!("(You can modify source assets and their .meta files to hot-reload changes!)"); + println!(); +} diff --git a/examples/audio/decodable.rs b/examples/audio/decodable.rs index 9ed6539d24c9e..94baaf28e87cd 100644 --- a/examples/audio/decodable.rs +++ b/examples/audio/decodable.rs @@ -3,15 +3,14 @@ use bevy::audio::AddAudioSource; use bevy::audio::AudioPlugin; use bevy::audio::Source; use bevy::prelude::*; -use bevy::reflect::{TypePath, TypeUuid}; +use bevy::reflect::TypePath; use bevy::utils::Duration; // This struct usually contains the data for the audio being played. // This is where data read from an audio file would be stored, for example. // Implementing `TypeUuid` will automatically implement `Asset`. // This allows the type to be registered as an asset. -#[derive(TypePath, TypeUuid)] -#[uuid = "c2090c23-78fd-44f1-8508-c89b1f3cec29"] +#[derive(Asset, TypePath)] struct SineAudio { frequency: f32, } diff --git a/examples/scene/scene.rs b/examples/scene/scene.rs index e4e7b508befa8..4e2f4aedd829e 100644 --- a/examples/scene/scene.rs +++ b/examples/scene/scene.rs @@ -1,15 +1,10 @@ //! This example illustrates loading scenes from files. -use bevy::{asset::ChangeWatcher, prelude::*, tasks::IoTaskPool, utils::Duration}; +use bevy::{prelude::*, tasks::IoTaskPool, utils::Duration}; use std::{fs::File, io::Write}; fn main() { App::new() - .add_plugins(DefaultPlugins.set(AssetPlugin { - // This tells the AssetServer to watch for changes to assets. - // It enables our scenes to automatically reload in game when we modify their files. - watch_for_changes: ChangeWatcher::with_delay(Duration::from_millis(200)), - ..default() - })) + .add_plugins(DefaultPlugins.set(AssetPlugin::default().watch_for_changes())) .register_type::() .register_type::() .register_type::() diff --git a/examples/shader/animate_shader.rs b/examples/shader/animate_shader.rs index e0cc03d2eeba4..a828420387ed6 100644 --- a/examples/shader/animate_shader.rs +++ b/examples/shader/animate_shader.rs @@ -3,8 +3,8 @@ use bevy::{ prelude::*, - reflect::{TypePath, TypeUuid}, - render::render_resource::*, + reflect::TypePath, + render::render_resource::{AsBindGroup, ShaderRef}, }; fn main() { @@ -34,8 +34,7 @@ fn setup( }); } -#[derive(AsBindGroup, TypeUuid, TypePath, Debug, Clone)] -#[uuid = "a3d71c04-d054-4946-80f8-ba6cfbc90cad"] +#[derive(Asset, TypePath, AsBindGroup, Debug, Clone)] struct CustomMaterial {} impl Material for CustomMaterial { diff --git a/examples/shader/array_texture.rs b/examples/shader/array_texture.rs index 39e0b94a9a42b..478119cfa9c57 100644 --- a/examples/shader/array_texture.rs +++ b/examples/shader/array_texture.rs @@ -1,7 +1,7 @@ use bevy::{ asset::LoadState, prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, render::render_resource::{AsBindGroup, ShaderRef}, }; @@ -65,7 +65,7 @@ fn create_array_texture( mut materials: ResMut>, ) { if loading_texture.is_loaded - || asset_server.get_load_state(loading_texture.handle.clone()) != LoadState::Loaded + || asset_server.load_state(loading_texture.handle.clone()) != LoadState::Loaded { return; } @@ -91,8 +91,7 @@ fn create_array_texture( } } -#[derive(AsBindGroup, Debug, Clone, TypeUuid, TypePath)] -#[uuid = "9c5a0ddf-1eaf-41b4-9832-ed736fd26af3"] +#[derive(Asset, TypePath, AsBindGroup, Debug, Clone)] struct ArrayTextureMaterial { #[texture(0, dimension = "2d_array")] #[sampler(1)] diff --git a/examples/shader/compute_shader_game_of_life.rs b/examples/shader/compute_shader_game_of_life.rs index 6d8383ee0c532..10c368771474b 100644 --- a/examples/shader/compute_shader_game_of_life.rs +++ b/examples/shader/compute_shader_game_of_life.rs @@ -106,7 +106,7 @@ fn prepare_bind_group( game_of_life_image: Res, render_device: Res, ) { - let view = &gpu_images[&game_of_life_image.0]; + let view = gpu_images.get(&game_of_life_image.0).unwrap(); let bind_group = render_device.create_bind_group(&BindGroupDescriptor { label: None, layout: &pipeline.texture_bind_group_layout, diff --git a/examples/shader/custom_vertex_attribute.rs b/examples/shader/custom_vertex_attribute.rs index aaca77df96aaa..c61c81b3b8ad0 100644 --- a/examples/shader/custom_vertex_attribute.rs +++ b/examples/shader/custom_vertex_attribute.rs @@ -3,7 +3,7 @@ use bevy::{ pbr::{MaterialPipeline, MaterialPipelineKey}, prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, render::{ mesh::{MeshVertexAttribute, MeshVertexBufferLayout}, render_resource::{ @@ -56,8 +56,7 @@ fn setup( } // This is the struct that will be passed to your shader -#[derive(AsBindGroup, Debug, Clone, TypeUuid, TypePath)] -#[uuid = "f690fdae-d598-45ab-8225-97e2a3f056e0"] +#[derive(Asset, TypePath, AsBindGroup, Debug, Clone)] pub struct CustomMaterial { #[uniform(0)] color: Color, diff --git a/examples/shader/fallback_image.rs b/examples/shader/fallback_image.rs index 7119324216748..b799b56d27e02 100644 --- a/examples/shader/fallback_image.rs +++ b/examples/shader/fallback_image.rs @@ -7,7 +7,7 @@ //! not panic. use bevy::{ prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, render::render_resource::{AsBindGroup, ShaderRef}, }; @@ -44,8 +44,7 @@ fn setup( }); } -#[derive(AsBindGroup, Debug, Clone, TypePath, TypeUuid)] -#[uuid = "d4890167-0e16-4bfc-b812-434717f20409"] +#[derive(AsBindGroup, Debug, Clone, Asset, TypePath)] struct FallbackTestMaterial { #[texture(0, dimension = "1d")] #[sampler(1)] diff --git a/examples/shader/post_processing.rs b/examples/shader/post_processing.rs index 7aed8e8f85d8c..fd6dfc3422173 100644 --- a/examples/shader/post_processing.rs +++ b/examples/shader/post_processing.rs @@ -6,7 +6,6 @@ //! This is a fairly low level example and assumes some familiarity with rendering concepts and wgpu. use bevy::{ - asset::ChangeWatcher, core_pipeline::{ clear_color::ClearColorConfig, core_3d, fullscreen_vertex_shader::fullscreen_shader_vertex_state, @@ -33,17 +32,12 @@ use bevy::{ view::ViewTarget, RenderApp, }, - utils::Duration, }; fn main() { App::new() .add_plugins(( - DefaultPlugins.set(AssetPlugin { - // Hot reloading the shader works correctly - watch_for_changes: ChangeWatcher::with_delay(Duration::from_millis(200)), - ..default() - }), + DefaultPlugins.set(AssetPlugin::default().watch_for_changes()), PostProcessPlugin, )) .add_systems(Startup, setup) diff --git a/examples/shader/shader_defs.rs b/examples/shader/shader_defs.rs index 1f45c43155203..a638c3569a054 100644 --- a/examples/shader/shader_defs.rs +++ b/examples/shader/shader_defs.rs @@ -3,7 +3,7 @@ use bevy::{ pbr::{MaterialPipeline, MaterialPipelineKey}, prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, render::{ mesh::MeshVertexBufferLayout, render_resource::{ @@ -74,8 +74,7 @@ impl Material for CustomMaterial { } // This is the struct that will be passed to your shader -#[derive(AsBindGroup, TypeUuid, TypePath, Debug, Clone)] -#[uuid = "f690fdae-d598-45ab-8225-97e2a3f056e0"] +#[derive(Asset, TypePath, AsBindGroup, Debug, Clone)] #[bind_group_data(CustomMaterialKey)] pub struct CustomMaterial { #[uniform(0)] diff --git a/examples/shader/shader_material.rs b/examples/shader/shader_material.rs index 6e5148d0eca79..59467914d8eaa 100644 --- a/examples/shader/shader_material.rs +++ b/examples/shader/shader_material.rs @@ -2,7 +2,7 @@ use bevy::{ prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, render::render_resource::{AsBindGroup, ShaderRef}, }; @@ -52,8 +52,7 @@ impl Material for CustomMaterial { } // This is the struct that will be passed to your shader -#[derive(AsBindGroup, TypeUuid, TypePath, Debug, Clone)] -#[uuid = "f690fdae-d598-45ab-8225-97e2a3f056e0"] +#[derive(Asset, TypePath, AsBindGroup, Debug, Clone)] pub struct CustomMaterial { #[uniform(0)] color: Color, diff --git a/examples/shader/shader_material_glsl.rs b/examples/shader/shader_material_glsl.rs index 5d33f470582ea..f24cfa5070334 100644 --- a/examples/shader/shader_material_glsl.rs +++ b/examples/shader/shader_material_glsl.rs @@ -3,7 +3,7 @@ use bevy::{ pbr::{MaterialPipeline, MaterialPipelineKey}, prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, render::{ mesh::MeshVertexBufferLayout, render_resource::{ @@ -46,8 +46,7 @@ fn setup( } // This is the struct that will be passed to your shader -#[derive(AsBindGroup, Clone, TypeUuid, TypePath)] -#[uuid = "4ee9c363-1124-4113-890e-199d81b00281"] +#[derive(Asset, TypePath, AsBindGroup, Clone)] pub struct CustomMaterial { #[uniform(0)] color: Color, diff --git a/examples/shader/shader_material_screenspace_texture.rs b/examples/shader/shader_material_screenspace_texture.rs index b0ec22368ea7c..ddee9a68097e0 100644 --- a/examples/shader/shader_material_screenspace_texture.rs +++ b/examples/shader/shader_material_screenspace_texture.rs @@ -2,7 +2,7 @@ use bevy::{ prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, render::render_resource::{AsBindGroup, ShaderRef}, }; @@ -65,8 +65,7 @@ fn rotate_camera(mut camera: Query<&mut Transform, With>, time: Res< cam_transform.look_at(Vec3::ZERO, Vec3::Y); } -#[derive(AsBindGroup, Debug, Clone, TypeUuid, TypePath)] -#[uuid = "b62bb455-a72c-4b56-87bb-81e0554e234f"] +#[derive(Asset, TypePath, AsBindGroup, Debug, Clone)] pub struct CustomMaterial { #[texture(0)] #[sampler(1)] diff --git a/examples/shader/shader_prepass.rs b/examples/shader/shader_prepass.rs index 1a723dfbcc71e..e2159f4277d34 100644 --- a/examples/shader/shader_prepass.rs +++ b/examples/shader/shader_prepass.rs @@ -6,7 +6,7 @@ use bevy::{ core_pipeline::prepass::{DepthPrepass, MotionVectorPrepass, NormalPrepass}, pbr::{NotShadowCaster, PbrPlugin}, prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, render::render_resource::{AsBindGroup, ShaderRef, ShaderType}, }; @@ -157,8 +157,7 @@ fn setup( } // This is the struct that will be passed to your shader -#[derive(AsBindGroup, TypePath, TypeUuid, Debug, Clone)] -#[uuid = "f690fdae-d598-45ab-8225-97e2a3f056e0"] +#[derive(Asset, TypePath, AsBindGroup, Debug, Clone)] pub struct CustomMaterial { #[uniform(0)] color: Color, @@ -206,8 +205,7 @@ struct ShowPrepassSettings { } // This shader simply loads the prepass texture and outputs it directly -#[derive(AsBindGroup, TypePath, TypeUuid, Debug, Clone)] -#[uuid = "0af99895-b96e-4451-bc12-c6b1c1c52750"] +#[derive(Asset, TypePath, AsBindGroup, Debug, Clone)] pub struct PrepassOutputMaterial { #[uniform(0)] settings: ShowPrepassSettings, diff --git a/examples/shader/texture_binding_array.rs b/examples/shader/texture_binding_array.rs index 90bf00afc705b..f395d9fbde9e4 100644 --- a/examples/shader/texture_binding_array.rs +++ b/examples/shader/texture_binding_array.rs @@ -3,7 +3,7 @@ use bevy::{ prelude::*, - reflect::{TypePath, TypeUuid}, + reflect::TypePath, render::{ render_asset::RenderAssets, render_resource::{AsBindGroupError, PreparedBindGroup, *}, @@ -74,7 +74,7 @@ fn setup( .iter() .map(|id| { let path = format!("textures/rpg/tiles/generic-rpg-tile{id:0>2}.png"); - asset_server.load(path) + asset_server.load(&path) }) .collect(); @@ -86,8 +86,7 @@ fn setup( }); } -#[derive(Debug, Clone, TypePath, TypeUuid)] -#[uuid = "8dd2b424-45a2-4a53-ac29-7ce356b2d5fe"] +#[derive(Asset, TypePath, Debug, Clone)] struct BindlessMaterial { textures: Vec>, } diff --git a/examples/tools/scene_viewer/animation_plugin.rs b/examples/tools/scene_viewer/animation_plugin.rs index e1f2970547961..41cb5925c825e 100644 --- a/examples/tools/scene_viewer/animation_plugin.rs +++ b/examples/tools/scene_viewer/animation_plugin.rs @@ -35,6 +35,7 @@ fn assign_clips( scene_handle: Res, clips: Res>, gltf_assets: Res>, + assets: Res, mut commands: Commands, mut setup: Local, ) { @@ -56,7 +57,7 @@ fn assign_clips( let clips = clips .iter() .filter_map(|(k, v)| v.compatible_with(name).then_some(k)) - .map(|id| clips.get_handle(id)) + .map(|id| assets.get_id_handle(id).unwrap()) .collect(); let animations = Clips::new(clips); player.play(animations.current()).repeat(); diff --git a/examples/tools/scene_viewer/main.rs b/examples/tools/scene_viewer/main.rs index 69df243382648..9cdeec38c4eb8 100644 --- a/examples/tools/scene_viewer/main.rs +++ b/examples/tools/scene_viewer/main.rs @@ -6,11 +6,10 @@ //! With no arguments it will load the `FlightHelmet` glTF model from the repository assets subdirectory. use bevy::{ - asset::ChangeWatcher, + asset::io::AssetProviders, math::Vec3A, prelude::*, render::primitives::{Aabb, Sphere}, - utils::Duration, window::WindowPlugin, }; @@ -30,6 +29,9 @@ fn main() { color: Color::WHITE, brightness: 1.0 / 5.0f32, }) + .insert_resource(AssetProviders::default().with_default_file_source( + std::env::var("CARGO_MANIFEST_DIR").unwrap_or_else(|_| ".".to_string()), + )) .add_plugins(( DefaultPlugins .set(WindowPlugin { @@ -39,11 +41,7 @@ fn main() { }), ..default() }) - .set(AssetPlugin { - asset_folder: std::env::var("CARGO_MANIFEST_DIR") - .unwrap_or_else(|_| ".".to_string()), - watch_for_changes: ChangeWatcher::with_delay(Duration::from_millis(200)), - }), + .set(AssetPlugin::default().watch_for_changes()), CameraControllerPlugin, SceneViewerPlugin, MorphViewerPlugin, @@ -79,7 +77,7 @@ fn setup(mut commands: Commands, asset_server: Res) { info!("Loading {}", scene_path); let (file_path, scene_index) = parse_scene(scene_path); - commands.insert_resource(SceneHandle::new(asset_server.load(file_path), scene_index)); + commands.insert_resource(SceneHandle::new(asset_server.load(&file_path), scene_index)); } fn setup_scene_after_load( diff --git a/examples/tools/scene_viewer/scene_viewer_plugin.rs b/examples/tools/scene_viewer/scene_viewer_plugin.rs index b386d06d57b09..3dd6b1273c834 100644 --- a/examples/tools/scene_viewer/scene_viewer_plugin.rs +++ b/examples/tools/scene_viewer/scene_viewer_plugin.rs @@ -92,7 +92,7 @@ fn scene_load_check( ) { match scene_handle.instance_id { None => { - if asset_server.get_load_state(&scene_handle.gltf_handle) == LoadState::Loaded { + if asset_server.load_state(&scene_handle.gltf_handle) == LoadState::Loaded { let gltf = gltf_assets.get(&scene_handle.gltf_handle).unwrap(); if gltf.scenes.len() > 1 { info!( diff --git a/examples/ui/font_atlas_debug.rs b/examples/ui/font_atlas_debug.rs index 4f8baf23ac118..04da0f848268a 100644 --- a/examples/ui/font_atlas_debug.rs +++ b/examples/ui/font_atlas_debug.rs @@ -1,7 +1,7 @@ //! This example illustrates how `FontAtlas`'s are populated. //! Bevy uses `FontAtlas`'s under the hood to optimize text rendering. -use bevy::{prelude::*, text::FontAtlasSet}; +use bevy::{prelude::*, text::FontAtlasSets}; fn main() { App::new() @@ -33,10 +33,10 @@ impl Default for State { fn atlas_render_system( mut commands: Commands, mut state: ResMut, - font_atlas_sets: Res>, + font_atlas_sets: Res, texture_atlases: Res>, ) { - if let Some(set) = font_atlas_sets.get(&state.handle.cast_weak::()) { + if let Some(set) = font_atlas_sets.get(&state.handle) { if let Some((_size, font_atlas)) = set.iter().next() { let x_offset = state.atlas_count as f32; if state.atlas_count == font_atlas.len() as u32 { diff --git a/tools/publish.sh b/tools/publish.sh index 3092af456838e..5e97f5e9b7e9d 100644 --- a/tools/publish.sh +++ b/tools/publish.sh @@ -15,6 +15,7 @@ crates=( bevy_time bevy_log bevy_dynamic_plugin + bevy_asset/macros bevy_asset bevy_audio bevy_core