Resources can not be found at compile time when added as dependency (Scala 3)

[rolang]

Scala 3 version of Dumbo.withResourcesIn is not able to list resources that are added as dependency at compile time (mentioned in #79 (comment) ).

[rolang]

☝️ cc @taig it should be fixed now from v0.5.2.
Let me know if you encounter any other issues.

[taig]

Unfortunately, it doesn't work for me in a real world scenario:

//> using repositories sonatype-s01:snapshots
//> using dep dev.rolang::dumbo:0.5.2
//> using dep de.hellobonnie::swan-mock-server:0.0.0+259-010141f9-SNAPSHOT

import _root_.dumbo.ResourceFilePath

ResourceFilePath.fromResourcesDir("migration/swan")

scala-cli dumbo.sc
Compiling project (Scala 3.5.2, JVM (21))
[error] ./dumbo.sc:7:1
[error] Exception occurred while executing macro expansion.
[error] java.nio.file.NoSuchFileException: [...]/Library/Caches/Coursier/v1/https/s01.oss.sonatype.org/content/repositories/snapshots/de/hellobonnie/swan-mock_3/0.0.0%252B259-010141f9-SNAPSHOT/swan-mock_3-0.0.0%252B259-010141f9-SNAPSHOT.jar
[error] 
[error] ResourceFilePath.fromResourcesDir("migration/swan")
[error] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The problem seems to come from getClass().getClassLoader().getResources which returns swan-mock_3-0.0.0%252B259-010141f9-SNAPSHOT.jar, but the correct path / file name is swan-mock_3-0.0.0%2B259-010141f9-SNAPSHOT.jar. I failed to find out why it's doing that.

[rolang]

@taig Ok I see. Thanks for providing the example, it helped investigation.

TL;DR It seems to be caused by double URL encoded + sign in the name. Removing the + from the version tag should also make it work.

Looks like coursier is storing the file name URL encoded. So a file name like swan-mock_3-0.0.0+259-010141f9-SNAPSHOT becomes 0.0.0%2B259-010141f9-SNAPSHOT on the file system. The + sign gets encoded to %2B (for encoding reference).

getClass().getClassLoader().getResources returns a collection of URLs, I guess when it get a file name like 0.0.0%2B259-010141f9-SNAPSHOT it applies URL encoding again and the % becomes %25, that's how it ends up with %252B instead of just %2B.

Can't tell for sure whether the file name should not have been URL encoded in the file system or getResources() shouldn't encode it again. I guess one could add a small workaround for that in dumbo to get it working 🤔 .

In any case it doesn't seem to be related to the usage of macros, I get the same behavior when applying it at runtime, the resources can't be found due to double URL encoded path.

[rolang]

Should be working now.

//> using repositories sonatype-s01:snapshots
//> using repositories sonatype:snapshots
//> using dep dev.rolang::dumbo:0.5.2-4-247449e-SNAPSHOT
//> using dep de.hellobonnie::swan-mock-server:0.0.0+259-010141f9-SNAPSHOT

import _root_.dumbo.ResourceFilePath

println(ResourceFilePath.fromResourcesDir("migration/swan"))

scala-cli dumbo.sc
Compiling project (Scala 3.5.2, JVM (21))
Compiled project (Scala 3.5.2, JVM (21))
List(/migration/swan/V0001__initial.sql)

[taig]

Thanks, I successfully migrated to 0.5.3 and could get rid of my custom resource scanner 👏

-def migrate(px: SxPool[IO], logger: Logger[IO])(database: String, migrations: String): IO[Unit] =
+def migrate(px: SxPool[IO], logger: Logger[IO])(database: String, migrations: List[ResourceFilePath]): IO[Unit] =
   given Console[IO] = Consoles(logger.append(Scope.Root / "dumbo" / database))
-  for
-    resources <- Resources
-      .list(migrations)
-      .filter(_.extName === ".sql")
-      .map(path => ResourceFilePath(s"/$path"))
-      .compile
-      .toList
-    _ <- Dumbo.withResources[IO](resources).withSession(px).runMigration
-  yield ()
+  Dumbo.withResources[IO](migrations).withSession(px).runMigration.void

[taig]

Had to revert due to a different issue that emerged 🙃
scalacenter/scalafix#2137

[rolang]

@taig One thing I occasionally need to do with sbt is to run copyResources beforehand. Usually happens when I have sbt shell open and run cleanup and then re-compile. It actually never happens in CI. Also closing / reopening sbt shell seems to fix it in most cases.

Have also this setting in some places to ensure that resources are copied beforehand.

(Compile / compile) := ((Compile / compile) dependsOn (Compile / copyResources)).value

The order in which resources are copied and the compilation is executed seem to vary in sbt depending on how you run the tasks... 🤔

[taig]

Thank you, that does indeed fix the problem 🙏