Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IDF caching for libraries #2850

Merged
merged 7 commits into from
Jul 3, 2024

Conversation

mikee47
Copy link
Contributor

@mikee47 mikee47 commented Jul 2, 2024

Cache esp-idf and tools for library builds. Be nice if we could use the cache from SmingHub/Sming directly but that sort of cross-repo sharing isn't supported.

Also discovered that cross-OS caching breaks because the cache action uses paths relative to the build directory, even though it's been given absolute paths. So, just simplify the thing and use a separate, combined cache for each OS/IDF combination. Total's about 5GB.

In all the .yml files we don't need to repeat steps for Ubuntu/MacOS as they're the same, so we can use a combined step for those, ie. not windows.

Scanlog tool:

  • Fix scanlog tool windows path normalisation - didn't account for change from c:/tools/ to d:/opt/, and that Windows idf path contains version.
  • Print warnings for each job by default, add 'm' flag to merge them

Also fix a few more warnings.

Copy link

what-the-diff bot commented Jul 2, 2024

PR Summary

  • Enhancements Made to the GitHub Workflow
    A default value has been set for the 'INSTALL_IDF_VER' environment variable in the library.yml workflow file. This simplifies the handling of this varibale. The workflow now also takes care of fixing permissions on non-Windows systems. Additionally, enhancements have been made to cache ESP-IDF and IDF build tools for Esp32 architecture, improving efficiency and speed of development workflows.

  • Codebase Clean-up
    Various unused parameters in functions across multiple files were found and removed. This includes setFileMap function in HttpMultipartResource.cpp, onStarted function in SwitchJoycon.h, decode function in Callback.cpp & InputStream::decode function in Stream.cpp, along with DummyOutputStream::write function in Stream.h and i2sCallback function in application.cpp in Basic_Audio sample. An unused function shutdown in HttpMultipartResource.h was also removed. This leads to cleaner code and fewer chances of misunderstanding and bugs.

  • Documentation Update
    The README.rst file now includes additional details explaining how to use warnings output in Visual Studio Code. This makes understanding and handling warning outputs easier for users.

  • Improvements in Scanlog.py
    Modifications were made in scanlog.py to introduce a new command-line argument -m, which enables merging of warnings from all jobs. The tools allows warnings output path replacement and removing prefixes. It will print merged or individual job warnings based on the -m flag. Also, it can now exclude warnings based on a list in a file, providing more control over warning management.

@mikee47
Copy link
Contributor Author

mikee47 commented Jul 2, 2024

I'm impressed with that summary from 'what-the-diff' - it's actually useful :-)

@mikee47 mikee47 changed the title Add IDF caching for libraries [WIP] Add IDF caching for libraries Jul 2, 2024
@mikee47
Copy link
Contributor Author

mikee47 commented Jul 2, 2024

Changing to WIP, library caching not quite working as expected for Ubuntu.

@mikee47
Copy link
Contributor Author

mikee47 commented Jul 2, 2024

Yep, turning build debugging on it's using relative paths when restoring the cache. Looks like the cache for IDF was saved in Windows and ended up being restored to a sub-folder somewhere. I'll have a dig about and see if the damn thing can use absolute paths like it's been given.

@mikee47 mikee47 changed the title [WIP] Add IDF caching for libraries Add IDF caching for libraries Jul 2, 2024
@mikee47
Copy link
Contributor Author

mikee47 commented Jul 2, 2024

Hang on, what's varibale ? I thought AI could spell...

@mikee47 mikee47 force-pushed the feature/library-ci-cache branch from 42a0876 to 928467a Compare July 2, 2024 15:50
Link warnings don't have a source reference, but include them
Use Parser class to manage state
Use methods for state
Speed up using regex
@slaff slaff merged commit f1d1a65 into SmingHub:develop Jul 3, 2024
47 of 48 checks passed
@mikee47 mikee47 deleted the feature/library-ci-cache branch July 3, 2024 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants