Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New functions to read elf files and parse modules (SP3) #155

Merged
merged 4 commits into from
Sep 26, 2023

Conversation

softwarecki
Copy link
Contributor

@softwarecki softwarecki commented Apr 6, 2023

Rewritten elf file reading and module parsing:

  • Set of new functions for reading elf files. A structure was created to represent a file, a section and a strings section. Created a new functions to read a elf file, retrieve a section header based on index or name, read a section contents based on a header or name, retrieve section name and functions that print file, section and program headers.
  • Set of a new functions to parse modules. A structure was created to represent the module, module section and section
    informations. Added a set of new functions for parsing a module and reading its contents.

Moved the module information to the new module structure. Removed elf.c file that is no longer in use.

SOF CI PR thesofproject/sof#7413

@marc-hb
Copy link
Contributor

marc-hb commented Apr 6, 2023

I don't understand, there are elfutils libraries already, why are we re-inventing this? Parsing binaries in C in 2023 looks like instant technical debt.

Copy link
Member

@lgirdwood lgirdwood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

src/module.c Show resolved Hide resolved
@lgirdwood
Copy link
Member

I don't understand, there are elfutils libraries already, why are we re-inventing this? Parsing binaries in C in 2023 looks like instant technical debt.

Probably not as simple to use elfutils for the required usage and nothing wrong with C here.

@plbossart
Copy link
Member

I don't understand, there are elfutils libraries already, why are we re-inventing this? Parsing binaries in C in 2023 looks like instant technical debt.

Probably not as simple to use elfutils for the required usage and nothing wrong with C here.

It's however a legitimate question to ask where this code comes from and how it was validated, no? Please tell me it didn't come from ChatGPT :-)

@pjdobrowolski
Copy link
Contributor

looks legit for me. I put tactical dot to watch for more changes.

marc-hb added a commit to marc-hb/rimage that referenced this pull request Apr 18, 2023
The deleted "install" target copied only the rimage executable and
left the config files behind. This gave the wrong impression that the
executable is useful without config files.

"Installing" also gave the wrong impression that rimage versions are
somewhat stable and relatively independent of SOF versions: they're
not. In fact there is no such thing as an rimage "version": everyone
should always use the exact rimage _git commit_ from the west manifest
or git submodule. There are no rimage "releases" and no semantic
versioning or anything like it, rimage is effectively part of the SOF
source and build and run directly from its build directory by
practically every SOF developer and SOF CI thanks to

  sof/src/arch/xtensa/CMakeLists.txt#ExternalProject_Add(rimage_ep ...
  sof/scripts/xtensa-build-zephyr.py#def build_rimage()
  sof/zephyr/CMakeLists.txt#set(RIMAGE_CONFIG_PATH ...

etc.

Providing an "install" target greatly increases the chances of
different people and CIs using different rimage versions which is the
last thing we want considering the many significant rimage changes
happening right now, examples:

- zephyrproject-rtos/zephyr#56099
- zephyrproject-rtos/zephyr#54700
- thesofproject/sof#7270
- thesofproject/sof#7304
- thesofproject/sof#7320
- thesofproject#155

While it's useful for multiple files (e.g.: config files), a CMake
target was always overkill to copy a single file. Someone or some
script who really wants to copy the rimage binary to some other place
that the build directory for some (discouraged) reason _can still do
so_ with a much more basic, simpler and more transparent file copy
command.

Finally, the default "bin" DESTINATION required root access which
is otherwise totally unncessary to build SOF.

Signed-off-by: Marc Herbert <[email protected]>
marc-hb added a commit to marc-hb/rimage that referenced this pull request Apr 18, 2023
The deleted "install" target copied only the rimage executable and
left the config files behind. This gave the wrong impression that the
executable is useful without config files.

"Installing" also gave the wrong impression that rimage versions are
somewhat stable and relatively independent of SOF versions: they're
not. In fact there is no such thing as an rimage "version": everyone
should always use the exact rimage _git commit_ from the west manifest
or git submodule. There are no rimage "releases" and no semantic
versioning or anything like it, rimage is effectively part of the SOF
source and build and run directly from its build directory by
practically every SOF developer and SOF CI thanks to

  sof/src/arch/xtensa/CMakeLists.txt#ExternalProject_Add(rimage_ep ...
  sof/scripts/xtensa-build-zephyr.py#def build_rimage()
  sof/zephyr/CMakeLists.txt#set(RIMAGE_CONFIG_PATH ...

etc.

Providing an "install" target greatly increases the chances of
different people and CIs using different rimage versions which is the
last thing we want considering the many significant rimage changes
happening right now, examples:

- zephyrproject-rtos/zephyr#56099
- zephyrproject-rtos/zephyr#54700
- thesofproject/sof#7270
- thesofproject/sof#7304
- thesofproject/sof#7320
- thesofproject#155

While it's useful for multiple files (e.g.: config files), a CMake
target was always overkill to copy a single file. Someone or some
script who really wants to copy the rimage binary to some other place
that the build directory for some (discouraged) reason _can still do
so_ with a much more basic, simpler and more transparent file copy
command.

Finally, the default "bin" DESTINATION required root access which
is otherwise totally unncessary to build SOF.

Signed-off-by: Marc Herbert <[email protected]>
kv2019i pushed a commit that referenced this pull request Apr 26, 2023
The deleted "install" target copied only the rimage executable and
left the config files behind. This gave the wrong impression that the
executable is useful without config files.

"Installing" also gave the wrong impression that rimage versions are
somewhat stable and relatively independent of SOF versions: they're
not. In fact there is no such thing as an rimage "version": everyone
should always use the exact rimage _git commit_ from the west manifest
or git submodule. There are no rimage "releases" and no semantic
versioning or anything like it, rimage is effectively part of the SOF
source and build and run directly from its build directory by
practically every SOF developer and SOF CI thanks to

  sof/src/arch/xtensa/CMakeLists.txt#ExternalProject_Add(rimage_ep ...
  sof/scripts/xtensa-build-zephyr.py#def build_rimage()
  sof/zephyr/CMakeLists.txt#set(RIMAGE_CONFIG_PATH ...

etc.

Providing an "install" target greatly increases the chances of
different people and CIs using different rimage versions which is the
last thing we want considering the many significant rimage changes
happening right now, examples:

- zephyrproject-rtos/zephyr#56099
- zephyrproject-rtos/zephyr#54700
- thesofproject/sof#7270
- thesofproject/sof#7304
- thesofproject/sof#7320
- #155

While it's useful for multiple files (e.g.: config files), a CMake
target was always overkill to copy a single file. Someone or some
script who really wants to copy the rimage binary to some other place
that the build directory for some (discouraged) reason _can still do
so_ with a much more basic, simpler and more transparent file copy
command.

Finally, the default "bin" DESTINATION required root access which
is otherwise totally unncessary to build SOF.

Signed-off-by: Marc Herbert <[email protected]>
@lgirdwood
Copy link
Member

@softwarecki any update ?

@softwarecki
Copy link
Contributor Author

@lgirdwood I'll try to finish work on this this PR soon.

@marc-hb
Copy link
Contributor

marc-hb commented Sep 19, 2023

Why parsing binaries in C in 2023 is (at best) a waste of time:

A structure was created to represent a file, a section and a strings
section. Created a new functions to read a elf file, retrieve a section
header based on index or name, read a section contents based on
a header or name, retrieve section name and functions that print file,
section and program headers.

Signed-off-by: Adrian Warecki <[email protected]>
@softwarecki softwarecki marked this pull request as ready for review September 22, 2023 13:20
@softwarecki
Copy link
Contributor Author

@plbossart: The code is the result of human work and was not created by chatGPT. The latter has the ability to give elaborate answers without any substance and sometimes has hallucinations. To perform a validation, I created a PR in SOF thesofproject/sof#7413.

@lgirdwood I think its ready now :)

lgirdwood
lgirdwood previously approved these changes Sep 22, 2023
@lgirdwood
Copy link
Member

@kv2019i fyi.

@marc-hb
Copy link
Contributor

marc-hb commented Sep 25, 2023

There are elfutils libraries already, why are we re-inventing this? Parsing binaries in C in 2023 looks like instant technical debt.

Ping, @softwarecki please explain how is this not re-inventing https://www.freshports.org/devel/elfutils/

@marc-hb marc-hb requested review from andyross and removed request for aborisovich September 25, 2023 04:02
A structure was created to represent the module, module section and section
informations. Added a set of new functions for parsing a module and reading
its contents.

Signed-off-by: Adrian Warecki <[email protected]>
Moved the module information to the new module structure. Used new
functions to parse module.

Signed-off-by: Adrian Warecki <[email protected]>
Removed elf.c file that is no longer in use.

Signed-off-by: Adrian Warecki <[email protected]>
@softwarecki
Copy link
Contributor Author

@marc-hb I concluded that it would be better to write a few simple functions for reading files than add a dependency to another (quite large) library. If this is a problem for you, you can use elfutils and create your own PR. It will be easier now, as I separated all elf file operations into a separate module.

Copy link
Contributor

@kv2019i kv2019i left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given this is replacing in-tree elf parser with another one and author of old code is approving, I'm good to go with this.

@pjdobrowolski pjdobrowolski merged commit 4fc431b into thesofproject:main Sep 26, 2023
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants