-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #296 from DSD-DBS/determine-memory-options
feat: Derive JVM-memory options automatically from host/container
- Loading branch information
Showing
17 changed files
with
194 additions
and
10 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -47,3 +47,6 @@ prometheus* | |
| volumes | ||
| *.egg-info/ | ||
| .env | ||
|
|
||
| **/set_memory_flags.py | ||
| !eclipse/set_memory_flags.py | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| <!-- | ||
| ~ SPDX-FileCopyrightText: Copyright DB InfraGO AG and contributors | ||
| ~ SPDX-License-Identifier: Apache-2.0 | ||
| --> | ||
|
|
||
| # Directory for startup scripts | ||
|
|
||
| This directory contains scripts that are executed during container startup. As | ||
| of now, we support scripts with file-ending `.sh` and `.py`. |
File renamed without changes.
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| <!-- | ||
| ~ SPDX-FileCopyrightText: Copyright DB InfraGO AG and contributors | ||
| ~ SPDX-License-Identifier: Apache-2.0 | ||
| --> | ||
|
|
||
| # Memory options for Eclipse | ||
|
|
||
| To specify fixed memory options for the JVM, you can pass the following | ||
| environment variables to the `docker run` commands: | ||
|
|
||
| - `MEMORY_MIN` (default `70%`), translated to `-Xms` for absolute values and | ||
| `-XX:InitialRAMPercentage` and `-XX:MinRAMPercentage` for percentage values. | ||
| Percentage values are calculated according to the total memory or the | ||
| requested memory by the container. | ||
| - `MEMORY_MAX` (default `90%`), translated to `-Xmx` for absolute values and | ||
| `-XX:MaxRAMPercentage` for percentage values. Percentage values are | ||
| calculated according to the total memory of the system or the total memory | ||
| available to the container. | ||
|
|
||
| If the value ends with a %, we assume that it's a percentage value. | ||
|
|
||
| - If used in a Kubernetes cluster, it determines the values from the Pod | ||
| requests/limits. | ||
| - If used on a host system, it determines the value from the host memory. | ||
|
|
||
| See also: | ||
|
|
||
| - https://stackoverflow.com/a/65327769 | ||
| - https://www.merikan.com/2019/04/jvm-in-a-container/#java-10 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| # SPDX-FileCopyrightText: Copyright DB InfraGO AG and contributors | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| """Set the memory flags in the *.ini file for Eclipse applications. | ||
| The script reads the following environment variable: | ||
| ECLIPSE_EXECUTABLE: The path to the eclipse executable. | ||
| MEMORY_MIN: The minimum memory that the JVM should use. | ||
| Check the set_memory_flags function for more information. | ||
| MEMORY_MAX: The maximum memory that the JVM should use. | ||
| Check the set_memory_flags function for more information. | ||
| The script is called during container startup. | ||
| """ | ||
|
|
||
| import os | ||
| import pathlib | ||
|
|
||
|
|
||
| def remove_all_memory_flags(file_content: list[str]) -> list[str]: | ||
| """Remove all memory flags from *.ini files. | ||
| To make sure that we don't have any conflicting memory flags in the | ||
| .ini file, we remove all flags that start with -Xm. | ||
| This will explicitely remove the -Xms and -Xmx flags. | ||
| """ | ||
|
|
||
| return [ | ||
| line for line in file_content if not line.lstrip().startswith("-Xm") | ||
| ] | ||
|
|
||
|
|
||
| def set_memory_flags( | ||
| file_content: list[str], memory_min: str, memory_max: str | ||
| ) -> None: | ||
| """Set the memory flags in *.ini. | ||
| If the value of memory_max ends with a %, we assume that it's a percentage value. | ||
| - If used in a Kubernetes cluster, it determines the values from the Pod requests/limits. | ||
| - If used on a host system, it determines the value from the host memory. | ||
| See Also | ||
| -------- | ||
| - https://stackoverflow.com/a/65327769 | ||
| - https://www.merikan.com/2019/04/jvm-in-a-container/#java-10 | ||
| """ | ||
|
|
||
| if memory_min.strip().endswith("%"): | ||
| append_flag_to_file( | ||
| file_content, "XX:InitialRAMPercentage=", memory_min.strip("%") | ||
| ) | ||
| append_flag_to_file( | ||
| file_content, "XX:MinRAMPercentage=", memory_min.strip("%") | ||
| ) | ||
| else: | ||
| append_flag_to_file(file_content, "Xms", memory_min) | ||
|
|
||
| if memory_max.strip().endswith("%"): | ||
| append_flag_to_file( | ||
| file_content, "XX:MaxRAMPercentage=", memory_max.strip("%") | ||
| ) | ||
| else: | ||
| append_flag_to_file(file_content, "Xmx", memory_max) | ||
|
|
||
|
|
||
| def print_vm_memory_usage_during_start(file_content: list[str]) -> None: | ||
| """Print the JVM memory options during start.""" | ||
|
|
||
| file_content.append("-XshowSettings:vm") | ||
|
|
||
|
|
||
| def append_flag_to_file( | ||
| file_content: list[str], flag: str, value: str | ||
| ) -> None: | ||
| """Append a flag to the *.ini file.""" | ||
|
|
||
| file_content.append(f"-{flag}{value}") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| eclipse_executable = pathlib.Path(os.environ["ECLIPSE_EXECUTABLE"]) | ||
| _memory_min = os.environ["MEMORY_MIN"].strip() | ||
| _memory_max = os.environ["MEMORY_MAX"].strip() | ||
|
|
||
| ini_path = eclipse_executable.with_suffix(".ini") | ||
|
|
||
| _file_content = ini_path.read_text("utf-8").splitlines() | ||
|
|
||
| _file_content = remove_all_memory_flags(_file_content) | ||
| set_memory_flags(_file_content, _memory_min, _memory_max) | ||
| print_vm_memory_usage_during_start(_file_content) | ||
|
|
||
| ini_path.write_text("\n".join(_file_content), encoding="utf-8") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters