A tool meant to allow for quick data extraction from StarCraft 2 replay files ".SC2Replay".
Note
This steps assumes that you have Docker installed on your machine and working. Additionally, you will either need to pull the docker image from DockerHub or build it locally as described in Build Docker Images.
The easiest way to run this tool is to use the provided Docker image release. Please note that the maps directory can be mounted to see what map files are downloaded, and what map files were pre-seeded with the Docker Image. Other command line options are described in CLI Options.
Commands below showcases the exampe usage of the tool. Steps:
- Pull the Docker image from DockerHub or Build Docker Images:
docker pull kaszanas/sc2infoextractorgo:latest- Place your
.SC2Replayfiles in the directory which you want to mount as the input directory. The default directory for replays within the container is/app/replays/input. - Pick one of the following commands.
- Run the docker container and mount the entire directory:
docker run -it --rm \ -v </path/to/your/directory>:/app \ sc2infoextractorgo:latest -help
- Run the Docker container and mount specific paths.
Replace the
<path>with the appropriate paths on your system:
docker run -it --rm \ -v </path/to/your/replays>:/app/replays/input \ -v </path/to/your/output>:/app/replays/output \ -v </path/to/your/logs>:/app/logs \ -v </path/to/your/maps>:/app/maps \ sc2infoextractorgo:latest -help
- When you execute the processing command, please note that you need to pass the appropriate paths to the tool that are withing the container. The default paths are:
- Input directory:
/app/replays/input - Output directory:
/app/replays/output - Logs directory:
/app/logs - Maps directory:
/app/maps - Additionally the tool outputs some auxiliary files that can be used for debugging purposes. These are created in the main directory of the container
/app/.
- Input directory:
Note
If you wish to see other files that are generated by the tool, you will have to mount the appropriate directories. The tool generates logs in the logs directory.
Note
These steps assume that you downloaded the release binary or you will have to compile the tool from source.
- Place
.SC2Replayfiles in./replays/input - Run
SC2InfoExtractorGo.exeas follows:
SC2InfoExtractorGo.exe -input ./replays/input -output ./replays/output- Verify the output in
./replays/output - If The output packages do not contain any processed replays, proceed to verify
./logs/.
To see the full list of available options, run the tool with the -help flag:
SC2InfoExtractorGo.exe -helpThe following flags are available:
-help
Show command usage
-game_mode_filter int
Specifies which game mode should be included from the processed files
in a format of a binary flag: AllGameModes: 0b11111111 (default 0b11111111) (default 255)
-input string
Input directory where .SC2Replay files are held. (default "./replays/input")
-log_dir string
Specifies directory which will hold the logging information. (default "./logs/")
-log_level int
Specifies a log level from 1-7:
Panic - 1, Fatal - 2,
Error - 3, Warn - 4,
Info - 5, Debug - 6,
Trace - 7 (default 4)
-maps_directory string
Directory where the maps will be downloaded as a result of the
replay processing. (default "./maps/")
-max_procs int
Specifies the number of logic cores of a processor that will be used for processing (default runtime.NumCPU()). (default 24)
-number_of_packages int
Provide a number of zip packages to be created and compressed
into a zip archive. Please remember that this number needs to be lower
than the number of processed files. If set to 0, will ommit the
zip packaging and output .json directly to drive. (default 1)
-only_map_download
Flag specifying if the tool is supposed to only download
the maps and not process the replays.
-output string
Output directory where compressed zip packages will be saved. (default "./replays/output")
-perform_chat_anonymization
Flag, specifying if the chat anonymization should be performed.
-perform_cleanup
Flag specifying if the tool is supposed to perform the cleaning
functions within the processing pipeline.
-perform_filtering
Flag, specifying if the pipeline ought to verify different hard coded game modes.
If set to false completely bypasses the filtering.
-perform_integrity_checks
Flag specifying if the software is supposed to check the hardcoded
integrity checks for the provided replays
-perform_player_anonymization
Flag specifying if the tool is supposed to perform player anonymization
functions within the processing pipeline.
If set to true please remember to download and run
an anonymization server: https://doi.org/10.5281/zenodo.5138313
-perform_validity_checks
Flag, specifying if the tool is supposed to use hardcoded validity checks
and verify if the replay file variables are within 'common sense' ranges.
-with_cpu_profiler string
Set path to the file where pprof cpu profiler will save its information.
If this is empty no profiling is performed.
If You have a pack of replays with nested directories and You would like to automatically flatten the directory structure, We have published a tool that can be used for that, please see SC2DatasetPreparator: https://doi.org/10.5281/zenodo.5296664
In order to anonymize the replays please make sure to download and run our open-source implementation of an anonymization server the SC2AnonServerPy: https://doi.org/10.5281/zenodo.5138313
This is required because of the multiprocessing nature of our code that needs to perform synchronization with an existing database of unique toons (player IDs) that are mapped to arbitrary incrementing integer.
Existing implementation downloads the maps from the Blizzard servers. This is to normalize the map names to English language. When there is no internet connection available, our tool should fallback to reading the map names from the files placed in the ./maps directory.
Currently the software supports some game mode filtering capabilities which can be used with -game_mode flag.
The flag itself is a binary flag where 0b11111111 is all game modes which is the default.
Other ways to set the flag:
0b00000001: 1v1 Ranked Games0b00000010: 2v2 Ranked Games0b00000100: 3v3 Ranked Games0b00001000: 4v4 Ranked Games0b00010000: 1v1 Custom Games0b00100000: 2v2 Custom Games0b01000000: 3v3 Custom Games0b10000000: 4v4 Custom Games
This repository is licensed under GNU GPL v3 license. If You would like to acquire a different license please contact me directly.
@software{BialeckiExtractor2021,
author = {Białecki, Andrzej and
Białecki, Piotr and
Krupiński, Leszek},
title = {{Kaszanas/SC2InfoExtractorGo: 1.2.0
SC2InfoExtractorGo Release}},
month = {jun},
year = {2022},
publisher = {Zenodo},
version = {1.2.0},
doi = {10.5281/zenodo.5296788},
url = {https://doi.org/10.5281/zenodo.5296788}
}