Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

undefined shifts #18

Open
michaelni opened this issue Jun 17, 2024 · 0 comments
Open

undefined shifts #18

michaelni opened this issue Jun 17, 2024 · 0 comments

Comments

@michaelni
Copy link

FFmpeg / Google ossfuzz did run into 2 bad shifts while encoding with libtheora through FFmpeg.

The first likely should be *8 instead of <<3, theres are 13 more matches to stride<< i dont know if these can be negative too

The 2nd case shifts into the sign bit which is undefined. It likely was meant to be unsigned

The stack traces from UndefinedBehaviorSanitizer are below

	state.c:654:19: runtime error: left shift of negative value -96
    #0 0x87b6f0 in oc_state_ref_bufs_init theora/lib/state.c:654:19
    #1 0x879e0c in oc_state_init theora/lib/state.c:736:17
    #2 0x85dc43 in oc_enc_init theora/lib/encode.c:1159:7
    #3 0x85dc43 in th_encode_alloc theora/lib/encode.c:1346:17
    #4 0x409499 in encode_init /src/ffmpeg/libavcodec/libtheoraenc.c:231:18
    #5 0x4d27ab in avcodec_open2 /src/ffmpeg/libavcodec/avcodec.c:326:19
    #6 0x4d11b5 in LLVMFuzzerTestOneInput /src/ffmpeg/tools/target_enc_fuzzer.c:153:15
    #7 0x436ab3 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
    #8 0x422212 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6
    #9 0x427abc in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9
    #10 0x450ff2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #11 0x79fee856e082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
    #12 0x4183dd in _start
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior state.c:654:19 in
	state.c:399:37: runtime error: left shift of 1 by 63 places cannot be represented in type 'ogg_int64_t' (aka 'long')
    #0 0x87ebd8 in oc_state_border_init theora/lib/state.c:399:37
    #1 0x87a9c4 in oc_state_frarray_init theora/lib/state.c:524:3
    #2 0x879de9 in oc_state_init theora/lib/state.c:735:7
    #3 0x85dc43 in oc_enc_init theora/lib/encode.c:1159:7
    #4 0x85dc43 in th_encode_alloc theora/lib/encode.c:1346:17
    #5 0x409499 in encode_init /src/ffmpeg/libavcodec/libtheoraenc.c:231:18
    #6 0x4d27ab in avcodec_open2 /src/ffmpeg/libavcodec/avcodec.c:326:19
    #7 0x4d11b5 in LLVMFuzzerTestOneInput /src/ffmpeg/tools/target_enc_fuzzer.c:153:15
    #8 0x436ab3 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
    #9 0x422212 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6
    #10 0x427abc in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9
    #11 0x450ff2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #12 0x7b6e9cb8d082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
    #13 0x4183dd in _start
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior state.c:399:37 in
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant