After checking some more I realised the problem is the way the program handles scene framerates and video framerates.
There is no transcoding or adjusting of any kind for video, it just sets the video frames to timeline frames, so if a video has 30 frames in 1 second, after importing into a 30fps scene it'll be mapped 1:1, but if you import that same video into a 60fps scene it will STILL be mapped 1:1, making the video play faster than it should.

example
600 frames scene @ 30fps, 20 seconds of duration
We add a 10 seconds long video @ 30fps, the video plays normally, each video frame is mapped to a scene frame, 1:1
600 frames scene @ 60fps, 10 seconds of duration
We add a 10 seconds long video @ 30fps, the video plays twice as fast, each video frame is mapped to a scene frame 1:1, instead of maintaining the video frame for 2 scene frames.

I'm currently looking through the code to see where exactly this takes place.

Found it, I'll try to submit a fix that takes scene framerate into account when calculating the video framerate later!

That would be awesome 👍