Rolling shutter compensation in-camera?

[TheBoogieKnight]

We've got a lot of cameras now that allow you to crop in to improve (or give) IBIS so I wondered if any companies have used the same idea in-camera to combat rolling shutter. The cameras have accelerometers and gyros built in so presumably this could be implemented fairly easily although it obviously wouldn't work on moving subjects!

[UncleBobsPhotography]

I guess rolling shutter compensation for a pan would be possible. For a sideways movement (truck?), the parallax effect would make it difficult since closer objects would require more rolling shutter compensation than objects farther away.

Rolling shutter is most obvious during pans, and the gyroscope should be able to notice the movement type, so why not?

[kye]

One challenge would be how to fill the frame.

If you straighten the lines (from this //////// to this ||||||||) then the problem is that you have to crop off the sides a bit (see diagram #2)

Then you'll either get black bars on the sides, or you'd have to crop in, but if it just cropped in and out based on horizontal movement then that would be very strange.  If you were doing it in post with accelerometer data then you could make good decisions about cropping and other things, so maybe that's the better approach - to save the accelerometer data in the footage then process it afterwards.

[TheBoogieKnight]

2 hours ago, kye said:

One challenge would be how to fill the frame.

If you straighten the lines (from this //////// to this ||||||||) then the problem is that you have to crop off the sides a bit (see diagram #2)

Then you'll either get black bars on the sides, or you'd have to crop in, but if it just cropped in and out based on horizontal movement then that would be very strange.  If you were doing it in post with accelerometer data then you could make good decisions about cropping and other things, so maybe that's the better approach - to save the accelerometer data in the footage then process it afterwards.

Hmm interesting. Even with pretty fast pans it doesn't seem like you'd need much of a crop so maybe a fixed crop would work. Saving the movement data definitely sounds like a potential avenue although maybe AI based solutions could calculate it from the image without even needing this data (warp stabilisers already do something similar). I've been using Denoise AI recently and it's pretty incredible how far we've come in terms of image processing.

[kye]

25 minutes ago, TheBoogieKnight said:

Hmm interesting. Even with pretty fast pans it doesn't seem like you'd need much of a crop so maybe a fixed crop would work. Saving the movement data definitely sounds like a potential avenue although maybe AI based solutions could calculate it from the image without even needing this data (warp stabilisers already do something similar). I've been using Denoise AI recently and it's pretty incredible how far we've come in terms of image processing.

I agree that AI could doo a reasonable job of it if implemented well.  I have wondered why there aren't more tools that fix issues in footage like this, but you're right that we've come a long way, and what's yet to come will dwarf everything still!