Apple is working on a control system for video processors that selectively control the operation of motion stablization processes, per a new patent (number 20110234825) at the US Patent & Trademark Office.
According to the invention, motion sensor data indicative of motion of a mobile device may be received and processed. A determination may be made by comparing processed motion sensor data to a threshold. Based on the determination, motion stabilization may be suspended on select portions of a captured video sequence. The inventors are Yuxin Liu, Xiaojin Shi, James Oliver Normile and Hsi-Jung Wu.
Here’s Apple’s background and summary of the invention: “Video stabilization is a class of video processing that removes unwanted shakiness from videos captured from portable camera devices such as smart phones, personal entertainment systems laptop computers and/or camcorder. The goal of video stabilization is to revise an original video sequence to mimic a sequence that would have been obtained if a camera captured the video from an ideal or a specified motion trajectory.
“Specifically, video stabilization techniques generate an idealized motion vector of a captured video sequence and then introduce motion compensation to a sequence of captured video to replicate the idealized motion vector. If, for example, a video stabilization algorithm estimated that a video sequence should exhibit no motion (e.g., ideally a camera would have been perfectly still during motion capture), then the motion compensation processes would estimate a global motion vector on each frame and perform processes to remove the global motion.
“Although video stabilization can improve the perceptual quality of a video sequence, it has its consequences. First, it can consume considerable resources at a capture device or processing device. Second, it can reduce the field of view of the final video sequence. Third and perhaps most importantly, video stabilization can impair perceived quality if the algorithm generates an incorrect estimate of idealized motion or an incorrect estimate of the source motion vector.
“Increasingly, consumer capture devices are provisioned with motion detection devices such as accelerometers and/or gyroscopes. The motion detection devices can provide metadata that indicates motion effects of a camera during video capture, however, even though the motion detectors provide data relating to global motion of the camera, the level of shakiness between frames often is comparable to the noise level of the motion detector data. Such high level of the noise in data prohibits directly use of accelerometer data in video stabilization.”
— Dennis Sellers
