Researchers can turn a single photo into a video
Sometimes photos cannot truly capture a scene. How
much more epic would that vacation photo of Niagara Falls be if
the water were moving? Researchers at the University of
Washington have developed a deep learning method that can do
just that: if given a single photo of a waterfall, the system creates
a video showing that water cascading down. All that’s missing is
the roar of the water and the feeling of the spray on your face.
This method can animate any flowing material, including
smoke and clouds. This technique produces a short video that
loops seamlessly, giving the impression of endless movement.
An expert says that a picture captures a moment frozen in
time, but a lot of information is lost in a static image. That makes
people wonder what led to that moment, and how things are
changing. If people think about the last time that they found
themselves fixated on something really interesting, chances are, it
was’t totally static.
What is special about that method is that it doesn’t require
any user input or extra information. All that is needed is a
picture. And it produces as output a high-resolution, seamlessly
looping video that quite often looks like a real video. Developing
a method that turns a single photo into a believable video has
been a challenge for the field.
The system consists of two parts: first, it predicts how
things were moving when the photo was taken, and then uses that
information to create the animation. Then the system uses that
information to determine if and how each pixel should move.
Finally, the researchers want their animation to loop seamlessly
to create a look of continuous movement. The animation network
follows a few tricks to keep things clean, including transitioning
different parts of the frame at different times and deciding how
quickly or slowly to blend each pixel depending on its
surroundings.
This method works best for objects with predictable fluid
motion, like water, fire or smoke. These types of motions obey
the same set of physical laws, and there are usually cues in the
image that tell us how things should be moving. Currently, the
technology struggles to predict how reflections should move or
how water distorts the appearance of objects beneath it. In the
future, the researchers would like to extend their work to operate
on a wider range of objects, like animating a person’s hair
blowing in the wind.
Internet: <www.sciencedaily.com> (adapted).