Adobe’s Project Cloak Removes Undesired Elements From Videos

The video is pretty cool, but it would be even better if your ex weren't on it? Adobe's Project Cloak may have what you need.

Sneaks, Technological Experiments, at MAX 2017

During its annual MAX conference, Adobe always gives the visitors a peek at the technologies that are in the works. Adobe calls them sneaks. One of the sneaks on MAX 2017 was called Project Cloak. Cloak is a pretty appropriate description of the function hidden behind the name. Cloak allows you to remove undesired elements from videos. This works in a way that's similar to the content based filling in Photoshop. As you know, it lets you mask items, and then have Photoshop remove them from the image. The area that was covered by the removed element is filled with the content of the background. Ideally, you can't even tell that there was something else in that spot before. In theory, we could do the same for videos in tedious frame-by-frame work. However, this would become very obvious when playing the video, as we have changed each frame, but didn't adjust them to each other. Thus, we need a technology that can consistently remove image elements across multiple frames.

Cloak Removes Elements Under All Circumstances

This is precisely what Project Cloak promises to do. The following video shows you what Cloak is capable of in six minutes: https://www.youtube.com/watch?v=TzBZWBht02I Comparatively, the most straightforward task is removing a static element from a video. Like the light pole in this image, for example: [caption id="attachment_103264" align="aligncenter" width="1024"] Look at the light pole right in front of the building.[/caption] Using Cloak, the pole can simply be masked, using a rough polygon selection, for instance. The result looks like this: [caption id="attachment_103263" align="aligncenter" width="1024"] The image on the right shows the result of the choice in the left-hand image.[/caption] The pole has vanished completely, and even in the playing video (see above), there is no indication of a pole possibly being there. The removal is residue free. Cloak can also handle more difficult challenges. For example, it can completely remove people or groups of people: [caption id="attachment_103262" align="aligncenter" width="1024"] If you prefer your desert video without any people...[/caption] The demand could also be eliminating the shadow of the recording drone on a t-shirt. If you can select it, you can remove it. This is a fantastic outlook for videographers.

It is Unclear When You Get to Use the Function

At the conference, Cloak was used as a part of After Effects. As it is only an experimental feature as of right now, you shouldn't expect to find the function in your After Effects anytime soon. Since the feature seems to work reliably already, though, it shouldn't take years until we can ban all of our exes and the in-laws from our videos. Let's rewrite our histories.