Training-Free Neural Matte Extraction for Visual Effects: Results and Failure Cases

This study introduces a training-free high quality neural matte extraction approach that specifically targets the assumptions of visual effects production.


This content originally appeared on HackerNoon and was authored by Rendering Technology Breakthroughs

:::info Author:

(1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com);

(2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com);

(3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com);

(4) Christoph Bregler, Google Research USA (Email: bregler@google.com).

:::

4 RESULTS AND FAILURE CASES

We use images, trimaps, and GT alpha from the dataset [Rhemann et al. 2009] to allow comparison to the ground truth. Fig. 1 shows a moderately challenging case with extensive hair, some of which is similar to the background color. Fig. 3 shows several additional examples. While the ground truth and estimated alpha maps appear identical at first glance, small differences can be seen upon enlargement. These may be due to imperfections in our algorithm, but also may reflect differences in color spaces between our algorithm and the ground-truth estimation process.

\ Fig. 2 shows the extrapolated foreground 𝐹ˆ and background 𝐵ˆ for a particular image. Note that the inpainted colors need to be correct only in the unconstrained region surrounding the object border in order to allow compositing over a new background, as is the case in this figure. The unrealistic extrapolated 𝐹ˆ over the pure background regions is ignored.

\ Figure 4: (Left) objects with holes can be a failurecase. (Middle) ground truth alpha map. (Right) estimated alpha map.

4.1 Failure cases

While objects with "holes" sometimes yield good mattes (e.g. the plant in Fig. 3), they are also a failure case (Fig. 4). In some cases the trimap can be adjusted to highlight the missing holes, though this would be laborious in cases such as the cup in Fig. 4.

\

:::info This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license.

:::

\


This content originally appeared on HackerNoon and was authored by Rendering Technology Breakthroughs


Print Share Comment Cite Upload Translate Updates
APA

Rendering Technology Breakthroughs | Sciencx (2024-07-06T12:15:14+00:00) Training-Free Neural Matte Extraction for Visual Effects: Results and Failure Cases. Retrieved from https://www.scien.cx/2024/07/06/training-free-neural-matte-extraction-for-visual-effects-results-and-failure-cases/

MLA
" » Training-Free Neural Matte Extraction for Visual Effects: Results and Failure Cases." Rendering Technology Breakthroughs | Sciencx - Saturday July 6, 2024, https://www.scien.cx/2024/07/06/training-free-neural-matte-extraction-for-visual-effects-results-and-failure-cases/
HARVARD
Rendering Technology Breakthroughs | Sciencx Saturday July 6, 2024 » Training-Free Neural Matte Extraction for Visual Effects: Results and Failure Cases., viewed ,<https://www.scien.cx/2024/07/06/training-free-neural-matte-extraction-for-visual-effects-results-and-failure-cases/>
VANCOUVER
Rendering Technology Breakthroughs | Sciencx - » Training-Free Neural Matte Extraction for Visual Effects: Results and Failure Cases. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2024/07/06/training-free-neural-matte-extraction-for-visual-effects-results-and-failure-cases/
CHICAGO
" » Training-Free Neural Matte Extraction for Visual Effects: Results and Failure Cases." Rendering Technology Breakthroughs | Sciencx - Accessed . https://www.scien.cx/2024/07/06/training-free-neural-matte-extraction-for-visual-effects-results-and-failure-cases/
IEEE
" » Training-Free Neural Matte Extraction for Visual Effects: Results and Failure Cases." Rendering Technology Breakthroughs | Sciencx [Online]. Available: https://www.scien.cx/2024/07/06/training-free-neural-matte-extraction-for-visual-effects-results-and-failure-cases/. [Accessed: ]
rf:citation
» Training-Free Neural Matte Extraction for Visual Effects: Results and Failure Cases | Rendering Technology Breakthroughs | Sciencx | https://www.scien.cx/2024/07/06/training-free-neural-matte-extraction-for-visual-effects-results-and-failure-cases/ |

Please log in to upload a file.




There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.