Depth Aware Tracking in Unity with Deckard Chromakey

Depth Aware chromakeying without any depth sensing hardware. As my new Deckard Chromakey for @Unity @unitygames has a tracking feature that works in realtime with extremely good performance, I have aplenty of power to do some other things. I'm experimenting a depth approximation that can be used for placing Chromakey plane on correct depth. So that a speaker in any virtual studio can come off behind a desk in a front of a desk. While staying on a ground, and casting shadows on virtual objects. Everything happens in realtime. Environment is mixed #Diffusionsplats made with @Scaniverse and 3D cubes inside Unity. As this is just a test, I didn't work a lot on visual fidelity, and this can be improved a lot. Also, I'm testing this with a video footage made in 2 minutes. It's a pretty rough test, and I would actually want a bigger chromakey studio right now 😉 Tracker tracks feet position with a help of fast compute shaders and algorythms that i've made for this task. Afterwards, based on results, it reprojects chromakeyed image onto a correct depth (defined by a virtual plane). This needs some setup: you have to know a height of a camera from a ground. Also, it works better if a camera is positioned higher and uses a wider lens.