i have managed to get halfway without any hassle, the final thing i need to solve is a chromakeying in 3-d space. and that’s where my main issues started to appear.
i created a plane in maya and exported as .obj file. then i imported it in max/MSP scene along with my 3-d cube elements. i even applied successfully a live texture onto my plane. next thing what i was working on is applying chromakeying on the texture feed. at that stage i thought what max will do is project only keyed image on the plane, there was a back thought of rest of the plane, but i had to run it through first test. what it came out with was a fine keyed texture on the 3-d plane, but “transparency” or unwanted areas were displayed in black. here is the patch showing this experiment::
if i look at the chromakey output window, keyed areas are black, too. so, when applied as a texture it will remain being black. i didn’t have any problem when keyed live footage was superimposed on top of another footage, in areas where it is black the underlying footage showed through. i did look into the explanation of chromakeying process again to see why black areas would appear on top of a plane. i was looking into alpha channel and transparency issues in jitter.
this is an information i found on jitter documentation: “ARGB (alpha, red, green, blue)- a 4 plane char data used in jitter to deal with colours and alpha channel. fourth plane is often useful for what is known as the alpha channel—a channel that stores information about how transparent a pixel should be when overlaid on another image. In Jitter, this is the most common way to describe a color: as a combination of exact intensities of red, green, and blue… For each pixel of an image—be it a video, a picture, or any other 2D matrix—we need at least three values, one for each of the three basic colors. Therefore, for onscreen color images, we most commonly use a 2D matrix with at least three planes…” all this information is relevant when referring to 2-d environment, not 3-d. my main issue is solving transparency on or in 3-d space but i am afraid that hasn’t been discussed much on max/MSP documentation. even-though in 3-d environment, jitter treats matrix chromakeying as 2-d and it doesn’t understands or i haven’t found the way how to make 3-d element become transparent on keyed areas.
in my first attempt i used 3-d plane which was overlaid with live texture. texture was keyed out and because nothing was displayed underneath it, it came out black. my aim is to achieve a transparency in black area, so i looked into different ways of importing video footage in 3-d environment, and there is another way of having a videoplane object instead of plane rendered out from maya. i used object which maps incoming video to a plane in 3-d space. this may be used to exploit hardware accelerated rotation, scaling, interpolation, and blending. here is a patch in which i used this jit.gl.videoplane object and chromakeyed it. unfortunately it came out with very similar undesired outcome – alpha as a black area and no transparency.
here is a close-up with non keyed green-screen footage:
where as this one is keyed
as a result i still don’t have any transparency which doesn’t allow to see 3-d cubes rotating behind the person. the scene is too big and black areas shouldn’t be there at all. so i need to try other options.
i was looking more into 3-d compositing in jitter and instead of importing plane made in maya or using a videoplane, i was looking at 3-d objects, and more precisely, grid shapes made in jitter using jit.gl.gridshape object tool. here is a list of 3-d models available.
the one i particularly am interested is a plane and possibility to overlay it with a texture. i quickly run through all of the shapes and check their parameters and transparency options. what is good about grid-shapes, they are !transparent objects with overlaying grid. i was quite happy to see anything what had some sort of transparency in 3-d space. here are some screenshots of gridshapes.
i managed to incorporate a gridshape object in same plane with my 3-d cubes. i even managed to overlay a gridshape with a texture, which i was trying to chromakey. i didn’t get my desired outcome. here is a patch of gridshape overlayed with a video texture::
the good thing is that i can see cubes in whole rotating in 3-d space and grid allows the see through, the bad thing is that my texture is almost invisible. it is slightly possible to track some strange light movement over the grid, but nobody even realizes this is my texture and it doesn’t help to achieve desired outcome.
after so many trials i started to look into forums and support files on internet. there is not much literature published about this software but there is a great deal of information available on max/MSP official website. there are amazing jitter recipes book which has many tutorial files with explanation and patches available to download and ready to test as well as forum. i managed to trace some posts with relevant subject i was working on. here is an extract i found on max/MSP Jitter webpage forum, and i managed to find one post written by person facing similar problem to mine::
[extracted taken from: http://www.cycling74.com/forums/topic.php?id=825 ]
very sorry to bother u again about something that must be quite simple…but i have done a test exporting a chromakeyed movie from jit.qt.record using the “animation” codec, and the alpha channel is black. is there a flag that needs to be set to enable alpha channel transparency? patch setup:jit.qt.chromakey –> jit.qt.record “write TEST.mov 15. animation max 600″
unfortunately nobody answered his question and i was left on my own findings how to solve my problem. one thing which i learned is that it is not quite possible to perform chromakeying on 3-d objects, even though it is possible to chromakey texture which will be overlaying a plane in 3-d space, i will not achieve transparency i am after. chromakeying works for two video sources, simple 2-d planes, a matrix which consists of pixels and max can deal with them very easily. max has done chromakeying in all cases without hassle, the problem i am facing is my false interpretation of chromakeying necessity. to achieve transparency in 3-d space i will need to look into alpha channel adjustments, alpha channel is a 4th channel which stores data about pixel transparency. in chromakeying process max/MSP never gets rid of pixels or makes them transparent, each chromakeyed pixel is always replaced with other pixel and will never have a faculty of being transparent. i was thinking wrong and have learnt my lesson. i have understood more processes in max/MSP and in order to achieve my final goal in presentation i will need to look in more depth and try another techniques.