Month: December 2022

This week, after getting the rough cut footage, I started on the special effects.
The first thing I did was export the video file as an image sequence so that I could parse each frame better. After that, I sorted each clip into categories and had a total of 30 shots to produce.
For the green screen keying, I mainly used the Red Gaint plug-in in AE, first using the colour grading tool to make the green more visible, then using the plug-in to subtract the green, and then using Key Cleaner to clean up the green edges so that I had a good character keying.
After that, I imported one of the frames into UE5 and used the base character model that came with it, put it into the scene and did the shot matching.
Finally, the video was exported and then imported into ae for compositing.

In addition, after completing the keying compositing of 30 shots, I also created 5 special effects shots, one where daiyu walks into the Taixu Fantasy Realm with a time portal, one where baoyu returns to the time portal in the Taixu Fantasy Realm, one where daiyu disappears from the Taixu Fantasy Realm to reach the snow scene, and the last one where baoyu disappears from the wedding scene to meet daiyu in the snow scene.

This week I’m shooting with the crew in a green screen studio, I’m the VFX director and I’m responsible for directing the VFX side of the shoot on set. I was in charge of directing the special effects on set, preventing any mishaps, making sure the characters were wrapped in green screen, etc.
Here are some pictures from the shoot.

This week I worked on building the fourth scene. According to the story version, the fourth scene is a snowy scene, because in the original Dream of the Red Chamber the first time baoyu and daiyu meet is in the snow.
Firstly, as with the previous scenes, I found some reference drawings.

As with the previous lake scene, I made this scene directly in UE5 because it is a natural landscape in the snow, which is very easy and fast to make in UE5. The snow looked too clean, so I added some footprints, which were done with UE5’s decal material.

In addition, the starry sky background is a copy of the texture from scene 1, which was rendered to a cooler tone using the colour grading tool. Again, to give the scene some movement, I added snowflakes using the particle system and added some height fog to give the scene a sense of atmosphere.

This week I worked on the third scene build. According to the storyboard, the third scene is an outdoor scene of an island on a lake. The main idea is to show the good times of baoyu and daiyu. So the scene should be quite beautiful and warm.
Firstly, as with the previous scenes, I found some reference drawings.

I made this scene directly in UE5 because it is more of a natural landscape. UE5 is very easy and fast to make natural landscapes, so I first found some tree and grass assets and made the terrain in worldcreator, and also made the mapping in Substance Painter.

Moreover, the lake I copied directly from the water surface of scene 1, including the sky as well, which I tinted to warm tones using the built-in colour grading tool. I also used UE5’s particle system to create a falling leaf effect to add a sense of movement to the scene.
The final scene turned out as follows.

This week I worked on the second scene build. According to the storyboard, the second scene is an indoor wedding scene.
Firstly, as with the previous scene, I found some reference drawings.

Next, I followed the reference drawings and built the scene in Maya. Firstly, I built the big frame as it is an indoor scene, and I designed the main body as a t-table. After that, I found some elements of a traditional Chinese wedding scene, such as chairs, tables, floor lamps and lights. Next, I imported the finished model into Substance painter for mapping, after which I integrated the model and mapping into UE5

The first time was when I felt that the overall lighting was too dark for a festive wedding, and the solution was to add two extra spotlights to illuminate the catwalk. The second time was that I felt that the lower stage of the t-stage was very empty and did not have a festive atmosphere, so I added two pieces of red gauze, which made the scene more festive.

This week I worked on the first scene build. According to the story version, the first scene is the heroine walking into a make-believe world – the Realm of the Too Void.
First, I looked for some reference drawings. My idea was to combine classical Chinese architecture with some cyberpunk elements.

Here are the references I found.

Next, I followed the reference drawings and found some existing assets on the internet that could help me build the scene more quickly. Next, I imported the assets into Maya and built them in Maya, after which I found that some of the assets had no mapping or the mapping felt different from what I wanted the scene to be, so I took the buildings, split the UV and imported them into substance painter to draw the mapping. This part took me a lot of time because some of the models he was merging together and some of them, like the tiled windows, had a large number of faces.

After doing the assets in Maya, I imported the models into UE5 for lighting and rendering, the advantage of UE5 is that I can see the results in real-time and can import millions of faces without getting stuck.

I then made a fluctuating water surface in UE5, using two dynamic water ripple maps and displacing nodes to create the effect of a plane with a water surface.

Finally, I added some special effects lighting to give the scene a cyberpunk feel and added height fog to add some mystery based on the reference.

This week I drew up a storyboard based on the outline of the story we discussed earlier.
For the storyboard, I started by finding some matching screenshots of the film and created them through a ps to the hand-drawn situation. Also, some were made in 3D software as a simple model and then assumed a camera. In this way, I could make the storyboards quickly and beautifully.
Here is the final version of the storyboard.