With the PS3 being announced for spring 2006, you might think the PS2 is at the point where it's reached its limit. At this time, the world's attention tends to go to the next generation machine, but this is actually the time where all the technology achieved over the lifetime of the current machine bears fruit, and is the age where the masterpiece games appear.
You could probably say that this winter, where SOTC (Shadow Of The Colossus) appeared, represents such a maturing period for PS2.
Shadow Of The Colossus is a good game in itself, but it is the technology which is operational within the machine, PS2-wise, that gives it the true "next-gen impression".
So now, we have brought together the information from the development of Shadow Of The Colossus, because we had heard discussion on the technology we used and wished to present it.
The HDR rendering and dynamic exposure / tone mapping algorithmEdit
After starting SOTC, the first impressive visuals you get are probably the view of the magnificent scenery seen outside from inside the sanctuary. It floods the external scenery which is seen from the interior to white, and light overflows through the opening of the columns which support the sanctuary. This is the trendy high dynamic range rendering (HDR - High Dynamic Range).
When you emerge from the dark sanctuary interior to the outside, the outside scene is drawn almost completely white, which settles shortly to its proper brightness balance. This is called the dynamic tone mapping, which is part of the HDR effect.The effect in pseudo-HDR rendering is seen everywhere. It means that brightness balance has been centred around the protagonist under the cliff, and so the relatively bright sky tends to become almost white.
HDR rendering and tone mapping is explained in the article about Half-Life 2: Lost Coast, but we will give a brief overview of the concept here.
In order to render a HDR image without being restricted by the possible color range of the television display, the HDR rendering indicates a process where tone mapping is used to get the proper brightness balance for the TV display.
Originally, 3D graphics performed on a computer, is something designed to make an image which is close to the actual world. There is a huge variation in brightness in the actual world, from intense brightness down to gloomy darkness, but the human eye and the camera cope by adjusting the exposure, shutter speed of the camera, and the opening of the pupil, etc. Regarding 3D graphics, we process the light intensity of the actual world using a similar large range with high accuracy (HDR rendering), and the simulation of the camera and the eye will happen at a later stage (tone mapping).
It is difficult to perform true HDR rendering for PC GPUs before the DirectX 8 generation which most current consoles use, as these render only to 16 million-color framebuffers containing 8-bit RGB channels. (Technically it is not impossible but it would be too slow to be of any practical use). Because of this, the pseudo-HDR rendering has become popular in game graphics.
With Xbox 360 and PS3, it is possible for each RGB channel to be 16-bit floating point (FP16), as the hardware was designed from scratch to be able to cope with HDR rendering. But there is no such potential for that on PS2. When you think of the graphics of Shadow Of The Colossus, pseudo-HDR rendering springs to mind.
Speaking of pseudo-HDR rendering, the glow effect where light overflows onto the surrounding areas during the post-processing (the bloom effect) is traditionally a general-purpose effect. Our implementation introduces exposure and simulation of the pupil with the proper brightness, which is not usually done. Furthermore, with SOTC, this is done according to the scene where the player is.
Behind the scenes of pseudo-HDR renderingEdit
|"To tell the truth, it uses the same basic principles as our previous title 'ICO'. When looking at it in a certain way, it would be fair to say it is a development of that."- Takuya Seki, SCE lead production department|
With Shadow Of The Colossus, the scene defines boxes (trigger boxes) everywhere on the map, (for this article I've decided to call them scene boxes for convenience), which when the player is inside, it dictates the settings for the effect from that area.
In other words, if you think back to the sanctuary example, "because the inside of the sanctuary is always dark, when you looked at the bright outside from there, it appears saturated", this is because the scene definition instructs it to "do just that".
|"You do the rendering itself normally. Afterwards, depending on the scene effect that has been defined, it just applies the effect where light starts overflowing in the outside scene."- Takuya Seki, SCE lead production department|
|Rendering a single frame in Shadow of the Colossus|
Occasionally, when you would move from a bright place to a dark place (and vice versa), it seems that the effect gradually moves to its proper brightness. How did we do this kind of dynamic tone mapping?
When you move from inside the sanctuary to the outside, because you have left the scene box in the sanctuary, the effect there stops being applied. This is done as a binary (on/off) decision, but if the scene were to change suddenly, it would look visibly unnatural. So, when the player leaves the box, there is a short interval where it gradually changes into the effect of the next scene box.
I perform the fake tone-mapping effect by changing smoothly from "Inside the sanctuary, outside scene is drawn bright" ==> "Outside the sanctuary, outside scene is drawn normally".
|"The downside to this method of scene boxes, which causes the PS2 to suffer, is that the time taken on the production side to set the boxes up is very high. SOTC has a very large world to manually set all this in."|
- Hajime Sugiyama, SCE lead production department
Motion Blur - with afterglow effect, and frame rateEdit
With SOTC, when the camera moves, the "motion blur effect" occurs.
The motion blur - there are various techniques for achieving this, but with SOTC, we adjust the current frame after rendering is done, by the velocity of the camera, and then simply accumulate these images. In the PS2, where there is little memory, and it is slow to access, this is a fairly reasonable technique.
Furthermore, this camera blur is not applied with respect to the character. If this camera blur was put on the character, the character would become too blurred and vague, which is not a good thing as it hinders gameplay.
But, when you look carefully, in SOTC when the player character runs etc., you can see (in the pictures below) that blur occurs regarding action. This camera blur occurs due to different processing reasons.
This is done by finding the difference between the bone positions in this frame and the previous frame, and forming a polygon from it. It applies the character image of the current frame as a texture for it.
On the occasion where the player has hung onto a colossus, and the colossus performs a large move, the view moves quickly from the sky to the land bewilderingly, but the sky brightness remains strongly visible. When the point of view is moving extremely, a similar effect happens, but this is when the human eye is looking at a bright light, and is known as the afterglow phenomenon. Because of this, the motion blur and the pseudo-HDR effect delays the brightness change, and the image remains. It seems to be a necessary hack to say "If the movement is excessive, keep the camera still". We just have to live with it. This is a unexpected by-product of step (6) in the HDR flow.
This afterglow effect is controlled by a value in every scene box. While playing you may see if you look carefully, that the afterglow does not happen in certain areas.
Well, SOTC is a game whose frame rate can increase and decrease wildly. Actually, we have incorporate the variable frame rate in the design - it increases and decreases with load balancing. Although there are cases when it reaches 60fps, there are also times when it falls to the 15fps range, but the motion blur helps to smooth over this, and the player's sensation of frame rate changes is held down to a minimum.
Generating Shadows via the self-shadowing stencil shadow volume techniqueEdit
Explanation of SOTC's shadow generation:
With shadow generation of this kind it is difficult enough to compute the hard-edged shadow (the penumbra), but we manage to generate your own shadow to fall not only on the ground, but also on your own body itself. For example, the shadow of the arm of main character is cast onto his own body. Because most PS2 games make do with casting the silhouette of the character onto the landscape, SOTC's shadows are considerably more immersive.
The shadow generation in SOTC adopts the same stencil shadow volume technique as DOOM3 and other games.
With this technique, first, we extrude the outline of the 3D object to form the contour along the direction of the light, forming a contained area (shadow volume) inside. Depth information of the scene from the camera's view is used for each pixel, to judge whether the pixel is inside or outside the shadow volume. The results are kept in a stencil buffer. Finally, using the contents of the stencil buffer, you perform the process that applies "shadow color" to the rendered image.
The quality of the shadow formed by this technique is high, but because more polygons are drawn than are actually visible, and the processing of the shadow volume formation, it becomes difficult when there is a lot to draw. Fortunately, with "Shadow Of The Colossus" the game mostly centers around the hero, the horse and the colossus when it comes to shadows. Seki tells how "It was possible to limit most of what needed processing".
By the way, even in our previous title "ICO", the stencil shadow volume technique of this kind was used, but with that the dynamic real time shadow was applied to just 2 people - ICO and the heroine.
Well, in the case of shadow volume generation, we find the outline of the 3D character from the light source and extrude this to form a shadow domain, but this processing is done on VU1 of the PS2's CPU. However, as the number of outline increases, this can take a very long amount of processing time.
In games such as DOOM3 on the PC, the model used for generating the shadow volume is almost equivalent to the character itself. But with SOTC, in order to speed this up, we made use of a simpler model with much fewer polygons in. The main character generally consists of 3,000 polygons, but the colossus can be around 18,000 polygons, depending on the type. But the model used for shadow generation will contain a substantially lower amount than this. For example, the simple model seen by the player will probably only use 1/40th of what the original model contained.
When in the game, if you look carefully, the shadow of the colossus appears to be less detailed than the actual model, but it is helped by the fact that the model and its penumbra are rarely projected in a way such they can be compared, so it looks fine. Perhaps this fine-tuning is the art of a true craftsman?
In addition, the penumbra generated from the stencil buffer is applied using a soft blur to get soft shadows. This is not physically correct as it does not take into account the projection distance of the shadow, but with the resolution of the PS2 it seems to work quite well without looking wrong.
Hair Growing On The Colossus using a fur shaderEdit
When you play Shadow Of The Colossus, it is the fur on the colossus that gives a striking impression. The 'fur shader' buzzword, when the programmable shader 1.x generation of GPUs appeared, caught on and even Xbox and PC games adopted it.
With SOTC, we implemented this without programmable shaders. During the 1.x generation, a popular technique was to construct fur using a multilayered shell system as the basis.
This rendering is done by constructing parallel layers, from the skin at which the hair grows and in fixed steps outwards. These translucent polygons accumulate using a cross-section texture of the fur, which build up to form the complete section of the hair. It is almost a kind of volume rendering effect.
Well, with this technique, the more polygons you use the more real the hair looks. But with SOTC, it tends to be limited to 3 - 6 layers due to the processing limits of the PS2.
The multilayer shell construction system does a good job of expressing the fluffy nature of fur, but when to imitate the high density of hair, you need to use a large number of layers. But, if you try to draw too many fur layers, mostly due to the amount of overdraw, this is too much to handle and starts to overload the video memory fill rate, which affects other drawing processes. Therefore in order to get a good balance between visual quality and workload, we add additional polygons of the hair drawn from the side, which we place vertically perpendicular to the skin.
The layered fur shells are easily visible at a sharp viewing angle, and the extra hair fins are equally visible at the opposite angle.
But by combining these two techniques, we create a very natural feeling of hair for the colossus.
When we implemented this layered fur on a GPU from the 1.x generation, we used a normal map to convert the direction into a texture lookup, giving a very realistic per-pixel lighting method. In addition, something that changes the shade drastically is anisotropic lighting (the relationship between the viewing angle and position of the light for each pixel), which gives the fur its characteristic highlight. Of course, this was not possible on the PS2.
However, with SOTC, we do the fur's lighting on the vector unit, to achieve the same results.
In addition, when the colossus moves, there are times when the skin goes into a strange shape and each layer of the fur deforms to match it. Therefore, correct simulation of the fur is not always done when the colossus moves, in order for it to not look wrong.
In any case, with PS2, the fact that the fur-effect is performed in real-time deserves special praise. We are pleased to draw attention to it as it is one of the key highlights of Shadow Of The Colossus' graphics.
LOD System and Landscape RenderingEdit
In SOTC, there are several colossuses, but in order to visit each one you have to travel around the immense landscape with your trusty steed.
It is during this time that we use the landscape rendering system to draw far away places, without cheating using distance fogging.
Landscape rendering is divided into 3 stages, the furthest background being either a rendered image or a texture, which is stuck on a distant polygon. The internal development team nicknamed this "Super Low".
Consequently, most of the medium range which goes from nearby to just past the furthest background, is rendered using a low-resolution landscape. This is managed in units of 600x600m, and as the player approaches it, once he gets nearer than a certain fixed distance, it changes to the high-res landscape model for the foreground. This switch is designed to blend well so it not obvious.
In addition before being combined down into one picture for the Super Low, most of the distant scenery being managed in 600x600m chunks has been reduced to around 1/30th - 1/100th of the polygons. Because there is little memory, we usually try and spread it around the landmark which represents that area. The combined image for the most distance scenery is done using this low-res model. With this, the low-resolution model appears seamlessly out of the Super Low image. By connecting the foreground and distant background well, we are able to produce a landscape rendering system which seems very natural.
Furthermore, the foreground landscape centered around the player is done with a high-res model which is managed in units of 100x100m.
The colossus is staggeringly large, but we did not use a LOD system based on view distance like the landscape because there was no memory to spare. We use the high accuracy model even when it is far away.
By the way, you wander about this immense land but do you notice that there are no loading time? We think that many players haven't noticed this, because you move around so smoothly. While SOTC uses a conventional resource management system, as you wander about the landscape, in fact, we perform dynamic background reading.
When you enter into a new area, the landscape data and textures etc. of the new area are read, the data which is no longer needed is thrown out. As this repeats, the memory eventually becomes fragmented into used and unused sections. Memory efficiency becomes very bad then.
With SOTC, during the vertical retrace period, and there is time to spare, we perform rearrangement processing of fragmented memory areas in the background. The program obtains information about the relocation addresses, so it can maintain the original setup. This isn't a visible technology like graphics, but because high-level technology is utilized even in such an area, the total quality of the game is heightened.
Collision With The Colossus, and the "deforming collision system"Edit
The "deforming collision system" buzzword was mentioned on an official website, and attracted huge attention. Here we explain it.
When it comes to collision, it often seems that the huge boss character found in a conventional game seems to be kind of "plain" in relation to the player and his actions.
For example, if you are crushed by that gigantic figure, then you just die..... or when you shoot, it just takes away damage.... and that's it.
|"With Shadow Of The Colossus, because we realized that the interaction would be very 'close', allowing the player character to hang onto the gigantic figure, a system of management and control was needed which was very unconventional. This is called the 'deforming collision system'. "- Hajime Sugiyama, SCE lead production department|
With 3D games, the collision shape is often separate from the obvious 3D graphics.
For example, a 3D game in which the character can climb up rock faces is now common, but this can need precise collision detection where the collision shape matches the rock model exactly.
On the one hand, if the game is a 3D shooter where the player shoots enemy characters, the collision decision is taken using a shape that roughly fits the shape of the enemy. It usually has a cube or sphere shape for each bone in the 3D model for the collision detection, to match the shape of the character, and that's it.
In SOTC, the colossus model is to some extent what could be called the "landscape", but when this model swings an arm around, or twirls his whole body around, using an inflexible collision model is not a great idea. If we were to use a system of cubes and spheres, it would be very apparent when you got up close to the colossus, which is not good.
So with SOTC, kind of like the example of the 3D shooter game, we use a 3D collision model which is close to the shape of the colossus. When the colossus changes its pose, the bone inside it moves, but the collision model is programmed to follow the movement of the bone - it becomes "deformed". This is where the "deforming collision" name comes from. Furthermore, the collision model is at no point actually drawn on-screen, but you can imagine it using the same triangles as for the displayed model.
Because it is difficult to perform complex collision detection on the vector unit, we simplify this in order to operate in real-time on the PS2.
For collision between the player and the colossus, we consider the whole player as a sphere-shaped collision model, and perform a point-to-polygon test against the colossus' collision model. And when colliding the colossus against the scenery, this time we use a sphere for the whole colossus and perform a point-to-polygon test against the scenery's collision model. We use the appropriate standpoint depending on "who vs. who" is involved.