Confetti (www.conffx.com), a provider of cutting edge 3D middleware and a Tools & Middleware provider for Xbox One, the all-in-one games and entertainment system from Microsoft, announces today that game developers for Xbox One will be able to access the full power of Aura (Dynamic Global Illumination System), Ephemeris (Dynamic Skydome System) and PixelPuzzle (Post-Processing Pipeline) to express all aspects of their creativity using more advanced technologies for their game.

Confetti is a think-tank for advanced real-time graphics research for the video game and movie industry. We are game developers with 10+ years of experience each.
We offer three industry leading middleware packages for licensing.

Aura -our Global Illumination system- is a fully dynamic Global Illumination system. It can simulate one or several bounces.

-          Aura doesn’t require pre-processing data, so artists work in real-time

-          Aura allows to destroy geometry randomly, so there are no limits to destructibility

-          Aura affects characters as well as the environment

-          Aura works with as many light sources as you want from directional, point or spot lights or any other light type

-          Very low memory footprint

Ephemeris -our Dynamic Skydome System- offers you a fully dynamic solution that is drag and drop. With sunrises, sunsets, blue sky, and a shimmering moon all in one package, you can insure that your game can support a 24 hour cycle without switching between different techniques.

It supports

-          Dynamic 24 hour time cycle

-          Physically based rendering model with multiple scatterings of the sun

-          Simulation of astronomically correct sun, stars, and moon

-          God rays

-          Aerial perspective

-          Dynamically lit volumetric clouds

PixelPuzzle – our Post-Processing Pipeline- is build as an extension to a regular in-game camera system. It allows you to create many different camera types with ,for example, different motion blur, depth of field, color filters etc. and then pick the camera you want on the fly. So you can switch between very different camera models in a game instantly and therefore enhance the way a story can be told.

PixelPuzzle also comes with various tonemappers to fit your game along with different color filters to enhance your scene. On top of that the system is easy to integrate to ensure you spend more time on your game.

More features are

-          Simulates camera controls for ease of use

-          Shaped Bokeh / Depth of field

-          Motion blur

-          HDR tonemapping

-          Bloom

-          Scotopic vision

-          Color filters

-          Camera Noise Simulation

-          Vignetting

-          Gamma correction

-          Easy to integrate

Services – we also offer real-time graphics related consulting and programming services. We have a long list of companies from the hardware and software industry that we worked with on cutting-edge graphics.
We offer game related consulting and programming services including optimizations and re-writing of whole game engines, if necessary. We worked in the past in any field related to GPUs for hardware vendors, game developers and software vendors. It will be hard to find a material or effect that we haven’t implemented into a game in the last four years. Our target platforms go from mobile games to latest console / PC platforms.

If you have a challenge that is hard to solve or requires in-depth knowledge, talk to us. We would love to spend time with you and analyze this challenge.

http://www.slideshare.net/WolfgangEngel/hair-intombraider-final

http://www.fmx.de/program.html#!/list?t=91

Blackfoot Blade at GDC

Gamasutra

Confetti is a think-tank for advanced real-time graphics research for the video game and movie industry. We are game developers with 10+ years of experience each.
We offer three industry leading middleware packages for licensing.

Aura -our Global Illumination system- is a fully dynamic Global Illumination system. It can simulate one or several bounces.
- Aura doesn’t require pre-processing data, so artists work in real-time
- Aura allows to destroy geometry randomly, so there are no limits to destructibility
- Aura affects characters as well as the environment
- Aura works with as many light sources as you want from directional, point or spot lights or any other light type
- Very low memory footprint

Ephemeris -our Dynamic Skydome System- offers you a fully dynamic solution that is drag and drop. With sunrises, sunsets, blue sky, and a shimmering moon all in one package, you can insure that your game can support a 24 hour cycle without switching between different techniques.
It supports
- Dynamic 24 hour time cycle
- Physically based rendering model with multiple scatterings of the sun
- Simulation of astronomically correct sun, stars, and moon
- God rays
- Aerial perspective
- Dynamically lit volumetric clouds

PixelPuzzle – our Post-Processing Pipeline- is build as an extension to a regular in-game camera system. It allows you to create many different camera types with ,for example, different motion blur, depth of field, color filters etc. and then pick the camera you want on the fly. So you can switch between very different camera models in a game instantly and therefore enhance the way a story can be told.
PixelPuzzle also comes with various tonemappers to fit your game along with different color filters to enhance your scene. On top of that the system is easy to integrate to ensure you spend more time on your game.
More features are
- Simulates camera controls for ease of use
- Shaped Bokeh / Depth of field
- Motion blur
- HDR tonemapping
- Bloom
- Scotopic vision
- Color filters
- Camera Noise Simulation
- Vignetting
- Gamma correction
- Easy to integrate

Services – we also offer real-time graphics related consulting and programming services. We have a long list of companies from the hardware and software industry that we worked with on cutting-edge graphics.
We offer game related consulting and programming services including optimizations and re-writing of whole game engines, if necessary. We worked in the past in any field related to GPUs for hardware vendors, game developers and software vendors. It will be hard to find a material or effect that we haven’t implemented into a game in the last four years. Our target platforms go from mobile games to latest console / PC platforms.
If you have a challenge that is hard to solve or requires in-depth knowledge, talk to us. We would love to spend time with you and analyze this challenge.
“PlayStation” is a registered trademark of Sony Computer Entertainment Inc.

http://blog.us.playstation.com/2013/02/20/playstation-meeting-2013-the-future-of-gaming-is-here-with-playstation-4/

Implementing Touch and Sensors

Confetti Celebrates Ultrabook™ Device Release of Blackfoot Blade*

Blackfoot Blade brings back the classic 3D helicopter shooter genre. You play through action packed 2 – 3 minute levels fighting against enemy helicopters and tanks.

Single-Player Missions

You –the commander of the most advanced helicopter ever developed- zip through canyons, deserts, forests, mountains, Caribbean islands, and snowy tundras to face the invading enemies. Your helicopter is equipped with heat-seeking missiles and a machine gun that can blast through the reinforced steel of tanks.

Fight with a Helicopter

Multi-Player Missions

In Blackfoot Blade you can already fly in one-versus-one multiplayer missions in case your machine has CCF support, which is a close proximity networking system that is still in beta.

Getting a sunburn

We demoed a multiplayer game at the Intel Developer Conference on September 12^th. You can see this demo, including two of us playing the prototype on stage by visiting the following link and skipping to 22 minutes in.

http://intelstudios.edgesuite.net/idf/2012/sf/ti/SPCS007/index.htm#

Dissolving in Fire

Hardware requirements:

This game is built for running on an Ivy Bridge processor. It should also work well with most DirectX 11 GPUs while running Windows 7 or 8 (not RT). We specifically developed on:
- Windows 8
- Intel Core i7-3427U Processor (3MB Cache, 1.8 GHz)
- HD Graphics 4000
- Resolution 1600×900
- Memory: 8 GB

Dessert Storm

The game uses touch-screen and tilt sensor support if available.

http://software.intel.com/en-us/articles/microsoft-directcompute-on-the-2nd-generation-intel-core-processor/

Let us know what you think.

http://software.intel.com/en-us/videos/channel/visual-computing/game-developers-discuss-ivy-bridge-directx11-support-at-gdc-2012/1622190917001

Many Area Lights / Screen-Space Soft ShadowsMany Area Lights / Screen-Space Soft Shadows

http://software.intel.com/en-us/articles/microsoft-directcompute-on-intel-ivy-bridge-processor-graphics/

Let us know what you think.

• 1. RawK III that shows our latest version of our Global Illumination middleware, PostFX pipeline and Particle system
• 2. SpeedTree – we teamed up with SpeedTree and integrated our Skydome middleware into their demo
• 3. Sponza that demos our Global Illumination middleware in the typical Sponza scene

We updated our website with new movies and screenshots in preparation for GDC.

Confetti helps to organize and sponsors an online conference at

http://altdevconf.org/

It is free and you can sign up for our talk -that will be given on Sunday 2pm- at

https://www4.gotomeeting.com/register/846958367

GPU Pro 3 should be available at GDC.

http://www.kgconf.com/eng/html/conference_view.html?idx=937

Screen-Space: Rules for Designing Graphics Sub-systems (Part I)

No Look-up Tables: Rules for Designing Graphics Sub-systems (Part II)

The third article “Even Error Distribution (Part III)” will be released soon. Additionally there is an article that is a primer on Points, Vertices and Vectors.

http://www.conffx.com/LightNShadows.zip

It is about 8.5Mb.

With only 3 months of production time, we couldn’t afford to have anything fail in the art pipeline. Most of the problems we ran into were coming from our source art; because we didn’t have a permanent artist on our team, we were sometimes pretty lost. The Mystic Game Development team was very patient and looked into all our issues and helped us to fix them. Overall the response time was impressive. Although we were in different time zones 9 hours apart, we always got very fast and immediate responses that were very well thought through and helpful. If you worked on a project with such a tight schedule, you can imagine how nervous we all have been.

Now looking back, it was a fantastic experience. Highly recommended!

http://altdevblogaday.org/2011/02/28/shadows-thoughts-on-ellipsoid-light-shadow-rendering/

The second is about my UCSD class:

http://altdevblogaday.org/2011/03/15/thoughts-on-the-knowledge-of-an-up-to-date-graphics-programmer/

and the third one is about the interview process:

http://altdevblogaday.org/2011/03/29/thoughts-on-the-interview-process-in-the-game-industry/

In essence what we were showing off was

• Deferred Lighting with lots of lights
• MLAA running on the CPU
• Ellipsoidal Light Shadow Maps
• Screen-Space Subsurface Scattering for the skin of all characters
• Depth of Field with a nice looking Bokeh

As you would expect everything in our demo was dynamic. We also covered our dynamic skydome system and larger parts of the PostFX pipeline in those slides. You can get the power-point slides from here.

http://software.intel.com/en-us/videos/confetti-special-effects-demo-at-gdc-2011/

• Deferred Lighting
• Ellipsoidal Shadow Maps
• CPU / GPU MLAA
• PostFX Camera System
• Screen-Space Material System

Here is the movie:

Some explanation about those techniques in there:

http://software.intel.com/en-us/articles/confetti-special-effects/

We will give six talks on GDC about those techniques. Expect more to come.

Those demos run on low-end graphics hardware.

Dynamic global illumination methods do not store any pre-computed data in texture maps or need any kind of pre-processing. One of the successful methods that came up over the last eight years was based on storing light data in cube maps [Mantiuk], that are located on a 3D grid. A good implementation can be found here. The first performance challenge of this technique was the need to fill up a lot of cube maps that represented the radiance for all directions. AMD’s / ATI’s implementation uses many slices of a 3D texture to store the cube map data. The 3D texture is later compressed by converting into spherical harmonics coefficients. This is done by integrating the light function stored in the cube map over the basis functions.
The conversion is done by rendering single points -representing each probe- into a volume texture. If you use four SH bands, 16 coefficients are necessary to represent a probe and therefore 4 volume render targets need to be filled.
While the scene is rendered into the framebuffer, the indirect lighting is applied by calculating the dot product between the SH coefficients from the previous pass and the SH coefficients for the transfer function to evaluate the integral.
The result of the light transfer function that uses the SH coefficients and the normal is then stored in four cube maps, and those cube maps are used when rendering.

There is a lot of memory transfers involved with this technique. Stay tuned for Part II

References
[Mantiuk] Rafal Mantiuk ,  Sumanta Pattanaik ,  Karol Myszkowski, “Cube-Map Data Structure for Interactive Global Illumination Computation”, In Proceedings of International Conference on Computer Vision and Graphics, http://www.mpi-sb.mpg.de/~mantiuk/papers/mantiuk02CubeMapDataStructureForGI.pdf

Course Objectives:
This course will cover techniques on how to implement real-time 3D graphics techniques in an efficient way on the Graphics Processing Unit (GPU).

Course Description:
This course focuses on algorithms and approaches for programming a GPU, including vertex, hull, tesselator, domain, geometry, pixel and compute shaders and CUDA. The list of topics is
- DirectX 11 API
- Deferred Lighting, Z Pre-Pass Renderer, Deferred Shading
- Programmable MSAA
- Post-Effect Pipeline
- Shadows (Cascaded Shadow Maps, Soft Shadows, Point Light Soft Shadows)
- GPU Particle System
- Global Illumination (Screen-Space, Reflective Shadow Maps)
- Order-Independent Transparency
- CUDA, DirectCompute

After an introduction into each of the algorithms, the students will learn step-by-step on how to implement those algorithms on the GPU.

In case you are currently not enrolled at UCSD, there is the UCSD Extension Concurrent Enrollment:

http://extension.ucsd.edu/about/index.cfm?vAction=concurrent

Skin rendering is still an unsolved problem for me because most solutions are much too expensive and blurring a map doesn’t cut it on the visual side for me. I liked very much the screen-space approach covered in GPU Pro in the article “Screen-Space Subsurface Scattering”. It looks very efficient and when many characters are on the screen its cost even goes down.

Anton Kaplanyan presented a range of different techniques in “CryENGINE3 Reaching the speed of Light” talk. He touched a wide range of topics that I would consider very valuable knowledge for a graphics programmer. I am sure many went through the same topics while shipping their game but the solutions presented are nicely backed-up by mathematical and observational proof.
The piecewise gamma approximation of the XBOX 360 is legendary. Changing the art pipeline to 16-bit textures is a simple but very efficient way to increase the quality of graphics substantially. Doing that for normal maps as well makes sense … you just have to convince the whole art team to do this …

I like the proposal for the normal map compression scheme. In that context I keep mentioning the fact that we still do not have a good HDR color format. What we need is a “gamma correctable” format that allows us to store a higher resolution than the DXT formats. If it is not gamma correctable it doesn’t matter because gamma compression is making a lot of difference. This perceptual compression scheme seems to be able to compress the equivalent of 17.3 bits into 8-bits. The argument that a HDR texture format that is not gamma compressable / decompressable could get close to the quality of a gamma compressed DXT format that is decompressed in the pixel shader is just not right.
Anton also describes their improvements with SSAO. They got it down to 1ms on the XBOX360.
Baking color transform values into 3D textures only makes sense in case you do not plan on changing those values afterwards. If you want to have a different color setting depending on location in the game, you will have challenges to switch between 3D textures smooth enough to make this work. The clip volumes described in the slides are quite interesting. This looks like a very good solution to one of the nagging challenges of Deferred Lighting. Anisotropic lighting is easy to implement as long as you use two geometry passes. You just run it in the second geometry pass with only the sun light or the main light source and do not apply the Deferred lights to those materials. In most cases it would look odd if for example hair would be lit by many light sources. If you do not want to do the second geometry pass (we don’t do it anymore),  the proposed solution is quite nice. You can even go further and pre-bake all the anisotropic properties into a texture. We did that in the anisotropic demo for the Qualcomm shader examples available on Qualcomm’s website. That comes down to one instruction: a multiply of rgb with a from a texture. Obviously Anton’s solution is better Another way to achieve a wider range of materials in a Deferred lighting engine is to store a material index in the G-Buffer. Because this is a dependent texture read, it will be quite slow on the XBOX 360. It looks similar to what was described in Game Programming Gems 3 in the article “Rendering with Handcrafted Shading Models”.

We are currently using MLAA for anti-aliasing in our engine. Combining Temporal Anti-Aliasing with Edge based anti-aliasing looks like a quality improvement.

http://www.facebook.com/pages/Confetti-Special-Effects-Inc/159613387880?v=wall

On a desktop machine you can go higher than this but honestly how many soft shadows do you need in a scene? The algorithm let’s you switch on and off softness per light, so you can switch off the soft part based on distance, moving speed of the light or any other criteria you can think off.
I think on a middle class PC graphics card you can have easily 128 shadows in a scene with decent quality. Some are soft and some are hard. My next effort is to find out how I can render faster into cube shadow maps or maybe use Dual-Paraboloid shadow maps.

http://bps10.idav.ucdavis.edu/

Having helped to ship games with Deferred Shading and later with Deferred Lighting gives me a good rough estimate on how those two compare with each other.

The presentation shows that the highest end graphics cards seem to max out with 1000 lights in the given scenario with the help of compute shader support. Most of my tests four years ago were done on a XBOX 360 or PS3 and later on lower end graphics cards. From what I remember in an artifical scenario 256 – 512 lights on a XBOX 360 / PS3 were possible in a similar setting that was described in the talk.
On low-end PC graphics cards like the 9600M GT of my MacBook Pro, we can run 8000 small point lights while a whole game level and colliding particles are rendered with more than 40 fps.
Comparing Deferred Shading with Deferred Lighting, I believe Deferred Lighting should be faster in all scenarios because you fetch fewer render targets and you do not resolve the lighting equation for each light.

Because the presenter used high-end NVIDIA and ATI cards I thought it would be cool to use an integrated INTEL GPU to show off Deferred Lighting and everyone could enjoy it. The drivers for those GPUs are really good now and our system only requires DX10, so we don’t use the compute shader. So I thought I give it a try on my two-and-a-half year old Lenovo X301 (http://www.notebookcheck.net/Lenovo-ThinkPad-X301.16099.0.html) with an Intel Graphics Media Accelerator (GMA) 4500MHD. This chipset is obviously not INTEL’s latest but to my surprise it ran our demo quite well. Wikipedia says this GPU has a theoretical memory bandwidth of 12.8 GB/s. The GPUs used in the SIGGRAPH presentation have a theoretical memory bandwidth of 153.6 GB/sec (ATI RADEON 5870) and 177.4 GB/sec (GeForce GTX 480) if they are standard GPUs; some vendors sell those GPUs with higher memory clock rates.
I chose to visualize 1000 small point lights without specular and the resolution of the notebook is set to 1024×768, so much smaller than what was used on SIGGRAPH. The particles, to which the point lights are attached to, also collide with the environment and bounce off. Nevertheless it was running during my tests with roughly 11 to 22 fps
Here are two screenshots and a shot of the laptop:

I think it would be cool to see more stuff running on INTEL integrated chip sets, after all it is fun to get things going on low-end GPUs … raise your hand if you’re with me

1. Calculate the Cube Shadow Map
2. Generate a coarse or dilated version from the Cube Shadow Map above by rendering into a smaller Cube Map. Each pixel of the coarse shadow map will approximate a block of pixels of the standard Cube Shadow Map.
3. Based on each coarse cube shadow map, calculate the penumbra size and blend the result into a screen-space texture.
4. Blend the shadow data of all cube maps into a screen-space shadow map.
5. Apply an anistropic Gaussian filter kernel to the screen-space shadow map while adjusting the kernel size based on the data in the penumbra size stored in the screen-space texture above.

Coarse Cube Shadow Map

Fernando [Fernando 2005] came up with the idea of doing a blocker search to find the average depth of the blockers. In this approach the blocker search is replaced by generating a minimum z map (min-z map) [Gumbau et al. 2010]. The min-z map approximates the distance between the light and the blockers.

Calculating the Penumbra Size

Fernando’s [Fernando 2005]  showed that assuming that the light source, blocker and receiver can be treated as parallel, the similar triangle approach can be used to estimate the penumbra size as shown in the following equation:

Following Gumbau [Gumbau et al. 2010] this work extends this idea by adding an additional parameter to this equation:

d_observer describes the distance to the observer and will be used to scale the screen-space filter kernel. d_blocker represents the content of the coarse screen-space shadow data.

Anisotropic Screen-Space Gauss Filter Kernel

Using a separable Gaussian Filter kernel allows to filter the shadow data with less texture fetches compared to the commonly used PCF based filter method. Compared to PCF filtering the Gaussian filter kernel performs with O(n+n) instead of O(n²). To determine the shape and the orientation of the filter kernel, the normal of the current pixel is fetched from the normal buffer and projected into eye space. To perform the anisotropic filtering in an efficient way, Geusebroek’s [Geusebroek et al. 2003] approach is applied.

Data Layout

This new algorithm is compliant with the concept of Deferred Lighting. The shadow and the coarse min-z data is stored in regular cube depth render targets. While the G-Buffer holds color, normal and depth data of the scene, there is a light buffer for the light data, a main back buffer and a separate –at least- 16-bit floating point buffer that holds the penumbra size values in screen-space. The main back buffer and the buffer that holds the screen-space penumbra size values covered in 3. can be filled up together if treated as a Multiple-Render Target (MRT).

In that case the first three channels of the main buffer are filled with the lit but un-shadowed scene while at the same time the screen-space shadow data mentioned in 4. can occupy the fourth channel of this render target.

More info at the Korean Game Developer Conference and in an article in GPU Pro 2 …

References

Randy Fernando, “Percentage-Closer Soft Shadows”, SIGGRAPH 2005 Sketch

Jan-Mark Geusebroek, Arnold W. M. Smeulders, J. van de Weijer, “Fast anisotropic Gauss filtering”, IEEE Transactions on Image Processing, Volume 12 (8), page 938-943, 2003

Jesus Gumbau, Miguel Chover and Mateu Sbert, “Screen Space Soft Shadows”, to appear in GPU Pro, AK Peters, 2010

You can find more screenshots of the mobile work we do at the
Confetti Special Effects fan page. More info is available soon on Qualcomm’s developer website or in the Paris Master Class

I hope to see some of you there.

Oolong Engine

Confetti Special Effects posted a couple of screenshots of several projects we are working on. So far we are showing of particle collisions, a next-gen dynamic skydome system with god rays, terrain self-shadowing, volumetric clouds and atmospheric scattering that is correct even when watched from space and a fully-dynamic global illumination system. Please check out our Facebook page:

Confetti Special Effects

In other news there are cool slides of a SPU-based implementation of the Light Pre-Pass renderer here:

A Bizarre Way to do Real-Time Lighting

There is a GPU Pro article that describes many of the implementation details. BTW: GPU Pro is on its way to the book sellers and looks fantastic. Color makes such a big difference for a book like this.

Kaori Kubota send me a link to an example implementation of Order-Independent Transparency where he changed the DirectX OIT example to use AMD’s linked list implementation. Framerate went on my machine from 10 to more than 1400 fps. He also wrote a documentation:

OIT

How to Create a Light Frustum
After splitting up the view frustum into slices constructing a tight light view frustum can be a challenge. There are several ways to do this.
Just using the eight points of the slice of the view frustum and constructing the light view frustum around this works well but what happens is that the quality of the shadow changes depending on how the camera is facing the sun. That gave sometimes very good results and sometimes bad results. Having the same error level distributed over the time of day in a game is an important requirement. So I build a sphere around the frustum slice that intersects the eight points of this slice. Then around this sphere the orthographic light view frustum is build. This follows Michael Valient’s article in ShaderX6. The orthographic light view frustum rotates around that sphere similar to a joint. The quality is evenly distributed independent of the angle of the camera and the light source. You can even use this in a flight simulator and fly loops and it will work appropriately.

Shadow Collector / Mask / Deferred Shadowing
After rendering into the shadow maps one intermediate step before the scene will be rendered can be to store all the shadow data in a screen-space texture. This can happen along with Z-buffer and / or Normal buffer rendering.
The main reason to do this is that you can’t fetch the texture data easily while rendering the main scene. Let’s say we have four cascades and therefore four shadow maps each 640×640. Fetching four maps of this size along with all the other maps that are necessary while rendering the scene will degrade performance substantially. So this needs to be done before the scene is rendered.
Having this screen-space texture also offers a good way to blur the data again. The reason why I called this screen-space texture shadow collector is that it can be used to collect all the shadows of the scene. There might be cloud shadows (just projected down), character shadows (for high-res self-shadowing), point and spot light shadows and all kind of special cases. All this data can be alpha blended into the shadow collector and then applied to the scene by fetching this texture. Obviously there might be other ways to store the data when you are using a Deferred Lighting approach.

How to Pick the Right Texture map
When you render into the shadow collector you need to have a good way to find out which texture map to pick. Michal Valient’s article has a great solution for this. I just do a sphere-pixel distance check for all four maps in the pixel shader. This is the sphere that you used to construct the light view frustum around. What I like about this is that the break between different maps is in circle form and not a line. This is easier to hide. There are probably smarter ways to do this.

How to Store the Shadow Map data
Storing the intial shadow data is best done in a texture atlas. There are numerous challenges on how to fetch data from a texture atlas that were described in a ShaderX3 article by Matthias Wloka. Nevertheless it is the best representative aside from a texture array that is only available on more recent hardware.
Jonathan Blow has a blog entry on how to store this data in the most efficient way.

There is an interesting discussion following his blog entry. Some people argue that the usage of a texture atlas is not necessary as long as you render the data immediately into the shadow collector. This way you render four times. On most platforms I worked on this is not efficient. You only want to render all the data coming from the Cascaded Shadow Maps once.
In other words Jonathan’s idea is really interesting. One person mentions the following link in this context .

Packing Square Tiles into One Texture

How to soften the Penumbra
On most hardware platforms you can’t run any filter kernel bigger than 4 taps. There are tricks to detect the areas where you can run bigger filter kernels but most of them didn’t work out well for me.
The amount of research that goes into this area is astonishing. It astonishes even more that nearly all solutions don’t work in any real-world game environment with a 24 hour time of day cycle.
The one solution I found working is Exponential Shadow maps. They only require a one channel depth buffer and you can render with the double-speed writes if you use the native depth buffer format. Unfortunately they suffer under light bleeding artifacts.
I used therefore a four-tap screen-space filter kernel that rotates and dithers the data and that worked well.

Future Development
There is hope that we can use eight-tap filter kernels on most platforms soon … but what will offer new opportunities is the ability of DX10 to render out to several maps in one render pass. Because Cascaded Shadow Maps is a shadow level-of-detail system that distributes shadow data quite evenly along the view frustum, I expect people to come up with better LOD schemes, in other words, more frustra or in other words multi-frustum shadow maps as already described by Tom Forsyth in 2004 are the future. ShaderX had interesting articles in this area that were called Virtual Shadow Maps.

Per-Pixel linked lists represent one pixel in the viewport / framebuffer with a list of structures consisting of the actual data and a link that links this list entry to the next one.  This list of elements is stored in a read/write structured buffer. A structured buffer is a buffer that contains elements of equal sizes. Use a structure with one or more member types to define an element. The buffer in the power-point slides is called “Fragment and Link Buffer”:

struct  FragmentAndLinkBuffer_STRUCT
{
  FramentData_STRUCT FragmentData;
  uint uNext;
}
RWStructuredBuffer<FragmentAndLinkBuffer_STRUCT>FLBuffer;

In addition to indexing, a structured buffer supports accessing a single member like this:

float4 myColor = FLBuffer[27].Color; // cache challenges here?

This buffer can now hold any number of list entries that represent one pixel. Each list entry refers to the previous list entry in the variable uNext. To make this work, a so called “Start Offset Buffer” is used. This buffer stores the offset into the “Fragment and Link Buffer” representing a pixel in the framebuffer. In case one pixel is “overwritten” by another pixel, this offset value is read from the “Start Offset Buffer”, written into the uNext variable of the new list entry and the offset value of the new list entry is then written into the “Start Offset Buffer” and so on. The slides show the following images (there is a nice animation in the slide show that is worth checking out):

The viewport is represented by the 2D grid on the left. For each pixel a value is written into the “Start Offset Buffer”. For the first pixel it is 0, then 1 and so on. This value represents the offset into the “Fragment and Link Buffer”. The first image on the left shows how the first value is written into the “Start Offset Buffer” and how this value represents the offset into the “Fragment and Link Buffer”. The second image shows how two more pixels are stored in one entry of a linked list. The third image on the right side shows how another list entry is added to the first entry in the “Fragment and Link Buffer”. The fourth entries uNext variable holds now 0 as the value.
Traversing the linked list is easy. Just following the offset value in the “Start Offset Buffer” of the pixel that needs to be rendered and then following the uNext entry leads to the fourth and then the first entry in the “Fragment and Link Buffer” as shown in the fourth image.

To implement Order-Independent Transparency with this technique, only transparent pixels need to be stored in a Per-Pixel Linked List. In the rendering phase, those list entries are stored in a back-to-front order and blended in a pixel shader. The blend mode can then be unique per-pixel.

It astonished me how much knowledge the people in the auditorium had. They asked me the right questions and it was fun just going on tangents on several topics not related to GI.

The students also showed me demos they did and those were impressive. They actually have to write their “engines” from scratch in C/C++ (no middleware allowed, e.g. no sound or physics middleware).

http://www.facebook.com/pages/Confetti-Special-Effects-Inc/159613387880?v=wall

We will also have a better website soon … very soon

With the DirectX 11 implementation it might also reduce the workload of the vertex shader because the shader transforms or animates only the coarse mesh. But if we add up the additional workload of the hull and domain shader it might be a wash.

For console developers, being able to store more world geometry on disc and in memory would be a great advantage. The reduction of the read memory bandwidth -while reading the data from memory- would also increase the efficiency.
The main question is if tessellating the geometry puts such a huge workload on the GPU that it is not feasible. I would love to have some real-world data here …

I would consider those now beta.

Please note that this overview has nothing to do with the way the hardware works. It is just a diagram that shows the data flow and the usage of the Direct3D 10 API to stream the data through several logical stages that might be represented in hardware by one unit. If you are interested in the actual hardware design, I would recommend reading

A Closer Look at GPUs

Course Objectives:
This course will cover techniques on how to implement 3D graphics
techniques in an efficient way on the Graphics Processing Unit (GPU).

Course Description:
This course focuses on algorithms and approaches for programming a
GPU, including vertex, hull, tesselator, domain, geometry, pixel and
compute shaders. After an introduction into each of the algorithms,
the students will learn step-by-step on how to implement those
algorithms on the GPU. Particular subjects may include geometry
manipulations, lighting, shadowing, real-time global illumination,
image space effects and 3D Engine design.

Example Textbook(s):
A list of reading assignments will be given out each week.

Laboratory work:
Programming assignments.

Very exciting

On DirectX9 and DirectX 10 class of hardware, one of the solutions that is mentioned to solve the problem of order-independent transparency is called Depth Peeling. It seems this techniques was first described by Abraham Mammen (“Transparency and antialiasing algorithms Implemented with the virtual pixel maps technique”, IEEE Computer Graphics and Applications, vol. 9, no. 4, pp. 43-55, July/Aug. 1989) and Paul Diefenbach (“Pipeline rendering: Interaction and realism through hardware-based multi-pass rendering”, Ph.D., University of Pennsylvania, 1996, 152 pages)(I don’t have access to those papers). A description of the implementation was given by Cass Everitt here. The idea is to extract each unique depth in a scene into layers. Those layers are then composited in depth-sorted order to produce the correct blended image.
In other words: the standard depth test gives us the nearest fragment/pixel. The next pass over the scene gives us the second nearest fragment/pixel; the pass after this pass the third nearest fragment/pixel. The passes after the first pass are rendered by using the depth buffer computed in the first pass and “peel away” depths values that are less than or equal to the values in that depth buffer. All the values that are not “peeled away” are stored in another depth buffer. Pseudo code might look like this:

const float bias 0.0000001;

// peel away pixels from previous layers
// use a small bias to avoid precision issues.
clip(In.pos.z – PreviousPassDepth – bias);

By using the depth values from the previous pass for the following pass, multiple layers of depth can be stored. As soon as all the depth layers are generated, for each of the layers the G-Buffer data needs to be generated. This might be the color and normal render targets. In case we want to store three layers of depth, color and normal data also need to be stored for those three depth layers.
Having a scene that has many transparent objects overlay each other, the number of layers increases substantially and therefore the memory consumption.

A more advanced depth peeling technique was named Dual Depth Peeling and described by Louis Bavoil et al. here. The main advantage of this technique is that it peels a layer from the front and a layer from the back at the same time. This way four layers can be peeled away in two geometry passes.
On hardware that doesn’t support independent blending equations in MRTs, the two layers per pass are generated by using MAX blending and writing out each component of a float2(-depth, depth) variable into a dedicated render target that is part of a MRT.

Nicolas Thibieroz describes in “Robust Order-Independent Transparency via Reverse Depth Peeling in DirectX 10″ in ShaderX6 a technique called Reverse Depth Peeling. While depth peeling extracts layers in a front-to-back order and stores them for later usage, his technique peels the layers in back-to-front order and can blend with the backbuffer immediately. There is no need to store all the layers compared to depth peeling. Especially on console platforms this is a huge advantage.
The order of operations is:

1. Determine furthest layer
2. Fill-up depth buffer texture
3. Fill-up normal and color buffer
4. Do lighting & shadowing
5. Blend in backbuffer
6. Go to 1 for the next layer

Another technique is giving up MSAA and using the samples to store up to eight layers of data. Kevin Myers et al. uses in the article “Stencil Routed A-Buffer” the stencil buffer to do sub-pixel routing of fragments. This way eight layers can be written in one pass. Because the layers are not ordered by depth they need to be sorted afterwards. The drawbacks are that the algorithm is limited to eight layers, allocates lots of memory (8xMSAA can be depending on the underlying implementation a 8x screen-size render target), requires hardware that supports 8xMSAA and the bitonic sort might be expensive. Giving up MSAA, the “light” case described above would be easily possible with this technique with satisfying performance but it won’t work on scenes where many objects are visible behind several other objects.

Another technique extends Dual Depth Peeling by attaching a sorted bucket list. The article “Efficient Depth Peeling via Bucket Sort” by Fang Liu et al. describes an adaptive scheme that requires two geometry passes to store depth value ranges in a bucket list, sorted with the help of a depth histogram. An implementation will be described in the upcoming book GPU Pro. The following image from this article shows the required passes.

The Initial Pass is similar to Dual Depth Peeling. Similar to other techniques that utilize eight render targets, 32:32:32:32 each, the technique has huge memory requirements.

To my knowledge those are the widely known techniques for order-independent transparency on DirectX 10 today. Do you know of any newer techniques suitable for DirectX 10 or DirectX 11 hardware?

For a general knowledge in programming the CPU:
- Write Great Code Volume 1: Understanding the Machine

For a better knowledge on how to program the GPU:
- DirectX documentation
- NVIDIA GPU Programming Guide
- ATI GPU Programming Guide

To learn about how to program certain effects in an efficient way:
- ShaderX – ShaderX7
- GPU Gems – GPU Gems 3
- GPU Pro and GPU Pro Blog

To start learning DirectX 10 API + Shader Programming:
- Introduction to 3D Programming with DirectX 10
- Programming Vertex, Geometry and Pixel Shaders

To start learning OpenGL & OpenGL ES:
- Khronos group

For general overview:
- Real-Time Rendering
- Fundamentals of Computer Graphics (this one also belongs in the math section)

To get started with C:
- C Programming Language

To learn C++
- C++ for Game Developers
- C++ Cookbook
- there is a long list of more advanced C++ books …

To maintain maximum precision during floating-point computations, most computations use normalized values. Keeping floating-point numbers normalized is beneficial because it maintains the maximum number of bits of precision in a computation. If several higher-order bits of the mantissa are all zero, the mantissa has that many fewer bits of precision available for computation. Therefore a floating-point computation will be more accurate if it involves only normalized values whose higher-order mantissa bit contains one.

The IEEE 754 32-bit floating-point format specifies special cases in case the bits in the exponent are all set to zeros or ones. If all exponent bits are set, then the number represents either =/- infinity or a NaN (not-a-number), depending on the mantissa value. If all exponent bits are zero, then the number is denormalized and automatically gets flushed to zero as specified in the Direct3D 10 single-precision floating-point specifications (see Nicolas Thibieroz, “Packing Arbitrary Bit Fields into 16-bit Floating-Point Render Targets in DirectX10″, ShaderX7).

When packing bit values, those cases need to be avoided.

http://graphics.cs.uiuc.edu/~kircher/publications.html

They assume a Light Pre-pass concept as covered here on this blog with three passes. The geometry pass where they fill up the buffer, the lighting pass where light properties are rendered into a light buffer and a material pass in which the whole scene is rendered again, this time re-constructing different materials.
Their approach adds several new techniques to the toolset used to do deferred lighting / Light Pre-Pass.

1. They use a much smaller G-Buffer and Light buffer with a size of 800×540 on the XBOX 360. This way their memory bandwidth usage and pixel shading cost should be greatly reduced.

2. To upscale the final light buffer, they use Discontinuity Sensitive Filtering. During the geometry pass, one 16 bit channel of the DSF buffer is filled with the linear depth of the pixel, the other 16 bit channel is filled with an ID value that semi-uniquely identifies continuous regions. The upper 8 bits are an object ID, assigned per-object (renderable instance) in the scene. Since 8 bits only allows 256 unique object IDs, scenes with more than this number of ob-jects will have some objects sharing the same ID.
The lower 8 bits of the channel contain a normal-group ID. This ID is pre-computed and assigned to each face of the mesh. Anywhere the mesh has continuous normals, the ID is also continuous. A normal is continuous across an edge if and only if the two triangles share the same normal at both vertices of the edge.
By comparing normal-group IDs the discontinuity sensitive filter can detect normal discontinuities without actually having to reconstruct and compare normals. Both the object ID and normal-group ID must exactly match the material pass polygon being rendered before the light buffer sample can be used (depth must also match withinan adjustable threshold).
During the material pass, the pixel shader computes the locations of the four light buffer texels that would normally be accessed if regular bilinear filtering would be used. These four locations are point sampled from the DSF buffer. The depth and ID values retrieved from the DSF buffer are compared against the depth and ID of the object being rendered. The results of this comparison are used to bias the usual bilinear filtering weights so as to discard samples that do not belong to the surface currently rendering. These biased weights are then used in custom bilinear filtering of the light buffer. Since the filter only uses the light buffer samples that belong to the object being rendered, the resulting lighting gives the illusion of being at full resolution. This same method works even when the framebuffer is multisampled (hardware MSAA), however sub-pixel artifacts can occur, due to the pixel shader only being run once per pixel, rather than once per sample.
The authors report that such sub-pixel artifacts are typically not noticeable.

3. The authors of this paper also implemented a technique that allows to render alpha polygons with the Light Pre-Pass / Deferred lighting. It is based on stippling and the usage of the DSF filtering.
During the geometry pass the alpha polygons are rendered using a stipple pattern, so that their G-Buffer samples are interleaved with opaque polygon samples.
In the material pass the DSF for opaque polygons will automatically reject stippled alpha pixels, and alpha polygons are handled by finding the four closest light buffer samples in the same stipple pattern, again using DSF to make sure the samples were not overwritten by some other geometry.
Since the stipple pattern is a 2×2 regular pattern, the effect is that the alpha polygon gets lit at half the resolution of opaque objects. Opaque objects covered by one layer of alpha have a slightly reduced lighting resolution (one out of every four samples cannot be used).

http://www.siggraph.org/s2009/sessions/courses/details/?id=12

I collected all the new development in this area, and added a few new things I found out while working on DirectX 10 / 11 implementations and will post a link to the slides here. Especially on the PS3 there is lots of new and interesting development (Judging from the number of games that will ship with this approach I want to believe that it is the most popular way to apply lots of lights in games now). I received a first draft of an article for ShaderX8 / GPU Pro from Steven Tovey about how they implemented the Light Pre-Pass in the upcoming game Blur on the PS3. They based their approach on work done by Matt Swoboda. The results look very cool. You can check out the screenshots on their website.

There is lots of progress happening with the Oolong Engine for the iPhone / iPod Touch. Check out the change list on

http://code.google.com/p/oolongengine

We got OpenGL ES 2.0 running and there is a new tutorial series that looks really cool.

In other news somehow my name was mentioned on “The Escapist”. Here is the link for your entertainment:

http://www.escapistmagazine.com/articles/view/columns/publishers-note/6250-Publishers-Note-Made-By-People.2

Overall I believe a fundamental shift to more generic method is necessary to solve those issues. This is one of the things I am looking into … so expect an update at some point in the future.

http://www.dimension3.sk/mambo/Articles/Deferred-Rendering-In-Killzone/View-category.php

What they do is read both samples in the multisampled render target, do the lighting calculations for both of them and then average the result and write it into the multi-sampled (… I assume it has to be multi-sampled because the depth buffer is multisampled) accumulation buffer. That somehow decreases the effectiveness of MSAA because the pixel averages all samples regardless of whether they actually pass the depth-stencil test. The multisampled accumulation buffer may therefore contain different values per sample when it was supposed to contain a unique value representing the average of all sample. Then on the other side they might only store a value in one of the samples and resolve afterwards … which would mean the pixel shader runs only once.
This is also called “on-the-fly resolves”.

It is better to write into each sample a dedicated value by using the sampling mask but then you run in case of 2xMSAA your pixel shader 2x … DirectX10.1+ has the ability to run the pixel shader per sample. That doesn’t mean it fully runs per sample. The MSAA unit seems to replicate the color value accordingly. That’s faster but not possible on the PS3. I can’t remember if the XBOX 360 has the ability to run the pixel shader per-sample but this is possible.

The pixel represented by a square has two triangles (blue and yellow) crossing some of its sample points. The black dot represents the pixel sample location (pixel center); this is were the pixel shader is executed. The cross symbol corresponds to the location of the multisamples where the depth tests are performed. Samples passing the depth test receive the output of the pixel shader. Those samples are replicated by the MSAA back-end into a multisampled render target that represents each pixel with -in that case- four samples. That means the render target size for an intended resolution of 1280×720 would be 2560×1440 representing each pixel with four samples but the pixel shader only writes 1280×720 times (assuming there is no overdraw) while the MSAA back-end replicates for each pixel four samples into the multisampled render target.
With deferred lighting there can be several of those multi-sampled render targets as part of a Multiple-Render-Target (MRT). In the so called Geometry stage, data is written into this MRT; therefore called G-Buffer. In case of 4xMSAA each of the render targets of the G-Buffer would be 2560×1440 in size.
In case of Deferred Lighting / Light Pre-Pass the G-Buffer holds normal and depth data. This data can never be resolved because resolving it would lead to incorrect results as shown by Nicolas in his article.
After the Geometry phase comes the Lighting or Shading phase in a Deferred Lighting/Light Pre-Pass/Deferred Shading renderer. In an ideal world you could blit each sample (not pixel) into the multisampled render target -that holds the result of the Shading phase- by reading the G-Buffer sample and performing all the calculations necessary on it.
In other words to achieve the best possible MSAA quality with those renderer designs, lighting equations would need to be applied on a per-sample basis into a multisampled render target and then later resolved.
This is possible with DirectX 10.1 graphics hardware (AMD’s 10.1 capable cards; didn’t try if S3 cards that support 10.1 can do this as well) that allows to execute a pixel shader at sample frequency.
To make this a viable option, this operation needs to be restricted to samples that belong to pixel edges. There are two passes necessary to make this work. One pass will use the pixel shader that runs operations performed on samples and in a second pass the pixel shader is run that performs operations per-pixel, which means the result of the pixel shader calculation is output to all samples passing the depth-stencil test.
To restrict the pixel shader that performs operations per-sample, a stencil test is used.
One interesting idea covered in the article is to detect edges with centroid sampling (available already on DirectX9 class graphics hardware). During the G-Buffer phase the vertex shader writes a variable unique to every pixel (e.g. pixel position data) into two outputs, while the associated pixel shader declares two inputs: one without and one with centroid sampling enabled. The pixel shader then compares the centroid-enabled input with the one without it. Differing values mean that samples were only partially covered by the triangle, indicating an edge pixel. A “centroid value” of 1.0 is then written out to a selected area of the G-Buffer (previously cleared to 0.0) to indicate that the covered samples belong to an edge pixel. Those values are then averaged while being resolved to find out the value per pixel. If the result is not exactly 0, then the current pixel is an edge pixel. This is shown in the following image from the article.
On the left the pixel shader input will always be evaluated at the center of the pixel regardless of whether it is covered by the triangle. On the right with centroid sampling, the two rightmost depth samples are covered by the triangle. The comparison of the values in the pixel shader will lead to the result that the samples were only partially covered by the triangle, indicating an edge pixel.
Because DirectX10 capable graphics hardware does not support the pixel shader running at sample frequency, a different solution needs to be developed here.
The best MSAA quality in that case is achieved by running the pixel shader multiple times per pixel, only enabling output to a single sample each pass. This can be achieved by using the OMSetBlendState() API. The results of this method would be identical to the DirectX 10.1 method but obviously due to the increased number of rendering passes and slightly reduced texture cache effectiveness more expensive.

Suitable C pseudo code could be:

float r = pow(pow(fabs(cos(m * o / 4)) / a, n2) + pow(fabs(sin(m * o / 4)) / b, n3), 1 / n1);

The result of this calculation is in polar coordinates. Please note the difference between the equation and the C code. The equation has a negative power value, the C doesn’t. To extend this result into 3D, the spherical product of several superformulas is used. For example, the 3D parametric surface is obtained multiplying two superformulas S1and S2. The coordinates are defined by the relations:

The sphere mapping code uses two r values:

point->x = (float)(cosf(t) * cosf(p) / r1 / r2);
point->y = (float)(sinf(t) * cosf(p) / r1 / r2);
point->z = (float)(sinf(p) / r2);

Distance runs on the axis called Z value. So 0 is close to the camera and 1.0 is far away. You can see how the near and far blur plane fade in and out with increasing of the value called Range. The equation to plot this in mathematica is rather simple. In practice it is a quite efficient approach to achieve the effect.

Plot3D[R*Abs[0.5 - z], {z, 1, 0}, {R, 0, 1},
PlotStyle -> Directive[Pink, Specularity[White, 50], Opacity[0.8]],
PlotLabel -> “Depth of Field”, AxesLabel -> {“Z value”, “Range”}]

My plan is to develop a few new algorithms and show the results here. It will be an exercise in thinking about new things for me. If you have any suggestions on what I should cover, please do not hesitate to post them in the comment line.

First of all: the book is great and definitely worth a read. It is written in a very informative, instructive and entertaining way (… if you know the guys that contributed to it you know that it is worth it ).

With that being said, let’s start with the review by looking at the Table of Content. I know that I usually spent more time than other people with reading the TOC. This is the best way for me to figure out what a book has to offer. A good TOC shows you the big picture of a book and allows you to see the pattern that the author chose on how to approach the topic. In most cases it even allows you to proof the underlying logic.
The book consists of 9 chapters. Each chapter consists of a analysis of facts by the author followed by an interview of a game industry veteran. The topics span from “How We got here” over “Anatomy of a Game-Dev Company”, “How Leaders are Chosen …”, “A Litmus Test for Leads”, “Leadership Types and Traits …” and then they go into more detail with the “The Project Team Leader …”, “The Department Leader …”, “Difficult Employees …”, “The Effect of Great Team Leadership” followed by a “Sample Skill Ladder” for artists in the appendix.

You might feel the need to discuss some of the details covered in each chapter but it is clear that this is the right formal approach to slice up the delicate topic of leadership in our industry.

When I first skipped through the book I wanted to figure out what kind of values the author has. After all a good leader makes it clear what kind of values he/she follows. I found it in the introduction. Here is the quote: “As will be seen, a major cause of people leaving a company is the perceived poor quality of their supervisors and senior management. The game business is a talent-based industry -the stronger and deeper your talent is, the better chances are of creating a great game. It is very difficult, in any hiring environment, to build the right mix of cross-disciplinary talent who function as a team at a high level; indeed, most companies never manage it. Once you get talented individuals on board, it’s critical not to lose them. Finding and nurturing compentent leaders who have the trust of the team will generate more retention than any addition of pool tables, movie nights, or verbal commitments to the value of “quality of life”.”
You might think this is the most obvious thing to say in the game industry.

Obviously the book wants to cover the process to setup a creative and great environment for all humans involved in the process of creating great games. Creating a great working environment starts with picking the right leaders that enable people by helping them to give their best. A great leader serves his/her people. He/she sees the best in everyone and has the ability to expose this talent. Many interviewees in the book also mention that humor is a leadership skill. I trained junior managers for BMW, Daimler, ABB and other companies back in Germany for two years on weekends and I always thought this is a strong skill. Making people laugh starts a lot of processes in the body that make people more relaxed and in general brighten up their day. Whoever can do this can certainly improve the morale and therefore efficiency of a team in seconds … priceless.

Managing a creative team is a completely different story than -for example- a sales team. The human factor in the relationship between people plays an important role. They have to create something together, while a sales person is on his own out in the field and comes back with a number and relies on a relationship with a potential customer that only lasts a few hours face-to-face time, a creative team stays together for years and has to overcome all the things that come up when humans have to live in a small space together. There is a complex social network in place that defines the relationships between those humans and it is important to keep the team running with all the constantly changing love/hate -and in-between- relationships on board. People on the team might even deal with difficult personal relationships and you end up with a mixture of chaos and randomness typical for family or close friends scenarios. In that context it was interesting to see what the interviewees thought about the question if leaders are born and / or can be trained to be successful in the game industry. Obviously someone who was active as a boy-scout leader, speaker/president of the students association at his university or volunteered to work with other people in general, already showed some level of social committment that is a good starting point for a leader ship role in our industry.

So defining and following the right values is a fundamental requirement for a book on leadership. Obviously after having set the values comes the part where those values need to be applied and used and this is where the book shines. It is hands-down and even if you do not agree with the author in every detail the fact that he wrote all this down earns the highest respect.

So now that I made it obvious that I am excited about this book, let’s think about how it might be improved in the future. A potential improvement I could see is to start the book with a target description. Not that the author fails to describe a target but I would appreciate it to go into more detail in this area.
What is the company you would want to work for? What is the environment you want to offer to make people as productive as possible? Obviously it is a hen / egg problem. Good people want to work in good teams and good teams consist of good people … there are social -soft skills- and knowledge -hard skills- attached to each person of that team.
A good team starts with a good leader who sets values and standards and hires the right people.

Assuming you are the leader of this future team, how would you create the environment for your dream team? How do you want people to feel when they are part of this team? What should they take home every night when they are exhausted? What do you want them to tell their wifes / better halves how it is to work with you as their leader?
A happy employee -fully enforced to be creative – should tell his wife/girlfriend that he works very hard but is treated fair and enjoys the family related benefits of the company.
He should tell his friends that he is working in a team where information is shared and where his potential is not only used as much as possible but also amplified. He needs to feel like he is growing with the team and the tasks.
He should tell his colleagues that he enjoys working with them and the team and that he enjoys coming into work every day and that he is excited about the project he is working on …

So if we make that into a list of items we could describe how an employee should feel about working in a company with good Leaders. Might be a great starting point for discussing leader core abilities.

http://www.shaderx7.com/

There is now the first draft of the cover and the Table of Content. Enjoy!

As before I will rest for a second when the new book comes out and think about what happened since I founded the series now eight years ago … my perception of time slows down for this second and I hear myself saying:”Chewbacca start the hyperdrive, let’s go to the next planet, I need to play cards, drink alcohol and find some entertainment … how about Tantoine?”

http://www.oolongengine.com

I updated the memory manager, the math library, upgraded to the latest POWERVR POD format and added to each example VBO support. Please also note that in previous updates a new memory manager was added, the VFP math library was added and a bunch of smaller changes were done as well.
The things on my list are: looking into the sound manager … it seems like the current version allocates memory in the frame and adding the DOOM III level format as a game format. Obviously zip support would be nice as well … let’s see how far I get.

http://wiki.gamedev.net/index.php/D3DBook:Book_Cover

If you have any suggestions, comments or additions to this book, please give me a sign or write it into the book comment pages.

http://gamearchitect.net/2008/09/19/good-middleware/

http://code.google.com/p/quake3-iphone/

There is a list of issues. If you have more spare time than me, maybe you can help out.

// Enumerates through all touch objects
for (UITouch *touch in touches)
{
[self _handleTouch:touch];
touchCount++;
}

_handleTouch might look like this:

- (void)_handleTouch:(UITouch *)touch
{
CGPoint location = [touch locationInView:self];
CGPoint previousLocation;

// if we are in a touchMoved phase use the previous location but then check if the current
// location is still in there
if (touch.phase == UITouchPhaseMoved)
previousLocation = [touch previousLocationInView:self];
else
previousLocation = location;

…
// fire event
// lower right corner .. box is 40 x 40
if (EVENTREGIONFIRE(previousLocation))
{
if (touch.phase == UITouchPhaseBegan)
{
// only trigger once
if (_bitMask ^ Q3Event_Fire)
{
[self _queueEventWithType:Q3Event_Fire value1:K_MOUSE1 value2:1];

_bitMask|= Q3Event_Fire;
}
}
else if (touch.phase == UITouchPhaseEnded)
{
if (_bitMask & Q3Event_Fire)
{
[self _queueEventWithType:Q3Event_Fire value1:K_MOUSE1 value2:0];

_bitMask^= Q3Event_Fire;
}
}
else if (touch.phase == UITouchPhaseMoved)
{
if (!(EVENTREGIONFIRE(location)))
{
if (_bitMask & Q3Event_Fire)
{
[self _queueEventWithType:Q3Event_Fire value1:K_MOUSE1 value2:0];

_bitMask^= Q3Event_Fire;
}
}
}
}
…

Tracking if the switch is on or off can be done with a bit mask. The event is send off to the game with a separate _queueEventWithType method.

• glEnable(GL_POINT_SPRITES_OES) – this is the global switch that turns point sprites on. Once enabled, all points will be drawn as point sprites.
• glTexEnvi(GL_POINT_SPRITES_OES, GL_COORD_REPLACE_OES, GL_TRUE) – this enables [0..1] texture coordinate generation for the four corners of the point sprite. It can be set per-texture unit. If disabled, all corners of the quad have the same texture coordinate.
• glPointParametervf(GLenum pname, const GLfloat * params) – this is used to set the point attenuation as described below.

The point size of a point sprite can be derived with the formula:
user_clamp represents GL_POINT_SIZE_MIN and GL_POINT_SIZE_MIN settings of the glPointParametervf(). impl_clamp represents an implementation-dependent point size range.
GL_POINT_DISTANCE_ATTENUATION is used to pass in params as an array containing the distance attenuation coefficients a, b, and c, in that order.
In case multisampling is used (not officially supported), the point size is clamped to have a minimum threshold, and the alpha value of the point is modulated by the following equation:
GL_POINT_FADE_THRESHOLD_SIZE specifies the point alpha fade threshold.
Check out the Oolong engine example Particle System for an implementation. It uses 600 point sprites with nearly 60 fps. Increasing the number of point sprites to 3000 lets the framerate drop to around 20 fps.

http://tog.acm.org/resources/shaderx/

• glMatrixMode(GL_MATRIX_PALETTE) – Set the matrix mode to palette
• glCurrentPaletteMatrix(n) – Set the currently active palette matrix and loads each matrix in the palette
• To enable vertex arrays
  glEnableClientState(MATRIX_INDEX_ARRAY)
  glEnableClientState(WEIGHT_ARRAY)
• To load the index and weight per-vertex data
  glWeightPointer()
  glMatrixIndexPointer()

On the iPhone there are up to nine bones per sub-mesh supported (check GL_MAX_PALETTE_MATRICES_OES). Check out the Oolong example MatrixPalette for an implementation.

Yann Lombard explains on how to pick a light source first that should cast a shadow. He is using distance, intensity, influence and other parameters to pick light sources.

He has a cache of shadow maps that can have different resolutions. His cache solution is pretty generic. I would build a more dedicated cache just for shadow maps.
After having picked the light sources that should cast shadows, I would only constantly update shadows in that cache that change. This depends on if there is an object with a dynamic flag in the shadow view frustum.
If you think about it how it happens when you approach a scene with lights that cast shadows:
1. the lights are picked that are close enough and appropriate to cast shadows -> shadow maps are updated
2. then while we move on, for the lights in 1. we only update shadow maps if there is an object in shadow view that is moving / dynamic; we start than with the next bunch of shadows while the shadows in 1 are still in view
3. and so on.

http://www.gamedev.net/community/forums/topic.asp?topic_id=517022

This is pretty cool. Culling stuff into the two hemispheres is obsolete here. Other than this the usual comparison between cube maps and dual-paraboloid maps applies:

• the number of drawcalls is the same … so you do not save on this front
• you loose memory bandwidth with cube maps because in worst case you render everything into six maps that are probably bigger than 256×256 … in reality you won’t render six times and therefore have less drawcalls than dual-paraboloid maps
• the quality is much better for cube maps
• the speed difference is not that huge because dual paraboloid maps use things like texkill or alpha test to pick the right map and therefore rendering is pretty slow without Hierarchical Z.

I think both techniques are equivalent for environment maps .. for shadows you might prefer cube maps; if you want to save memory dual-paraboloid maps is the only way to go.

Update: just saw this article on dual-paraboloid shadow maps:

http://osman.brian.googlepages.com/dpsm.pdf

The basic idea is that you do the WorldSpace -> Paraboloid transformation in the pixel shader during your lighting pass. That avoids having the paraboloid co-ordinates interpolated incorrectly.

• The maximum texture size is 1024×1024
• 2D texture are supported; other texture formats are not
• Stencil buffers aren’t available

As far as I know stencil buffer support is available in hardware. That means the Light Pre-Pass renderer can only be implemented with the help of the scissor (hopefully available). As a side note: one of the other things that do not seem to be exposed is MSAA rendering. With the unofficial SDK it seems like you can use MSAA.
Texture filtering is described on page 99 of the iPhone Application programming guide. There is also an extension for anisotropic filtering supported, that I haven’t tried.

The pixel shader of the iP* platform is programmed via texture combiners. There is an overview on all OpenGL ES 1.1 calls at

http://www.khronos.org/opengles/sdk/1.1/docs/man/

The texture combiners are described in the page on glTexEnv. Per-Pixel Lighting is a popular example:

glTexEnvf(GL_TEXTURE_ENV,
// N.L
.. GL_TEXTURE_ENV_MODE, GL_COMBINE);
.. GL_COMBINE_RGB, GL_DOT3_RGB); // Blend0 = N.L

.. GL_SOURCE0_RGB, GL_TEXTURE); // normal map
.. GL_OPERAND0_RGB, GL_SRC_COLOR);
.. GL_SOURCE1_RGB, GL_PRIMARY_COLOR); // light vec
.. GL_OPERAND1_RGB, GL_SRC_COLOR);

// N.L * color map
.. GL_TEXTURE_ENV_MODE, GL_COMBINE);
.. GL_COMBINE_RGB, GL_MODULATE); // N.L * color map

.. GL_SOURCE0_RGB, GL_PREVIOUS); // previous result: N.L
.. GL_OPERAND0_RGB, GL_SRC_COLOR);
.. GL_SOURCE1_RGB, GL_TEXTURE); // color map
.. GL_OPERAND1_RGB, GL_SRC_COLOR);
Check out the Oolong example “Per-Pixel Lighting” in the folder Examples/Renderer for a full implementation.

// “View” for games in applicationDidFinishLaunching
// get screen rectangle
CGRect rect = [[UIScreen mainScreen] bounds];

// create one full-screen window
_window = [[UIWindow alloc] initWithFrame:rect];

// create OpenGL view
_glView = [[EAGLView alloc] initWithFrame: rect pixelFormat:GL_RGB565_OES depthFormat:GL_DEPTH_COMPONENT16_OES preserveBackBuffer:NO];

// attach the view to the window
[_window addSubView:_glView];

// show the window
[_window makeKeyAndVisible];

The screen dimensions are retrieved from a screen object. Erica Sadun compares the UIWindow functionality to a TV set and the UIView to actors in a TV show. I think this is a good way to memorize the functionality. In our case EAGLView, that comes with the Apple SDK, inherits from UIView and adds all the OpenGL ES functionality to it. We attach this view than to the window and make everything visible.
Oolong assumes a full-screen window that does not rotate. It is always in widescreen view. The reason for this is that otherwise the accelerometer usage -to drive a camera with the accelerometer for example- wouldn’t be possible.
There is a corresponding dealloc method to this code that frees all the allocated resources again.
The anatomy of a Oolong engine example uses mainly two files. A file with “delegate” in the name and the main application file. The main application file has the following methods:
- InitApplication()
- QuitApplication()
- UpdateScene()
- RenderScene()
The first pair of methods do one-time device dependent resource allocations and deallocations, while the UpdateScene() prepares scene rendering and the last method actually does what the name says. If you would like to extend this framework to handle orientation changes, you would add a pair of methods with names like InitView() and ReleaseView() and handle all orientation dependent code in there. Those methods would always been called when the orientation changes -only once- and at the start of the application.

One other basic topic is the usage of C++. In Apple speak this is called Objective-C++. Cocoa Touch wants to be addressed with Obj-C. So native C or C++ code is not possible. For game developers there is lots of existing C/C++ code to be re-used and its usage makes games easier to port to several platforms (quite common to launch an IP on several platforms at once). The best solution to this dilemma is to use Objective-C where necessary and then wrap to C/C++.
If a file has the postfix *.mm, the compiler can handle Objective-C, C and C++ code pieces at the same time to a certain degree. If you look in Oolong for files with such a postfix you will find many of them. There are whitepapers and tutorials available for Objective-C++ that describe the limitations of the approach. Because garbage collection is not used on the iP* device I want to believe that the challenges to make this work on this platform are smaller. Here are a few examples on how the bridge between Objective-C and C/C++ is build in Oolong. In our main application class in every Oolong example we bridge from the Objective-C code used in the “delegate” file to the main application file like this:

// in Application.h
class CShell
{
..
bool UpdateScene();

// in Application.mm
bool CShell::UpdateScene()
..

// in Delegate.mm
static CShell *shell = NULL;

if(!shell->Update()) printf(“Update error\n”);

An example on how to call an Objective-C method from C++ can look like this (C wrapper):

// in PolarCamera.mm -> C wrapper
void UpdatePolarCamera()
{
[idFrame UpdateCamera];
}
-(void) UpdateCamera
{
..
// in Application.mm
bool Cshell::UpdateScene()
{
UpdatePolarCamera();
..

The idea is to retrieve the id for a class and then use this id to address a function in the class from the outside.
If you want to see all this in action, open up the skeleton example in the Oolong Engine source code. You can find it at
Examples/Renderer/Skeleton
Now that we are at the end of this tip I would like to refer to a blog that my friend Canis wrote. He talks about memory management here. This blog entry applies to the iP* platforms quite well:

http://www.wooji-juice.com/blog/cocoa-6-memory.html

• .app folder holds everything without required hierarchy
• .lproj language support
• Executable
• Info.plist – XML property list holds product identifier > allows communicate with other apps and register with Springboard
• Icon.png (57×57) set UIPrerenderedIcon to true in Info.plist to not receive the gloss / shiny effect
• Default.png … should match game background; no “Please wait” sign … smooth fade
• XIB (NIB) files precooked addressable user interface classes >remove NSMainNibFile key from Info.plist if you do not use it
• Your files; for example in demoq3/quake3.pak

If the game boots very fast a good mobile phone experience could be guaranteed by making a screenshot when the user ends the app and then using that screenshot while booting up the game and bringing it to the state it was before.
Every iP* app is sandboxed. That means that only certain folders, network resources and hardware can be accessed. Here is a list of folders that might be affected by your application:

• Preferences files are in var/mobile/Library/Preferences based on the product identifier (e.g. com.engel.Quake.plist); updated when you use something like NSUserDefaults to add persistance to game data like save and load
• App plug-in /System/Library (not available)
• Documents in /Documents
• Each app has a tmp folder
• Sandbox spec e.g. in /usr/share/sandbox > don’t touch 

The sandbox paradigm is also responsible for a mechanism that stops your game if it eats up too many resources of the iPhone. I wonder under which conditions this is going to happen.

• To work in one window only, use the “All-in-One” mode if you miss Visual Studio (http://developer.apple.com/tools/xcode/newinxcode23.html)
  You have to load Xcode, but not load any projects. Go straight to Preferences/General Tab, and you’ll see “Layout: Default”. Switch that to “Layout: All-In-One”. Click OK. Then, you can load your projects.
• Apple+tilde – cycle between windows in the foreground
• Apple+w – closes the front window in most apps
• Apple+tab – cycle through windows

Please note that Apple did a revolutionary thing on the new MacBook Pro’s (probably also the new MacBook’s) … there is no Apple key anymore. It is now called command key.

For everyone who prefers hotkeys to start applications you might check out Quicksilver. Automatically hiding and showing the Dock gives you more workspace. If you are giving presentations about your work, check out Stage Hand for the iPod touch / iPhone.

For reference you should have POWERVR SDK for Linux downloaded. It is a very helpful reference regarding the MBX chip in your target platforms.

Not very game or graphics programming related but very helpful is Erica Sadun’s book “The iPhone Developer’s Cookbook”. She does not waste your time with details you are not interested in and comes straight to the point. Just reading the first section of the book is already pretty cool.
You want to have this book if you want to dive into any form of Cocoa interface programming.
The last book I want to recommend is Andrew M. Duncan’s “Objective-C Pocket Reference”. I have this usually lying on my table if I stumble over Objective-C syntax. If you are a C/C++ programmer you probably do not need more than this. There are also Objective-C tutorials on the iPhone developer website and on the general Apple website.

If you have any other tip that I can add to the website I would mention it with your name.

• GPU: PowerVR MBXLite with VGPLite with 103 Mhz
• ~DX8 hardware with vs_1_1 and ps_1_1 functionality
• Vertex shader is not exposed
• Pixel shader is programmed with texture combiners
• 16 MB VRAM – not mentioned anywhere
• CPU: ARM 1176 with 412 Mhz (can do 600 Mhz)
• VFP unit 128-bit Multimedia unit ~= SIMD unit
• 128 MB RAM; only 24 MB for apps allowed
• 320×480 px at 163 ppi screen
• LIS302DL, a 3-axis accelerometer with 412 Mhz (?) update rate
• Multi-Touch: up to five fingers
• PVRTC texture compression: color map 2-bit per pixel and normal map 4-bit per-pixel

The interesting part is that the CPU can do up to 600 Mhz, so it would be possible to increase the performance here in the future.
I wonder how the 16 MB VRAM are handled. I assume that this is the place where the VBO and textures are stored. Regarding the max size of apps of 24 MB; I wonder what happens if an application generates geometry and textures dynamically … when does the sandbox of the iPhone / iPod touch stop the application. I did not find any evidence for this.

http://msdn.microsoft.com/en-us/library/dd285359.aspx

Running Crysis on a 8 core CPU with a resolution of 800×600 at 7.2 fps is an achievement. If this would be hand-optimized very well, it would be the best way to write code for. 4 – 8 cores will be a common target platform in the next two years. Because it can be switched off if there is a GPU, this is a perfect target for game developers. What this means is that you can write a game with the DirectX 10 API and not only target all the GPUs out there but also machines without GPU … this is one of the best developments for the PC market since a long time. I am excited!

The other interesting consequence from this development is: if INTELs “Bread & Butter” chips run games with the most important game API, it would be a good idea if INTEL would put a bunch of engineers behind this and optimize WARP (in case they haven’t already done so). This is the big game market consisting of games like “The Sims” and “World of Warcraft” and similar games that we are talking about here. The high-end PC gaming market is much smaller.