Bruno Evangelista

This year Siggraph took place in LA between 5 and 9 of August and it was pretty cool as in previous years I attended. I took a couple of notes during the conference that I though might be helpful for people couldn’t attend – although I planned to post them months ago. Hope you enjoy! =)

---

Talk: Pointed Illumination http://s2012.siggraph.org/attendees/sessions/100-119

Progressive Lightcuts for GPU (Phd students, Universität des Saarlandes / Intel):
[Lightcuts] http://www.graphics.cornell.edu/~bjw/lightcutSigTalk05.pdf
[Progressive lightcuts] http://dl.acm.org/citation.cfm?id=2343047

• Light cuts adaptively approximate complex global illumination through clusters of virtual point lights (VPLs): area light, sky light, indirect illumination, etc. Steps: convert illumination to VPLs, build a light tree, for each eye ray choose a cut that best approximates the illumination. (optimization to path tracing?)
• This work tries to “group” the contribution of lights that don’t contribute much to the final scene, e.g. are far away. (progressively averaging the lightcut images?)
• There’s issues with VPL’s too near the surface that cause spikes in the reflection intensity (very bright aliasing), they change the attenuation formula to clamp the intensity near the surface.

SGRT: Scalable Mobile GPU Architecture Based on Ray Tracing (Samsung)
http://dl.acm.org/citation.cfm?id=2343048

• Presented an architecture for real-time raytracing (based on recent papers from Samsung).
• Fixed-function hardware for traversal and intersection (T&I engine [Nah et al. 2011]) handles bounding volume hierarchy (split bounding volume, surface area heuristic, etc), ray-AABB intersection, and supports dynamic scenes.
• Samsung Reconfigurable Processor for shading [Lee et al. 2011], supports shaders written in C-like language with branches, allowing recursive ray-tracing.
• Uses Samsung multiplatform kernel and adaptive restart trail.

Point-Based Global Illumination Directional Importance Mapping (Dreamworks)

• Raytracing was not used at all in Madagascar 3, their solution was to use many point lights (convert area lights to points?) and they baked them (all of them?) to environment maps (or HDR env maps).
• They presented an importance sampling scheme to increase quality without paying the cost of handling all lights. The result video showed nice improvements on shadows, they mentioned quality similar to 4 hours of rendering in 1.5 hours with importance mapping.
• Showed some videos of things that would be difficult to render without ray-tracing (character walking on energy cables with energy rays, motorbike shadows).

Ill-Loom-inating Handmade Fabric in “Brave” (Pixar)

• Started presenting some early work they did to handle fabric in brave, as previous techniques were not good enough handling “frayed edges”, “holes”, etc.
• His early work looked a lot like Relief Mapping, he showed demos of animation fabric with small deformations, stretching, etc. It looked pretty good. They dropped this technique as it was difficult to apply to arbitrary surfaces due to silhouette issues (again, same flaws as Relief Mapping that were only addressed for quadric surfaces).
• Final work was based on render-man shaders that were used to generate/tessellate the surface on-the-fly.

---

Technical papers: Image Processing http://s2012.siggraph.org/attendees/sessions/100-49
http://kesen.realtimerendering.com/sig2012.html

Decoupling Algorithms From Schedules for Easy Optimization of Image-Processing Pipelines (MIT, Adobe, Stanford)
www.halide-lang.org

• Decouples algorithms from schedules, allowing you to easily code and try many different scheduling approaches. New language called Halide which provides a compiler to generate code for many different architectures.
• The scheduling is done mixing some of the available “modes”, 1 to 1 (fusion), 1 to N or N to 1. Also, the output must be in a grid. Other data structures, such as lists, trees, etc, are not supported yet.
• It’s also difficult to write custom functions, such as a Fourier transform, using the Halide language. To handle that they would enable the “import” of functions from external libraries (apparently not currently possible)
• Results were very good and they improved running time of a few algorithms (e.g. “Local laplacian filter”), and they did better than Intel tools (Intel TBB?)

Adaptive Manifolds for Real-Time High-Dimensional Filtering (UFRGS)

• They apply high-dimensional manifold filtering over many rendered images, for example, scenes rendered with path tracing.
• They talk about their algorithm being “aware of edges” and “temporal coherent”, so it might be possible to use for AA as well. The only issue is that for some of the results (in a video for example) they also use future frames.
• Linear cost: 70ms for a 10MP image, and 7ms for a 1MP image (still too much for real time).

High-Quality Image Deblurring With Panchromatic Pixels (Kodak)

• They talked about generation blur kernels, gathering it from images and handling invalid kernels. My understanding is that their algorithm uses two images from the same scene captured with different levels of exposure, and use it to calculate the blur kernel.
• They also said something about using infrared sensors not sure what. They also show that their algorithm fails when there’s too much blur.

Practical temporal Consistency for Image-Based Graphics Application (Disney Zurich)

• Talked about optical flow, temporal anti-aliasing and motion aware filtering. They compared their result with MPEG and show that it is better for their scenes.

---

Talks: Surf & Turf http://s2012.siggraph.org/attendees/sessions/100-121

From a Calm Puddle to a Stormy Ocean: Rendering Water in Uncharted (NaughtyDog)
http://dl.acm.org/citation.cfm?id=2343050

• They show the water simulation system created for Uncharted, which appears to be divided in a few parts to handle waves, ocean, etc.
• From what I get they can use Gerstner Waves or Point Particles to add displacement/deformation and simulate ripples and waves. They can control how much stretching is going-on.
• In the in-game indoor cinematics the water has so many particles on top of it that it looks much more impressive than the water itself.
• Many objects in the ocean are not attached to anything and move according to the water. For that they integrate all forces applied over each object (pretty cool).

What if the Earth Was Flat: The Globe UI System in SSX (EA)
http://dl.acm.org/citation.cfm?id=2343051

• This talk covered the development of the UI in the SSX game. I wasn’t really impressed by it and the problems they solved look quite old to me.
• Relaxed Cone Relief Mapping was used to render all the details in their globe, as I remember it, this accelerates the per-pixel ray-tracing by using a safe-cone (don’t intersect the geometry) and a relaxed code (intersects the geometry only once). The silhouettes in the zoom-in image look better than I expected.
• They shown how they fixed issues using perspective interpolation instead of just linear interpolation on the sphere…

Adaptive Level-of-Detail System for End of Nations (Petroglyph Games)
http://dl.acm.org/citation.cfm?id=2343052

• They presented a lod system that handles LOD in many different subsystems and has a nice level of scalability.
• Their LOD system would handle lod on meshes, materials, effects, screen resolution (need to confirm?), etc. They have an in-game menu where they would force an specific render time and the game would scale details to make it real-time again.
• The results were good, they would remove most of the particles (in a smart way), remove effects, and even do things like rendering a different subset of all objects per frame. Those changes were also fast enough that they wouldn’t introduce problems while “teleporting” to other maps of the map that were running smoothly.

Screen Space Decals in Warhammer 40,000: Space Marine (Relic)
http://dl.acm.org/citation.cfm?id=2343053

• They presented an algorithm to render decals by projecting a 3D box on top of their G-Buffer and rendering the decals in screen space.
• There were some issues presented when the projected box intersects the near clipping plane of the camera, or then the normal of the surface changes too heavily (corners) under the projected box.
• Final results were pretty impressive were this system was used to add a lot of details to many parts of the game.

SquareEnix released a Final Fantasy XIV benchmark application, which according to them, displays actual maps and characters to assign a score to your PC and rate its performance for actual play of FINAL FANTASY XIV for Windows.

Being curious about the character’s polygon count and texture maps, I decided to get some information about the game using PIX (Performance Investigator for DirectX), a tool that comes with the DirectX SDK. Using PIX, I ran the benchmark and when an interesting image appeared (shown in the screenshot below) I captured a Direct 3D frame, which contains all the API calls, data buffers (including vertices, textures and render targets) and shaders.

First I would like to say that every time I captured a Direct3D frame it came with thousands of API calls, making it difficult to find out where the interesting information really was. Also, I believe that because it is a benchmark the game is doing some repeated/unnecessary work. The following picture shows some of the rendering calls captured with PIX.

Next, I would like to investigate the rendering of the character who appears in the upper right part of the first picture. This character is draw inside the MultiScreenManager::Draw method as a set of parts/chunks. Note that in the previous picture the MultiScreenManager::Draw call was not expanded because it has lots of calls inside. The table below shows all the rendered parts/chunks of the investigated character.

Part/Chunk Name	Vertex Count	Triangle Count	Vertex/Triangle Rate
Head	1043	1322	0.788
Eye	50	64	0.781
Upper Body Cloth1	1000	840	1.190
Upper Body Cloth2	240	184	1.304
Upper Body Cloth3	654	752	0.869
Trousers	332	536	0.619
Hands	462	628	0.735
Foot Cloth	1008	434	2.322
Foot	214	168	1.273
Hair	939	1009	0.930
TOTAL	5942	5937	1.000

As you can see, this character has almost 6K polygons divided into 10 chunks. All the chunks were drawn using an indexed triangle list with three DXT1 textures for diffuse color, normal map and specular map (or a mask, in some cases). I will not post any picture of the character textures but his face, for example, has three 128×256 textures.

The analyzed chunks have an average rate of one vertex per triangle, however, in a closed mesh with lots of triangles, each vertex is shared on average by 6 faces. Therefore, it might be possible to achieve a 0.5 vertex/triangle rate but I will leave this discussion for a future post.

Yesterday, I read a recent paper (May 2010) from Morten Mikkelsen at Naughty Dog about a new bump mapping technique that does not require a given surface parametrization.

The idea of his work, as I understand it, was to calculate the surface parametrization at the time the surface’s fragments are begin processed in the Pixel Shader. For that purpose, he uses ddx and ddy functions to differentiate the surface’s eye-space position received from the Vertex Shader inside each fragment processed. That way, for an arbitrary triangular mesh, he does not need to store the usual per-vertex UV, tangent or bitangent (also known as binormal) parametrization vectors.

One disadvantage of this technique is that it relies entirely on GPUs that support Shader Model 3 but I wonder if it would not be possible/viable to differentiate the surface in a deferred pass using the PS3 SPUs.

I implemented Mikkelsen’s Bump Mapping technique inside my old Detailed Surfaces Viewer project, making it easy for me to compare his technique with two others: Normal Mapping and Relief Mapping. My implementation differs from Mikkelsen because I still sample the surface’s height from a normal/height texture using an interpolated UV coordinate, and then use the ddx and ddy functions over it.

The first issue that I noticed in Mikkelsen technique was that I needed to scale the height map values to increase the normal displacement and make it looks like Normal Mapping. The following pictures show a flat surface rendered with Normal Mapping (left) and Relief Mapping (right) using the same normal/height RGBA texture/map.

The following pictures show the same flat surface rendered with Mikkelsen’s technique using the same normal/height texture. In these pictures the height values acquired from the normal/height texture were scaled using a different factor varying is ascending order between 1, 4, 16 and 32.

As you can see, the results obtained exhibits low displacement and low noise in the first pictures but better displacement and higher noise in the last ones. I would like to further analyze this noise issue to make sure I didn’t do anything wrong and to find out how to deal with it. Note that I’m using trilinear filtering here and that the height map is the same being used by the Relief Mapping technique.

The following pictures compare Mikkelsen’s Bump Mapping without parametrization (first line), Normal Mapping (second line) and Relief Mapping (third line) techniques using two other normal/height maps. Note that for the first technique a height scale factor of 16x was used for both maps.

I will update the Detailed Surfaces Viewer project as soon as I deal with the noise problem.

Almost two yeast have passed since my last post. During this time many big things happened in my live: I left my city (Belo Horizonte) and moved to Campinas to work at Tectoy Digital, I finished my M.Sc. in Computer Science, and finally, I got married! In the mean time, I also went to Siggraph 2009 as a Student Volunteer, and a new version of our XNA book was published. So many things have happened that I will not be able to talk about all of them now.

I was trying to update this website for a long time but after joining Tectoy Digital I could not find any free time to do that. Fortunately, I could find some time to work on this website again in the past weeks and after a lot of word it is finally ready! I tried to port all the content from my previous website to this new one but I decided to drop some old posts and projects leaving more space for the new stuff that is coming.

This new site was made using Wordpress and it has some features that I believe everyone will enjoy, such as commenting the posts or rating it. In the following weeks I will try to post some materials (projects and talks) that I worked in the past two years, as well as some interesting news. I hope you enjoy this new website!

Siggraph is the world’s premiere conference for Computer Graphics and Interactive Techniques. It is a huge conference, attended by tens of thousands of people, and this year it will be held on Los Angeles. Well, this year I will attend to Siggraph for the first time and as a Student Volunteer!!!

Siggraph’s Student Volunteer is great program that provides students with a Full Conference Pass which allows you to go almost anywhere in the conference. As a volunteer you can apply for 18 or 30 hours of work, and if you apply for 30 hours you can also be awarded with housing. It was the second time that I applied for the volunteer program, and fortunately, I was accepted! =D

Are you attending to Siggraph this year? I would really like to meet you there. It will be my first time there and I would like to meet a lot of people.

Yes, they made it! The Mother Gaia Studio team (a brazilian team) is the winner of the Game Development category at Imagine Cup 2008. They took home the Game Development prize with the City Rain game, a game with a strong environmental focus. As I said in my old post, Brazil always perform well on Imagine Cup, and they made me no regreat my words!

It was the first time Imagine Cup featured a Game Development category, where students from all around the world competed creating a XNA games with the theme: “Imagine a world where technology enables a sustainable environment.”

Besides the Game Development category another brazilian team reached the forth place in the Software Design category. Check the Imagine Cup 2008 winners here.

About an year ago while working on the XNA TPS game I was surprised how my old single core Athlon 64 2.0 GHz processor could handle my game at 100+ fps (frames per second), while I was gettings less than 40 fps on the 3.2 GHz Xbox 360’s Xeon processor. (Note that my game was CPU bound and not optimized!)

Performance issues are common for XNA developers on the Xbox 360 – specially when your entire game was developed on Windows – and even professional developers, like Torpex team, had to face them. In general, these performance issues are due to: .NET Compact Framework (which runs on the Xbox) and Xbox 360’s Xeon PowerPC architecture. In this post, I will discuss about Xeon’s architecture and why it might run your game slower than your desktop Intel/AMD x86 processor. Let’s start looking at the Xeon specs at Xbox.com:

• Three symmetrical cores running at 3.2 GHz each.
• Two hardware threads per core; six hardware threads total.

Looking at these specs you may wonder how powerfull this processor is: “Wow, six hardware threads running at 3.2Ghz cores!“. In fact it is really powerfull, if you know how to effectively take advantage of its architecture! Do not expect a great performance from your ordinary (not optimized) single thread code.

When a manufacturer like IBM design a chip they are concerned about its surface size (or die size). The die size is the space they have to put everything the chip needs: logic units, control units, registers, cache and so on. But bigger chips means heat, high power consuption and high prices (they make money by getting as many chips from a silicon wafer as possible). In order to put a triple-core CPU in a small chip in early 2005, IBM had to remove some hardware optimizations from its cores, relying more on software optimizations. So, what makes Xeon processor different?

• In-order-execution: When one of the operands of an instruction is not available the processor waits untill it gets available (what can take longer if the data it is not in cache). In this case, an out-of-order execution would allow other instuctions to execute reducing latency.
• Poor branch prediction: When a conditional branch is found, branch prediction tries to guess if it will be taken or not, before evaluating its condition. Then, the next program instructions can be pre-fetched but a branch misprediction may cause bubbles in the pipeline. Some compilers allows developers to label branchs that are “likely” to be taken, generating a more optimized code. Branch intensive codes like IA and Game Logic may suffer here!
• Small cache size: Xeon features 32KB/32KB (instructions/data) L1 cache and 1MB L2 cache. It is indeed a small cache (my laptop’s Intel Core 2 Duo Mobile processor has a 3MB L2 cache) and if you do not handle it your program might suffer from lots of cache miss.

Furthermore, .NET Compact Framework JIT (just in time) compiler makes things more difficult for XNA developers, generating very unoptimized codes! But I will leave this discussion for my next post! =) Finally, I hope in the future (maybe on XNA 3.0) we can get access to the VMX 128 units on the Xbox. These units are a great help for math intensive code and the Xbox 360 features three of them.

Wanna learn more about the Xeon architecture? Check the “Inside the Xbox 360″ articles part 1 and part 2, or also check these threads (1, 2) on Creator’s Club.

One of the things that have been higly discussed on game developer foruns in Brazil last week was the fact that Ubisoft is opening a new studio in Brazil. The new studio will open its doors next month with an initial staff of 20 employees, aiming to grow to more than 200 developers within the next four years.

The new studio will be situated in Sao Paulo, the biggest economic city in Brazil, where they will initially work on “family titles” for consoles and portables. Heading up the new studio will be Bertrand Chaverot, previously the managing director of Ubisoft’s external development department for Europe, the Middle East, and Asia.

Are you an avid brazilian game developer looking for a job at one of the biggest worldwide publishers? Send your resume to: jobs.brazil@ubisoft.com, your check their open positions here. Right now, they are looking for great 2D and 3D artists and animators.

I recently bought an Acer Aspire 5920G laptop. It is my first laptop and it comes with a nice GeForce 8600M GT graphics card. Although the GeForce 8600M is not the most powerfull graphics card on the market it is a good one for a laptop, being able to handle DirectX10 and Shader Model 4.0 pretty well.

One of the things that drew my attention on this laptop was how old its drivers were, and I mean REALLY OLD! Its graphics card driver dates April 2007. My first thought was “lets check for new graphics driver at nVidia” but after some searching I found that many graphics drivers for laptops are provided directly by their vendor (this is because each laptop may use a modified driver version). Going to Acer driver download page I found that I was already using their latest drivers… O.o

Because nVidia has a Unified Driver Architecture (UDA) I thought it might be possible to install their latest drivers on my laptop and after some search I found the solution at laptopvideo2go. There you can find modded INF files that allows you to intall the latest nVidia drivers on your laptop. I tried their Forceware 177.35 driver mod for Windows Vista and it worked great! If you have a laptop make it a try.

The Dream-Build-Play Challenge is a worldwide challenge on game development for the Xbox 360® using the XNA Game Studio. It is one of the biggest contests on game development and has prizes totaling $75,000 USD and the opportunity to publish your game on the Xbox LIVE® Arcade.

To compete you should create an Xbox 360® game using the XNA Game Studio 2.0 and submitting it between September 1 and 23.

Another great thing is that this year they will provide a 12-month XNA Creators Club trial membership for all the participants of the contest, great isn’t it? Also, according to the XNA Team Blog we will begin to see the winner games of the Dream-Build-Play 2007 contest later this year on Xbox LIVE Arcade.

Microsoft has announced the Gamefest 2008 conference, which is its premier conference on game development. The conference will take place in Seattle, Washington, on July 22-23 and will be composed of eleven different tracks, including: Casual Games, CPU and System Programming, LIVE and Networking, Visual Arts and XNA Game Studio. If you are interested in hearing the latest news about gaming on Windows®, Xbox 360™, and ZUNE™ it is the right place to go.

Although I never had a change to go to Gamefest in the US, last year I participated of the Gamefest Brazil, which was great! I also had the opportunity to gave two XNA’s lectures there: “Creating a 3D Game with XNA” and “Creating Visual Effects for Games with Shaders and XNA”, which you can find in the Publications section.

Next Friday, I’m going to give a lecture about XNA at the CGGT (Computer Graphics and Game Development) group at my university (UFMG). In this lecture I will focus on all the great features that came out with XNA 2.0, and the new features that are coming with XNA 3.0:

• Networking with XNA
• XNA for Xbox Live Arcade
• Community Games
• Zune Support

Also, I will take advantage of this opportunity to show why managed code will become the next big thing in game development, and why I definitely agree with some statements made by Tim Sweeney’s:

• “Productivity is just as important as performance”
• “We will gladly sacrifice 10% of our performance for 10% higher productivity”
• “We never use Assembly language”

If you want to stay tuned with the CGGT group activities, you should subscribe to our e-mail list (in Portuguese).

The Mother Gaya Studio, a brazillian team, reached the finals in the Game Development category of the Imagine Cup 2008 with the game City Rain. And now, they are going to Paris for the Game Development finals. You can see the City Rain gameplay in the following video:

“With a strong environmental focus, City Rain is a game that can be used to teach the players how to build a sustentable city.”

Brazil has always performed very well in the Imagine Cup, having teams in the finals of different categories every year. Also, brazillians were more than 30% of the total registered students in the Imagine Cup 2008, with 63.757 registered students. Cool isn’t it?

You can check the other finalists of the Game Development category below:

• BAAM! Studios – Germany
• Drunk Puppy – Belgium
• ECOThink – France
• Gomz – Korea
• SiSS – Germany

Last year, I was invited by Alexandre Lobão — a great writer and programmer—to participate in the development of a XNA book. Besides me, José Antônio Leal de Farias — Microsoft MVP and founder of Sharp Games (biggest Brazilian XNA community) — also joined the team.

After some months of work, we could finally put the book together — which was published by APRESS. The book foreword was written by Amintas Neto, who is the Academic Relations Manager of Microsoft Brazil and a great enthusiast of XNA.

You can check the book summary below:

• Game Planning and Programming Basics
• 2-D Graphics, Audio, and Input Basics
• Creating Your First 2-D Game
• Improving Your First 2-D Game
• Basics of Game Networking
• Rock Rain Live!
• 3-D Game Programming Basics
• Rendering Pipeline, Shaders, and Effects
• Lights, Camera, Transformations!
• Generating a Terrain
• Skeletal Animation
• Creating a Third-Person Shooter Game
• Closing Words

You can find more information about this book at the APRESS website, where you can also download the first book chapter for free here. I would like to thank everyone who participated in the development of this book, specially the APRESS team for their great support.

Last year, when I start to work on the XNA TPS game I spend some time studying computer animation, or to be more specific, skeletal animation (also known as skinning).

My duty was to implement an efficient and flexible way to animate the game characters using skeletal animation. While some actions like walking and running are simple to reproduce and can be pre-made in a modeling tool (such as Maya), other actions like aiming a target at any position might need some kind of procedural animation, or inverse kinematics (IK).

While developing the first prototype of the XNA TPS animation library I was very concerned about performance (yeah, it was a big mistake). Because of that, the animations were stored through time frames and poses, where each time frame stores a time and a pose, and each pose stores the configuration of all the skeleton’s bones. Moreover, I decided to bake all the bones, storing their absolute configuration instead of their local configuration. This system was indeed very fast: you can find a time frame in O(logN) using a binary search and then use it. However, it used a lot of memory and wasn’t flexible!

On the second prototype I was looking for different approaches to reproduce animations, such as splitting the animation in channels (where each bone has its own channel) or using a simpler and efficient approach like the XNA’s Skinned Model Sample. At that time I implemented something similar to the XNA’s sample (deadlines don’t leave me many options).

Some weeks ago I decided to put together what I’ve learned about animations in a skeletal animation library. Then, I come up with the XNAnimation, which is a skeletal animation library for the XNA. XNAnimation 0.1 is the first version of this library and there’s still a lot of work to be done. You can download it with samples at the Projects section.