www.tombraiderforums.com

Go Back   www.tombraiderforums.com > Tomb Raider Modding > Tomb Raider Level Editor > Software Development

Reply
 
Thread Tools
Old 08-06-15, 05:07   #1501
Cochrane
Gold Contributor
 
Cochrane's Avatar
 
Join Date: Apr 2006
Location: Germany
Posts: 15,586
Default

I fixed TR3 loading right now. Will have to think a bit about the other crashes. For me, these things don't crash, but that may be because the OpenGL driver here is more forgiving of my mistakes than yours is. I have a vague idea what might be going on, gotta test that.

The issues of rooms not drawing are not caused by me.

Yes, we could totally merge Lara into one mesh with bone weights. It's not going to be trivial, but it is doable. I'm not sure how well such skinning would work if the model isn't designed for it, but if we have volunteers willing to try it out and provide the scripts, yeah, that would work.

For hair, it'll be easier, and I'm already planning to do something like that. But don't expect it in the next two hours or anything.

Edit to add: Just pushed a new commit that may solve the crashes you're seeing with the inventory and item notifier.
__________________
GŁter auf die Bahn!

Last edited by Cochrane; 08-06-15 at 05:28.
Cochrane is offline   Reply With Quote
Old 08-06-15, 08:30   #1502
Lwmte
Explorer
 
Join Date: Aug 2010
Location: Russia
Posts: 889
Default

I tested it, and unfortunately, it doesn't fix this issue, if I use nvidia 635M graphics card. However, it works if I switch to Intel integrated graphics! I will try to update ForceWare, maybe it makes a difference (but I can't say that currently I'm using outdated driver).

Another bug I noted is that some animated textures are now broken (for ex., in TR3 Jungle, waterfalls are messed up). And scrolling waterfall textures in TR4/5 are broken too (like ones in Angkor Wat).

Also, Bullet debug mode is now colored completely red, while previously it had Bullet-specific color coding (I wrote about that on Github).

Talking about "forced" skinning and looking at original TR1/2/3 Lara models, I can say that they were built in a manner to "mask" meshed nature of the model, i. e. with big overlaps, etc. The problem is, there could be a situation where you have no vertices in two linked meshes to produce realistic result - for example, Lara's butt have only three potential connection points to link her legs, while legs themselves have five. So it's much harder than I thought (however, with hair meshes it's much more simple, as hair meshes generally have 4 vertices at their potential connection points).

Last edited by Lwmte; 08-06-15 at 08:38.
Lwmte is offline   Reply With Quote
Old 08-06-15, 15:25   #1503
TeslaRus
Student
 
TeslaRus's Avatar
 
Join Date: Jan 2013
Posts: 194
Default glitches

Missed rooms is not incorrect frustum's reason; I tested last frustum manager on old (before VAO build) and all works correct + in latest commit You can see that all frustums exists and no one missed, but room mesh was missed. +I found reason of debug drawer rendering objects, that are not exists and fix it (I hope )
TeslaRus is offline   Reply With Quote
Old 08-06-15, 19:17   #1504
Cochrane
Gold Contributor
 
Cochrane's Avatar
 
Join Date: Apr 2006
Location: Germany
Posts: 15,586
Default

Hm, weird. I'll look into it.

----

I promised to explain what VAOs are all about, so here it is:

Background which you probably already know: To draw something, you need textures, shaders and vertex data. Vertex array objects are about the last part. The vertex data consists of information for each vertex, such as position, color and so on (collectively known as attributes), and of the indices that tell you in which order the vertices are drawn. Both parts are stored buffer objects, also known as vertex buffer objects in this role. Generally there's two, one for the indices and one for the attributes, but you can have more, for example putting one attribute into its own buffer (OpenTomb does that for texture coordinates for animated textures. The animated attribute is one buffer, the part that stays static is another).

(Note: You always draw from a buffer object. If you think you don't, that just means the driver is creating one for you while you're not looking, which is probably slower than using your own where you know how and when it will be used.)

All very simple, but it requires an awful lot of setup for every draw call. Before any draw call, you have to say what attributes you want and for each what format it has and where to get it from. This is annoying, and it takes time.

Vertex array objects are special objects to solve this problem. A VAO simply encapsulates all that state. Conceptually, it is simply a list, telling you what attributes are enabled, their format, the buffer they're from and so on, and also what buffer to get the indices from (some people will tell you that VAOs don't contain the indices. That's just plain wrong). Binding a VAO means the driver will do all that setup for you, and generally faster and more optimized than you can (though don't expect any huge speedups here).

That's really all there is to it. It's a way of changing this code:

Code:
glBindBuffer(GL_ARRAY_BUFFER, 10);
glEnableClientState(GL_VERTEX_ARRAY);
glVertexPointer(3, GL_FLOAT, 42, 0);
glEnableClientState(GL_NORMAL_ARRAY);
glNormalPointer(GL_FLOAT, 42, 5);

glBindBuffer(GL_ARRAY_BUFFER, 9);
glEnableClientState(GL_TEX_COORD_ARRAY);
glTexCoordPointer(2, GL_FLOAT, 8, 0);

glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, 11);

glDrawWhatever();
(example code, do not use) to this:

Code:
glBindVertexArray(12);
glDrawWhatever();
Tricky parts

- Some hardware doesn't support VAOs. On them, a different implementation is used that does all the manual annoying binding. Look at vertex_array.cpp for the details. manual_vertex_array in particular is a fully functional implementation of vertex array objects done on the client side. We still need the other version because OpenGL 3.2 Core Profile allows only that one.
- I'm no longer using glVertexPointer and friends. VAOs do support them, but Core Profile doesn't. Instead I use generic numbered attributes. The numbers (specified in enums) match to names of variables declared as attribute in the shader. That matching happens in shader_description.cpp (look for glBindAttribLocationARB).
- The extension has a weird history: It was created by Apple, so the functions had names like glBindVertexArrayAPPLE. Then it became part of OpenGL 3.0 and lost the APPLE suffix, so it was just glBindVertexArray. Then that was turned into an extension so older cards could support it. For that, they didn't add a suffix, so it stays glBindVertexArray there, too. Which is a long way of saying: There is no glBindVertexArrayARB. Never has been, never will be. Don't worry about it.
- The selected code will work both with the APPLE version of the extension and the official suffix-less one. This is probably irrelevant for non-Mac users, but doesn't hurt either.
- VAOs absolutely require that everything is in a vertex buffer, otherwise it doesn't work. Getting that done was the main task, and is the reason why VAOs are not yet used for the GUI (that's next).
- Since not everything is VAO yet, VAOs have to be unbound before drawing something that is not VAO yet. If you don't do that, the effect may be nothing at all (the VAO simply ignored), or a complete crash. Which one you get depends on your driver. Ah, trying to fix random crashes that only happen on some strangers computer a thousand miles away. Isn't graphics programming awesome?

----

In other news: I have no plans to port the Mac version OpenTomb to Apple's new graphics API Metal. That would serve no purpose whatsoever, because Metal is built to solve problems that OpenTomb doesn't have. In particular, OpenTomb doesn't spend a lot of time calling OpenGL functions (which is what Metal can reduce); it spends about 30% of its time in Character_GetHeightInfo. If you ever think performance is a problem, try optimizing that function first, either how it works or by making sure it gets called less often.

----

Edit to add: Fixed that bug with the frustums. I think a better fix might be possible, but I don't understand the frustum code well enough (for one thing, it seems that frustums should really be named portals unless I'm getting something very wrong), so this will have to do for now.
__________________
GŁter auf die Bahn!

Last edited by Cochrane; 08-06-15 at 20:33.
Cochrane is offline   Reply With Quote
Old 08-06-15, 21:12   #1505
TeslaRus
Student
 
TeslaRus's Avatar
 
Join Date: Jan 2013
Posts: 194
Default

Quote:
Originally Posted by Cochrane View Post
...Edit to add: Fixed that bug with the frustums. I think a better fix might be possible, but I don't understand the frustum code well enough (for one thing, it seems that frustums should really be named portals unless I'm getting something very wrong), so this will have to do for now.
Thanks for bug fixing and explanations: it is interesting and useful. I will spent some time to study VAO OpenGL logic. Portal like a static window between rooms, but frustum is something like clipped (by other's frustums) window with own clip planes and that clip planes depends on camera position; Clip plane contain (or based on) camera position and portal (frustum) edge.
And as I understood BSP tree this time not using polygon splitting because new, generated by splitting, vertexes can't be correctly processed by anim textures shader (polygon splitting is slow, but it absolute correctly generates every transparency scene)?
TeslaRus is offline   Reply With Quote
Old 09-06-15, 14:14   #1506
SLAYER
Archaeologist
 
SLAYER's Avatar
 
Join Date: Jun 2006
Location: Jordan
Posts: 2,101
Default

Minor issue: Bartoli's hideout (Venice) shouldn't have an ambiance track playing. deleting VENICE_autoexec.lua fixes it.
__________________
Right now, I'm losing patience
SLAYER is online now   Reply With Quote
Old 10-06-15, 03:24   #1507
Lwmte
Explorer
 
Join Date: Aug 2010
Location: Russia
Posts: 889
Default

Cochrane, thanks for such detailed explanation! To be honest, I never learned too much about OpenGL, mostly it was random reading about some fancy things like shadows, environment maps, bloom/ssao/etc., but "core" functionality always remained a mystery to me.

Quote:
Originally Posted by Cochrane View Post
Before any draw call, you have to say what attributes you want and for each what format it has and where to get it from. This is annoying, and it takes time.
Vertex array objects are special objects to solve this problem. A VAO simply encapsulates all that state.
So basically, with VAOs you can get rid of all state changes for the price of creating specific VAOs for every possible purpose in game? This seems rational.


Quote:
Originally Posted by Cochrane View Post
The extension has a weird history: It was created by Apple, so the functions had names like glBindVertexArrayAPPLE. Then it became part of OpenGL 3.0 and lost the APPLE suffix, so it was just glBindVertexArray.
You mean whole VAO was invented by Apple? Well, then it seems Russian wikipedia is wrong on that subject, as it says VAO term was coined by ATI! But whatever, as long as it's in official OpenGL specification, then it's fine!

Quote:
Originally Posted by Cochrane View Post
Ah, trying to fix random crashes that only happen on some strangers computer a thousand miles away. Isn't graphics programming awesome?
Ha! I am sure this will pay off when you come to some neat feature business after streamlining core renderer!

Quote:
Originally Posted by Cochrane View Post
In other news: I have no plans to port the Mac version OpenTomb to Apple's new graphics API Metal.
Honestly, I don't see any reason to port it to Metal, even if it needed to be ported, because Metal seems very... well, temporary, after announcement of Vulkan (which also made Mantle temporary as well). It's all in the very far future, however.


Quote:
Originally Posted by Cochrane View Post
it spends about 30% of its time in Character_GetHeightInfo. If you ever think performance is a problem, try optimizing that function first, either how it works or by making sure it gets called less often.
Now that's very surprising, I can't see anything complicated in that function, could it be Bullet's ray test who eats up all CPU there?

By the way, if we talk about profiling, I wonder how much does it take to find specific entity pointer in red-black tree array of entities? Because when I started to write entity functions, I realized that many script functions take entity ID as argument, and then just get entity pointer in C by calling World_GetEntityByID function (which in turn scans through red-black tree).

It's not so obvious that this function gets called a lot, because so far we have less than 1% of entity functions programmed, but the more entities we will program, the more obvious it will be.

I have absolutely no idea if it's right idea or not, but I thought that we may implement some kind of "cache" for World_GetEntityByID, which will remember last entity ID passed to it and corresponding entity pointer, and if next call will contain same entity ID in argument, it will simply return same pointer? It should speed it up a bit, as entity functions tend to use numerous commands in a row involving same (usually function owner's) entity ID. The problem is, every time red-black tree is rearranged (in case new entity was spawned or removed or whatever), pointer may get trashed, so this operation is risky. We may need to "reset" cache every time red-black tree rearrange happens. But I have no other ideas of optimizing it.

Quote:
Originally Posted by SLAYER View Post
Minor issue: Bartoli's hideout (Venice) shouldn't have an ambiance track playing. deleting VENICE_autoexec.lua fixes it.
Sorry, seems like leftover of experimenting with background ambiences from TR1. I have fixed it, as well as added hair spawn commands for all levels in TR2-5. Check it out!
Lwmte is offline   Reply With Quote
Old 10-06-15, 05:33   #1508
Cochrane
Gold Contributor
 
Cochrane's Avatar
 
Join Date: Apr 2006
Location: Germany
Posts: 15,586
Default

Quote:
Originally Posted by TeslaRus View Post
Thanks for bug fixing and explanations: it is interesting and useful. I will spent some time to study VAO OpenGL logic. Portal like a static window between rooms, but frustum is something like clipped (by other's frustums) window with own clip planes and that clip planes depends on camera position; Clip plane contain (or based on) camera position and portal (frustum) edge.
And as I understood BSP tree this time not using polygon splitting because new, generated by splitting, vertexes can't be correctly processed by anim textures shader (polygon splitting is slow, but it absolute correctly generates every transparency scene)?
That's true. I have no idea how to implement splitting with VAOs, so I left it out. I hope this will not cause problems. If it does, I will look at it again.

Quote:
Originally Posted by Lwmte View Post
Cochrane, thanks for such detailed explanation! To be honest, I never learned too much about OpenGL, mostly it was random reading about some fancy things like shadows, environment maps, bloom/ssao/etc., but "core" functionality always remained a mystery to me.

So basically, with VAOs you can get rid of all state changes for the price of creating specific VAOs for every possible purpose in game? This seems rational.
There somehow isn't a lot of good documentation about the core parts of OpenGL around; mostly it's about effects or horribly outdated and still mentioning glBegin.

Yes, that is exactly the point of VAOs.

Quote:
Originally Posted by Lwmte View Post
You mean whole VAO was invented by Apple? Well, then it seems Russian wikipedia is wrong on that subject, as it says VAO term was coined by ATI! But whatever, as long as it's in official OpenGL specification, then it's fine!
I think the ATI version was different somehow, and for the standard they picked the Apple one. But I don't know any of the details, I'm afraid.

Quote:
Originally Posted by Lwmte View Post
Ha! I am sure this will pay off when you come to some neat feature business after streamlining core renderer!
It already did in some places, like with the room shader, per-pixel lighting and so on. So I think this will be fun.

Quote:
Originally Posted by Lwmte View Post
Honestly, I don't see any reason to port it to Metal, even if it needed to be ported, because Metal seems very... well, temporary, after announcement of Vulkan (which also made Mantle temporary as well). It's all in the very far future, however.
Yes, that is probably true. I think Metal is interesting because it's the only next-gen API with official documentation out in the open, so you can read it sort of as a preview of what Vulkan might be like. But other than that, I don't think it has a bright long-term future ahead of it.

Quote:
Originally Posted by Lwmte View Post
Now that's very surprising, I can't see anything complicated in that function, could it be Bullet's ray test who eats up all CPU there?
Yes. The function spends all its time in Bullet.

Quote:
Originally Posted by Lwmte View Post
By the way, if we talk about profiling, I wonder how much does it take to find specific entity pointer in red-black tree array of entities? Because when I started to write entity functions, I realized that many script functions take entity ID as argument, and then just get entity pointer in C by calling World_GetEntityByID function (which in turn scans through red-black tree).

It's not so obvious that this function gets called a lot, because so far we have less than 1% of entity functions programmed, but the more entities we will program, the more obvious it will be.

I have absolutely no idea if it's right idea or not, but I thought that we may implement some kind of "cache" for World_GetEntityByID, which will remember last entity ID passed to it and corresponding entity pointer, and if next call will contain same entity ID in argument, it will simply return same pointer? It should speed it up a bit, as entity functions tend to use numerous commands in a row involving same (usually function owner's) entity ID. The problem is, every time red-black tree is rearranged (in case new entity was spawned or removed or whatever), pointer may get trashed, so this operation is risky. We may need to "reset" cache every time red-black tree rearrange happens. But I have no other ideas of optimizing it.
So far the method hasn't shown up in any noticeable way when profiling. I'll be very surprised if it ever does, because it's a good implementation of a good algorithm. If it ever does become an issue, I'd probably try with a hashtable first, but there's no point thinking about that now.

If you care about performance, here's a trace I did just now, of Lara sliding down in Jungle.TR2:
https://gist.github.com/cochrane/d87b49fe3d6f73186a09

It's kind of gigantic and difficult to parse, contains a lot of boring stuff (such as the details of Apple's OpenGL drivers) and you have to know what to look for (for example some functions show up at many different places). But RB_SearchNode shows up for a total of three samples of all three seconds, meaning it took about three milliseconds in total. We're actually spending more time doing matrix multiplication.

In general, if you're interested in performance, I suggest you find yourself a decent performance profiling tool. I'm using one supplied by Apple. Both AMD and Intel have similar (possibly better) tools for Windows and Linux, and at least the AMD one (CodeXL) is free. Without such a tool, fixing performance issues is basically impossible, because your instincts about where to look will always be wrong. Every single time. Trust me on this.
__________________
GŁter auf die Bahn!

Last edited by Cochrane; 10-06-15 at 05:38.
Cochrane is offline   Reply With Quote
Old 12-06-15, 00:00   #1509
Lwmte
Explorer
 
Join Date: Aug 2010
Location: Russia
Posts: 889
Default

Some news on ragdolls! I have optimized script and code itself, which made ragdoll look much more natural. Also I've made Lara's own velocity transfer to ragdoll, so if you set her to ragdoll state in mid-air jump, then she will preserve her speed.

However, there are two bugs to fix: sometimes Lara "tunnels" through level geometry with some of her bodies and gets stuck at wall or floor (this is common bug in contemporary games with ragdolls, when a ragdoll stucks into closed door or something like this). Second bug is, ragdoll bodies are not colliding with each other, which is kinda strange, since collision mask is set properly).

Here are demo videos with these two bugs: one, two.

By the way, it's so funny to see classic Lara in ragdoll state (even though actual mesh rendering is still not implemented), that I did random video with infinitely falling Lara with some trippy background and music!


Quote:
Originally Posted by Cochrane View Post
There somehow isn't a lot of good documentation about the core parts of OpenGL around; mostly it's about effects or horribly outdated and still mentioning glBegin.
Funny that something similar is happening with Bullet documentation, actually there is NO proper documentation for Bullet, and examples are outdated. It gave me LOTS of pain with ragdolls, as certain parameters produce completely different results with only slight change of value (like cone twist constraint relaxation factor, value of 0.7 is stable, while 0.6 produce total mess, 0.0 completely breaks joint limits, etc.).

So I wonder, how all these "professional" and "game industry" folks out there write their code, if there is no proper documentation? Everybody just learns it by trial and error?

Quote:
Originally Posted by Cochrane View Post
It already did in some places, like with the room shader, per-pixel lighting and so on. So I think this will be fun.
Yes, can't wait for more such stuff! Lighting already looks great, although it needs some fixing, I can imagine how good it will look like with some simple modern day tricks.

Quote:
Originally Posted by Cochrane View Post
Yes. The function spends all its time in Bullet.
If it's actually ray test function which acts as a bottleneck here, it's really weird - shouldn't ray test be simple as hell? But at least we know the bad guy now!

Quote:
Originally Posted by Cochrane View Post
RB_SearchNode shows up for a total of three samples of all three seconds, meaning it took about three milliseconds in total. We're actually spending more time doing matrix multiplication.
Then I won't worry about it at all, when it comes to more entity programming - actually I asked TeslaRus about this search "cache" optimizing, and he already did that.

Last edited by Lwmte; 12-06-15 at 09:45.
Lwmte is offline   Reply With Quote
Old 12-06-15, 22:11   #1510
godmodder
Explorer
 
godmodder's Avatar
 
Join Date: Feb 2006
Location: Belgium
Posts: 532
Default

This project is coming along nicely. Maybe I should join you guys

Should you need a level editor: feel free to use the code of my level editor from IrisEngine on Sourceforge. Of course, I could always help with that.
Wouldn't that be cool, having a level editor for OpenTomb?
__________________
C++ is the only place where friends get to see your private members.

Last edited by godmodder; 12-06-15 at 22:12.
godmodder is offline   Reply With Quote
Reply

Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT. The time now is 08:07.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.