www.tombraiderforums.com  

Go Back   www.tombraiderforums.com > Tomb Raider Modding > Tomb Raider Modding > XNALara

Reply
 
Thread Tools
Old 11-09-12, 15:20   #21
Hiddenus
Explorer
 
Join Date: Feb 2010
Location: southern Poland
Posts: 643
Default

From what I see in your program user will be able to use bones named unused (from which few are quite useful in keeping arms twists look correctly and are not used in XNALara )
That's great. But I can't use it anyway
Hiddenus is offline   Reply With Quote
Old 11-09-12, 15:23   #22
Cochrane
Golden
 
Cochrane's Avatar
 
Join Date: Apr 2006
Location: Germany
Posts: 16,723
Default

My current plan is that you can use the preferences to set whether you can see unused bones or not. Hiding them can be a good idea for novice users, but in the end, the more power the users have, the better.
__________________
Güter auf die Bahn!
Cochrane is offline   Reply With Quote
Old 11-09-12, 15:27   #23
TRDaz
Tomb Raider
 
TRDaz's Avatar
 
Join Date: May 2011
Posts: 10,458
Default

I think this is really great! Even though I wont be using it (dont have a mac, and wont buy one xD), I still think that you are doing a great job Cochrane
TRDaz is offline   Reply With Quote
Old 12-09-12, 18:34   #24
Cochrane
Golden
 
Cochrane's Avatar
 
Join Date: Apr 2006
Location: Germany
Posts: 16,723
Default

After my many complaints about Microsoft in the other thread, perhaps for fairness's sake, I should also list my complaints about Apple here. And if not for fairness's sake, then just because I was annoyed and want to get this off my chest.


It starts with SceneKit. SceneKit is a new framework in Mac OS X for 3D app development, a very simple, high-level API (definitely easier than XNA). You give it your models, tell it where they are, and it figures out the rest for itself. I know enough OpenGL to not need this, but I also know enough OpenGL to know how annoying it can be, so I wanted to try it for another project and maybe XNALara.

The problem is that SceneKit works well if you have Collada files, and support for anything else is very limited. You can load in your own geometry, but if you want to have more complicated things, God help you. For example, I decided I needed my own shaders. That is possible, with the SCNProgram class and the associated SCNProgramDelegate. There are just a few questions:

- Do my shaders have to be GLSL 1.20 or GLSL 1.50? The two versions are very different in a lot of respects, and most importantly, not compatible (so the name 1.5 was chosen badly, but if I start complaining about OpenGL API design, we'll be here until tomorrow.)
- How do I set textures for the shader to use? Do I have to load them myself? If yes, can I assume that once I've loaded them, they're always available, or do I have to load them once per context?
- When setting uniform values, what mesh am I setting them for? Oh, that's secret? Well, thank you.

These are not some bits and pieces; these are important key factors, and Apple doesn't tell me them. There is also no sample code using SCNProgram. So in short: SceneKit is unusable for actual 3D rendering. (That reminds me, I need to file a bug report with Apple about that. Maybe they'll notice and write real documentation for 10.9).


Next up: What is wrong with this vertex shader?
Code:
#version 150

uniform mat4 matrices[25];
uint index1;

void main()
{
	mat4 matrix = matrices[index1];
}
Well, it doesn't do anything, but that's not the point (this is just a test case I wrote for the bug report to Apple. rdar://12228253 ). If you try to compile it in your program, the compiler crashes, and your entire program with it (In OpenGL, you have to compile shaders at run time, not before). This is not due to a syntax error; if I add one, that will get reported correctly. After a lot of trial and error, I found it: index1 should be int, not uint, because with uint, it crashes.


Finally, see https://developer.apple.com/graphics.../capabilities/ (a very useful table, and I can't thank the Apple OpenGL engineer who developed it enough). Below the OpenGL and GLSL version, you see the extensions supported by those renderers, and all of them are links. Click on them to get to the specification. That is, all of them except APPLE_container_object_shareable. There is no other documentation for it either. If you Google search for the term and for GL_APPLE_container_object_shareable (that's the full name; it's common in OpenGL to drop the GL prefix if everyone knows that it should be there), all you'll find is that document, a document explaining that this was added to a header (but not why), a few people who post OpenGL reports and… me, on Twitter, asking if anyone knows anything about it. No, I never got a reply on that one.

Why does it matter? You need to have one OpenGL context per window, but by default, each OpenGL context has its own data and state. You can make two contexts share their data, though, which is essential for my support of multiple render windows here. But in their infinite wisdom, the guys who make OpenGL have decided that you can't share container objects, which are data objects that contain other ones - the most important being Vertex Array Objects. That is incredibly annoying. So an extension with a name saying that container objects might be shareable, under unspecified circumstances, sounds as if it could be an amazing help (and at the same time a fresh breath of common sense). But what do I have to do to share objects? Enable it once? Enable it per object? Are there any special restrictions?

Well, in the end I just went ahead and tried, and it turns out that sharing happens for all container objects automatically, whether you enable it or not. I.e. the way any reasonable OpenGL developer would expect things to happen until they read the specification and went WTF. So that's awesome. Why isn't there a single line of text anywhere explaining this? (This one was filed as documentation bug rdar://12260745 ).


I've encountered other issues, including one related to header path includes ( rdar://12227623 , closed as duplicate) and NSTreeController with an outline view… after looking at the documentation, I decided to just roll my own data source. But all those require way more explanation than I want to write right now.
__________________
Güter auf die Bahn!
Cochrane is offline   Reply With Quote
Old 13-09-12, 07:00   #25
XNAaraL
Professor
 
XNAaraL's Avatar
 
Join Date: Apr 2009
Location: The worthwhile problems are the U can really solve, the ones U can really contribute something to
Posts: 3,207
Default

That is incredibly annoying!

BTW: XNA (not XNALara) is not complicated.
You can load a model (.fbx, .x ...) with this simple code:
Code:
// Set the 3D model to draw.
Model myModel;

// The aspect ratio determines how to scale 3d to 2d projection.
float aspectRatio;

protected override void LoadContent() {
    // Create a new SpriteBatch, which can be used to draw textures.
    spriteBatch = new SpriteBatch(GraphicsDevice);

    myModel = Content.Load<Model>("Models\\p1_wedge");
    aspectRatio = graphics.GraphicsDevice.Viewport.AspectRatio;
}
and you can display the model with this code
Code:
// Set the position of the model in world space, and set the rotation.
Vector3 modelPosition = Vector3.Zero;
float modelRotation = 0.0f;

// Set the position of the camera in world space, for our view matrix.
Vector3 cameraPosition = new Vector3(0.0f, 50.0f, 5000.0f);

protected override void Draw(GameTime gameTime) {
    graphics.GraphicsDevice.Clear(Color.CornflowerBlue);

    // Copy any parent transforms.
    Matrix[] transforms = new Matrix[myModel.Bones.Count];
    myModel.CopyAbsoluteBoneTransformsTo(transforms);

    // Draw the model. A model can have multiple meshes, so loop.
    foreach (ModelMesh mesh in myModel.Meshes) {
        // This is where the mesh orientation is set, as well 
        // as our camera and projection.
        foreach (BasicEffect effect in mesh.Effects) {
            effect.EnableDefaultLighting();
            effect.World = transforms[mesh.ParentBone.Index] * 
                Matrix.CreateRotationY(modelRotation)
                * Matrix.CreateTranslation(modelPosition);
            effect.View = Matrix.CreateLookAt(cameraPosition, 
                Vector3.Zero, Vector3.Up);
            effect.Projection = Matrix.CreatePerspectiveFieldOfView(
                MathHelper.ToRadians(45.0f), aspectRatio, 
                1.0f, 10000.0f);
        }
        // Draw the mesh, using the effects set above.
        mesh.Draw();
    }
    base.Draw(gameTime);
}
The input handling:
Code:
protected override void Update(GameTime gameTime) {
    // Allows the game to exit
    if (GamePad.GetState(PlayerIndex.One).Buttons.Back == 
        ButtonState.Pressed) {
             this.Exit();
    }
    modelRotation += (float)gameTime.ElapsedGameTime.TotalMilliseconds *
        MathHelper.ToRadians(0.1f);

    base.Update(gameTime);
}

EDIT:
Render the image as .PNG or .JPG (or any other format)
Code:
private void SaveImage(string fileName) {
    //force a frame to be drawn (otherwise back buffer is empty)
    game.GameDraw(new GameTime());

    Microsoft.Xna.Framework.Graphics.ResolveTexture2D resolveTarget;
    Microsoft.Xna.Framework.Graphics.SurfaceFormat format = game.GraphicsDevice.PresentationParameters.BackBufferFormat;
    Microsoft.Xna.Framework.Rectangle rec = game.Window.ClientBounds;
    resolveTarget = new Microsoft.Xna.Framework.Graphics.ResolveTexture2D(game.GraphicsDevice, rec.Width, rec.Height, 1, format);

    game.GraphicsDevice.ResolveBackBuffer(resolveTarget);

    string filenameLower = fileName.ToLower();
    if (filenameLower.EndsWith(".png")) {
        resolveTarget.Save(fileName,
            Microsoft.Xna.Framework.Graphics.ImageFileFormat.Png);
    }
    if (filenameLower.EndsWith(".jpg")) {
        resolveTarget.Save(fileName,
            Microsoft.Xna.Framework.Graphics.ImageFileFormat.Jpg);
    }

    resolveTarget.Dispose();
}
There exist many programs similar to XNALara:
Kane's Model Viewer for XNA
XNA Model Viewer
...


All are written with XNA 4.x and with the .ANX framework, you can use this application on MS Windows/DirectX9/10 or 11, MacOs opengl, Linux ... using the virtuell render system http://mitohnehaare.de/2011/11/29/ei...work/#more-988
Quote:
The ANX.Framework is a framework which is source compatible with Microsoft's XNA framework 4.0. Source compatibility means, that you can "translate" a XNA game project to be a ANX game project by simply replacing all namespaces from Microsoft.XNA.Framework to ANX.Framework. The advantage of ANX is simply that you are able to swap the RenderSystem, the InputSystem and the AudioSystem. By swapping this systems you are no more limited to run your game using DirectX9 which XNA is using. ANX comes with a DirectX10 RenderSystem as a default. A DirectX 11, DirectX 11.1 and a OpenGL 3 RenderSystem is currently in development. This will make it possible to run your games on Linux and other plattforms which are supported by OpenGL etc. simply by swapping the namespaces.
__________________
Link removed. - Why ? google, google “Google is your friend!”

Last edited by XNAaraL; 13-09-12 at 07:55.
XNAaraL is offline   Reply With Quote
Old 13-09-12, 19:50   #26
Cochrane
Golden
 
Cochrane's Avatar
 
Join Date: Apr 2006
Location: Germany
Posts: 16,723
Default

Oh, I never meant to imply that XNA is hard (although the way it's used in XNALara, without many of its higher-level facilities, is almost the same as OpenGL in terms of difficulty). My point was rather that SceneKit is even easier. Just useless for any actual work.

As for ANX, I think you've mentioned it before. But personally, I don't like Windows software that has been ported to Mac via Mono and the like. It never looks and feels just perfect. Writing everything yourself is still the way to go.



In terms of code updates, I got OBJ support. That was incredibly frustrating. Over my life, I've written several OBJ parsers, and am now reusing the latest one - which has the side effect of adding the first C++ into my code. Oh well, nothing wrong with C++ as long as one uses it like C with classes and container objects. Still, I needed to really update my code. Now I support multiple material library files (XNALara doesn't do that), negative offsets (XNALara always did support that, but I generally didn't include this feature) and so on. Still, there are OBJ files that won't load in my program, for example because they hardcode full Windows paths to mtllib files. There is only so much bug compatibility I'm willing to have with XNALara, and I'd rather have support for for most non-XNALara obj files than the very few custom objects specifically for XNALara that are still available as OBJ. And let me tell you this: I have way better OBJ support now than XNALara. With real materials and such.

In the long term (but not the first version), I hope to implement COLLADA support. That's finally a format that makes sense and is properly documented. It also includes things like bone weights.

I've also started work on the export renderer. That should not be hard, in theory, but you know what they say about famous last words. I'll also add localization support via the Wil Shipley method, so it should be really easy to provide new languages for the application interface (I plan to offer English and German by default). And there are some other plans I have that I'm really excited about, but I'd like to keep that a surprise for the time being…
__________________
Güter auf die Bahn!

Last edited by Cochrane; 13-09-12 at 19:52.
Cochrane is offline   Reply With Quote
Old 14-09-12, 09:59   #27
XNAaraL
Professor
 
XNAaraL's Avatar
 
Join Date: Apr 2009
Location: The worthwhile problems are the U can really solve, the ones U can really contribute something to
Posts: 3,207
Default

Sounds good.

About COLLADA support: I do not like it. Collada makes sense and is properly documented, but it offers many possibilities and there is a lot of work to implement them all. There are too many models, which are rejected by many tools as faulty. The XNALara models were exported as .dae first. And Blender could not loaded this files correctly. The armature works wrong. Therefore XNALara was developed.
Quote:
Originally Posted by Dusan View Post
The Collada/DAE files won't work in Blender. The importer script keeps throwing errors no matter what I try.

But as I wrote in one of my previous posts, TRU bones seem to be incompatible with Blender bones anyways. That's why I started looking for another 3D program in the first place.
I'll contact developers of Blender to find out whether this incompatibility is a bug or a feature...
__________________
Link removed. - Why ? google, google “Google is your friend!”
XNAaraL is offline   Reply With Quote
Old 14-09-12, 11:18   #28
Cochrane
Golden
 
Cochrane's Avatar
 
Join Date: Apr 2006
Location: Germany
Posts: 16,723
Default

I'm not yet certain about Collada support; I've tried reading the specification and got stuck. But the skinning support in there is perfectly compatible with what XNALara uses if you do a tiny bit of maths. Specifically, if you look at page 4-8 of the specification and compare it with CalcAbsTransform in Armature.cs, you'll see that XNALara has:

v*BSM is what is stored in the file
IBMi is bone.invMove
JM[i] is bone.absTransform

Note that both the COLLADA specification and the XNALara code use column vectors and thus multiply the wrong way around. But it's the same for both, so in this context, it makes no difference.

I'm not sure what problem Dusan had with Blender, or whether Blender uses a different model for skinning than COLLADA does, but COLLADA supports what XNALara does.

Of course, the variation in the files is the major issue. I'll have to take a good look at that; I'll probably sift through the official conformance tests to find out how difficult it is to catch them all.

But what alternative is there to COLLADA? OBJ? That stopped being a good idea twenty years ago. It is nowhere near expressive enough for the needs of modern 3D applications. And all the proprietary, closed formats are, well, proprietary and closed. COLLADA is by no means my dream format, but it's the closest thing to a standard there is.

----

Edit to add: This isn't fun without a screenshot. I've now added export of images. There are still some bugs with it, but the general principle works. And the images I can generate are large. How large? I've had no trouble with more than 100 Megapixels.

https://raw.github.com/cochrane/GLLa...endertest.jpeg

Warning: This image is 3.7 MB in size and 12810 x 8780 pixels, and it seems to completely break the JPEG rendering in Safari. It may help to download it.

Of course, this resolution is way too large to be practical. But it's fun! And it's also fast; I can render such an image in less than five seconds. There are two main tricks to that: Tile size and multithreading.

First of all, you can't render an image this size on a graphics card. It's just too big. So what you do is you divide it into tiles that can be rendered, and render each at a time. This opens the question of how big a tile should be. XNALara, for example, uses 256x256 pixels, so a tile doesn't use too much VRAM (and of course smaller ones if something is left over).

I use, depending on the graphics card, 4096x4096 or 8192x8192 tiles (again, this is the maximum). At that size, even for the biggest files, I need to draw only four times, which outweighs any disadvantages of the big format. Once I get more complicated scenes, it is possible that I may need to reduce the size, but not at the moment.

The second part is getting the images from the graphics card back into the main memory. This sounds simple, but the CPU has to wait for the graphics card to be finished, and a general rule in graphics programming is that you don't want to let the CPU wait for the GPU or vice versa. I simply use two threads: One does the drawing, another one waits for the GPU to be finished with its result and downloads them. That other threads has to wait, but because the first can continue anyway, there is no problem with that.

The funny thing is that multithreading with OpenGL is actually fairly easy, but the majority of people don't know it.

Now most of the time is spent on JPEG compression. I'm thinking about moving that to a background thread, too.

----

Second edit to add: I've decided that I won't support multisampling for this rendering to image. Instead, users can simply render the image larger and then scale it down with Photoshop. The results won't be worse, performance may be a bit worse but it's still acceptable in all cases, and the image quality will be the same - and I won't have to spend hours trying to figure out what the problem is now.

----

Things from saturday:

In another thread, I was asked about Linux support, and I thought for general reference, I'd add my reply here again, too. Also Windows, while I'm at it.

The short answer is that this program will most likely never run on Linux or Windows. Sorry.

The long answer: I'm not going to port it, but since it's open source (remember, you can download the code already), anyone could give it a try. But it wouldn't be fun for them.

This app uses Cocoa, the framework that is used for application development on Mac OS X (for you Windows guys, you could compare it to .NET, but Cocoa is always installed and not a separate component). It uses it for absolutely all features, so it is not possible to replace Cocoa with anything else.

There is a version of Cocoa for Linux, called GNUstep. It also has some Windows support, and for Windows only, there's also Cocotron. Both are interesting projects, but they don't support most of Cocoa's newer features, and this app makes very heavy use of the newest features.

So anyone porting it would either have to remove the use of new features from my app, or add those new features to GNUstep or Cocotron. Either option is an insane amount of work. I strongly recommend against trying it.

If someone really wants a Linux version of XNALara, I'd recommend looking at oglLara. Alternatively, someone could take any of the frameworks available on Linux, like Qt, Gtk, Tk, Motif (for those rocking it old school), perhaps even Java, and write their own app, using my source code and XNALara's source code as reference, and perhaps copy some pieces of code where it makes sense (like my shaders, or the way I've handled the hardcoded model parameters). If anyone (who knows what they're doing) wants to give it a try, I'll be happy to help as far as I can.

------

In other news, I now support all but two of the original models. One of the most annoying parts of XNALara is how almost everything is hardcoded, at least for non-generic_item models. Somehow, I have to include that information in my program, otherwise the models won't load. My way is to have per-model configuration files. These files can be easily modified, they're a standard XML format so other tools can read them (such as a hypothetical TuxLara), and in the future, it will be possible for modelers and porters and modders to provide their own configuration files, for much, much more freedom than is currently possible in XNALara.

That left me only to write all these configuration files. I had tried writing them by hand, but they're just too many, so in the end I wrote a program that parses the XNALara code and automatically generates me the necessary files. This tool is also included in the source code now, although I'm fairly certain that nobody will ever need to run it again (unless XNALara adds more models that are supported by default).

Two models are still lacking, because their code is a bit more complicated (CroftManorBack and SkydomeThailand). I'll add these per hand later.
__________________
Güter auf die Bahn!

Last edited by Cochrane; 15-09-12 at 17:34. Reason: Originally included that image in full resolution in this post. That was not my idea, nor very useful.
Cochrane is offline   Reply With Quote
Old 16-09-12, 15:18   #29
Cochrane
Golden
 
Cochrane's Avatar
 
Join Date: Apr 2006
Location: Germany
Posts: 16,723
Default

Sorry for the double post, but the last one was getting a bit long (by the way, if you're interested, there may be stuff in there you didn't read yet).

I'm basically done. Well, not completely, of course, I'm still miles away from feature parity with XNALara (although I do have a lot stuff that XNALara doesn't). But all the features that I wanted in the first preview release are implemented as of committ 98f8c69fac. I could make a release now. If any of you are interested, have a Mac and Xcode, you can give it a whirl. But personally, I don't feel that confident yet, so my next step will be, (to use a term all german readers know well) a stress test.

Specifically, I'll be trying to create something fun with the program (and Photoshop), and see what bugs I hit, and what things annoy me so much that I want to change them right away. I have full confidence that there will be lots. After that, it's on to the public preview release, after which I'll try to find what bugs need fixing first, and what features should be implemented next, based on user feedback.
__________________
Güter auf die Bahn!
Cochrane is offline   Reply With Quote
Old 16-09-12, 23:29   #30
Lyle Croft
Relic Hunter
 
Lyle Croft's Avatar
 
Join Date: Jun 2011
Posts: 5,485
Default

i have to upgrade now? T.T i'm a different os x
__________________
Sitting over here starin in your face
Lyle Croft is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT. The time now is 17:56.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.