Tomb Raider Forums  

Go Back   Tomb Raider Forums > Tomb Raider Level Editor and Modding > Tomb Raider Level Editor > Software Development

Reply
 
Thread Tools
Old 17-09-17, 21:58   #1
zdimension
Member
 
zdimension's Avatar
 
Joined: Feb 2016
Posts: 68
Default Tomb Raider 5 Decompilation Project

Tomb Raider 5 Decompilation Project

TL;DR AT THE END

The goal of this project is to decompile entirely Tomb Raider 5 to C/C++ code. There are many reasons for that; first, having the source code (or a functionally equivalent code) of the game allows modding it much more extensively than what the existing patches (TREP, TRNG, etc.) can do, because where those patches will inject code, and patch parts of the memory using hacks and workarounds, here, we can simply modify the code and recompile. Second, it is almost certain that Square Enix / Crystal Dynamics are not in possession of the code. We know that the source code of TR1 and TR2 (used to make the mobile versions) have been given to SE by former Core Design employees, but we are almost sure that the source code of the other games have been lost. This project allows us to preserve a part of history that would be lost otherwise.

How?

A lot of information required to decompile the game comes from the symbol files contained in the leaked TR4-5 PSX SDK. A symbol file is a file contains a list of memory address, and the name of "what is at that address". When you load the executable binary file of the game, you only get the raw binary code, without any function or variable name. A symbol file allows us to put back the original function and variable names, hence making the reverse engineering process much easier.
The symbol files contained in the leaked SDK are of a special format that contains much more information than simply the names, and thanks to Steffen Ohrendorf (OpenTomb & EdisonEngine developer) we have been able to get even more than names: function definitions, variable types and even the original file names.
Thus, our current source tree is very similar (for the PSX version, it's identical) to the original one.

Source tree

The repository is divided in multiple folders (exactly like the original source code):
  • GAME = main game code, that is common between platforms
  • SPEC_* = platform-specific code
    • SPEC_PC = PC version
    • SPEC_PSX = PSX version (requires the Psy-Q SDK and toolchain)
    • SPEC_PSXPC = PSX version that runs on PC (used for debugging purposes)
    • (maybe one day) SPEC_CE = Dreamcast

Current state of the project

As of today (2017-09-17), all three trees compile without errors. The PSX and PSXPC run up to the main game loop, where they just loop and do nothing.
The PSX version can currently go up to the first loading screen (with the spinning CD icon), but a lot of work still needs to be done since on PSX, all the loading code is located in a separate file that is dynamically injected at runtime.
The PC version can currently go up to the main game thread and the level loading part (title level) but the level loading code is not finished yet, so the game crashes a little bit, unless you comment out the main thread code, then the game just loops indefinitely like its PSX counterpart.

Who?
  • Gh0stBlade: main developer, now works mainly on PSX version
  • zdimension: main developer of PC version

Legal stuff

All code here comes from decompilation of publicly available (this includes the leaked SDK) binaries and ressources of Tomb Raider 5. Additional informations about the game's internal mechanisms come from the TRosettaStone 3 project. Square Enix plz don't sue us. We're basically retrieveing the game's source for free whereas your developers would want money.

TL;DR

We're decompiling TR5 so that you can mod it more easily (and that we, developers, can understand it better and make better tools).
It is not finished at all.
There is still a lot of work to do.
It may take months, it may take years.

Want to help?

Currently we're searching for people that know C and C++, and that can use IDA Pro or similar reverse engineering tools. Knowledge of x86 and/or MIPS assembly is a plus.

Shut up and take my money!

GitHub:
Binaries:
None

Screenshots:
click here

Last edited by zdimension; 20-02-19 at 17:28.
zdimension is offline   Reply With Quote
Old 18-09-17, 00:18   #2
HD86
Member
 
Joined: Apr 2014
Posts: 260
Default

I am going to say my opinion and I know many people will not agree. If you want to mod the game extensively, you should just build a new game or play one of the recent Tomb Raider releases.

I personally like the classic Tomb Raider game as it is and I don't want to see it modified extensively, because that will most likely ruin it. It will become a different game. I can't imagine that you are going to be modest in modifying it after you manage to decompile it.
HD86 is offline   Reply With Quote
Old 18-09-17, 01:44   #3
Boobandie
Member
 
Joined: Jul 2012
Posts: 4,286
Default

Why rag on the project? No matter what we do you can play Chronicles as is. This is more for the benefit of fan games and a historical record than it is for just modifying TR5. Though once it is decompiled it could lead to mobile and IOS ports of TR5, or a linux version/better functionality on new operating systems.

Personally I can't wait to see where this goes, the TR5 engine is a lot better than TR4 and I've always been annoyed that it wasn't the version included as the Level Editor.
Boobandie is offline   Reply With Quote
Old 18-09-17, 06:01   #4
A_De
Member
 
A_De's Avatar
 
Joined: Jan 2012
Posts: 1,320
Default

Quote:
Originally Posted by HD86 View Post
I am going to say my opinion and I know many people will not agree. If you want to mod the game extensively, you should just build a new game or play one of the recent Tomb Raider releases.

I personally like the classic Tomb Raider game as it is and I don't want to see it modified extensively, because that will most likely ruin it. It will become a different game. I can't imagine that you are going to be modest in modifying it after you manage to decompile it.
You were right
Just imagine now, how much the game will be ruined by TRNG's plugins, for example

To be serious: my best wishes to the decompiling team. I really want to see the "different game" some day
A_De is offline   Reply With Quote
Old 18-09-17, 07:41   #5
Caesum
Member
 
Caesum's Avatar
 
Joined: Aug 2008
Posts: 12,070
Default

Quote:
Originally Posted by HD86 View Post
If you want to mod the game extensively, you should just build a new game or play one of the recent Tomb Raider releases.
That is kinda silly tbh. Using that logic we should suggest barracuda415 to stop and remove his TombATI project, because if one wants to mod the game he can just play TR Anniversary, no? I mean it doesn't even make sense.

Recompiling an engine gives a variety of new possibilities. It's not only about modding the existing game, it's making it possible to create an entire new one based on the source of TR5, or modifying TRLE in order to give it the features of TR5 engine. TR5 has been heavily modified, especially graphically. The main difference is that integers were replaced by floating points, making the entire game less shaky and much better looking on PC version.

Even your beloved TR5 (and mine btw) can benefit greatly if we can manage to reclaim the source code. TR5 is a Windows 98 game which doesn't work well on modern hardware. Heck, at this point TR1 with TombATI is more compatible than TR2-5. The source gives the possibility to modify the executable in order to make it work on modern PCs. No dirty hacks needed, no wrappers, no emulation. Modifying the game can also make it possible to restore PSX-exclusive effects like echo, footprints, dynamic Dreamcast shadows, special effects. It can also allow rewriting the rendering engine to something more modern than DirectX 5 and thus more compatible.

Also, understanding the engine can make it possible to modify enough to be able to make new adventures in it. Maybe even one would be able to replace the old and quirky TRLE engine with it. The source makes it possible to add new features without dirty hacks that do more harm than good (TRNG). We already have the official Room Editor for TR5 along with several tools and PSX devkit.

At last, decompiling the source will give new information for programmers to improve and extend their custom engines, eg. OpenTOMB and OpenLARA.
Caesum is offline   Reply With Quote
Old 18-09-17, 07:43   #6
Dustie
Member
 
Dustie's Avatar
 
Joined: Apr 2005
Posts: 9,208
Default

Quote:
Originally Posted by HD86 View Post
I am going to say my opinion and I know many people will not agree. If you want to mod the game extensively, you should just build a new game or play one of the recent Tomb Raider releases.

I personally like the classic Tomb Raider game as it is and I don't want to see it modified extensively, because that will most likely ruin it. It will become a different game. I can't imagine that you are going to be modest in modifying it after you manage to decompile it.
You are right about building a new game, some of the builders would definitely love to see a completely new engine at their disposal and with a decompiled code the path to making it happen is pretty much open. A lot the ideas we have oftentimes make us conclude that we might as well make ourselves a new game if we want to make them happen.

But I think the core (sic!) purpose here is to be able to patch certain things up, increase compatibility with newer systems and possibly introduce certain graphical tweaks and extensions, which would add up to the classic experience, not ruin it. I'm convinced such efforts come from passion to the classic Tomb Raider, not the hate for it.
Dustie is offline   Reply With Quote
Old 18-09-17, 10:14   #7
Lwmte
Member
 
Lwmte's Avatar
 
Joined: Aug 2010
Posts: 1,810
Default

I can only welcome and applause for this effort. And I can't understand people who possibly say that they are happy with current state of the game, and this is why:

1. As already said by Caesum, TR5 is still Win9x application. It's only a matter of time before we can't use it with modern hardware. What's the point of having broken binary which only works through virtual machine or with quirky workarounds like compatibility modes or 16-bit color depth (say hi to winroomedit)?

2. TR5 is the less known engine in old-school TR engine family. File formats are less known, internal processing is less known, lots of stuff altered, replaced, lots of workarounds were written, and so on. Decompiling it will clarify many things that are currently vaguely known.

3. As we already have TR1 and TR2 decompilation projects in progress, having TR5 decompilation is like pinning down whole engine family from both sides - that is, if you have decompiled TR1 or TR2 and decompiled TR5, it means that you (sort of) decompiled TR3 and TR4 as well Also, as said by Caesum, it will be a great benefit for developing OT/OL engines.

4. Don't forget about some stupid TR5 bugs, especially in final section. You will finally get the chance to fix them.

All in all, decompiling original engine always foreshadow great changes. Updated compatibility, graphics, ability to fork "vanilla" engine into new one, etc. etc. But decompiling is the hardest thing, it's really easier to write engine from scratch than decompiling existing one. Fortunately, from what I see, progress is quite fast. So my best wishes to the team!
Lwmte is offline   Reply With Quote
Old 18-09-17, 14:16   #8
HD86
Member
 
Joined: Apr 2014
Posts: 260
Default

The current TR game is what makes this community exists. If the source code becomes available, we will have a plethora of new TR games as many people will try to build their own versions of the game. The community will splinter because many will be playing different games.

Anyway, I am of course not against upgrading the game so that it remains compatible with new systems. Technical improvements are welcome as long as they don't change the gameplay.

Last edited by HD86; 18-09-17 at 15:09.
HD86 is offline   Reply With Quote
Old 18-09-17, 14:19   #9
Zebra
Member
 
Zebra's Avatar
 
Joined: Jun 2007
Posts: 26,911
Default

^How is that differernt to what's been happening with the official games for years? People have long been discussing whether LAU or the reboot games (or AoD, for that matter) can be called Tomb Raider so that's really nothing new and I don't think it's much of a problem. People will just keep using their own personal definitions of what is and isn't Tomb Raider and move on with their lives .

This project is awesome. I hope you'll be successful in recreating the source code !
Zebra is offline   Reply With Quote
Old 18-09-17, 16:28   #10
DunHill
Inactive
 
Joined: Apr 2017
Posts: 205
Default

This project is amazing, I agree with many people here. It could replace the TR4 engine with many new features.
I'm so delighted to see so many people working on the modding community. With the new engines, the new Tomb Editor... All of this will bring so much more possibilities to the TRLE world

Quote:
Originally Posted by HD86 View Post
The current TR game is what makes this community exists. If the source code becomes available, we will have a plethora of new TR games as many people will try to build their own versions of the game. The community will splinter because many will be playing different games.

Anyway, I am of course not against upgrading the game so that it remains compatible with new systems. Technical improvements are welcome as long as they don't change the gameplay.
Tomb Raider 5 was built on the same old grid system, it wouldn't make sense to change the gameplay to something modern
I think that the main goal here is to keep the same game, just more compatible with modern hardware
Though, If the code becomes available and we are able to add graphical features and gameplay changes, I think we could find a balance between old and modern. Let me explain myself :
We have to keep the easy way of building thanks to the grid system, for people who don't want to learn c++ or game design to build on newer engines such as Unreal engine. Plus, it will be even easier for people to edit wads and objects with the new editor.
BUT we can find a way to make the gameplay more fluid while keeping it adapted to the grid system.
And Core Design had the right idea when building Anniversary Edition : they imported TR1 maps, modified them graphically, but kept the grid-based system for the jumps and so on while having way more responsive controls.
DunHill is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT. The time now is 14:46.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.
Tomb Raider Forums is not owned or operated by CDE Entertainment Ltd.
Lara Croft and Tomb Raider are trademarks of CDE Entertainment Ltd.