BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Discussions about all other tools (CusPop, SAV/HOF editors) and hex editing.
Post Reply
BlackDaemon
Efendi
Efendi
Posts: 8
Joined: May 22nd, 2018, 2:48 pm

BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by BlackDaemon »

Hello there :) Some people may know me as Project "Timeline" author :P Recently i was working on reseaching BF format, and finally achieved my main goal to make game work with larger bin/bf file, than original ones. For example, you could extract some .bin, decompress it, replace original texture with larger, compress it and build .bf file, which would work :P Current code is rough, there is still issues with proper compression/decompression of some .bin files, but basic stuff, like all_weapons .bin file should work ;) I hope, that would bring more researching/modding activity for PoP SoT trilogy series :roll: Attached recent tool build to attachments, have fun reseaching/modding! :D
Spoiler: show
Image
Attachments
bf_repacker_2018_05_23_1419.rar
Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker
(99.92 KiB) Downloaded 1715 times
Last edited by BlackDaemon on May 23rd, 2018, 1:22 pm, edited 3 times in total.
Falcury
Calif
Calif
Posts: 565
Joined: June 25th, 2009, 10:01 pm

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by Falcury »

Sounds like a breakthrough. Nice! :)

I can't get it to run properly, though. When I try to extract prince.bf (from the GOG.com release of WW), I get this error:

Code: Select all

See the end of this message for details on invoking 
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at bf_repacker.MainForm.printPaths()
   at bf_repacker.MainForm.ReadBF(String path)
   at bf_repacker.MainForm.btn_read_bf_Click(Object sender, EventArgs e)
   at System.Windows.Forms.Control.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ButtonBase.WndProc(Message& m)
   at System.Windows.Forms.Button.WndProc(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.3101.0 built by: NET472REL1LAST_B
    CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v4.0.30319/mscorlib.dll
----------------------------------------
bf_repacker
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/work/prince/bf_repacker/bf_repacker.exe
----------------------------------------
System.Windows.Forms
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.3056.0 built by: NET472REL1
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.3056.0 built by: NET472REL1
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.3056.0 built by: NET472REL1
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System.Configuration
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.3056.0 built by: NET472REL1
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Core
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.3101.0 built by: NET472REL1LAST_B
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
----------------------------------------
System.Xml
    Assembly Version: 4.0.0.0
    Win32 Version: 4.7.3056.0 built by: NET472REL1
    CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
    <system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.
This is the contents of the log:
Spoiler: show
Program started.
Selected path to BF file: C:\work\prince\bf_repacker\prince.bf
Selected path to extract folder: C:\work\prince\bf_repacker\prince.bf_unpacked
Reading BF file: C:\work\prince\bf_repacker\prince.bf
Reading header...done!
Reading FileIdOffset table...done!
Skipping FileIdOffset table dummy area...done!
Reading FileEntry table...done!
Skipping FileEntry table dummy area...done!
Checking folders count in FolderEntry table...done!
Reading FolderEntry table...done!
Skipping FolderEntry table dummy area...done!
Skipping remaining dummy area and reading header total size block...done!
Looking for size.grs table offset...done!
Checking files count in size.grs table...done!
Reading size.grs table...done!
Linking FileEntry and size.grs table file Ids...done!
Printing file paths...
BlackDaemon
Efendi
Efendi
Posts: 8
Joined: May 22nd, 2018, 2:48 pm

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by BlackDaemon »

Yeah, i just few minutes ago was checking WW bf file, and found issue with folders detection :D Just fixed it, updated attachments with fixed build :P

Upd: one more workaround for WW bf, also spotted odd thing: in size.grs table allocated more size for all_weapons_wow_ff0610da.bin, than it's actual data :? Tested with actual data size value - it worked without crashes so far.
Falcury
Calif
Calif
Posts: 565
Joined: June 25th, 2009, 10:01 pm

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by Falcury »

Now it works.
I opened up a random file (105_Accessoires_wow_ff087cde.dec) in a hex editor.
It in itself contains various filenames, for example, 105_AileNord_Ombrages_feuilles_Cam01.gao.
Funny, they seem to have used a mixture of French and English (perhaps unsurprising for Ubisoft Montreal).
BlackDaemon
Efendi
Efendi
Posts: 8
Joined: May 22nd, 2018, 2:48 pm

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by BlackDaemon »

Unfortunately, reversing .bin files is a long-term task, because basically there is no public info about it's structure. PersianRug tool isn't finished, and it's source wasn't released.

Very basic example of data structure inside ACT_Enemy_HumanGarde_wow_ff0205de.bin :P
Spoiler: show
Image
Falcury
Calif
Calif
Posts: 565
Joined: June 25th, 2009, 10:01 pm

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by Falcury »

Looking at some of the WW data files...
In the ROOT/Bin directory there are 73 files named _{some address?}.dec. These appear to be files containing the English text of the menu items, the subtitles, etc.
At first sight they appear to be entirely identical, except for the first 16 bytes where the file ID is different. (Then, why would there need to be 73 identical files?) One exception is _fdae0164.dec, which is slightly larger and appears to have some extra stuff at the beginning of the file.
Looking at a random file (_fdac0225.dec), and scrolling down past some unknown data, the first block of zero-terminated strings is located at offset 6E30:

Code: Select all

Pause
Continue
Options
Back to main menu
Game Over
Retry
Quit
Press START to return to game
Do you want to save the game?
Repeat Vision
Save Game
No
Yes
Please reconnect the controller\nto any controller port and\n press START to continue.
press START to continue.
{185}   Select\n{186}   Go back
Play the original Prince of Persia tm?
Current progress will be lost.\nAre you sure you want to quit?
The original Prince of persia unlocked
Continue without saving\n\nAre you sure ?
A saved game already exist at this location.\nDo you want to overwrite it?
Saving the game . . .\nPlease do not turn off your Xbox console.
Prince of persia\nthe shadow and the flame\nunlocked
You are about to leave\nPrince of Persia : the Sands of Time tm,\nto play Prince of Persia,\n the shadow and the flame tm.\nYou will have to reload your last saved\ngame in the main menu to continue playing.\n\nAre you sure you want to proceed?
Funny, there is still Sands of Time stuff in there that they didn't delete :P But what's that about the Shadow and the Flame? Don't tell me there's a similar easter egg in Warrior Within (or, a second easter egg in SoT) that we know nothing about??

Here is another interesting string in the same file (offset A16D):
You gain\0Wooden stick\0This sturdy wooden stick is a weak\0replacement weapon.\0\0\0It can only chain 2-combo attacks.
Apparently there was a cutscene where you triumphantly get the wooden stick as a weapon, similar to how you get the other swords. :P
BlackDaemon
Efendi
Efendi
Posts: 8
Joined: May 22nd, 2018, 2:48 pm

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by BlackDaemon »

Files without name (which has only fileId) mostly contains various text stuff, yep :P My guess about discovered text, that it's one of skeletons, which Ubisoft left in a closet :) Like reference to removed cutscene for fake ending :P

Code: Select all

(10,50)Such terrible screams.\n I hope I am not too late. (60,50)Farzaad, my first mate!\n I thought all had perished at sea... (110,30)Farzaad,\n what happened to you? (140,30)The beast... it… (170,30)...you must…you must… (200,30)What, my friend?\n What must I do? (230,30)...RUN!
BlackDaemon
Efendi
Efendi
Posts: 8
Joined: May 22nd, 2018, 2:48 pm

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by BlackDaemon »

Initial command-line port :P

Code: Select all

Extract BF: -e <bf_file> <extracted_files_folder>
Decompress BIN files: -d <bf_file> <extracted_files_folder>
Compress BIN files: -c <bf_file> <extracted_files_folder>
Build BF: -b <bf_file> <extracted_files_folder> <new_bf_file>
Attachments
bf_repacker_cli_2018_06_25_1101.rar
BF Repacker, command-line version
(70.58 KiB) Downloaded 498 times
Olio
Scholar Scribe
Scholar Scribe
Posts: 2
Joined: November 12th, 2018, 11:40 am

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by Olio »

Hi, I'm fascinated with your work, but I beg you to upload the code to GitHub.
Olio
Scholar Scribe
Scholar Scribe
Posts: 2
Joined: November 12th, 2018, 11:40 am

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by Olio »

One more question, where I can read about encoding this file format? I want to improve this app a bit, but I need to obtain some knowledge how .bf works. Thanks.
Voodooman
Scholar Scribe
Scholar Scribe
Posts: 2
Joined: February 8th, 2019, 1:42 am

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by Voodooman »

Man, can you please update you game to add support for Beyond Good and Evil, both original PC version and PS3 and Xbox HD ports? Im trying to make HD mod, and lack of tools killing attempts.

Also, could you add support for all the other games that used jade engine? Rayman Raving Rabbids (possibly all the other sequels used same engine), King Kong Sd and Gamers Edition, Zombie (U), TNMT (from Ubi), James Cameron Avatar movie game (probably only Wii version). But 1st add BGE1 PC and Xbox 360 and PS HD support please!

Right im gettin error related to UT8:

D:\Program Files (x86)\Ubisoft\Beyond Good and Evil>bf_repacker_cli.exe -e sally_clean.bf sally_clean

Необработанное исключение: System.ArgumentException: Буфер выходных символов не достаточен для хранения закодированных символов, кодирование 'Unicode (UTF-8)' резерв 'System.Text.DecoderReplacementFallback'.
Имя параметра: chars
в System.Text.Encoding.ThrowCharsOverflow()
в System.Text.Encoding.ThrowCharsOverflow(DecoderNLS decoder, Boolean nothingDecoded)
в System.Text.UTF8Encoding.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, DecoderNLS baseDecoder)
в System.Text.DecoderNLS.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, Boolean flush)
в System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex, Boolean flush)
в System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex)
в System.IO.BinaryReader.InternalReadOneChar()
в System.IO.BinaryReader.Read()
в System.IO.BinaryReader.ReadChar()
в bf_repacker_cli.Program.FileEntry.readFileEntrys(Header header, BinaryReader buf)
в bf_repacker_cli.Program.ReadBF(String path)
в bf_repacker_cli.Program.Main(String[] args)
`
GUI also does not work. Same with bin tool.
ravio
Scholar Scribe
Scholar Scribe
Posts: 1
Joined: May 12th, 2019, 11:12 pm

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by ravio »

Hi Blackdaemon!

Very good work! It's really awesome!

I take the opportunity to ask you if you could help me about the decompression of .bf files. You seem to have studied the structure closely and probably be the only person on the internet to help me! To be honest, I do not have any notion of how to extract an archive without any tools, I am a 3D artist and I work a lot with Blender and Unity, but unfortunately my programming abilities are limited to unity scripts ...

So here, I'm trying to properly decompress the .bf file of the game POP: The Forgotten Sands on Wii. As you probably know the Wii version of the game offers environments and a scenario completely alternative to the PC / X360 / PS opus.

I tried several tools to extract .BF archives: PersianRug 2.5 / BFextractor / QuickBms (all .bf script i've find) / and finally BFrepacker. However, none of them work properly on this game.

Only PersianRug can read some fragments of the archive, but the few .bin files that can be extracted are unreadable and do not contain any 3D (mesh) or 2D (texture) data. Yet POP: The Forgotten Sands on Wii was realized with the Jade Engine, which implies that the structure of the .bf files on WII must have some differences with those of the other supports.

If you ever want an example here is the only one ".bf" file of the game "djinn_bin_wii.bf" (600mb) extracted from the iso with the Dolphin emulator.
https://mega.nz/#!58Y2GSTY!U4TnFxDNsCNs ... 7WdpM18Dvk

I really hope you can help me because it's the only 3D POP game I can not extract 3D data from. Unfortunately I really, really needed it, because I work on a project whose goal is to integrate the main areas of the series in an large world with the Unity engine. Be careful however, the project is not really an open world, nor a game for copyright reasons. Only an environment that we can travel. To make it easier I do with POP on Unity what Cryzenx does with TLOZ on UE4 (https://www.youtube.com/channel/UC-86Wp ... s2gWJSuX-w).I will give you more details in PM.



PS: Sorry for my english ^^
Dastan
Scholar Scribe
Scholar Scribe
Posts: 2
Joined: December 31st, 2019, 1:08 pm

Re: BF Repacker - Prince of Persia (SoT/WW/T2T) gamedata packer/unpacker

Post by Dastan »

Hi, i can't rebuild a bf file after making some bin modification. can you post a tutorial for this part?
Post Reply