Accessing Borderlands 3 Data

Among the challenges of modding Borderlands 3, in addition to the fact that there's no viable modding platform yet, is getting data out of the game. It's unfortunately not as straightforward as BL2 or TPS, where we had some relatively-easily-available tools to generate data dumps that are used in apps like BLCMM and FT/BLCMM Explorer. This page will go over the various methods of retrieving data from BL3 that we've got available as of early January, 2020.

• Extracting Raw Datafiles
• Basic Raw Datafile Info
• Advanced Raw Datafile Info (JohnWickParse)
• Console Access in Borderlands 3 Itself
• Getting Data from the Console

Extracting Raw Datafiles

The first, and most straightforward thing you can do is unpack all the various *.pak files which live in OakGame/Content/Paks (and in AdditionalContent/*/Paks for DLCs). These pak files are encrypted, so you'll need to find the encryption key online (it won't be listed on this page). Unpacking the data can be done with the relevant version of the UnrealPak utility which comes with Unreal Engine. You'll want to use the same version of the engine that BL3 uses, which is 4.20.3.

You'll first need to create a crypto.json file (named however you like, actually) with contents that look like the following:

{
    "$types": {
    "UnrealBuildTool.EncryptionAndSigning+CryptoSettings, UnrealBuildTool, Version=4.0.0.0, Culture=neutral, PublicKeyToken=null": "1",
    "UnrealBuildTool.EncryptionAndSigning+EncryptionKey, UnrealBuildTool, Version=4.0.0.0, Culture=neutral, PublicKeyToken=null": "2"
    },
    "$type": "1",
    "EncryptionKey": {
    "$type": "2",
    "Name": null,
    "Guid": null,
    "Key": "<encryption key found online>"
    },
    "SigningKey": null,
    "bEnablePakSigning": false,
    "bEnablePakIndexEncryption": true,
    "bEnablePakIniEncryption": true,
    "bEnablePakUAssetEncryption": false,
    "bEnablePakFullAssetEncryption": false,
    "bDataCryptoRequired": true,
    "SecondaryEncryptionKeys": []
}

Then you'll unpack using the commandline:

UnrealPak.exe filename.pak -extract extractdir -cryptokeys=crypto.json

The pak file will get extracted into the extractdir directory. The order in which paks should be extracted is important in some cases. The base game Pak files are all named like pakchunk4-WindowsNoEditor.pak. A patch later added pakchunk4-WindowsNoEditor_0_P.pak, which should be extracted after the original file. Then there was a pakchunk4-WindowsNoEditor_1_P.pak in a patch after that, and so on. So extract the ones without _P first, then extract all the _0_P files next, and so on.

UE4 organizes its objects into directories, and the extracted data will be extracted in a similar way, though the extracted directories won't exactly match the object paths. A Python script exists which generates the commands necessary to move the extracted objects to their "proper" location so that the directories match the objects, though it does assume you're running it on Linux. That script can be found here. If you're using it, it'd be best to run it inbetween the various *_P extractions, so that the objects are sure to overwrite themselves properly. It's definitely not necessary to do that reorganization step; it just means that it might take more effort to track down the datafiles you're looking for, since the directories won't always match the filenames.

Basic Raw Datafile Info

Without even having to parse the datafiles, you can get a reasonable amount of information just by using the utility strings to look for strings inside the .uasset files specifically. The strings found inside the .uasset files will include all the object names that that object references, so you can infer quite a bit of information about how objects relate to each other just by that method. That's actually the basis for the interactive Borderlands 3 Object Refs page, which is a very useful resource for browsing around the data.

Other useful information that can be found with strings includes the attribute names which can be found in the object. You can also find ingame text by running strings on the .uexp files, such as item/weapon names, card text, and the like.

Advanced Raw Datafile Info (JohnWickParse)

You can get even more info out of the datafiles by using a utility called JohnWickParse. It's an asset parser for Fortnite, another UE4 game, but it can often work just fine on Borderlands 3 datafiles as well. You should be able to download it from the "releases" page on its github site. If you prefer to compile yourself: clone the github repo, compile it using cargo build from the commandline, and then execute the compiled binary which'll be found in target/debug.

To actually run JWP to generate some human-readable data, run:

john-wick-parse serialize <object_name>

You'll need to have the .uasset and .uexp file in the directory where you run it -- the object_name should be the base filename without extension. When you run the serialization process, it should hopefully output a new file named object_name.json in the same directory. For some objects it won't work at all, unfortuantely -- for instance, you'll never be able to serialize the BPChar character objects, or mission objects. But for many of them, you'll have a JSON file which contains a lot of useful info in it. Many objects will only serialize partially, so some parts will be filled in and others blank. So you'll end up needing to compliment these serializations with the strings output as well. You'll want to edit the JSON using an editor which prettifies the output, or process it using an app like jq or site like jsonformatter.org, to make it easier to read.

Console Access in Borderlands 3 Itself

Finally, getting data out of the BL3 ingame console is possible, though it's not nearly as convenient as in BL2/TPS. First off, there isn't a simple hex-edit to enable the console; instead, you've got to inject a DLL into the running BL3 process. Most Windows users will find it most convenient to do DLL injection via Cheat Engine, though any other DLL injector should also presumably do the trick, such as Xenos, Extreme Injector, or the one which comes bundled with the Universal Unreal Engine 4 Console Unlocker (UUU). For Linux users who run BL3 via Wine/Proton, note that the UUU-provided injector is the only one I've found which works reliably in that environment (see here for my problems using CE with BL3, in Linux).

Once you have an injection app, you need a DLL to inject which will unlock the console. At the moment (Jan 12, 2020), I'm only aware of one which actually works, which is the Universal Unreal Engine 4 Console Unlocker (UUU). As noted above, that comes with its own DLL injector as well, which is convenient. There was also another console-unlocker DLL available at fearlessrevolution.com, but that DLL stopped working with one of BL3's more recent patches, so as of writing, you're better off with UUU.

To inject the DLL, regardless of injector, just start up BL3, wait until you're at the main menu, then launch the injector and use it on the BL3 process. Give it a few seconds, and you should be able to hit tilde/backtick a few times to cycle through the available console modes ingame. In my experience it works just fine if you inject while ingame as well, so you probably don't have to be on the main menu specifically.

Getting Data from the Console

Once you have console access, there's unfortunately a very limited set of commands you can use which are useful for getting data. Namely, the mainstay of object dumping from BL2/TPS -- obj dump -- is not available on the BL3 console. Instead, the only command usable for data retrieval is getall. In BL2/TPS that was primarily just used to get lists of objects, which you'd then obj dump, but in BL3 it'll be your window to in-game data.

The UE4 getall command can be used just to list objects, like this command to list all item pools currently loaded in memory (this is one kind of object which JohnWickParse can serialize easily, but checking values in-game is often helpful even then):

getall itempooldata

You can get specific and provide the exact attribute you're looking for, like so:

getall itempooldata balanceditems

When you're looking for one specific object's attribute, though, you'll want to narrow the results down a bit, which isn't always possible, depending on the kind of object, but for ItemPoolData objects, it is. You can specify an extra name=foo parameter at the end, like so:

getall itempooldata balanceditems name=itempool_artifacts

That command would show you the contents of the itempool which drops Artifacts in-game. The name to use, to narrow down the getall output to something reasonable, will be found after the last slash in the object names, so you can generally get them right from the object list itself. (Though again, in some cases you'll find that all the objects that show up share a common name, so you might be stuck anyway.)

Note that when you use the name=foo construct at the end, you need to specify an attribute name as well, otherwise it won't work. If you don't actually care about the attributes and you just want the object list, you can put in any bit of text for the attribute and you'll still get the object list.

One final wrinkle that you'll have to contend with is that the BL3 console does not wrap text to the next line, if there's enough output to reach the righthand side of the screen. Anything beyond that point will just be cut off and not visible ingame. Fortunately, you can get the missing text, by using a memory viewer like Cheat Engine. You'll have to attach to the running Borderlands 3 process and then search memory for some bit of the getall output that you can see, and then look at the rest of the line in the CE memory editor. Note that the console output text looks like it's encoded internally in memory as UTF-16 strings, so you may have to change some dropdowns on whatever app you're using to search in memory.