Skip to content

Commit

Permalink
Merge branch 'C++Kit'
Browse files Browse the repository at this point in the history
  • Loading branch information
@Minty-Meeo
Minty-Meeo committed Oct 19, 2021
2 parents 9d8b919 + 5680ab3 commit b9d7179
Show file tree
Hide file tree
Showing 4 changed files with 1,467 additions and 40 deletions.
113 changes: 92 additions & 21 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,34 +1,46 @@
# What is DOL C-Kit?
DOL C-Kit is a toolkit for compiling C code (or assembly) using DevkitPPC to inject into a GameCube/Wii \*.dol executable. It has been written in such a way that it can be adapted to many different games. You will need [Python 3](https://www.python.org/downloads/) and [DevKitPPC](https://devkitpro.org/wiki/Getting_Started) installed to use it. As well, DOL C-Kit is dependent on [pyelftools](https://github.com/eliben/pyelftools), JoshuaMK's fork of [dolreader](https://github.com/JoshuaMKW/dolreader), and [geckocode-libs](https://github.com/JoshuaMKW/geckocode-libs).
DOL C-Kit is a toolkit for compiling C/C++ code (or assembly) using DevkitPPC to inject into a GameCube/Wii \*.dol executable. It has been written in such a way that it can be adapted to many different games. You will need [Python 3](https://www.python.org/downloads/) and [DevKitPPC](https://devkitpro.org/wiki/Getting_Started) installed to use it. As well, DOL C-Kit is dependent on [pyelftools](https://github.com/eliben/pyelftools), JoshuaMK's fork of [dolreader](https://github.com/JoshuaMKW/dolreader), and [geckocode-libs](https://github.com/JoshuaMKW/geckocode-libs).

Credit to Yoshi2 for creating the original GC C-Kit. DOL C-Kit couldn't exist without it.

# How to use it / The Project Class
## How to use it
DOL C-Kit is a Python module. To install on Windows, run INSTALL.bat as administrator. To install on Linux, run INSTALL.sh as superuser.

The Project class automates the tedious parts of compiling, linking, and injecting custom code into a \*.dol executable. The Project class constructor has two optional parameters to set the "base_addr" and "verbose" member variables. By default, they are set to None and False, respectively.
## The Project Class
`from dol_c_kit import Project`

The Project class has many member variables that may be directly modified:
* src_dir: Path to C code (or assembly) source files. Default is "".
* obj_dir: Path to output \*.o files and other files generated by DOL C-Kit to. Default is "".
* devkitppc_path: Change this if DevKitPPC is not installed at its default location.
* project_name: Name used for certain files generated by DOL C-Kit. Default is "project".
* gcc_flags: Non-crucial flags passed to powerpc-eabi-gcc. Defaults include "-w", "-std=c99", "-O1", and "-fno-asynchronous-unwind-tables".
* as_flags: Non-crucial flags passed to powerpc-eabi-as. Defaults include "-w".
* ld_flags: Non-crucial flags passed to powerpc-eabi-ld. Defaults include nothing.
* base_addr: The location new data will be put at. This is set by the constructor, but may be modified directly as well.
* sda_base: The value used for the \_SDA\_BASE\_ symbol. This is set by the set\_sda\_bases method, but may be modified directly as well.
* sda2_base: The value used for the \_SDA2\_BASE\_ symbol. This is set by the set\_sda\_bases method, but may be modified directly as well.
* verbose: Flag for additional information printing. This is set by the constructor, but may be modified directly as well.
The Project class automates the tedious parts of compiling, linking, and injecting custom code into a \*.dol executable.

By shifting forward the stack, db_stack, and OSArenaLo, space for new data can be allocated. To do this, a patching function modifying a given game's "\_\_init_registers", "OSInit", and "\_\_OSThreadInit" functions must be written. The project's save_dol function passes two parameters to this patching function: a DolFile class, and the base_addr of your project.

## Step 1: Populate the project
* `add_c_file(filepath, gcc_flags=(), use_global_flags=True)`<br>
Add a C source file to the project. Two optional arguments may be given: gcc_flags is a tuple of strings passed to powerpc-eabi-gcc as flags, and use_global_flags determines if the gcc_flags member of the Project class are used for this source file.
### Class constructor
* `Project(self, base_addr=None, verbose=False)`<br>
The Project class constructor has two optional parameters. base_addr and verbose set their respective class members upon class construction.

* `add_asm_file(filepath, as_flags=(), use_global_flags=True)`<br>
Add an assembly source file to the project. Two optional arguments may be given: as_flags is a tuple of strings passed to powerpc-eabi-as as flags, and use_global_flags determines if the as_flags member of the Project class are used for this source file.
### Class members
The Project class has many member variables that may be directly modified:
* `src_dir` Path to C code (or assembly) source files. Default is "".
* `obj_dir` Path to output \*.o files and other files generated by DOL C-Kit to. Default is "".
* `devkitppc_path` Change this if DevKitPPC is not installed at its default location.
* `project_name` Name used for certain files generated by DOL C-Kit. Default is "project".
* `c_flags` Non-crucial flags passed to powerpc-eabi-gcc. Defaults include "-w", "-std=c99", "-O1", and "-fno-asynchronous-unwind-tables".
* `cpp_flags` Non-crucial flags passed to powerpc-eabi-g++. Defaults include "-w", "-std=c++98", "-O1", "-fno-asynchronous-unwind-tables", and "-fno-rtti".
* `asm_flags` Non-crucial flags passed to powerpc-eabi-as. Defaults include "-w".
* `linker_flags` Non-crucial flags passed to powerpc-eabi-ld. Defaults include nothing.
* `base_addr` The location new data will be put at. This is set by the constructor, but may be modified directly as well.
* `sda_base` The value used for the \_SDA\_BASE\_ symbol. This is set by the set\_sda\_bases method, but may be modified directly as well.
* `sda2_base` The value used for the \_SDA2\_BASE\_ symbol. This is set by the set\_sda\_bases method, but may be modified directly as well.
* `verbose` Flag for additional information printing. This is set by the constructor, but may be modified directly as well.

### Step 1: Methods to populate the project
* `add_c_file(filepath, flags=(), use_global_flags=True)`<br>
Add a C source file to the project. Two optional arguments may be given: flags is a tuple of strings passed to powerpc-eabi-gcc as flags, and use_global_flags determines if the c_flags member of the Project class are used for this source file.

* `add_cpp_file(filepath, flags=(), use_global_flags=True)`<br>
Add a C++ source file to the project. Two optional arguments may be given: flags is a tuple of strings passed to powerpc-eabi-g++ as flags, and use_global_flags determines if the cpp_flags member of the Project class are used for this source file.

* `add_asm_file(filepath, flags=(), use_global_flags=True)`<br>
Add an assembly source file to the project. Two optional arguments may be given: flags is a tuple of strings passed to powerpc-eabi-as as flags, and use_global_flags determines if the asm_flags member of the Project class are used for this source file.

* `add_obj_file(filepath, do_cleanup=False)`<br>
Add an unlinked object file to the project. This object file must be in the obj_dir, not the src_dir. The cleanup method WILL DELETE FILES added by the add_obj_file method if the optional do_cleanup argument is True.
Expand Down Expand Up @@ -69,7 +81,7 @@ Give your project a game-specific patching function to use to allocate space for
* `set_sda_bases(sda_base, sda2_base)`<br>
Set the \_SDA\_BASE\_ and \_SDA2\_BASE\_ symbols. These values get passed to the linker. They are also important for the @sda and @sda2 modifiers for Immediate16Hooks.

## Step 2: Build the project
### Step 2: Methods to build the project
* `build_dol(in_dol_path, out_dol_path)`<br>
Compile, assemble, and link all source files, hooks, and supported Gecko Codes into a \*.dol executable. If no base_addr is specified, the ROM end will automatically be detected and used. A new text section will be allocated to contain the new data. If no text sections are available, a data section will be allocated instead.<br>
Note: Automatic ROM end detection does not work for DOLs that allocate space for .sbss2.
Expand All @@ -82,3 +94,62 @@ Generate a CodeWarrior-like symbol map from the project. Run this after buildin

* `cleanup()`<br>
Delete unimportant files created by DOL C-Kit. This includes unlinked \*.o files, and <project_name>.o, <project_name>.bin, and <project_name>.map.

# How to work with mangled symbols (C++)
In C++, there is the concept of mangled symbol names. For example, the function signature `int foo::bar(MyClass arg1)` becomes the symbol `_ZN3foo3barE7MyClass`. DOL C-Kit provides faculties to make working with mangled symbols easy.

## The LDPlusPlus class
`from dol_c_kit import LDPlusPlus`

Just like a usual linker script to give dol-side symbols a value for C and ASM, one is needed for C++ as well. LDPlusPlus makes this easier by mangling function signatures in bulk for you.

### Class constructor
* `LDPlusPlus(abi)`<br>
The LDPlusPlus class constructor has one parameter. abi is the desired ABI (an enum value). At the moment, only ABI.Itanium is available.

### Methods
* `assign(prototype, value)`<br>
[Assign](https://sourceware.org/binutils/docs/ld/Simple-Assignments.html) a value to a symbol in the linker script.

* `provide(prototype, value)`<br>
[Provide](https://sourceware.org/binutils/docs/ld/PROVIDE.html) a value for a symbol in the linker script.

* `save(filepath)`<br>
Save the linker script to a given filepath.

## mangle (and itanium_mangle)
`from dol_c_kit import mangle, itanium_mangle`

To mangle an individual signature, use the mangle function (or itanium_mangle function). This is useful for hooks which take a symbol name as an argument.

* `mangle(prototype, abi)`<br>
Returns a mangled symbol from a given signature prototype in a given ABI (an enum value). At the moment, only ABI.Itanium is available.

* `itanium_mangle(prototype)`<br>
Returns a mangled symbol from a given signature prototype in the Itanium ABI.

## Limitations and workarounds of the mangler
Writing a C++ mangler that has no concept of user defined types, or really any context at all, was a struggle that came with a few compromises.
* Function pointer types are not supported.
* Certain kinds of template instancing are not supported.
* Typedefs are not supported (yet).
* To mangle certain edge cases, special tokens starting with "$$" are used:
* `$$vtable` is used for vtable signatures. e.g. `ClassA::$$vtable` will mangle to `_ZTV6ClassA`.
* `$$rtti` is used for typeinfo signatures. e.g. `ClassA::$$rtti` will mangle to `_ZTI6ClassA`.
* `$$vtt_structure` is used for vtable structure signatures. e.g. `ClassA::$$vtt_structure` will mangle to `_ZTT6ClassA`.
* `$$rtti_name` is used for typeinfo names. e.g. `ClassA::$$rtti_name` will mangle to `_ZTS6ClassA`.
* `$$ctor` is used for (the assumed default) object constructors. e.g. `ClassA::$$ctor` will mangle to `_ZN6ClassAC1Ev`.
* `$$ctor1` is used for complete object constructors. e.g. `ClassA::$$ctor1` will mangle to `_ZN6ClassAC1Ev`.
* `$$ctor2` is used for base object constructors. e.g. `ClassA::$$ctor2` will mangle to `_ZN6ClassAC2Ev`.
* `$$ctor3` is used for complete object allocating. e.g. `ClassA::$$ctor3` will mangle to `_ZN6ClassAC3Ev`.
* `$$dtor` is used for (the assumed default) object destructors. e.g. `ClassA::$$dtor` will mangle to `_ZN6ClassAD1Ev`.
* `$$dtor0` is used for deleting destructors. e.g. `ClassA::$$dtor0` will mangle to `_ZN6ClassAD0Ev`.
* `$$dtor1` is used for complete object destructors. e.g. `ClassA::$$dtor1` will mangle to `_ZN6ClassAD1Ev`.
* `$$dtor2` is used for base object destructors. e.g. `ClassA::$$dtor2` will mangle to `_ZN6ClassAD2Ev`.
* `$$unary` is used for operator overloads that are ambigious without context.
* `operator+ $$unary` will produce the "positive" operator override rather than the "add" operator override.
* `operator- $$unary` will produce the "negative" operator override rather than the "subtract" operator override.
* `operator& $$unary` will produce the "reference" operator override rather than the "bitwise AND" operator override.
* `operator* $$unary` will produce the "dereference" operator override rather than the "multiply" operator override.

There are likely other cases that the mangler won't cover, but for 99% of signatures, it should work.
6 changes: 5 additions & 1 deletion dol_c_kit/__init__.py
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
__version__ = "2.4.1"
__version__ = "3.0.0"
__author__ = "Minty Meeo"
__credits__ = "Yoshi2 (RenolY2)"

Expand All @@ -24,5 +24,9 @@
from dol_c_kit.doltools import write_li
from dol_c_kit.doltools import write_lis
from dol_c_kit.doltools import write_nop
from dol_c_kit.mangle import ABI
from dol_c_kit.mangle import LDPlusPlus
from dol_c_kit.mangle import mangle
from dol_c_kit.mangle import itanium_mangle

from dol_c_kit.devkit_tools import Project
57 changes: 39 additions & 18 deletions dol_c_kit/devkit_tools.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -287,12 +287,14 @@ def __init__(self, base_addr=None, verbose=False):
self.obj_dir = ""
self.project_name = "project"
self.c_files = []
self.cpp_files = []
self.asm_files = []
self.obj_files = []
self.linker_script_files = []
self.gcc_flags = ["-w", "-std=c99", "-O1", "-fno-asynchronous-unwind-tables",]
self.as_flags = ["-w",]
self.ld_flags = []
self.c_flags = ["-w", "-std=c99", "-O1", "-fno-asynchronous-unwind-tables",]
self.cpp_flags = ["-w", "-std=c++98", "-O1", "-fno-asynchronous-unwind-tables", "-fno-rtti",]
self.asm_flags = ["-w",]
self.linker_flags = []
self.symbols = {}
self.verbose = verbose

Expand All @@ -307,11 +309,14 @@ def __init__(self, base_addr=None, verbose=False):

# Add stuff

def add_c_file(self, filepath, gcc_flags=(), use_global_flags=True):
self.c_files.append((filepath, gcc_flags, use_global_flags))
def add_c_file(self, filepath, flags=(), use_global_flags=True):
self.c_files.append((filepath, flags, use_global_flags))

def add_asm_file(self, filepath, as_flags=(), use_global_flags=True):
self.asm_files.append((filepath, as_flags, use_global_flags))
def add_cpp_file(self, filepath, flags=(), use_global_flags=True):
self.cpp_files.append((filepath, flags, use_global_flags))

def add_asm_file(self, filepath, flags=(), use_global_flags=True):
self.asm_files.append((filepath, flags, use_global_flags))

def add_obj_file(self, filepath, do_cleanup=False):
self.obj_files.append((filepath, do_cleanup))
Expand Down Expand Up @@ -549,25 +554,38 @@ def cleanup(self):

# Private stuff

def __compile(self, infile, gcc_flags, use_global_flags):
def __compile(self, infile, flags, use_global_flags):
args = [self.devkitppc_path+"powerpc-eabi-gcc", "-c", self.src_dir+infile, "-o", self.obj_dir+infile+".o", "-I", self.src_dir]
if use_global_flags:
for flag in self.gcc_flags:
for flag in self.c_flags:
args.append(flag)
for flag in gcc_flags:
for flag in flags:
args.append(flag)
if self.verbose:
print(args)
subprocess.call(args)
self.obj_files.append((infile+".o", True))
return True

def __assemble(self, infile, as_flags, use_global_flags):
def __compileplusplus(self, infile, flags, use_global_flags):
args = [self.devkitppc_path+"powerpc-eabi-g++", "-c", self.src_dir+infile, "-o", self.obj_dir+infile+".o", "-I", self.src_dir]
if use_global_flags:
for flag in self.cpp_flags:
args.append(flag)
for flag in flags:
args.append(flag)
if self.verbose:
print(args)
subprocess.call(args)
self.obj_files.append((infile+".o", True))
return True

def __assemble(self, infile, flags, use_global_flags):
args = [self.devkitppc_path+"powerpc-eabi-as", self.src_dir+infile, "-o", self.obj_dir+infile+".o", "-I", self.src_dir]
if use_global_flags:
for flag in self.as_flags:
for flag in self.asm_flags:
args.append(flag)
for flag in as_flags:
for flag in flags:
args.append(flag)
if self.verbose:
print(args)
Expand All @@ -594,7 +612,7 @@ def __link_project(self):
for filename, do_cleanup in self.obj_files:
args.append(self.obj_dir+filename)
args.extend(("-Map", self.obj_dir+self.project_name+".map"))
for flag in self.ld_flags:
for flag in self.linker_flags:
args.append(flag)
if self.verbose:
print(args)
Expand Down Expand Up @@ -631,11 +649,14 @@ def __build_project(self):
is_linked = False
is_processed = False

for filepath, gcc_flags, use_global_flags in self.c_files:
is_built |= self.__compile(filepath, gcc_flags, use_global_flags)
for filepath, flags, use_global_flags in self.c_files:
is_built |= self.__compile(filepath, flags, use_global_flags)

for filepath, flags, use_global_flags in self.cpp_files:
is_built |= self.__compileplusplus(filepath, flags, use_global_flags)

for filepath, as_flags, use_global_flags in self.asm_files:
is_built |= self.__assemble(filepath, as_flags, use_global_flags)
for filepath, flags, use_global_flags in self.asm_files:
is_built |= self.__assemble(filepath, flags, use_global_flags)

if is_built == True:
is_linked |= self.__link_project()
Expand Down
Loading

0 comments on commit b9d7179

Please sign in to comment.