Skip to content

Scriptcomp optimizations refactor #149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
niv merged 3 commits into from
Aug 17, 2025
Merged

Scriptcomp optimizations refactor #149

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f40104f
scriptcomp: constant folding is a core language feature, no longer co…
mtijanic Aug 5, 2025
e9eeb6a
scriptcomp: optimization flags rework. Now default to _SAFE, which is…
mtijanic Aug 5, 2025
831726c
nwn_script_comp: Make the CLI optimization flags reflect the backend …
mtijanic Aug 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions neverwinter/nwscript/compiler.nim
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,14 @@ type
# Needs to match scriptcomp.h
OptimizationFlag* {.pure.} = enum
RemoveDeadCode = 0x1
FoldConstants = 0x2
MeldInstructions = 0x4
RemoveDeadBranches = 0x8
MeldInstructions = 0x2
Copy link

Copilot AI Aug 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The enum value for MeldInstructions has changed from 0x4 to 0x2. This could cause issues if the enum values are persisted or used in binary interfaces. Ensure this change is intentional and documented.

Suggested change
MeldInstructions = 0x2
MeldInstructions = 0x4

Copilot uses AI. Check for mistakes.
RemoveDeadBranches = 0x4

const
OptimizationFlagsO0* = {}
OptimizationFlagsO2* = fullSet(OptimizationFlag)
OptimizationFlagsO1* = {RemoveDeadCode}
OptimizationFlagsO2* = {RemoveDeadCode, RemoveDeadBranches}
OptimizationFlagsO3* = {RemoveDeadCode, RemoveDeadBranches, MeldInstructions}

proc scriptCompApiNewCompiler(
src, bin, dbt: cint,
Expand Down
2 changes: 1 addition & 1 deletion neverwinter/nwscript/compilerapi.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ extern "C" void scriptCompApiInitCompiler(CScriptCompiler* instance, const char*
{
instance->SetGenerateDebuggerOutput(writeDebug);
instance->SetOptimizationFlags(
writeDebug ? CSCRIPTCOMPILER_OPTIMIZE_NOTHING : CSCRIPTCOMPILER_OPTIMIZE_EVERYTHING);
writeDebug ? CSCRIPTCOMPILER_OPTIMIZE_NOTHING : CSCRIPTCOMPILER_OPTIMIZE_AGGRESSIVE);
instance->SetCompileConditionalOrMain(1);
instance->SetIdentifierSpecification(lang);
instance->SetOutputAlias(outputAlias);
Expand Down
17 changes: 10 additions & 7 deletions neverwinter/nwscript/native/scriptcomp.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,16 +51,19 @@ class CScriptCompilerIdentifierHashTableEntry;

// Removes any functions that cannot possibly be called by any codepath
#define CSCRIPTCOMPILER_OPTIMIZE_DEAD_FUNCTIONS 0x00000001
// Merges constant expressions into a single constant where possible.
// Note: Only affects runtime expressions, assignments to const variables are always folded.
#define CSCRIPTCOMPILER_OPTIMIZE_FOLD_CONSTANTS 0x00000002
// Post processes generated instructions to merge sequences into shorter equivalents
Copy link

Copilot AI Aug 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bit flag value 0x00000002 is now being reused for MELD_INSTRUCTIONS after removing FOLD_CONSTANTS. Consider adding a comment explaining this change or ensuring this doesn't break binary compatibility with existing code.

Suggested change
// Post processes generated instructions to merge sequences into shorter equivalents
// Post processes generated instructions to merge sequences into shorter equivalents
// Note: The bit flag value 0x00000002 was previously used for FOLD_CONSTANTS.
// After removing FOLD_CONSTANTS, this value is now reused for MELD_INSTRUCTIONS.
// Ensure this does not break binary compatibility with existing code or data.

Copilot uses AI. Check for mistakes.
Copy link
Collaborator Author

@mtijanic mtijanic Aug 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No ABI requirements, can reuse these freely.

#define CSCRIPTCOMPILER_OPTIMIZE_MELD_INSTRUCTIONS 0x00000004
#define CSCRIPTCOMPILER_OPTIMIZE_MELD_INSTRUCTIONS 0x00000002
// Removes the jump and the dead branches in if (CONST) constructs
#define CSCRIPTCOMPILER_OPTIMIZE_DEAD_BRANCHES 0x00000008
#define CSCRIPTCOMPILER_OPTIMIZE_DEAD_BRANCHES 0x00000004

// Optimization groups - roughly corresponding to -O0, -O1, -O2, -O3. Each group includes the previous one.
// Safe - Known good, used by the game and toolset
// Aggressive - Probably good, used by the external compiler
// Experimental - Untested or known to break something
#define CSCRIPTCOMPILER_OPTIMIZE_NOTHING 0x00000000
#define CSCRIPTCOMPILER_OPTIMIZE_EVERYTHING 0xFFFFFFFF
#define CSCRIPTCOMPILER_OPTIMIZE_SAFE (CSCRIPTCOMPILER_OPTIMIZE_DEAD_FUNCTIONS)
#define CSCRIPTCOMPILER_OPTIMIZE_AGGRESSIVE (CSCRIPTCOMPILER_OPTIMIZE_SAFE | CSCRIPTCOMPILER_OPTIMIZE_DEAD_BRANCHES)
#define CSCRIPTCOMPILER_OPTIMIZE_EXPERIMENTAL (CSCRIPTCOMPILER_OPTIMIZE_AGGRESSIVE | CSCRIPTCOMPILER_OPTIMIZE_MELD_INSTRUCTIONS)


class CScriptCompilerIncludeFileStackEntry
Expand Down Expand Up @@ -558,7 +561,7 @@ class CScriptCompiler
CScriptParseTreeNode *m_pGlobalVariableParseTree;
int32_t AddToGlobalVariableList(CScriptParseTreeNode *pGlobalVariableNode);

BOOL ConstantFoldNode(CScriptParseTreeNode *pNode, BOOL bForce=FALSE);
BOOL ConstantFoldNode(CScriptParseTreeNode *pNode);

CScriptParseTreeNode *TrimParseTree(CScriptParseTreeNode *pNode);

Expand Down
11 changes: 4 additions & 7 deletions neverwinter/nwscript/native/scriptcompcore.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,7 +307,7 @@ CScriptCompiler::CScriptCompiler(RESTYPE nSource, RESTYPE nCompiled, RESTYPE nDe

m_sLanguageSource = "";
m_sOutputAlias = "OVERRIDE";
m_nOptimizationFlags = CSCRIPTCOMPILER_OPTIMIZE_EVERYTHING;
m_nOptimizationFlags = CSCRIPTCOMPILER_OPTIMIZE_SAFE;
m_nIdentifierListState = 0;

m_pSRStack = NULL;
Expand Down Expand Up @@ -1544,11 +1544,8 @@ int32_t CScriptCompiler::OutputError(int32_t nError, CExoString *psFileName, int
// Destructively modify a node and all its children to decay it into a single
// CONSTANT operation, if possible.
// This function is safe to call multiple times on the same node.
BOOL CScriptCompiler::ConstantFoldNode(CScriptParseTreeNode *pNode, BOOL bForce)
BOOL CScriptCompiler::ConstantFoldNode(CScriptParseTreeNode *pNode)
{
if (!bForce && !(m_nOptimizationFlags & CSCRIPTCOMPILER_OPTIMIZE_FOLD_CONSTANTS))
return FALSE;

if (!pNode)
return FALSE;

Expand Down Expand Up @@ -1596,9 +1593,9 @@ BOOL CScriptCompiler::ConstantFoldNode(CScriptParseTreeNode *pNode, BOOL bForce)

// In case of complex expression, start folding at the leaf nodes
// e.g.: C = 3 + 2*4 - First fold 2*4 into 8, then 3+8 into 11
ConstantFoldNode(pNode->pLeft, bForce);
ConstantFoldNode(pNode->pLeft);
if (pNode->pRight)
ConstantFoldNode(pNode->pRight, bForce);
ConstantFoldNode(pNode->pRight);

// Can only fold if the operands are constants.
if (pNode->pLeft->nOperation != CSCRIPTCOMPILER_OPERATION_CONSTANT_INTEGER &&
Expand Down
6 changes: 3 additions & 3 deletions neverwinter/nwscript/native/scriptcompfinalcode.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1003,7 +1003,7 @@ int32_t CScriptCompiler::TraverseTreeForSwitchLabels(CScriptParseTreeNode *pNode
{
int32_t nCaseValue;

ConstantFoldNode(pNode->pLeft, TRUE);
ConstantFoldNode(pNode->pLeft);
// Evaluate the constant value that is contained.
if (pNode->pLeft != NULL &&
pNode->pLeft->nOperation == CSCRIPTCOMPILER_OPERATION_NEGATION &&
Expand Down Expand Up @@ -1250,7 +1250,7 @@ int32_t CScriptCompiler::GenerateIdentifiersFromConstantVariables(CScriptParseTr
pNode->pRight->pLeft->pLeft->pLeft != NULL)
{
CScriptParseTreeNode *pNodeConstant = pNode->pRight->pLeft->pLeft->pLeft;
ConstantFoldNode(pNodeConstant, TRUE);
ConstantFoldNode(pNodeConstant);

int32_t nConstantOperation = pNodeConstant->nOperation;
int32_t nSign = 1;
Expand Down Expand Up @@ -1561,7 +1561,7 @@ int32_t CScriptCompiler::PreVisitGenerateCode(CScriptParseTreeNode *pNode)
{
int32_t nCaseValue;

ConstantFoldNode(pNode->pLeft, TRUE);
ConstantFoldNode(pNode->pLeft);
// Evaluate the constant value that is contained.
if (pNode->pLeft != NULL &&
pNode->pLeft->nOperation == CSCRIPTCOMPILER_OPERATION_NEGATION &&
Expand Down
15 changes: 12 additions & 3 deletions nwn_script_comp.nim
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ import shared
import neverwinter/nwscript/compiler
import neverwinter/resman

proc fmtOptFlags(flags: set[OptimizationFlag], indentSize: int = 36): string =
indent(join(toSeq(flags).mapIt("+" & $it), "\n"), indentSize)

const ArgsHelp = """
Compile one or more scripts using the official compiler library.

Expand Down Expand Up @@ -31,10 +34,14 @@ Usage:
-y Continue processing input files even on error.
-j N Parallel execution (default: all CPUs).

-O N Optimisation levels [default: 2]
-O N Optimisation levels [default: 1]
0: Optimise nothing
2: Aggressive optimisations; fastest and smallest code size:
""" & indent(join(toSeq(OptimizationFlagsO2).mapIt("+" & $it), "\n"), 36) & """
1: Safe optimisations only, as used by the game and toolset
""" & fmtOptFlags(OptimizationFlagsO1, 36) & "\n" & """
2: Aggressive optimisations; faster and smaller code size
""" & fmtOptFlags(OptimizationFlagsO2, 36) & "\n" & """
3: Experimental optimisations; untested or known to break something
""" & fmtOptFlags(OptimizationFlagsO3, 36) & "\n" & """


--max-include-depth=N Maximum include depth [default: 16]
Expand Down Expand Up @@ -98,7 +105,9 @@ globalState.params = Params(
debugSymbols: globalState.args["-g"].to_bool,
optFlags: (case parseInt($globalState.args["-O"])
of 0: OptimizationFlagsO0
of 1: OptimizationFlagsO1
of 2: OptimizationFlagsO2
of 3: OptimizationFlagsO3
else: raise newException(ValueError, "No such optimisation flag: " & $globalState.args["-O"])
),
continueOnError: globalState.args["-y"].to_bool,
Expand Down