draft: Final touches (closes #147, #135, #133, #132, #124, #111, #100, #92, #82, #81, #57, #54)

[recanman]

This pull request contains the final changes needed for the package migration.
Package name will be changed to monero-integrations/monero-crypto (namespace MoneroIntegrations\MoneroCrypto)

This will also bump the minimum PHP version to 8.1.
A fork of refring's work. This will be called monero-integrations/monero-rpc (namespace MoneroIntegrations\MoneroRpc).

Tasks left:

• Finish refactoring the other classes and add unit tests for them
  • Varint
  • Base58
  • Mnemonic
  • Subaddress
  • Ed25519
  • Cryptonote
• Create a workflow for publishing on Packagist
• Create a new repository in monero-integrations for the MoneroRpc package
• Write some documentation/redo the README

Most of my work is usually offline/on my local git server and I may not push to this PR's branch frequently.

I've spent some time with refring's code and tested it (it is very good). Merging it is just a matter of creating a new repository (I will CC serhack at the appropriate time)

[recanman]

An extremely important task is to have tests for these cryptographic operations (to make sure results are consistent of course). If anyone seeing this is willing to give an expected input/output for each function in Cryptonote.php while I'm refactoring, that would be really great (if it isn't done, I'll do it myself. Just in case anyone else wants to speed up the migration process).

[recanman]

Rewriting it with sodium does not seem feasible at the moment as certain functions are not too well-documented. I've decided to add types, tests, and clean up the code for ed25519 and Cryptonote.
The scalarmult function in the ed25519 class is recursive and incredibly slow (15+ seconds). I'll fix that as well.

Here's my progress on Cryptonote.php:

• keccak_256
• gen_new_hex_seed
• sc_reduce
• hash_to_scalar
• derive_viewKey
• gen_private_keys
• pk_from_sk
• gen_key_derivation
• derivation_to_scalar
• stealth_payment_id
• txpub_from_extra
• derive_public_key
• is_output_mine
• encode_address
• verify_checksum
• decode_address
• integrated_addr_from_keys
• address_from_seed
• generate_subaddr_secret_key
• generate_subaddress_spend_public_key
• generate_subaddr_view_public_key
• generate_subaddress
• deserialize_block_header

[recanman]

Just added the publishing workflow. I'm about 80% done with the rest of the refactoring, but yet to polish it up and push it here.
@serhack, could you create a new repository under this organization? My name suggestion would be monerophp-rpc.

[serhack]

@recanman I created the new repository

[sneurlax]

@recanman I'd like to help completing any remaining tasks, performing tests, adding unit tests, etc. ... I'll be reviewing this as it progresses and the other repo as well, if there's any areas still needing work please feel free to direct me towards it.

[recanman]

I really appreciate your help. I've been a bit busy.

[recanman]

I think that helping to implement the remaining functions would be great. I have struggled with signature and checkvalid.

[serhack]

Thanks to both of you.

[sneurlax]

I think that helping to implement the remaining functions would be great. I have struggled with signature and checkvalid.

@recanman I'm working on this. I'm syncing testnet and getting set up to run ... basing off of your recanman/monerophp/tree/refactor

phpunit throws

1) BCMATH_BigIntegerTest::testCreate
Error: Class 'MoneroIntegrations\MoneroCrypto\BigInteger' not found

and so on due to the classes not being found / imported properly. I'm just running phpunit in the root monerophp folder, do I need any args or what else do I need to do? This is on a fresh Ubuntu 20.04 with XAMPP and PHP on the commandline and PHPUnit installed according to those linked docs

Later I can update the documentation. For now I'm going to poke around and any tips would be appreciated

Also, can you recommend an IDE? I usually use JetBrains IDEs but meh

[sneurlax]

Just documenting as I go for reference later... I installed Composer, ran composer update, and composer install in monerophp. At this point I realized I had PHP 7.4 on the commandline (XAMPP installed 8.4 but only for itself), so I updated PHP to 8.3.10. Then I had to install PHP's bcmath extension as in sudo apt install php8.3-bcmath

sudo apt install php8.3-curl speeds composer up

but I still get

PHP Fatal error:  Cannot redeclare MoneroIntegrations\MoneroCrypto\Cryptonote::$network_prefixes in /home/user/src/monerophp/src/Cryptonote.php on line 61

which I'll work on massaging... Are you using a globally-installed PHPUnit or are you installing PHPUnit with Composer? And I see you're on Windows by your file format, I'll have to set my git to use Windows-style line endings... I also have a Windows dev box which I can work on for convenience sake. We should document the setup required for both Windows and Linux anyways. I also have a fresh macOS :D

[recanman]

Let me check my local machine, sorry about that.

[recanman]

@sneurlax I am not on Windows?

I use VSCodium mainly.

[recanman]

After running git stash I get the same error, fix is coming up.

By the way, try composer run test:unit instead of composer run test. That will run unit tests directly as there as some lint errors.

[sneurlax]

Ahh I was just running phpunit. composer run test:unit has much the same result tho even after this most recent commit

I have to take a break for awhile, if there are no more pushes by that time I'll start tinkering on what's here, thanks for this, love the approach to reimplement crypto fundamentals in PHP. I'll also contribute over in the rpc-centric repo. Cheers

[recanman]

I have pushed all of my local changes.
If you're able to figure out derive_viewKey, signature and checkvalid, that would be great.
I'll post some resources related to how these functions are calculated and hopefully you'll have some luck.

Most of the remaining Cryptonote functions are dependent on derive_viewKey.

(CI now passes! 😄)

[recanman]

https://getmonero.dev/cryptography/asymmetric/private-key.html
https://getmonero.dev/cryptography/asymmetric/public-key.html

I am rewriting the README to get it back up-to-date, and also to reduce confusion on how to run tests.

[sneurlax]

OK, just confirming that I got the tests working basically:

$ php -v
PHP 8.3.10 (cli) (built: Aug  2 2024 15:30:49) (NTS)
...
$ php -m
[PHP Modules]
bcmath
...
gmp
...
$ git clone https://github.com/recanman/monerophp.git
Cloning into 'monerophp'...
..
$ cd monerophp/
$ git checkout recanman/refactor
$ composer install
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Package operations: 17 installs, 0 updates, 0 removals
...
$ $ composer test:unit
> phpunit --testsuite unit
PHPUnit 10.5.29 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.10
Configuration: /home/user/src/monerophptesting/monerophp/phpunit.xml

.....W.....................WPHP Warning:  Constant S_MATH_BIGINTEGER_MODE already defined in /home/user/src/monerophptesting/monerophp/tests/unit/GMP_BigIntegerTest.php on line 13
PHP Stack trace:
PHP   1. {main}() /home/user/src/monerophptesting/monerophp/vendor/bin/phpunit:0
PHP   2. include() /home/user/src/monerophptesting/monerophp/vendor/bin/phpunit:122
PHP   3. PHPUnit\TextUI\Application->run($argv = [0 => '/home/user/src/monerophptesting/monerophp/vendor/bin/phpunit', 1 => '--testsuite', 2 => 'unit']) /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/phpunit:104
PHP   4. PHPUnit\TextUI\TestRunner->run($configuration = class PHPUnit\TextUI\Configuration\Configuration { private readonly array $cliArguments = []; private readonly ?string $configurationFile = '/home/user/src/monerophptesting/monerophp/phpunit.xml'; private readonly ?string $bootstrap = NULL; private readonly bool $cacheResult = TRUE; private readonly ?string $cacheDirectory = '/home/user/src/monerophptesting/monerophp/.phpunit.cache'; private readonly ?string $coverageCacheDirectory = '/home/user/src/monerophptesting/monerophp/.phpunit.cache/code-coverage'; private readonly PHPUnit\TextUI\Configuration\Source $source = class PHPUnit\TextUI\Configuration\Source { private readonly ?string $baseline = NULL; private readonly bool $ignoreBaseline = FALSE; private readonly PHPUnit\TextUI\Configuration\FilterDirectoryCollection $includeDirectories = class PHPUnit\TextUI\Configuration\FilterDirectoryCollection { ... }; private readonly PHPUnit\TextUI\Configuration\FileCollection $includeFiles = class PHPUnit\TextUI\Configuration\FileCollection { ... }; private readonly PHPUnit\TextUI\Configuration\FilterDirectoryCollection $excludeDirectories = class PHPUnit\TextUI\Configuration\FilterDirectoryCollection { ... }; private readonly PHPUnit\TextUI\Configuration\FileCollection $excludeFiles = class PHPUnit\TextUI\Configuration\FileCollection { ... }; private readonly bool $restrictDeprecations = FALSE; private readonly bool $restrictNotices = FALSE; private readonly bool $restrictWarnings = FALSE; private readonly bool $ignoreSuppressionOfDeprecations = FALSE; private readonly bool $ignoreSuppressionOfPhpDeprecations = FALSE; private readonly bool $ignoreSuppressionOfErrors = FALSE; private readonly bool $ignoreSuppressionOfNotices = FALSE; private readonly bool $ignoreSuppressionOfPhpNotices = FALSE; private readonly bool $ignoreSuppressionOfWarnings = FALSE; private readonly bool $ignoreSuppressionOfPhpWarnings = FALSE }; private readonly bool $pathCoverage = FALSE; private readonly ?string $coverageClover = NULL; private readonly ?string $coverageCobertura = NULL; private readonly ?string $coverageCrap4j = NULL; private readonly int $coverageCrap4jThreshold = 30; private readonly ?string $coverageHtml = NULL; private readonly int $coverageHtmlLowUpperBound = 50; private readonly int $coverageHtmlHighLowerBound = 90; private readonly string $coverageHtmlColorSuccessLow = '#dff0d8'; private readonly string $coverageHtmlColorSuccessMedium = '#c3e3b5'; private readonly string $coverageHtmlColorSuccessHigh = '#99cb84'; private readonly string $coverageHtmlColorWarning = '#fcf8e3'; private readonly string $coverageHtmlColorDanger = '#f2dede'; private readonly ?string $coverageHtmlCustomCssFile = NULL; private readonly ?string $coveragePhp = NULL; private readonly ?string $coverageText = NULL; private readonly bool $coverageTextShowUncoveredFiles = FALSE; private readonly bool $coverageTextShowOnlySummary = FALSE; private readonly ?string $coverageXml = NULL; private readonly string $testResultCacheFile = '/home/user/src/monerophptesting/monerophp/.phpunit.cache/test-results'; private readonly bool $ignoreDeprecatedCodeUnitsFromCodeCoverage = FALSE; private readonly bool $disableCodeCoverageIgnore = FALSE; private readonly bool $failOnDeprecation = TRUE; private readonly bool $failOnEmptyTestSuite = FALSE; private readonly bool $failOnIncomplete = FALSE; private readonly bool $failOnNotice = FALSE; private readonly bool $failOnRisky = FALSE; private readonly bool $failOnSkipped = FALSE; private readonly bool $failOnWarning = FALSE; private readonly bool $stopOnDefect = FALSE; private readonly bool $stopOnDeprecation = FALSE; private readonly bool $stopOnError = FALSE; private readonly bool $stopOnFailure = FALSE; private readonly bool $stopOnIncomplete = FALSE; private readonly bool $stopOnNotice = FALSE; private readonly bool $stopOnRisky = FALSE; private readonly bool $stopOnSkipped = FALSE; private readonly bool $stopOnWarning = FALSE; private readonly bool $outputToStandardErrorStream = FALSE; private readonly int $columns = 80; private readonly bool $noExtensions = FALSE; private readonly ?string $pharExtensionDirectory = NULL; private readonly array $extensionBootstrappers = []; private readonly bool $backupGlobals = FALSE; private readonly bool $backupStaticProperties = FALSE; private readonly bool $beStrictAboutChangesToGlobalState = FALSE; private readonly bool $colors = TRUE; private readonly bool $processIsolation = FALSE; private readonly bool $enforceTimeLimit = FALSE; private readonly int $defaultTimeLimit = 1; private readonly int $timeoutForSmallTests = 1; private readonly int $timeoutForMediumTests = 10; private readonly int $timeoutForLargeTests = 60; private readonly bool $reportUselessTests = TRUE; private readonly bool $strictCoverage = FALSE; private readonly bool $disallowTestOutput = FALSE; private readonly bool $displayDetailsOnIncompleteTests = FALSE; private readonly bool $displayDetailsOnSkippedTests = FALSE; private readonly bool $displayDetailsOnTestsThatTriggerDeprecations = TRUE; private readonly bool $displayDetailsOnTestsThatTriggerErrors = FALSE; private readonly bool $displayDetailsOnTestsThatTriggerNotices = FALSE; private readonly bool $displayDetailsOnTestsThatTriggerWarnings = FALSE; private readonly bool $reverseDefectList = FALSE; private readonly bool $requireCoverageMetadata = FALSE; private readonly bool $registerMockObjectsFromTestArgumentsRecursively = FALSE; private readonly bool $noProgress = FALSE; private readonly bool $noResults = FALSE; private readonly bool $noOutput = FALSE; private readonly int $executionOrder = 0; private readonly int $executionOrderDefects = 0; private readonly bool $resolveDependencies = TRUE; private readonly ?string $logfileTeamcity = NULL; private readonly ?string $logfileJunit = NULL; private readonly ?string $logfileTestdoxHtml = NULL; private readonly ?string $logfileTestdoxText = NULL; private readonly ?string $logEventsText = NULL; private readonly ?string $logEventsVerboseText = NULL; private readonly ?array $testsCovering = NULL; private readonly ?array $testsUsing = NULL; private readonly bool $teamCityOutput = FALSE; private readonly bool $testDoxOutput = FALSE; private readonly ?string $filter = NULL; private readonly ?array $groups = []; private readonly ?array $excludeGroups = []; private readonly int $randomOrderSeed = 1723586420; private readonly bool $includeUncoveredFiles = TRUE; private readonly PHPUnit\TextUI\Configuration\TestSuiteCollection $testSuite = class PHPUnit\TextUI\Configuration\TestSuiteCollection { private readonly array $testSuites = [...] }; private readonly string $includeTestSuite = 'unit'; private readonly string $excludeTestSuite = ''; private readonly ?string $defaultTestSuite = NULL; private readonly array $testSuffixes = [0 => 'Test.php', 1 => '.phpt']; private readonly PHPUnit\TextUI\Configuration\Php $php = class PHPUnit\TextUI\Configuration\Php { private readonly PHPUnit\TextUI\Configuration\DirectoryCollection $includePaths = class PHPUnit\TextUI\Configuration\DirectoryCollection { ... }; private readonly PHPUnit\TextUI\Configuration\IniSettingCollection $iniSettings = class PHPUnit\TextUI\Configuration\IniSettingCollection { ... }; private readonly PHPUnit\TextUI\Configuration\ConstantCollection $constants = class PHPUnit\TextUI\Configuration\ConstantCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $globalVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $envVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $postVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $getVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $cookieVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $serverVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $filesVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... }; private readonly PHPUnit\TextUI\Configuration\VariableCollection $requestVariables = class PHPUnit\TextUI\Configuration\VariableCollection { ... } }; private readonly bool $controlGarbageCollector = FALSE; private readonly int $numberOfTestsBeforeGarbageCollection = 100; private readonly ?string $generateBaseline = NULL; private readonly bool $debug = FALSE }, $resultCache = class PHPUnit\Runner\ResultCache\DefaultResultCache { private readonly string $cacheFilename = '/home/user/src/monerophptesting/monerophp/.phpunit.cache/test-results'; private array $defects = []; private array $times = ['BCMATH_BigIntegerTest::testCreate' => 0.002, 'BCMATH_BigIntegerTest::testCreateSafe' => 0, 'BCMATH_BigIntegerTest::testSpaces' => 0, 'BCMATH_BigIntegerTest::testOp' => 0.001, 'BCMATH_BigIntegerTest::testBig' => 0.002, 'Base58Test::testEncode' => 0.001, 'Base58Test::testDecode' => 0.001, 'CryptonoteTest::testKeccak256' => 0.001, 'CryptonoteTest::testGenNewHexSeed' => 0, 'CryptonoteTest::testScReduce' => 0, 'CryptonoteTest::testEncodeAddress' => 0.002, 'CryptonoteTest::testVerifyChecksum' => 0.004, 'CryptonoteTest::testDecodeAddress' => 0.003, 'CryptonoteTest::testIntegratedAddrFromKeys' => 0.002, 'Ed25519Test::testPoint' => 0.001, 'Ed25519Test::testH' => 0, 'Ed25519Test::testExpMod' => 0, 'Ed25519Test::testInv' => 0, 'Ed25519Test::testXrecover' => 0.017, 'Ed25519Test::testEdwardsAndEncodePoint' => 0.001, 'Ed25519Test::testScalarMult' => 0.002, 'Ed25519Test::testPublicKey' => 0.873, 'Ed25519Test::testEncodeInt' => 0, 'Ed25519Test::testBit' => 0, 'Ed25519Test::testHint' => 0, 'Ed25519Test::testIsOnCurve' => 0.001, 'Ed25519Test::testDecodeInt' => 0, 'Ed25519Test::testDecodePoint' => 0.017] }, $suite = class PHPUnit\Framework\TestSuite { private string $name = '/home/user/src/monerophptesting/monerophp/phpunit.xml'; private array $groups = []; private ?array $requiredTests = NULL; private array $tests = []; private ?array $providedTests = NULL; private ?PHPUnit\Runner\Filter\Factory $iteratorFilter = NULL; private bool $wasRun = TRUE }) /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/TextUI/Application.php:202
PHP   5. PHPUnit\Framework\TestSuite->run() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:64
PHP   6. PHPUnit\Framework\TestSuite->run() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:380
PHP   7. PHPUnit\Framework\TestSuite->run() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:380
PHP   8. PHPUnit\Framework\TestSuite->invokeMethodsBeforeFirstTest($emitter = class PHPUnit\Event\DispatchingEmitter { private readonly PHPUnit\Event\Dispatcher $dispatcher = class PHPUnit\Event\DeferringDispatcher { private readonly PHPUnit\Event\SubscribableDispatcher $dispatcher = class PHPUnit\Event\DirectDispatcher { ... }; private PHPUnit\Event\EventCollection $events = class PHPUnit\Event\EventCollection { ... }; private bool $recording = FALSE }; private readonly PHPUnit\Event\Telemetry\System $system = class PHPUnit\Event\Telemetry\System { private readonly PHPUnit\Event\Telemetry\StopWatch $stopWatch = class PHPUnit\Event\Telemetry\SystemStopWatch { ... }; private readonly PHPUnit\Event\Telemetry\MemoryMeter $memoryMeter = class PHPUnit\Event\Telemetry\SystemMemoryMeter { ... }; private readonly PHPUnit\Event\Telemetry\GarbageCollectorStatusProvider $garbageCollectorStatusProvider = class PHPUnit\Event\Telemetry\Php83GarbageCollectorStatusProvider { ... } }; private readonly PHPUnit\Event\Telemetry\Snapshot $startSnapshot = class PHPUnit\Event\Telemetry\Snapshot { private readonly PHPUnit\Event\Telemetry\HRTime $time = class PHPUnit\Event\Telemetry\HRTime { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $memoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $peakMemoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\GarbageCollectorStatus $garbageCollectorStatus = class PHPUnit\Event\Telemetry\GarbageCollectorStatus { ... } }; private PHPUnit\Event\Telemetry\Snapshot $previousSnapshot = class PHPUnit\Event\Telemetry\Snapshot { private readonly PHPUnit\Event\Telemetry\HRTime $time = class PHPUnit\Event\Telemetry\HRTime { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $memoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\MemoryUsage $peakMemoryUsage = class PHPUnit\Event\Telemetry\MemoryUsage { ... }; private readonly PHPUnit\Event\Telemetry\GarbageCollectorStatus $garbageCollectorStatus = class PHPUnit\Event\Telemetry\GarbageCollectorStatus { ... } }; private bool $exportObjects = FALSE }, $testSuiteValueObjectForEvents = class PHPUnit\Event\TestSuite\TestSuiteForTestClass { private readonly string ${PHPUnit\Event\TestSuite\TestSuite}name = 'GMP_BigIntegerTest'; private readonly int ${PHPUnit\Event\TestSuite\TestSuite}count = 5; private readonly PHPUnit\Event\Code\TestCollection ${PHPUnit\Event\TestSuite\TestSuite}tests = class PHPUnit\Event\Code\TestCollection { private readonly array $tests = [...] }; private readonly string $className = 'GMP_BigIntegerTest'; private readonly string $file = '/home/user/src/monerophptesting/monerophp/tests/unit/GMP_BigIntegerTest.php'; private readonly int $line = 8 }) /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:357
PHP   9. GMP_BigIntegerTest::setUpBeforeClass() /home/user/src/monerophptesting/monerophp/vendor/phpunit/phpunit/src/Framework/TestSuite.php:643
PHP  10. define($constant_name = 'S_MATH_BIGINTEGER_MODE', $value = 'gmp') /home/user/src/monerophptesting/monerophp/tests/unit/GMP_BigIntegerTest.php:13
........EEE.......                    46 / 46 (100%)

Time: 00:00.962, Memory: 12.00 MB

There were 3 errors:

1) MnemonicTest::testEncode
Error: Call to undefined function MoneroIntegrations\MoneroCrypto\gmp_init()

/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:100
/home/user/src/monerophptesting/monerophp/tests/unit/MnemonicTest.php:35

2) MnemonicTest::testEncodeWithChecksum
Error: Call to undefined function MoneroIntegrations\MoneroCrypto\gmp_init()

/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:100
/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:120
/home/user/src/monerophptesting/monerophp/tests/unit/MnemonicTest.php:40

3) MnemonicTest::testDecode
Error: Call to undefined function MoneroIntegrations\MoneroCrypto\gmp_add()

/home/user/src/monerophptesting/monerophp/src/Mnemonic.php:168
/home/user/src/monerophptesting/monerophp/tests/unit/MnemonicTest.php:45

ERRORS!
Tests: 46, Assertions: 228, Errors: 3, Warnings: 2.
Script phpunit --testsuite unit handling the test:unit event returned with error code 2

so I've begun working and will be back with updates as they come.

note that this doesn't imply any review and these sorts of changes will require thorough review