Expand ARM Architecture Compatibility

[halibobo1205]

Background

Java-Tron currently only supports the x86 architecture. Nevertheless, ARM architecture has gained significant traction recently, especially in cloud computing and mobile devices. ARM processors are known for their energy efficiency and cost-effectiveness, making them increasingly popular in data centers, cloud computing, and edge computing scenarios. It would be great to have the option to run Java-Tron on the ARM architecture.

Key developments in ARM architecture:

• Apple's transition from Intel to its ARM-based processors for Macs, dubbed Apple Silicon
• AWS Graviton, Microsoft Azure Ampere Altra, Alibaba Cloud Yitian 710 and Google Axion processors used in cloud computing
• Increasing adoption of ARM in supercomputers and high-performance computing

ARM advantages:

• Better performance per watt, low energy consumption, and low cost
  • 23% Cost savings and 36% performance gain by deploying GitLab on Arm-based AWS Graviton2
• Lower total cost of ownership for data centers
• Growing ecosystem and software support

Related Issues and PRs

• RocksDB Linux ARM support. #5845
• M3 MacBook Pro / aarch64 unable to run java-tron node. #5815
• Java-tron - Release linux/arm64 builds #4692
• Why my node stopped sync? #4592
• run private chain on mac(m2) with errors #4548
• FullNode.jar doesn't start without any error message #4534
• Building on ARM64 fails #4525
• Update protoc-gen-grpc-java dependency to > v1.26.0 to be able to build in ARM64 #4248
• Run java-tron on ARM - AWS Arm-based Graviton2 instances / Apple M1 #4132
• Tron compile with Apple Silicon MacBook Pro m1 #3834
• Build failed Raspbian #363

Scope of Impact

• Build and deployment processes
• Core application code
• Third-party dependencies
• Development and testing environments

Current Progress Summary

1. JDK version

   • x86 is limited to JDK 1.8
   • arm is limited to JDK 17

2. Native code

   • RocksDBJNI: org.rocksdb:rocksdbjni:9.7.4
   • zksnark-java-sdk: io.github.tronprotocol:zksnark-java-sdk:1.0.0

3. Third-party dependencies

   • protoc-gen-grpc-java: GreatVoyage-v4.7.3 released Oct 25, 2023, upgraded io.grpc:protoc-gen-grpc-java from 1.9.0 to 1.52.1 for ARM64 architecture by feat(net):update com.google.protobuf and io.grpc version #5254
   • lombok: 1.18.12->1.18.34(Lombok Changelog)
   • aspectjrt: 1.8.13->1.9.8 AspectJ Java version compatibility
   • add javax.annotation:1.3.2 and javax.jws:1.1, JEP 320: Remove the Java EE and CORBA Modules

4. Floating-point arithmetic

   • Migrate pow operation from java.lang.Math to java.lang.StrictMath in GreatVoyage-v4.7.7(Epicurus)
   • Replace java.math with the cross-platform consistent java.strictMath in GreatVoyage-v4.8.0(Kant)
   • For historical data(the Bancor trading pair), Hardcoded Special Cases(48 POW calculation instances)

5. Build and deployment process

   • Update build scripts to support ARM architecture.
   • Ensure CI/CD pipelines can be built and tested in ARM environments.
   • Docker support

6. only supports RocksDB

   • FullNode.jar and Toolkit.jar only support RocksDB on ARM
   • Tookit.jar db archive,db convert commands are not supported on arm
   • ArchiveManifest.jar is not supported on arm

7. x86 vs ARM Performance Comparison

Test Category	Test Type	Winner	Performance Difference	Notes
Database Read	Random Read	X86	X86 outperforms ARM by ~60%	Significant advantage in random I/O
Database Read	Sequential Read	ARM	ARM outperforms X86 by ~6%	Better sequential access efficiency
Serialization	Serialization	ARM	ARM outperforms X86 by ~170%	Substantial advantage in data serialization
Serialization	Deserialization	ARM	ARM outperforms X86 by ~3%	Minimal but consistent advantage
Transaction	TRX Transfer	X86	X86 outperforms ARM by ~23.8%	Better TRX transaction processing
Transaction	USDT Transfer	Comparable	No significant difference	Similar performance on both platforms
Signature Verification	Serial Processing	X86	X86 outperforms ARM by ~14%	Single-threaded verification advantage
Signature Verification	Parallel Processing	ARM	ARM outperforms X86 by ~18%	Multi-threaded verification advantage

8. Related PRs
   • feat(architecture): support arm64 based on JDK17 #6327
   • refactor(db): unify database engine initialization #6421
   • feat(db): update to rocksdb 9 for arm #6440
   • feat(exit): throw TronError if JDK does not support #6455
   • feat(jvm): optimize JVM GC logging for arm64 #6457
   • feat(runtime): enforce Java 17 for ARM64 #6459
   • feat(db/rocksdb): improve resource management with try-with-resources #6472
   • feat(protocol/protoc): support macOS ARM64 natively compile #6502

[angrynurd]

I am totally in favor of extending ARM architecture compatibility. This will allow Java-Tron to run on more platforms and take advantage of the benefits of the ARM architecture, such as higher energy efficiency and lower cost.
In my opinion, we can start with the following:

1. Prioritize ARM support for key dependencies: For example, RocksDB/LevelDB is an important database component in Java-Tron and it is critical to ensure its compatibility on ARM.
2. Establish an ARM test environment: We need to establish a dedicated ARM test environment to ensure the stability and performance of Java-Tron on ARM.
3. Collaborate with the community: We can work with the community to solve ARM compatibility issues and share experiences and best practices.

[tomatoishealthy]

It sounds great, but I am a novice in ARM architecture. I am curious about the challenges of supporting ARM architecture.

Can you list something like a task list in the future? It is convenient to clearly understand the current status and future challenges.

[halibobo1205]

Here are some common considerations:

Important

1. JDK version compatibility
   Ensure the JDK version supports ARM Architecture. Consider using ARM-optimized JDK distributions.

• Linux got support in JDK 9(non-LTS) by JEP 237
• Windows got support in JDK 16(non-LTS) by JEP 388
• Macs got support in JDK 17(LTS) by JEP 391

Important

2. Native code
JNI (Java Native Interface) or other native code.
These native code components need to be recompiled or upgraded for ARM architecture.

• LevelDBJni
• RocksDBJni
• zksnark-java-sdk

Tip

3. Endianness
x86 is little-endian, while some ARM processors may be big-endian.
Check if any operations in the code(such as TVM) depend on a specific endianness, especially when handling binary data.

Tip

4. Memory alignment:
ARM architecture may have different memory alignment requirements than x86.
Check for code(such as TVM) that assumes specific memory alignments.

Tip

5. Atomic operations and concurrency
Some atomic operations(TVM) may be implemented differently on different architectures.
Review concurrent code to ensure it works correctly on ARM as well.

Caution

6. Floating-point arithmetic
ARM and x86 may have subtle differences in floating-point precision and behavior.
For applications that rely on precise floating-point calculations, comprehensive testing is necessary.

Tip

7. Performance optimization
x86-specific performance optimizations(TVM) may no longer be applicable on ARM.
Consider using ARM-specific optimization techniques.

Important

8. Third-party dependencies
Ensure all third-party libraries and dependencies support ARM architecture.
Some incompatible dependencies may need to be updated or replaced.

• protoc-gen-grpc-java

Important

9. Build and deployment process:

• Update build scripts to support ARM architecture.
• Ensure CI/CD pipelines can be built and tested in ARM environments.
• Docker support

Tip

10. Hardware feature dependencies:
Check if the code(TVM) relies on x86-specific hardware features.
Alternatives may need to be found for ARM.

Tip

11. System calls and OS interactions
If the code makes direct system calls, adjustments may be needed for ARM.

Important

12. Cross-platform testing

• Establish comprehensive test suites to ensure the functionality works correctly on ARM.
• Conduct performance benchmarking to compare x86 and ARM performance differences.

[317787106]

@halibobo1205 Do you want to support ARM Architecture and latest JVM version at the same time ? Or just support ARM Architecture using JDK8 ?

[halibobo1205]

@317787106 JVM officially supports ARM:

• Linux got support in JDK 9 by JEP 237
• Windows got support in JDK 16 by JEP 388
• Macs got support in JDK 17 by JEP 391

According to Oracle Java SE Support Roadmap, JDK9 and JDK16 are non-LTS, and JDK 17 is LTS. Based on the above information, I propose that ARM support JDK17 as a minimum.

Warning

This is the last planned update of JDK 17 under the NFTC. Updates after September 2024 will be licensed under the Java SE OTN License (OTN) and production use beyond the limited free grants of the OTN license will require a fee.

[angrynurd]

Here are some common considerations:

---

Regarding JDK version compatibility.
You recommend using an ARM-optimized JDK distribution. What specific ARM-optimized JDK distributions do you recommend? What are their performance and stability advantages?

[abn2357]

When is the expected completion time for this work? It sounds like a big project.

[halibobo1205]

@endiaoekoe I propose that ARM support JDK17 as a minimum.

[halibobo1205]

@abn2357 Tron currently only supports JDK 8, based on the above information, JDK17 supports ARM fully, perhaps Tron needs to upgrade JDK17 first, which is another big project.

[zeusoo001]

@halibobo1205 It sounds great, and I look forward to your implementation. I see that there may be subtle differences in floating point precision and behavior between ARM and x86. When supporting it, be sure to ensure data consistency. Also investigate whether there are other places that may cause data inconsistency.

[Murphytron]

This issue has been added to the core devs community call #22, welcome to share the latest progress @halibobo1205, and discuss together with @endiaoekoe @tomatoishealthy @317787106 @zeusoo001 @abn2357.

[halibobo1205]

1. JDK version compatibility
After some brief research, I found ARM 64-bit versions of JDK 8 available. cc @endiaoekoe @317787106 @abn2357

Provider	Linux	Mac	Windows	Notes
Oracle	✓	✓	✗	• Official support • Requires payment for commercial use
Eclipse Temurin	✓	✗	✗	• Free OpenJDK • Regularly updated and supported by the Adoptium community
Azul Zulu	✓	✓	✗	• Free OpenJDK • Full enterprise version requires payment
BellSoft Liberica	✓	✓	✗	• Free OpenJDK for all users • Relatively less well-known
Amazon Corretto	✓	✓	✗	• Free OpenJDK • long-term support by Amazon • Amazon runs Corretto internally on thousands of production services

[halibobo1205]

6. Floating-point arithmetic known issues:

• x87 FPU for some Java Math
  • Don't use x87 FPU on x86-64
  • Misuse of Math library without strictfp

Unfortunately, Tron does use Math.pow() for floating-point calculations for the Bancor trading pair in ExchangeProcessor:

private long exchangeToSupply(long balance, long quant) {
    logger.debug("balance: " + balance);
    long newBalance = balance + quant;
    logger.debug("balance + quant: " + newBalance);

    double issuedSupply = -supply * (1.0 - Math.pow(1.0 + (double) quant / newBalance, 0.0005));
    logger.debug("issuedSupply: " + issuedSupply);
    long out = (long) issuedSupply;
    supply += out;

    return out;
  }

  private long exchangeFromSupply(long balance, long supplyQuant) {
    supply -= supplyQuant;

    double exchangeBalance =
        balance * (Math.pow(1.0 + (double) supplyQuant / supply, 2000.0) - 1.0);
    logger.debug("exchangeBalance: " + exchangeBalance);

    return (long) exchangeBalance;
  }

Test case

 @Test
  public void testPow() {
    double x = 29218;
    double q = 4761432;
    double ret = Math.pow(1.0 + x / q, 0.0005);
    double ret2 = StrictMath.pow(1.0 + x / q, 0.0005);

    System.out.printf("%s%n", doubleToHex(ret)); //  3ff000033518c576
    System.out.printf("%s%n", doubleToHex(ret2)); // 3ff000033518c575
    Assert.assertEquals(0, Double.compare(ret, ret2)); // fail in jdk8_X86, success in jdk8_ARM64
  }

  public static String doubleToHex(double input) {
    // Convert the starting value to the equivalent value in a long
    long doubleAsLong = Double.doubleToRawLongBits(input);
    // and then convert the long to a hex string
    return Long.toHexString(doubleAsLong);
  }

Tron Should Use StrictMath to Avoid Cross-Platform Consistency Issues. To help ensure the portability on ARM for Java-Tron, I suggest a new proposal to convert Math to StrictMath. cc @zeusoo001