Fix Template Literal String Conversion for ECMAScript Compliance

[dilbertKocik]

Summary

This commit addresses ECMAScript specification compliance issues with template literal string conversion by introducing a dedicated STRING_CONCAT token and operation that properly converts objects to strings using toString() semantics instead of the default ToPrimitive behavior.

Problem

Template literals like ${Object(1n)} were incorrectly using the default ToPrimitive conversion (which calls valueOf() first) instead of string conversion semantics (which calls toString() first). This caused Test262 compliance failures where custom toString() methods on BigInt and Symbol wrapper objects were not being invoked correctly.

Before: Template literals were transformed into regular ADD operations, causing ${Object(1n)} to return "1" instead of the expected custom toString() result.

After: Template literals use dedicated STRING_CONCAT operations that properly call ScriptRuntime.toString(), ensuring ECMAScript compliance.

Changes Made

1. New Token Type: Added Token.STRING_CONCAT to distinguish template literal string concatenation from arithmetic addition operations.

2. IRFactory Enhancement: Modified transformTemplateLiteral() to use STRING_CONCAT instead of ADD tokens, ensuring template literal substitutions follow string conversion semantics.

3. Interpreter Support: Implemented DoStringConcat instruction class that directly converts operands to strings using ScriptRuntime.toString() before concatenation.

4. Optimizer Support: Added STRING_CONCAT handling in BodyCodegen and OptRuntime.concat() method for optimized execution paths.

5. Test Improvements: Removed 2 failing Test262 tests from the exclusion list:

   • built-ins/BigInt/wrapper-object-ordinary-toprimitive.js
   • built-ins/Symbol/prototype/Symbol.toPrimitive/removed-symbol-wrapper-ordinary-toprimitive.js

Technical Details

• Root Cause: Template literals were incorrectly sharing the same code path as arithmetic addition, causing wrong ToPrimitive hint usage
• Solution: Separate template literal string concatenation from arithmetic operations at the token level
• Performance: Direct string conversion eliminates unnecessary ToPrimitive calls and type checking overhead
• Compliance: Ensures template literals follow ECMAScript specification for string conversion semantics

Impact

• ✅ Resolves ES2015 Issue with Template Literal Object > string substitution #1406
• ✅ Fixes ECMAScript compliance for template literal string conversion
• ✅ Improves Test262 conformance (2 additional tests now pass)
• ✅ Maintains backward compatibility for existing code
• ✅ Optimizes template literal performance by avoiding unnecessary type conversions

This change brings Rhino's template literal implementation into full compliance with the ECMAScript specification while maintaining performance and backward compatibility.

[rbri]

@dilbertKocik many thanks for this PR.
Even if only two more tests are successful, I think that's a real step forward.

The CI failures are because you have manually edited the test262.properties. The idea of this is to regenerate the file - see https://github.com/mozilla/rhino/tree/master/tests#updating-the-test262properties-file for more

It will be great if you can regenerate the file according to the docu - i'm sure the ci build will pass then and we can merge your great PR.

[rbri]

@dilbertKocik maybe you have an idea about a good place in the code to document the idea behind the new token a bit. Hope this will help others in the future to understand why we have the two add tokens.

[rbri]

@dilbertKocik can you please rebase and force push again, i see some conflicts here

[dilbertKocik]

@rbri Yeah, I see it. I rebased my branch locally but I'm seeing a large diff in the test262.properties file (approximately 500 lines). I'm trying to figure out why that's happening. All the other changes look the same as the PR. I tried regenerating the file but it looks like the same large diff. I'll try to sort this out on Monday.

[dilbertKocik]

@rbri Can you tell me if this test update makes sense? I don't understand how so many tests could be affected. But this is what I get when I run the generator per the documentation.

[rbri]

@dilbertKocik will have a look

the ci fails because your new comments are violating the spotless rules

Run './gradlew :rhino:spotlessApply' to fix these violations.

[rbri]

@dilbertKocik looks like you have reverted the update of the linked test262 project as part of your commits. Therefore you get this many differences. Not sure how to sort this out - maybe the easiest way is to start a new branch from head and copy your changed files and make a new pr out of this

[dilbertKocik]

@rbri Thanks for the help. Starting from head and cherry picking was great advice.

[gbrail]

Thanks!

I'd really like to see the actual "concat" logic not be duplicated between interpreted and compiled mode -- see the comments. Thanks!

[gbrail]

See below, but this is an opportunity to consolidate the "concat" logic into one call into ScriptRuntime rather than duplicating it in two places.

[gbrail]

Could you please put this function in "ScriptRuntime" and then use it in both the interpreter code (see above) and in the bytecode? That way we only have the logic in here once. Yes, it's only three lines of code but it still means that we can put them together in the same place to change them in the future if we want.

[dilbertKocik]

This is a good idea. Thanks for the suggestion.

[rbri]

@dilbertKocik thanks a lot, hope you provide more geat stuff like this in the future