Skip to content

Map TIMESTAMP_MILLIS to FROM_UNIXTIME#135

Merged
gauravdawar-e6 merged 1 commit intoe6data:mainfrom
tkaunlaky-e6:Timestamp_Millis
Jul 31, 2025
Merged

Map TIMESTAMP_MILLIS to FROM_UNIXTIME#135
gauravdawar-e6 merged 1 commit intoe6data:mainfrom
tkaunlaky-e6:Timestamp_Millis

Conversation

@tkaunlaky-e6
Copy link

@tkaunlaky-e6 tkaunlaky-e6 commented Jul 21, 2025

Summary

  • Refactored TIMESTAMP_MILLIS mapping to use existing UnixToTime expression instead of creating a new class
  • Maps Databricks TIMESTAMP_MILLIS(x) to E6 FROM_UNIXTIME_WITHUNIT(x, 'milliseconds')
  • Enhanced E6 generator to handle UnixToTime with scale parameter

Changes

  1. Removed TimestampMillis class from expressions.py - Avoids unnecessary class proliferation
  2. Updated Databricks parser - TIMESTAMP_MILLIS now directly creates UnixToTime with scale='milliseconds'
  3. Enhanced E6 generator - from_unixtime_sql now checks for scale parameter and generates FROM_UNIXTIME_WITHUNIT when appropriate

Test plan

  • All 857 tests pass
  • Existing TIMESTAMP_MILLIS tests verify the transpilation works correctly
  • Databricks TIMESTAMP_MILLIS(1230219000123) → E6 FROM_UNIXTIME_WITHUNIT(1230219000123, 'milliseconds')

gauravdawar-e6
gauravdawar-e6 previously approved these changes Jul 31, 2025
Copy link
Collaborator

@gauravdawar-e6 gauravdawar-e6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Adithyak-0926
Adithyak-0926 previously approved these changes Jul 31, 2025
ranjanankur314
ranjanankur314 previously approved these changes Jul 31, 2025
Copy link

@ranjanankur314 ranjanankur314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

- Add TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions to Spark parser (Spark 3.1+)
- Add TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions to Databricks parser
- Convert microseconds to milliseconds by dividing by 1000 in E6 dialect
- Use proper UnixToTime scale constants for consistency across dialects
Copy link
Collaborator

@gauravdawar-e6 gauravdawar-e6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@gauravdawar-e6 gauravdawar-e6 merged commit 496b24b into e6data:main Jul 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants