Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timestamp Data Model #1121

Merged
merged 16 commits into from
Jul 11, 2023
15 changes: 12 additions & 3 deletions partiql-types/src/main/kotlin/org/partiql/value/datetime/Date.kt
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,25 @@ public data class Date private constructor(
val month: Int,
val day: Int
) {

public companion object {
public fun of(year: Int, month: Int, day: Int): Date {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JvmStatic and @Throws

Also, side note: as it is, we are already computing the local date whenever we invoke of(). Why not have that be part of the init of Date and we can retain the value of LocalDate? As it stands, whenever we attempt to access property localDate we end up computing it a second time.

if (year < 1 || year > 9999) throw DateTimeException("Expect Year Field to be between 1 to 9999, but received $year")
if (year < 1 || year > 9999)
throw DateTimeFormatException("Expect Year Field to be between 1 to 9999, but received $year")
try {
LocalDate.of(year, month, day)
} catch (e: java.time.DateTimeException) {
throw DateTimeException(e.localizedMessage)
throw DateTimeFormatException(e.localizedMessage, e)
}
return Date(year, month, day)
}
}

private val localDate: LocalDate by lazy {
LocalDate.of(year, month, day)
}

public fun plusDays(daysToAdd: Long): Date {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this required to be public?

val newDate = this.localDate.plusDays(daysToAdd)
return of(newDate.year, newDate.monthValue, newDate.dayOfMonth)
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package org.partiql.value.datetime

/**
* This is its own class, because in the future we may want to migrate the time and date in the base,
* and be able to compare/case between those types. (having all three extend from Datetime interface)
*/
// TODO: Consider model Date, Time, Timestamp to have a common parent.
internal object DateTimeComparator {

internal fun compareTimestamp(left: Timestamp, right: Timestamp): Int {
return when {
left.timeZone != null && right.timeZone != null -> left.epochSecond.compareTo(right.epochSecond)
// for timestamp without time zones, assume UTC and compare
left.timeZone == null && right.timeZone == null -> {
left.copy(timeZone = TimeZone.UtcOffset.of(0))
.compareTo(right.copy(timeZone = TimeZone.UtcOffset.of(0)))
}

else -> throw DateTimeComparisonException(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this more of a TODO?

SQL-1999 Section 4.7 states:

Items of type datetime are mutually comparable only if they have the same <primary datetime
field>s.

Both Timestamp WITH/WITHOUT TZ contain the same <primary datetime field>s.

In the limited time, I couldn't determine which cast takes place for the comparison (in SQL-1999), but a quick Google search led me to how DB2 does it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather this casting to be handled by application, in this case probably the planner. I.e., cast(<a timestamp without time zone), TIMESTAMP WITH TIME ZONE) > <a timestamp with time zone>.

left,
right,
"Can not compare between timestamp with time zone and timestamp without time zone",
)
}
}
}
35 changes: 21 additions & 14 deletions partiql-types/src/main/kotlin/org/partiql/value/datetime/Time.kt
Original file line number Diff line number Diff line change
Expand Up @@ -24,32 +24,39 @@ public data class Time private constructor(
hour: Int,
minute: Int,
second: BigDecimal,
timeZone: TimeZone?,
precision: Int?
timeZone: TimeZone? = null,
precision: Int? = null
): Time {
try {
val hour = ChronoField.HOUR_OF_DAY.checkValidValue(hour.toLong()).toInt()
val minute = ChronoField.MINUTE_OF_HOUR.checkValidValue(minute.toLong()).toInt()
ChronoField.HOUR_OF_DAY.checkValidValue(hour.toLong())
ChronoField.MINUTE_OF_HOUR.checkValidValue(minute.toLong())
// round down the second to check
val wholeSecond = ChronoField.SECOND_OF_MINUTE.checkValidValue(second.setScale(0, RoundingMode.DOWN).toLong())
ChronoField.SECOND_OF_MINUTE.checkValidValue(second.setScale(0, RoundingMode.DOWN).toLong())
val arbitraryTime = Time(hour, minute, second, timeZone, null)
if (precision == null) {
return arbitraryTime
}
if (precision == null) { return arbitraryTime }
return arbitraryTime.toPrecision(precision)
} catch (e: java.time.DateTimeException) {
throw DateTimeException(e.localizedMessage)
} catch (e: IllegalStateException) {
throw DateTimeException(e.localizedMessage)
} catch (e: IllegalArgumentException) {
throw DateTimeException(e.localizedMessage)
throw DateTimeFormatException(e.localizedMessage, e)
yliuuuu marked this conversation as resolved.
Show resolved Hide resolved
}
}
}

/**
* Counting the time escaped from midnight 00:00:00 in seconds ( fraction included)
*/
val elapsedSecond: BigDecimal by lazy {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

internal

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And, roundToPrecision can use this value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may not need to be internal as this is a commonly exposed property in java's time package.
Maybe keep it as is and wait for the evaluation PR to confirm?

BigDecimal.valueOf(this.hour * SECONDS_IN_HOUR + this.minute * SECONDS_IN_MINUTE).plus(this.second)
}

private fun toPrecision(precision: Int) =
when {
second.scale() == precision -> this
second.scale() == precision -> this.copy(
hour = hour,
minute = minute,
second = second,
timeZone = timeZone,
precision = precision
)
second.scale() < precision -> paddingToPrecision(precision)
else -> roundToPrecision(precision)
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ public sealed class TimeZone {
* Notice if the time zone is a negative offset, then both [tzHour] and [tzMinute] needs to be negative.
*/
public fun of(tzHour: Int, tzMinute: Int): UtcOffset {
if (abs(tzHour) > MAX_TIME_ZONE_HOURS) throw DateTimeException("Except Timezone Hour to be less than 24, but received $tzHour")
yliuuuu marked this conversation as resolved.
Show resolved Hide resolved
if (abs(tzHour) > MAX_TIME_ZONE_MINUTES) throw DateTimeException("Except Timezone Minute to be less than 60, but received $tzMinute")
if (abs(tzHour) > MAX_TIME_ZONE_HOURS) throw DateTimeFormatException("Except Timezone Hour to be less than 24, but received $tzHour")
if (abs(tzMinute) > MAX_TIME_ZONE_MINUTES) throw DateTimeFormatException("Except Timezone Minute to be less than 60, but received $tzMinute")
return UtcOffset(tzHour * 60 + tzMinute)
}

public fun of(totalOffsetMinutes: Int): UtcOffset {
if (abs(totalOffsetMinutes) > MAX_TOTAL_OFFSET_MINUTES) throw DateTimeException("Expect total offset Minutes to be less than or equal to $MAX_TOTAL_OFFSET_MINUTES, but received $totalOffsetMinutes")
if (abs(totalOffsetMinutes) > MAX_TOTAL_OFFSET_MINUTES) throw DateTimeFormatException("Expect total offset Minutes to be less than or equal to $MAX_TOTAL_OFFSET_MINUTES, but received $totalOffsetMinutes")
return UtcOffset(totalOffsetMinutes)
}
}
Expand Down
182 changes: 167 additions & 15 deletions partiql-types/src/main/kotlin/org/partiql/value/datetime/Timestamp.kt
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
package org.partiql.value.datetime

import java.math.BigDecimal
import java.time.LocalDate
import com.amazon.ion.Timestamp as TimestampIon

/**
* This class is used to model both Timestamp Without Time Zone type and Timestamp With Time Zone Type.
*
* Two timestamp values are equal if and only if all the fields (including precision) are the same.
*
* Use [compareTo] if the goal is to check equivalence (refer to the same point in time).
*/
public data class Timestamp(
val year: Int,
Expand All @@ -17,29 +22,176 @@ public data class Timestamp(
val precision: Int?
yliuuuu marked this conversation as resolved.
Show resolved Hide resolved
) {
public companion object {
/**
* The intention of this API is to create a Timestamp using a Date and a Time component.
* It is assumed that the time component has already been rounded,
* meaning this API will not be responsible, if the rounding requires carrying in to day field.
*
* For example: the result of
* Timestamp.of(Date.of(2023, 06, 01), Time.of(23, 59, 59.9, TimeZone.of(0,0), 0)
* will result in
* 2023-06-01T00:00:00+00:00
*
* If the desired result is `2023-06-02T00:00:00+00:00`, use [of]
*/
@JvmStatic
public fun of(date: Date, time: Time): Timestamp =
Timestamp(
date.year, date.month, date.day,
time.hour, time.minute, time.second,
time.timeZone, time.precision
)

public fun of(ionTs: TimestampIon) {
if (ionTs.localOffset == null) {
Timestamp(
ionTs.year, ionTs.month, ionTs.day,
ionTs.hour, ionTs.minute, ionTs.decimalSecond,
TimeZone.UnknownTimeZone,
null
)
} else {
Timestamp(
ionTs.year, ionTs.month, ionTs.day,
ionTs.hour, ionTs.minute, ionTs.decimalSecond,
TimeZone.UtcOffset.of(ionTs.localOffset),
null
)
public fun of(
yliuuuu marked this conversation as resolved.
Show resolved Hide resolved
year: Int,
month: Int,
day: Int,
hour: Int,
minute: Int,
second: BigDecimal,
timeZone: TimeZone?,
precision: Int? = null
): Timestamp {
val date = Date.of(year, month, day)
val arbitraryTime = Time.of(hour, minute, second, timeZone)
val roundedTime = Time.of(hour, minute, second, timeZone, precision)
// if the rounding result and the original result differs in more than 1 second, then we need to carry to date
return when ((arbitraryTime.elapsedSecond - roundedTime.elapsedSecond).abs() > BigDecimal.ONE) {
true -> of(date.plusDays(1L), roundedTime)
false -> {
of(date, roundedTime)
}
}
}

@JvmStatic
public fun of(ionTs: TimestampIon): Timestamp {
val timestamp = when (ionTs.localOffset) {
null ->
Timestamp(
ionTs.year, ionTs.month, ionTs.day,
ionTs.hour, ionTs.minute, ionTs.decimalSecond,
TimeZone.UnknownTimeZone,
null
)

else ->
Timestamp(
ionTs.year, ionTs.month, ionTs.day,
ionTs.hour, ionTs.minute, ionTs.decimalSecond,
TimeZone.UtcOffset.of(ionTs.localOffset),
null
)
}
return timestamp.let {
it.ionRaw = ionTs
it
}
}
}

/**
* Returns an Ion Equivalent Timestamp
* [ionTimestampValue] takes care of the precision,
* the ion representation will have exact precision of the backing partiQL value.
*/
val ionTimestampValue: TimestampIon by lazy {
yliuuuu marked this conversation as resolved.
Show resolved Hide resolved
when (val timeZone = this.timeZone) {
null -> throw DateTimeException("Timestamp without Time Zone has no corresponding Ion Value")
TimeZone.UnknownTimeZone -> TimestampIon.forSecond(
year, month, day,
hour, minute, second,
TimestampIon.UNKNOWN_OFFSET
)

is TimeZone.UtcOffset -> TimestampIon.forSecond(
year, month, day,
hour, minute, second,
timeZone.totalOffsetMinutes
)
}
}

/**
* Returns a BigDecimal representing the Timestamp's point in time that is
* the number of Seconds (*including* any fractional Seconds)
* from the epoch.
*
* Since PartiQL support a wider range than Unix Time,
* timestamp value that is before 1970-01-01T00:00:00Z will have a negative epoch value.
*
* If a timestamp does not contain Information regarding timezone,
* there is no way to assign a point in time for such value, therefore the method will throw an error.
*
* If a timestamp contains unknown timezone, by semantics its UTC value is known, we return the UTC value in epoch.
*
* This method will return the same result for all Timestamp values representing
* the same point in time, regardless of the local offset.
*/
val epochSecond: BigDecimal by lazy {
yliuuuu marked this conversation as resolved.
Show resolved Hide resolved
when (val timeZone = this.timeZone) {
null -> throw DateTimeException("Timestamp without time zone has no Epoch Second attribute.")
TimeZone.UnknownTimeZone -> getUTCEpoch(0)
is TimeZone.UtcOffset -> getUTCEpoch(timeZone.totalOffsetMinutes)
}
}

val epochMillis: BigDecimal by lazy {
epochSecond.movePointRight(3)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exception to be caught?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be safe.

}

/**
* For backward compatibility issue, we track the original Ion Input
*/
@Deprecated(
"We will not store raw Ion Timestamp Value in the next release.",
replaceWith = ReplaceWith("ionTimestampValue")
)
var ionRaw: TimestampIon? = null
johnedquinn marked this conversation as resolved.
Show resolved Hide resolved

/**
* Comparison method for timestamp value
* If one value is timestamp with time zone and the other is timestamp without time zone:
* Error
* One may not directly compare a timestamp value with timezone to a timestamp value without time zone.
* If both value are timestamp with timezone:
* The two value are considered equivalent if they refer to the same point in time.
* If both value are timestamp without timezone:
* The two value are consider equivalent if all the fields (exclude precision) are equivalent.
* Another way to interpret this is, two timestamp without timezone are equivalent
* if and only if they are equivalent when converted to the same time zone
*
* It is worth to distinguish between the [equals] method and the [compareTo] method.
*
* For example:
* The [equals] method will return false for the following timestamp with time zone values, but [compareTo] will return 0.
* ```
* // Timestamp with time zone
* Timestamp(1970, 1, 1, 0, 0, BigDecimal.valueOf(0, 3), 0, 0, null) // arbitrary precision 1970-01-01T00:00:00.000Z
* Timestamp(1970, 1, 1, 0, 0, BigDecimal.valueOf(0, 1), 0, 0, null) // arbitrary precision 1970-01-01T00:00:00.0Z
* Timestamp(1970, 1, 1, 0, 0, BigDecimal.valueOf(0, 3), 0, 0, 3) // precision 3, 1970-01-01T00:00:00.000Z
* Timestamp(1970, 1, 1, 0, 0, BigDecimal.valueOf(0, 1), 0, 0, 1) // precision 1, 1970-01-01T00:00:00.0Z
* Timestamp(1969, 12, 31, 23, 59, BigDecimal.valueOf(59.9, 1), 0, 0, 0) // precision 0, 1970-01-01T00:00:00Z
* Timestamp(1970, 1, 1, 1, 0, BigDecimal.valueOf(0, 1), 1, 0, 1) // precision 1, 1970-01-01T01:00:00.0+01:00
* ```
* The [equals] method will return false for the following timestamp without time zone values, but [compareTo] will return 0.
* ```
* // Timestamp with time zone
* Timestamp(1970, 1, 1, 0, 0, BigDecimal.valueOf(0, 3), null) // arbitrary precision 1970-01-01T00:00:00.000
* Timestamp(1970, 1, 1, 0, 0, BigDecimal.valueOf(0, 1), null) // arbitrary precision 1970-01-01T00:00:00.0
* Timestamp(1970, 1, 1, 0, 0, BigDecimal.valueOf(0, 3), 3) // precision 3, 1970-01-01T00:00:00.000
* Timestamp(1970, 1, 1, 0, 0, BigDecimal.valueOf(0, 1), 1) // precision 1, 1970-01-01T00:00:00.0
* Timestamp(1969, 12, 31, 23, 59, BigDecimal.valueOf(59.9, 1), 0) // precision 0, 1970-01-01T00:00:00
* ```
*
*/
public fun compareTo(other: Timestamp): Int = DateTimeComparator.compareTimestamp(this, other)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could have Timestamp implement Comparable<DateTime> (or parameterized by Timestamp) instead.


private fun getUTCEpoch(totalOffsetMinutes: Int): BigDecimal {
val epochDay = LocalDate.of(year, month, day).toEpochDay()
val excludedSecond = epochDay * SECONDS_IN_DAY + hour * SECONDS_IN_HOUR + minute * SECONDS_IN_MINUTE
// since offset does not include second field, we can adjust it here, and leave bigDecimal calculation later
val adjusted = excludedSecond - totalOffsetMinutes * SECONDS_IN_MINUTE
return second.plus(BigDecimal.valueOf(adjusted))
}
}
20 changes: 17 additions & 3 deletions partiql-types/src/main/kotlin/org/partiql/value/datetime/Util.kt
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,24 @@ internal const val MILLIS_IN_HOUR = 60 * MILLIS_IN_MINUTE
internal const val MILLIS_IN_DAY = 24 * MILLIS_IN_HOUR
internal const val SECONDS_IN_MINUTE = 60L
internal const val SECONDS_IN_HOUR = 60 * SECONDS_IN_MINUTE
internal const val SECONDS_IN_DAY = 24 * SECONDS_IN_HOUR
internal const val MAX_TIME_ZONE_HOURS: Int = 23
internal const val MAX_TIME_ZONE_MINUTES: Int = 59
internal const val MAX_TOTAL_OFFSET_MINUTES: Int = MAX_TIME_ZONE_HOURS * 60 + MAX_TIME_ZONE_MINUTES

public class DateTimeException(
public override val message: String
) : Throwable()
public open class DateTimeException(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want others to be able to implement DateTimeException? If not, we can make sealed and potentially move to a dedicated file for better visibility. Also, a KDoc would be nice.

public override val message: String? = null,
public override val cause: Throwable? = null
) : RuntimeException()

public class DateTimeFormatException(
public override val message: String? = null,
public override val cause: Throwable? = null
) : DateTimeException(message, cause)

public class DateTimeComparisonException(
public val timestamp1: Timestamp,
public val timestamp2: Timestamp,
public val reason: String? = null,
public override val cause: Throwable? = null,
) : DateTimeException("Can not compare $timestamp1 with $timestamp2. $reason", cause)
Loading