Add a ZonedDateTime type

[cromefire]

Add a type that aligns with the normal calendar and combines TimeZone information with a date.

[cromefire]

@dkhalanskyjb

The same argument can be made for date + time.

It can't because 1) waiting for an hour at 23:31 will start the next day 2) the set of valid times depends on the date.

1. It's just a PITA to have APIs using Pair.
2. How would you e.g. fix the offset (toOffsetDateTime()) or get an Instant from that pair?
3. How would you handle parsing and serialization?

Not true.

Well for everything else, just use OffsetDateTime.

[dkhalanskyjb]

It's just a PITA to have APIs using Pair.

True. However, there's a difference between introducing an OffsetDateTime or a ZonedDateTime with their whole semantics and behavior, and adding a small wrapper that's almost like a pair, but with a couple of helper functions. Therefore, we need to decide which path to take. A small wrapper seems like a good approach, but if there are other use cases for ZonedDateTime, we may just introduce it instead.

How would you e.g. fix the offset (toOffsetDateTime()) or get an Instant from that pair?

I'm not sure what "fixing an offset" means, but to get an Instant from a Pair<LocalDateTime, UtcOffset>, you can do the following:

    fun ldtAndOffsetToInstant(pair: Pair<LocalDateTime, UtcOffset>): Instant {
        val (ldt, offset) = pair
        return ldt.toInstant(offset)
    }

How would you handle parsing and serialization?

At the moment, we don't provide means to handle this.

[cromefire]

True. However, there's a difference between introducing an OffsetDateTime or a ZonedDateTime with their whole semantics and behavior, and adding a small wrapper that's almost like a pair, but with a couple of helper functions.

The helper functions definite the semantics of the class, so I don't see a difference here.

I'm not sure what "fixing an offset" means

Convert from a TimeZone (dynamic-ish offset) to a fixed offset.

At the moment, we don't provide means to handle this.

I'd define this as a goal though.

[dkhalanskyjb]

The helper functions definite the semantics of the class, so I don't see a difference here.

The difference is that there are fundamental classes that define the problem area—we've chosen Instant, LocalDateTime, TimeZone, etc. for this—and then there are just helper classes. One has to learn to use the fundamental classes, as they represent something that one has to know to use the library, but helper classes are just there for grouping functionality together.

Convert from a TimeZone (dynamic-ish offset) to a fixed offset.

Timezones have different offsets at different times. Daylight saving time is an example.

I'd define this as a goal though.

Sure, we are planning an API for parsing and formatting, I said this several times in the other thread.

[wkornewald]

Sorry, noticed too late, moving discussion here.

Do you have an example? My impression is that it adapts the Instant to ensure that the LocalDateTime+TimeZone is stable.

Yep, sure.

ZonedDateTime.of(LocalDateTime.parse("2021-03-27T02:16:20"), ZoneId.of("Europe/Berlin")).plusDays(1).plusDays(1).toString() ==
"2021-03-28T03:16:20+02:00[Europe/Berlin]"

Because an intermediate computation (plusDays(1)) fell into the gap, all future results get shifted by an hour in the LocalDateTime.

Wow. This looks more like a bug. Try going from summer to winter time:

val d2 = ZonedDateTime.parse("2021-10-31T02:16:20+02:00[Europe/Berlin]")
// => 2021-10-31T02:16:20+02:00[Europe/Berlin]

d2.plusDays(1)
// => 2021-11-01T02:16:20+01:00[Europe/Berlin]

d2.plusHours(24)
// => 2021-11-01T01:16:20+01:00[Europe/Berlin]

ZonedDateTime.ofInstant(d2.toInstant().plus(1, ChronoUnit.DAYS), ZoneId.of("Europe/Berlin"))
// => 2021-11-01T01:16:20+01:00[Europe/Berlin]

it's actually LocalDateTime+Offset or LocalDateTime+ZoneId depending on what input you provide

Not the case. I've shown before that, on deserialization, the LocalDateTime part gets changed to preserve the Instant, see above. Now I've shown also that the gap gets corrected.

Your first example just happened to auto-correct the invalid offset that is disallowed by the zone - and this is documented as an edge case. It just recomputed to a different offset, but that doesn't mean it stores an Instant.

The same argument can be made for date + time.

It can't because 1) waiting for an hour at 23:31 will start the next day 2) the set of valid times depends on the date.

You're already suggesting to use Pair<LocalDateTime, TimeZone> then my counter-question is why not have Triple<Date, Time, TimeZone>? You could even define extension functions like plusDays over Triple, so isn't this "just as good"?

No Triple/Pair is not just as good. Apart from real types being more convenient they also improve the type safety and allow type hierarchies. A ZonedDateType could be a sealed class with child classes OffsetDateTime and RegionDateTime for cases where you have to ensure via types that only the expected format is used. You can't do that with Pair/Triple (which also just from a readability point of view is horrible) and since Pair/Triple use generics you can quickly run into ugly problems because of type erasure.

should stay unmodified in LocalDateTime and auto-correct the underlying Instant because people in that country keep going to work at the same LocalDateTime and even airports might adapt their active hours (some disallow flights after midnight for example). That's why Java's ZonedDateTime is not Instant+TimeZone, but LocalDateTime+TimeZone.

Your argument: "A is good, and ZonedDateTime is good, therefore ZonedDateTime is A", whereas my argument is "ZonedDateTime is not A, A is good, therefore ZonedDateTime is not good". It's easy to observe that ZonedDateTime is not A.

No my argument is: Humans often think in terms of LocalDateTime+TimeZone, so we need a solution for that. And ideally it also has a bug-free plusDays() implementation that takes DST and zone changes into account. 😄

It's pretty reliable at representing LocalDateTime and that's the intended use-case.

Not true.

Actually it is true, but you seem to have found a bug (or kind-of bad design). Let's imagine there was no bug. 😅

[cromefire]

The helper functions definite the semantics of the class, so I don't see a difference here.

The difference is that there are fundamental classes that define the problem area—we've chosen Instant, LocalDateTime, TimeZone, etc. for this—and then there are just helper classes. One has to learn to use the fundamental classes, as they represent something that one has to know to use the library, but helper classes are just there for grouping functionality together.

Well I guess in that would make it something in between. It's not a primitive/fundamental type, but it'll probably be a bit more than just a helper class I guess? Depends on if you still classify it as such. I've never argued for it to be a fundamentally brand new type, most functions would probably just be one liner and the biggest benefits would be documentation and clearer (and easier) code. Most things can already be done today if you where willing to handle both separately, but with the compound type it'll be a lot easier.

Convert from a TimeZone (dynamic-ish offset) to a fixed offset.

Timezones have different offsets at different times. Daylight saving time is an example.

Yes, that's why you can only convert it to an offset for a single point of time and even that may not be unambiguous.

I'd define this as a goal though.

Sure, we are planning an API for parsing and formatting, I said this several times in the other thread.

Yeah I'd probably add some default support though, if there is a common format being used for this.

[dkhalanskyjb]

Wow. This looks more like a bug.

This behavior is documented fairly well, and, I think, it's the only consistent way to implement this given the constraints of ZonedDateTime.

It just recomputed to a different offset, but that doesn't mean it stores an Instant.

LocalDateTime + UtcOffset uniquely identify an Instant, so it could as well just store an Instant.

You could even define extension functions like plusDays over Triple, so isn't this "just as good"?

Nope, it's not just as good: LocalDate and LocalTime are tightly connected and are constantly changed together. Pair represents disjoint data, where one component does not rely on the other.

Your first example just happened to auto-correct the invalid offset that is disallowed by the zone - and this is documented as an edge case.

It's only an edge case if you consider it an edge case if two machines have different timezone databases. If you attempt to send a ZonedDateTime to another machine, which considers it invalid, it will disregard the LocalDateTime part, not the Instant part. This should be a good giveaway of where the priorities lie for ZonedDateTime.

No my argument is: Humans often think in terms of LocalDateTime+TimeZone, so we need a solution for that.

Sure. Do we agree that ZonedDateTime, at least as defined in Java, is not that solution?

Actually it is true, but you seem to have found a bug (or kind-of bad design). Let's imagine there was no bug.

Imagining it requires implementing a whole new data structure, unfortunately, as this behavior is inherent in ZonedDateTime. Here we're discussing ZonedDateTime, however.

Yes, that's why you can only convert it to an offset for a single point of time and even that may not be unambiguous.

For each moment in time, there's an unambiguous offset for each TimeZone, unless the rules change. Ambiguity can happen with LocalDateTime + TimeZone.

[cromefire]

LocalDateTime + UtcOffset

Well yes for that case, the same is not true for LocalDateTime + ZoneId. What you are describing is OffsetDateTime, which indeed can(/should?) be stored as Instant with an offset.

Sure. Do we agree that ZonedDateTime, at least as defined in Java, is not that solution?

It was pretty fine last time I used it. We can probably get something more lightweight going, but the basic principles seemed fine.

Ambiguity can happen with LocalDateTime + TimeZone.

Yes that is the point. Because it should not represent a moment in time.

[dkhalanskyjb]

Well yes for that case, the same is not true for LocalDateTime + ZoneId.

ZonedDateTime also stores ZoneOffset.

It was pretty fine last time I used it.

It could be fine if you used it as Instant + TimeZone, which it is.

Yes that is the point. Because it should not represent a moment in time.

I was answering these words:

you can only convert it to an offset for a single point of time and even that may not be unambiguous.

Nope, for a single point in time (that is, Instant), the offset is unambiguous.

[wkornewald]

Wow. This looks more like a bug.

This behavior is documented fairly well, and, I think, it's the only consistent way to implement this given the constraints of ZonedDateTime.

It just recomputed to a different offset, but that doesn't mean it stores an Instant.

LocalDateTime + UtcOffset uniquely identify an Instant, so it could as well just store an Instant.

Not quite.

Future zone changes have different results depending on whether you pick LocalDateTime or Instant because the meaning of a future ZonedDateTime changes. In the LocalDateTime case the local time is stable and the serialization is stable (because you exactly reproduce the formatted string). In the Instant case the UTC time is stable, but the serialization is unstable and even incorrect from a typical end-user point of view because the formatted time suddenly changes, but users want only the offset to change.

Human-friendly calendar-like computations (plusDays etc.) must be based on the LocalDateTime.

Though, maybe java.time.ZonedDateTime doesn't treat all of this as well as I'd like to have, so maybe we should shift the discussion a little bit.

You could even define extension functions like plusDays over Triple, so isn't this "just as good"?

Nope, it's not just as good: LocalDate and LocalTime are tightly connected and are constantly changed together. Pair represents disjoint data, where one component does not rely on the other.

When adding durations to LocalDateTime+TimeZone you also can't treat them as disjoint data. The dependency stems from what you want to do with the data.

Your first example just happened to auto-correct the invalid offset that is disallowed by the zone - and this is documented as an edge case.

It's only an edge case if you consider it an edge case if two machines have different timezone databases. If you attempt to send a ZonedDateTime to another machine, which considers it invalid, it will disregard the LocalDateTime part, not the Instant part. This should be a good giveaway of where the priorities lie for ZonedDateTime.

I'd consider your example a much more extreme edge case than the actual intention of LocalDateTime+TimeZone: Correcting zone changes for events in the future. That's a much more likely issue.

No my argument is: Humans often think in terms of LocalDateTime+TimeZone, so we need a solution for that.

Sure. Do we agree that ZonedDateTime, at least as defined in Java, is not that solution?

Yes we surely agree that Java's implementation is not ideal. In my mind the ZonedDateTime should better match how humans think.

Actually it is true, but you seem to have found a bug (or kind-of bad design). Let's imagine there was no bug.

Imagining it requires implementing a whole new data structure, unfortunately, as this behavior is inherent in ZonedDateTime. Here we're discussing ZonedDateTime, however.

For zoneddatetime + duration arithmetic and compareTo/equals we could need some custom logic. And then I'd rather discuss how things should be instead of java.time.

[cromefire]

ZonedDateTime should better match how humans think

That is the central point I think. This type should be used for user facing stuff and therefore work how a normal person would expect it to.

[wkornewald]

Just FYI, in the JavaScript ecosystem the proposed Temporal API also could provide some interesting insights: https://tc39.es/proposal-temporal/docs/ambiguity.html

[wkornewald]

I've created an initial PR #175 with a sketch of what the API could look like.

@dkhalanskyjb @cromefire Would be nice to have your feedback and possibly counter-PRs or PRs on top of my PR. :)

[rocketraman]

Here is a use case that is easy with a Java ZoneDateTime, but I'm not even sure is possible to do correctly with kotlinx-datetime.

Given a LocalDateTime and a time zone, I want to identify whether the given LocalDateTime is repeated due to a "fall back", for example when DST ends.

For example, given zone America/Toronto, local time Nov 7th, 2021 1:30 am repeats because at 2am the time "falls back" to 1am again (reference: https://www.timeanddate.com/time/zone/canada/toronto).

With Java ZonedDateTime this can easily be seen:

ZonedDateTime.of(2021, 11, 7, 0, 30, 0, 0, ZoneId.of("America/Toronto")).let { zdt ->
  (0L until 4L).map { zdt.plusHours(it) }.joinToString("\n")
}

outputs:

2021-11-07T00:30-04:00[America/Toronto]
2021-11-07T01:30-04:00[America/Toronto]
2021-11-07T01:30-05:00[America/Toronto]
2021-11-07T02:30-05:00[America/Toronto]

where we can clearly see 1:30 am is repeated twice with two different offsets.

I can solve the problem easily on the JVM using another feature of ZonedDateTime which gives me access to both the earlier and later offsets at a time which is part of an overlap:

ZonedDateTime.of(2021, 11, 7, 0, 30, 0, 0, ZoneId.of("America/Toronto")).let { zdt ->
  (0L until 4L)
    .map { zdt.plusHours(it) }
    .map { it.withEarlierOffsetAtOverlap() != it.withLaterOffsetAtOverlap() }
}

which outputs as expected:

[false, true, true, false]

so we can easily solve the original problem like this:

fun LocalDateTime.isRepeatedJava(zone: TimeZone) =
  toJavaLocalDateTime().atZone(zone.toJavaZoneId()).let { 
    it.withEarlierOffsetAtOverlap() != it.withLaterOffsetAtOverlap()
  }

The closest equivalent I've come up with for kotlinx-datetime would be something like this:

fun LocalDateTime.isRepeatedKotlin() =
  this == toInstant(zone).plus(1.hours).toLocalDateTime(zone)

however this makes all sorts of incorrect assumptions that a) toInstant(zone) always returns the first instant when the local date time is in an overlap (by contrast, ZonedDateTime in java specifies this type of behavior in its documentation), and b) that the overlap period is always 1 hour, no more and no less.

While practically that implementation will likely work for most cases, its uncomfortable to say the least.

[dkhalanskyjb]

by contrast, ZonedDateTime in java specifies this type of behavior in its documentation

Not sure what you mean, we also document this behavior:

kotlinx-datetime/core/common/src/TimeZone.kt

Lines 158 to 159 in 5a920ea

	* - There are two possible instants that can have this date/time components in the [timeZone]. In this case the earlier
	* instant is returned.

I'm not even sure is possible to do correctly with kotlinx-datetime.

I can think of a way to replace the assumption that the overlap period is always 1 hour by an assumption that there's at most one transition in a given day:

• add 24 hours to the instant,
• add 1 day to the instant in the given time zone,
• check the difference between the instants in hours, it will be the length of the overlap period.

However, it's true that this is also not completely future-proof.

We are planning to solve this issue by adding a parameter to LocalDateTime.toInstant that allows the user to choose which instant to use in case of an overlap. Then, your use case would be covered: you'd just pass different values of that parameter and compare the results. CC @ilya-g: we have a specific use case for configuring the LocalDateTime.toInstant strategy.

[kevinb9n]

(source: My team has spent multiple person-years trying to eradicate date/time bugs in Google's codebase.)

FWIW, the abseil library for C++ takes a stand against the existence of this type, and I agree with them. I think such a type is ambiguous, dangerous, and unnecessary.

It's dangerous because it becomes easy to write natural-looking code, where no time zone is seen, that asks a question like "what is one day after x?" in two subtly different ways. Since the data type has a zone tucked away, it's capable of answering both versions of the question, and to most users the difference won't be clear.

The cleaner approach (that kotlinx has now!) is for all code to be unambiguously either about physical time or about civil time, not a mixture of the two. A time zone is used only as a (nearly-invertible) function for going back and forth between the two.

Note, the alternative is not necessarily Pair. Most often, the zone is stored at some broader scope. This forces you to think about which of Instant or LocalDateTime is the one you really want to store, since that broader time zone setting might change. But you'd really better do that anyway in case the zone data changes.

[cromefire]

either about physical time or about civil

Well right now it only has physical time, civil time is only DIY, you have to implement a means of storage and helpers on your own, because sometimes the timezone is stored in a broader context (in which case, yes just use LocalDateTime, that's what it's made for), but that notion falls apart when communicating among devices or handling thing centrally, off device. Yes you could probably refactor APIs, but ultimately you're mostly dealing with pre existing APIs you can't change (the source of all of this). It's less about what could you do in an empty space and more about having something that is actually useful and can be used with other things, right now to use kotlinx-datetime, you're usually just implementing this type your self, because you need something along this line to be compatible with the world.

[justingrant]

I led the design work for JavaScript's new Temporal.ZonedDateTime type, and I stumbled upon this issue while doing research for the upcoming IETF standardization of the ZonedDateTime string serialization format. Reading the issue above, it sounds like the introduction of ZDT into Kotlin is just as contentious as it was to JavaScript! 😄

If you're interested, feel free to read through tc39/proposal-temporal#700 which is the original design doc for the Temporal.ZonedDateTime type. Some decisions in that PR were superseded later, but it remains a good summary of the decisions we made and why we made them.

If you do go down the path of ZonedDateTime, I'll share some advice that would have been helpful for us when we started designing this API 2 years ago:

• Most of the hardest problems (e.g. dealing with repeated/skipped hours due to DST, order of operations for duration arithmetic, etc.) have industry standards like RFC 5545 that can help resolve those problems. You'll have better compatibility if you avoid re-inventing the wheel.
• We constantly had to remind ourselves (and each other) that we weren't designing the API for our own skill level and preferences, but instead for the "median developer" who'd be less experienced, less conscientious, and more bug-prone. We tried to design the API for the worst developer we'd ever worked with.
• Like you, we struggled to decide between Java's "all-in-one" ZDT type vs. Abseil Time's separation of wall-clock vs. exact time types. All of us API-design nerds were comfortable with Abseil's model, but ultimately we recognized that real-world "median developers" would not realistically be able to puzzle out DST-safe date math algorithms on their own. So we opted for Java's opinionated, all-in-one model. But we added escape hatches in the form of optional flags that could control behaviors like disambiguating skipped clock times or dealing with offset vs. time zone conflicts. Developers who didn't know what they were doing would get predictable defaults, while experts could easily customize behavior.
• We spent a lot of time investigating common bug patterns in other APIs, and we tried to defensively design the API to make those bugs harder or impossible. For example, a really common bug was using a non-TZ-aware date/time type (like LocalDateTime in Java) to perform arithmetic which breaks around DST transitions. So we removed the direct conversion between Temporal.Instant and Temporal.PlainDateTime so that developers could only convert from Instant to ZonedDateTime. This made DST-safe operations the path of least resistance for lazy developers.
• Dealing with conflicts between offsets and time zones was our hardest design problem. We're pretty happy with the solution we ended up with. See the offset option in ZonedDateTime.from.
• The DST-safe algorithms for addition, subtraction, and (especially) calculating the difference between ZDT instances were really hard to get right! Having a solid test suite of corner cases was critical to success. Feel free to borrow our tests which cover all kinds of weird stuff like in 2011 when the country of Samoa switched to the opposite side of the International Date Line, which was essentially a 24-hour-skipped DST transition. Or the case in Brazil in 2019 when DST was abruptly cancelled on short notice, leaving previously-recorded valid times with conflicts between offset and time zone. Fun!
• There are no perfect solutions to some of these problems. Some algorithms required "coin-flip" tradeoffs that made sense to some people and seemed weird to others. In the end, we decided to focus more on consistency and less on abstract perfection. Even if some behaviors were surprising to some developers, at least they were consistently surprising.
• Before designing the API, we first got consensus on the goals of the new type and we defined specific in-scope and out-of-scope use cases. In retrospect, doing this made it much vastly easier to make hard tradeoffs later because we could always refer back to those original decisions for guidance. BTW, here's the goals we started with (slightly edited to match later changes).
  • Represent a particular instant in time in a particular time zone, which matches most real-world use cases because most apps are focused on users doing things at a particular time in a particular place.
  • "Feel like Temporal" - Use similar patterns and naming as the rest of the library.
  • "Feel like PlainDateTime" - Provide a familiar API that is mostly a superset of PlainDateTime with only a handful of mostly-forward-compatible differences that are required to accommodate the presence of a time zone. Porting code from PlainDateTime to ZonedDateTime should be easy.
  • Make it easier to work with Temporal for junior developers who may not understand best practices for working with date/time data and time zones.
  • Make it easier to build standards-compliant calendar apps. RFC 5545 and related standards-track work like JSCalendar have opinionated definitions of how durations and time/date math should behave in order to ensure interoperability between calendaring apps, but today these durations are very hard to work with in Temporal. ZonedDateTime has built-in support for RFC 5545 durations.
  • Provide DST-safe behavior by default, even for developers who are unsure of DST best practices. By default, hybrid (RFC 5545-compliant) durations will be used for math methods, which provides the safest DST experience for developers who don't understand DST issues well enough to know which duration kind to pick.
  • Provide opt-in options to handle use cases where defaults won't suffice, including dealing with future times stored before time zone definition changes, or handling DST corner cases.
  • Bidirectionally interoperate with other Temporal types. It should be easy to use ZonedDateTime with other types, especially PlainDate and PlainTime.

Anyway, I hope you find the above useful, and I wish you good luck in designing a really fun and challenging API! If there's any questions you have, feel free to reach out. I'm happy to help.

[cromefire]

I led the design work for JavaScript's new Temporal.ZonedDateTime type, and I stumbled upon this issue while doing research for the upcoming IETF standardization of the ZonedDateTime string serialization format. Reading the issue above, it sounds like the introduction of ZDT into Kotlin is just as contentious as it was to JavaScript! 😄

Would potentially good to be compatible with as much stuff as possible and you definitely seem to have put in more work that we did discussing it on like a few days.

[dkhalanskyjb]

Hi @justingrant!

Thanks for reaching out, and congratulations on landing this controversial functionality!

I'm a bit confused by your message though, because either I'm misunderstanding it greatly, or you're not aware that we already have this functionality, just with a different API.

Let's break this down point by point.

• We do have DST-safe datetime arithmetic performed on Instant + TimeZone: https://github.com/Kotlin/kotlinx-datetime#instant-arithmetic As ISO-8601 (which is a superset of the linked RFC-5545 article) prescribes, this arithmetic behaves like wall-clock arithmetic when date-based units are added, and like operating on real time when time-based units are added.
• We do strive to make the common cases easy to use, though not necessarily for the sake of "the worst developer we've seen" but for people who have more important things to do other than to research how datetime works—like actually writing the code they're interested in. This mindset has the advantage of not sweeping the inevitable complexity under the rug for the sake of dumbing down the API, while avoiding rendering the API useless by making it too flexible and requiring research to do even the typical things.
• Ditto: the developers don't have to "puzzle out DST-safe date math algorithms on their own", we provide them already.
• We do not provide the ability to do non-timezone-aware arithmetics and don't plan on adding them, exactly because of them being error-prone, so we're safe in this regard.
• Yeah, we encountered the issue of dealing with time gaps and time overlaps with our LocalDateTime.toInstant implementation, and we're discussing the possible solutions internally. Could you please provide the design rationale for this decision? It could help a lot.
• Thank you for the test suite! I'll go and see whether any of the tests you have would be good to add to our own suite of tests for Instant arithmetics.
• "In the end, we decided to focus more on consistency and less on abstract perfection"—if ZonedDateTime (as it is implemented everywhere) was consistent, we wouldn't be arguing about this, but it's surprisingly tricky to actually develop a mental model for how it behaves. See this exact discussion (above) where a long-time extensive user of ZonedDateTime thought that its inherent behavior is a bug: Add a ZonedDateTime type #163 (comment). We'd like to avoid the developers forming inaccurate mental models, and making the developers use Instant + TimeZone arithmetic explicitly avoids the confusing behavior, making everything explicit. At this point, I think that if we do end up implementing something to represent the datetime in a timezone, it will not be modeled by Instant + TimeZone, exactly because of such confusing and opaque behavior.

[justingrant]

• We do have DST-safe datetime arithmetic performed on Instant + TimeZone

Oh, this is an interesting solution! Sorry for missing this, my knowledge of Kotlin is limited. It's cool to see how different people have come up with different solutions to the problem of developers ignoring DST. Your solution (every method that would have required a ZDT instead requires both an Instant and a TimeZone) seems reasonable to me, as does omitting math from LocalDateTime.

we encountered the issue of dealing with time gaps and time overlaps with our LocalDateTime.toInstant implementation, and we're discussing the possible solutions internally. Could you please provide the design rationale for this decision? It could help a lot.

Our solution to this was for the default behavior to match what's specified in RFC 5545 and what was used by the pre-Temporal JS specification. So the default behavior was kind of a no-brainer because we just followed existing practice.

Here's the text from RFC 5545 Section 3.3.5:

  If, based on the definition of the referenced time zone, the local
  time described occurs more than once (when changing from daylight
  to standard time), the DATE-TIME value refers to the first
  occurrence of the referenced time.  Thus, TZID=America/
  New_York:20071104T013000 indicates November 4, 2007 at 1:30 A.M.
  EDT (UTC-04:00).  If the local time described does not occur (when
  changing from standard to daylight time), the DATE-TIME value is
  interpreted using the UTC offset before the gap in local times.
  Thus, TZID=America/New_York:20070311T023000 indicates March 11,
  2007 at 3:30 A.M. EDT (UTC-04:00), one hour after 1:30 A.M. EST
  (UTC-05:00).

And here's the text from the pre-Temporal ECMAScript language spec:

When tlocal represents local time repeating multiple times at a negative time zone transition (e.g. when the daylight saving time ends or the time zone offset is decreased due to a time zone rule change) or skipped local time at a positive time zone transitions (e.g. when the daylight saving time starts or the time zone offset is increased due to a time zone rule change), tlocal must be interpreted using the time zone offset before the transition.

Temporal also provide a disambiguation option for the caller to request different behavior. The choices are this option are: (copying from the docs)

• 'compatible' (the default): Acts like 'earlier' for backward transitions and 'later' for forward transitions.
• 'earlier': The earlier of two possible times.
• 'later': The later of two possible times.
• 'reject': Throw a RangeError instead.

Was that the info you were looking for?

a long-time extensive user of ZonedDateTime thought that its inherent behavior is a bug: Add a ZonedDateTime type #163 (comment)

Yeah, intermediate values are confusing. The case you linked to seems like the easy version of this confusing behavior, because the caller is making separate method calls, so they can inspect the results to understand more about what's happening with the intermediate values.

IMO the more confusing case is adding a Duration that contains both time and date units to a ZonedDateTime, so the intermediate results are internal to the operation. In that case, we chose to follow RFC 5545's order of operations (largest units first) which made it deterministic: first add the date units, then adjust for DST (if needed), then add the time units.

Note that we don't support the DST disambiguation option in add and subtract methods, because we were already following the RFC 5545 standard for how those operations should behave so we just adopted RFC 5545's disambiguation behavior too for those operations. We could have offered this option, we just didn't think there would be a lot of demand so we left it out. It can always be added in a V2 if needed.

An even harder case we had no explicit guidance from any existing standard was when calculating differences (until and since methods) between two ZonedDateTime instances. What we chose to do there was to define difference calculations using add and subtract. For example, d = a.until(b) was defined as the value of d that when added to a, would yield b. This allowed us to tell ourselves that we were following some standard. 😄

As ISO-8601 (which is a superset of the linked RFC-5545 article) prescribes, this arithmetic behaves like wall-clock arithmetic when date-based units are added, and like operating on real time when time-based units are added.

Oh, I wasn't aware that this behavior was also specified in ISO 8601. Do you have a citation for the version and section number? The newer ISO 8601 specs (esp. the -1 and -2 supplements) are quite, er, voluminous so it's not surprising that we missed that.

[wkornewald]

@dkhalanskyjb The Instant + TimeZone approach is buggy because it can't correctly describe future dates (human, not physical). Imagine you have an appointment in Berlin at 2023-03-01T15:00+01:00[Europe/Berlin] and add it to your calendar. The corresponding Instant + Zone is 2023-03-01T14:00 + Europe/Berlin. As you can see, the Instant loses the human-relevant local time information and converts it to UTC. This loss of information is irreversible and leads to bugs. If the time zone offset is changed by the government because Europe switches to permanent summer time, what happens to the Instant + TimeZone? It'll suddenly show my calendar entry as 16:00 local time although the appointment should still be at 15:00. So it behaves incorrectly from a human perspective. You can only solve this with a proper ZonedDateTime which stores the same information as originally entered by a human: local time with zone id (and if possible also offset for disambiguating DST transitions). Humans don't convert to UTC in their heads and we shouldn't either.

In other words, Instant is only useful when working with physical time while ZonedDateTime could represent time in the way humans think. They are not the same and while the Instant arithmetic API might work 90% of the time it's still overall the wrong approach which leads to incorrect future date handling.

Apart from the correctness issue it's very inconvenient to always work with two separate values. You even need three values to represent the same level of detail as ZonedDateTime. This separation also makes parsing/serialization more difficult unnecessarily. How do I work with my JSON data which contains zone information? Do I have to create an ad-hoc data class and serializer which is a mediocre version of ZonedDateTime? Basically the whole world uses a combined string/field that includes the datetime and zone. Why would anyone want to deal with two (or three) separate values all the time instead of having just one?

[dkhalanskyjb]

[regarding the handling of gaps and overlaps] So the default behavior was kind of a no-brainer because we just followed existing practice. [...] Was that the info you were looking for?

We already have that behavior by default as well. I was interested in the disambiguation option: could you please clarify, why these specific values? In particular, does any user actually require something other than the default behavior? Why and when?

An even harder case we had no explicit guidance from any existing standard was when calculating differences (until and since methods) between two ZonedDateTime instances.

ISO-8601 provides an unambiguous definition of a difference (called "duration" there). See 3.1.1.8 for the definition of the duration, 3.1.2 for the specific definitions of time units used in the document, and 5.5.2 for the definition of the composite duration. The version I'm looking at is ISO 8601-1, edition 2019-02. This is also the answer for

Oh, I wasn't aware that this behavior was also specified in ISO 8601. Do you have a citation for the version and section number?

esp. the -1 and -2 supplements

I don't think 8601-1 is a supplement. Looking at the document, it says "part 1: basic rules". What is the actual ISO 8601 then?

[ilya-g]

@dkhalanskyjb AFAIK, ISO doesn't specify how to calculate a difference between two ZonedDateTime instances, primarily because it doesn't consider such entity as ZonedDateTime at all.

[dkhalanskyjb]

The duration is specified for Instant values, and Temporal.ZonedDateTime.until only supports date-based units if the time zones of the arguments are the same, so it behaves completely like Instant.until. So, I think the standard is still applicable.

[justingrant]

disambiguation option: could you please clarify, why these specific values? In particular, does any user actually require something other than the default behavior?

Good question. We were mostly following the lead of Abseil Time (the C++ library) which has fine-grained control for operations like this. We didn't have specific use cases in mind-- but we had heard from a few users that they wanted customized control.

I don't think 8601-1 is a supplement. Looking at the document, it says "part 1: basic rules". What is the actual ISO 8601 then?

Sorry, it's been two years since I read the ISO 8601 specs in detail. I believe you're correct.

The version I'm looking at is ISO 8601-1, edition 2019-02. This is also the answer for

OK I'll take a look. Thanks for the pointer.

@dkhalanskyjb The Instant + TimeZone approach is buggy because it can't correctly describe future dates (human, not physical). Imagine you have an appointment in Berlin at 2023-03-01T15:00+01:00[Europe/Berlin] and add it to your calendar. The corresponding Instant + Zone is 2023-03-01T14:00 + Europe/Berlin. As you can see, the Instant loses the human-relevant local time information and converts it to UTC.

If you're working with timezone-aware data, you need to choose whether the data is Instant + TimeZone or (using Java/Kotlin terms here) LocalDateTime + TimeZone. There is no perfect choice. Some use cases work better with Instant + TimeZone, while others work better with LocalDateTime + TimeZone. We decided that Instant + TimeZone was the right combo because it's unambiguous. In order to make LocalDateTime + TimeZone unambiguous, we'd have needed to store the offset too. This could be done, of course, but it would have added to the complexity of the type.

To be clear, this was a judgement call. We could have chosen the other option, but we didn't.

[wkornewald]

@dkhalanskyjb The Instant + TimeZone approach is buggy because it can't correctly describe future dates (human, not physical). Imagine you have an appointment in Berlin at 2023-03-01T15:00+01:00[Europe/Berlin] and add it to your calendar. The corresponding Instant + Zone is 2023-03-01T14:00 + Europe/Berlin. As you can see, the Instant loses the human-relevant local time information and converts it to UTC.

If you're working with timezone-aware data, you need to choose whether the data is Instant + TimeZone or (using Java/Kotlin terms here) LocalDateTime + TimeZone. There is no perfect choice. Some use cases work better with Instant + TimeZone, while others work better with LocalDateTime + TimeZone. We decided that Instant + TimeZone was the right combo because it's unambiguous. In order to make fLocalDateTime + TimeZone unambiguous, we'd have needed to store the offset too. This could be done, of course, but it would have added to the complexity of the type.

To be clear, this was a judgement call. We could have chosen the other option, but we didn't.

If you want correctness you really need both the offset and zone ID. The added "complexity" is very minor here and I hope the solution for kotlinx.datetime won't be "let's make it simpler even if it means it's only 90% correct". The current solution is just unacceptable for calendar apps and similar use-cases.

[dkhalanskyjb]

At this point, this discussion could benefit from being more structured. So, we opened a topic in GitHub Discussions: #237 Please chime in!

The format of GitHub Discussions is new for us, so we don't have a set of best practices in place, but initially, let's try the following:

• One comment per each concern.
• Before commenting on your concern, please check if it was already commented on and if so, reply to that discussion instead.

[cromefire]

BTW, draft-ietf-sedate-datetime-extended is nearing completion, which would extend RFC3339 with time zone information which would enable compatible exchange of zoned date times. This is what JS Temporal also implements and I hope it'll spread to other date time frameworks as well (like NodaTime for example).

[Ayfri]

Any update ?

[dkhalanskyjb]

@Ayfri, please use Instant or share your use cases in #237