Clarify the ordering of the StopTimeUpdates

[RachM]

The ordering of the StopTimeUpdates must correspond to the stop_sequence. The stop_sequence can be provided two ways: 1) in the StopTimeUpdate or 2) in the static GTFS feed.

I want to explicitly call out the two ways in this section, to ensure providers know that the ordering must comply to the StopTimeUpdate stop_sequences, and failing that, the stop_sequences in the GTFS static feed.

Announced on GTFS-realtime Google Group at: https://groups.google.com/forum/#!topic/gtfs-realtime/5WdAtq5wrFA

[barbeau]

+1 for this - when running a lot of feeds through our validator I'm continuously surprised at how many feeds have the out-of-order problem.

Suggestion - I'd call out specifically where in GTFS too, something like:
"or in stop_times.txt of the corresponding GTFS..."

Also, from quickly skimming the spec right now on mobile, I'm not sure if it ever explicitly says that the pairing of stop_sequence and stop_id in StopTimeUpdate must matching the pairing in stop_times.txt. If it doesn't it would be worth adding that here and in StopTimeUpdate, as I've seen that error frequently too.

[barbeau]

I suppose order could be enforced in gtfs-realtime-bindings as well when stop_sequence is provided within StopTimeUpdate.

[RachM]

Great suggestion about calling out where in the GTFS - done.

The description of stop_sequence and stop_id both start off with "Must be the same as in stop_times.txt in the corresponding GTFS feed.", which seems ok to me.

[barbeau]

👍

[jxeeno]

I can understand why explicitly calling out stop_times.txt in GTFS is a good idea, but perhaps simply stating that the StopTimeUpdates needs to be "in the order they occur in the trip" is sufficient?

I'm considering edge cases where there are changes made to the stopping sequence in realtime vs GTFS. This can occur in lasso loops services like the City Circle in Sydney.

Trains might be scheduled to go around the loop in direction A but was changed to direction B for operational reasons. Currently, there's no requirement for the feed to contain the stop_sequence field if stops aren't visited more than once.

If the GTFS stop_times.txt stop_sequence is used to order the StopTimeUpdates, it would use the direction A stopping order rather than the "order they occur in the trip" (direction B).

Alternatively, we could require producers to provide stop_sequence if the stopping order changes from what was supplied in the GTFS stop_times.txt feed?

[barbeau]

Currently, there's no requirement for the feed to contain the stop_sequence field if stops aren't visited more than once.

This requirement was added in GTFS-rt v2.0 - stop_sequence docs now say:

stop_sequence is required for trips that visit the same stop_id more than once (e.g., a loop) to disambiguate which stop the prediction is for.

[barbeau]

I'm considering edge cases where there are changes made to the stopping sequence in realtime vs GTFS. This can occur in lasso loops services like the City Circle in Sydney.

I believe the best way to handle this in GTFS-realtime is by canceling the scheduled trip, and creating a new ADDED trip with the new stop order.

[jxeeno]

This requirement was added in GTFS-rt v2.0

Yep, that's what I said 🙂 Sorry, the double negative there probably didn't help.

I believe the best way to handle this in GTFS-realtime is by canceling the scheduled trip, and creating a new ADDED trip with the new stop order.

Depends on the situation I think? If the trip is already in progress when the route changes, cancelling and adding would break user journeys which depend on a persistent trip id.

This is probably a separate discussion, but my understanding of the current spec is that minor route variations such as changes to stop order can still be conveyed with a SCHEDULED schedule_relationship (i.e. without cancelling and adding a new trip) because it is "close enough to the scheduled trip to be associated with it".

[barbeau]

Whoops! Sorry, I missed the double negative.

Yes, I think this is a good discussion to have in a separate thread, and another good candidate for at least a best practices document. I've been tracking these items via issues in our validator repo until I can formalize them further -
for this see CUTR-at-USF/gtfs-realtime-validator#287.

Tl;dr I think that would break some consumers, as my assumption was changes in trip geometry aren't currently supported in GTFS-rt. At a minimum I think it should be accompanied by a ServiceAlert with detour effect. But, let's continue discussion in above issue until it can be transferred elsewhere.

[IvanVolosyuk]

Depends on the situation I think? If the trip is already in progress when the route changes, cancelling and adding would break user journeys which depend on a persistent trip id. This is probably a separate discussion, but my understanding of the current spec is that minor route variations such as changes to stop order can still be conveyed with a SCHEDULED schedule_relationship (i.e. without cancelling and adding a new trip) because it is "close enough to the scheduled trip to be associated with it".

That's a good point, but there are certain issues with that approach: - It is hard to tell the intended use of this construct from the common error. - Shape of the trip doesn't match the new stops configuration - thus if we project the progress on the trip to the old shape we might see something strange, e.g. 10x speeds forward or backward. The stop order changes doesn't look minor changes to me. ADDED trip with different shape looks much safer approach. I think maintaining using journey should be done slightly separately. E.g. instead of using ADDED for schedule relations - implement REPLACED (with a reference to the original trip). Sorry for the off-topic, this should probably be discussed separately.

…

[barbeau]

• It is hard to tell the intended use of this construct from the common error.

Yes!! I'd like to get away from some of the implicit uses of GTFS-rt that are very difficult to distinguish from errors when validating feeds. IMHO feed data quality is probably collectively the biggest challenge that we face, and moving towards more explicit data definitions/use cases would help. But, yes, off topic for this proposal...

[jxeeno]

The stop order changes doesn't look minor changes to me. ADDED trip with
different shape looks much safer approach. I think maintaining using
journey should be done slightly separately. E.g. instead of using ADDED for
schedule relations - implement REPLACED (with a reference to the original
trip).

Yeah, that's fair enough. I think this proposal is a step in the right direction in making the spec more robust. I agree it's hard to distinguish between feed errors and intended alterations.

As part of this (or as a separate PR), can we look into clarifying what constitutes as "close enough to the scheduled trip" means in the definition of SCHEDULED in the GTFS-realtime v2 spec? That would resolve some of the ambiguities I raised earlier.

Broadly, I'd also be in favour of seeing the return of something like REPLACEMENT so that things like stop sequence alterations and even track/platform changes (as discussed in Google Groups) can be easily identified by the consumer and not flagged as a potential error.

[barbeau]

As part of this (or as a separate PR), can we look into clarifying what constitutes as "close enough to the scheduled trip" means in the definition of SCHEDULED in the GTFS-realtime v2 spec?

I've always interpreted SCHEDULED as exactly matching the GTFS data in terms of stop_id/sequence pairing and shape - it's the default value.

However, it sounds like others (including Google) aren't interpreting it this way:
https://groups.google.com/d/msg/gtfs-realtime/BZOfsVeI2Cc/gIUjGbkCBAAJ

I agree that this needs to be clarified in a new proposal, preferable along with a way to specify replacement trips.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This pull request has been closed due to inactivity. Pull requests can always be reopened after they have been closed. See the Specification Amendment Process.