Deprecate TripUpdate.schedule_relationship = ADDED, add TripUpdate.schedule_relationship = NEW / REPLACEMENT to specify new / replaced trips which do not run on a schedule from the GTFS static.

gtfs-realtime/best-practices/best-practices.md

@@ -52,7 +52,7 @@ All entities should only be removed from a GTFS Realtime feed when they are no l
 General guidelines for trip cancellations:
 
 * When canceling trips over a number of days, producers should provide TripUpdates referencing the given `trip_ids` and `start_dates` as `CANCELED` as well as an Alert with `NO_SERVICE` referencing the same `trip_ids` and `TimeRange` that can be shown to riders explaining the cancellation (e.g., detour).
-* If no stops in a trip will be visited, the trip should be `CANCELED` instead of having all `stop_time_updates` being marked as `SKIPPED`.  
+* If no stops in a trip will be visited, the trip should be `CANCELED` instead of having all `stop_time_updates` being marked as `SKIPPED`, unless the trip was a `NEW` or `DUPLICATED` trip and was subsequently canceled. 
 
 | Field Name | Recommendation |
 | --- | --- |
@@ -73,7 +73,13 @@ For example, a `VehiclePosition` entity has `vehicle_id:A` and `trip_id:4`, then
 
 | Field Name | Recommendation |
 | --- | --- |
-| `schedule_relationship` | The behavior of `ADDED` trips are unspecified and the use of this enumeration is not recommended. |
+| `schedule_relationship` | The behavior of `ADDED` trips are unspecified and the use of this enumeration is not recommended.<br/>If the trip is not scheduled to run originally, use `NEW` if it doesn't follow the stopping pattern of an existing trip, or `DUPLICATED` if it is a copy of an existing trip.<br/>If the trip runs on a modified schedule or stops, but can be associated to an original scheduled trip in the GTFS static, use `REPLACEMENT` and specify the full list of stop times for the modified trip. |
+
+### TripProperties
+
+| Field Name | Recommendation |
+| --- | --- |
+| `trip_headsign` | Should always be provided for a trip with `TripDescriptor.schedule_relationship` = `NEW`, and provided for a trip with `TripDescriptor.schedule_relationship` = `REPLACEMENT` if the trip is diverted. |
 
 ### VehicleDescriptor
 
@@ -100,6 +106,7 @@ For example, a `VehiclePosition` entity has `vehicle_id:A` and `trip_id:4`, then
 | Field Name | Recommendation |
 | --- | --- |
 | `delay` | If only `delay` is provided in a `stop_time_update` `arrival` or `departure` (and not `time`), then the GTFS [`stop_times.txt`](https://gtfs.org/reference/static#stopstxt) should contain `arrival_times` and/or `departure_times` for these corresponding stops. A `delay` value in the realtime feed is meaningless unless you have a clock time to add it to in the GTFS `stop_times.txt` file. |
+| `scheduled_time` | If the trip is a new or replacement trip, and the trip will run according to a schedule (which can be a modified schedule in case of a replacement trip), `scheduled_time` should be provided for all timepoints. If a duplicated trip runs have different run or dwell times compared to the original, `scheduled_time` can also be used to specify them. |
 
 ### VehiclePosition
 

gtfs-realtime/proto/gtfs-realtime.proto

@@ -190,6 +190,13 @@ message TripUpdate {
     // To specify a completely certain prediction, set its uncertainty to 0.
     optional int32 uncertainty = 3;
 
+    // Scheduled time for a NEW, REPLACEMENT, or DUPLICATED trip.
+    // In Unix time (i.e., number of seconds since January 1st 1970 00:00:00
+    // UTC).
+    // Optional if TripUpdate.schedule_relationship is NEW, REPLACEMENT or DUPLICATED, forbidden otherwise.
+    // NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
+    optional int64 scheduled_time = 4;
+
     // The extensions namespace allows 3rd-party developers to extend the
     // GTFS Realtime Specification in order to add and evaluate new features
     // and modifications to the spec.
@@ -272,6 +279,32 @@ message TripUpdate {
       // NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
       optional string assigned_stop_id = 1;
 
+      // The updated headsign of the vehicle at the stop.
+      // NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
+      optional string stop_headsign = 2;
+
+      enum DropOffPickupType {
+        // Regularly scheduled pickup/dropoff.
+        REGULAR = 0;
+
+        // No pickup/dropoff available
+        NONE = 1;
+
+        // Must phone agency to arrange pickup/dropoff.
+        PHONE_AGENCY = 2;
+
+        // Must coordinate with driver to arrange pickup/dropoff.
+        COORDINATE_WITH_DRIVER = 3;
+      }
+
+      // The updated pickup of the vehicle at the stop.
+      // NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
+      optional DropOffPickupType pickup_type = 3;
+
+      // The updated drop off of the vehicle at the stop.
+      // NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
+      optional DropOffPickupType drop_off_type = 4;
+
       // The extensions namespace allows 3rd-party developers to extend the
       // GTFS Realtime Specification in order to add and evaluate new features
       // and modifications to the spec.
@@ -379,6 +412,14 @@ message TripUpdate {
     // NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future. 
     optional string shape_id = 4;
 
+    // Specifies the headsign for this trip when it differs from the original.
+    // NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
+    optional string trip_headsign = 5;
+
+    // Specifies the name for this trip when it differs from the original.
+    // NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
+    optional string trip_short_name = 6;
+
     // The extensions namespace allows 3rd-party developers to extend the
     // GTFS Realtime Specification in order to add and evaluate new features
     // and modifications to the spec.
@@ -793,14 +834,10 @@ message TripDescriptor {
     // enough to the scheduled trip to be associated with it.
     SCHEDULED = 0;
 
-    // An extra trip that was added in addition to a running schedule, for
-    // example, to replace a broken vehicle or to respond to sudden passenger
-    // load.
-    // NOTE: Currently, behavior is unspecified for feeds that use this mode. There are discussions on the GTFS GitHub
-    // [(1)](https://github.com/google/transit/issues/106) [(2)](https://github.com/google/transit/pull/221)
-    // [(3)](https://github.com/google/transit/pull/219) around fully specifying or deprecating ADDED trips and the
-    // documentation will be updated when those discussions are finalized.
-    ADDED = 1;
+    // This value has been deprecated as the behavior was unspecified. 
+    // Use DUPLICATED for an extra trip that is the same as a scheduled trip except the start date or time, 
+    // or NEW for an extra trip that is unrelated to an existing trip.
+    ADDED = 1 [deprecated = true];
 
     // A trip that is running with no schedule associated to it (GTFS frequencies.txt exact_times=0).
     // Trips with ScheduleRelationship=UNSCHEDULED must also set all StopTimeUpdates.ScheduleRelationship=UNSCHEDULED.
@@ -809,8 +846,9 @@ message TripDescriptor {
     // A trip that existed in the schedule but was removed.
     CANCELED = 3;
 
-    // Should not be used - for backwards-compatibility only.
-    REPLACEMENT = 5 [deprecated = true];
+    // A trip that replaces an existing trip in the schedule.
+    // NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
+    REPLACEMENT = 5;
 
     // An extra trip that was added in addition to a running schedule, for example, to replace a broken vehicle or to
     // respond to sudden passenger load. Used with TripUpdate.TripProperties.trip_id, TripUpdate.TripProperties.start_date,
@@ -819,7 +857,10 @@ message TripDescriptor {
     // (in calendar.txt or calendar_dates.txt) is operating within the next 30 days. The trip to be duplicated is
     // identified via TripUpdate.TripDescriptor.trip_id. This enumeration does not modify the existing trip referenced by
     // TripUpdate.TripDescriptor.trip_id - if a producer wants to cancel the original trip, it must publish a separate
-    // TripUpdate with the value of CANCELED or DELETED. Trips defined in GTFS frequencies.txt with exact_times that is
+    // TripUpdate with the value of CANCELED or DELETED. If a producer wants to replace the original trip, a value of 
+    // `REPLACEMENT` should be used instead.
+    //
+    // Trips defined in GTFS frequencies.txt with exact_times that is
     // empty or equal to 0 cannot be duplicated. The VehiclePosition.TripDescriptor.trip_id for the new trip must contain
     // the matching value from TripUpdate.TripProperties.trip_id and VehiclePosition.TripDescriptor.ScheduleRelationship
     // must also be set to DUPLICATED.
@@ -839,6 +880,10 @@ message TripDescriptor {
     // real-time predictions.
     // NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
     DELETED = 7;
+
+    // An extra trip unrelated to any existing trips, for example, to respond to sudden passenger load.
+    // NOTE: This field is still experimental, and subject to change. It may be formally adopted in the future.
+    NEW = 8;
   }
   optional ScheduleRelationship schedule_relationship = 4;
 
@@ -864,8 +909,8 @@ message TripDescriptor {
     extensions 9000 to 9999;
   }
 
-  // Linkage to any modifications done to this trip (shape changes, removal or addition of stops). 
-  // If this field is provided, the `trip_id`, `route_id`, `direction_id`, `start_time`, `start_date` fields of the `TripDescriptor` MUST be left empty, to avoid confusion by consumers that aren't looking for the `ModifiedTripSelector` value. 
+  // Linkage to any modifications done to this trip (shape changes, removal or addition of stops).
+  // If this field is provided, the `trip_id`, `route_id`, `direction_id`, `start_time`, `start_date` fields of the `TripDescriptor` MUST be left empty, to avoid confusion by consumers that aren't looking for the `ModifiedTripSelector` value.
   optional ModifiedTripSelector modified_trip = 7;
 
   // The extensions namespace allows 3rd-party developers to extend the
@@ -1134,7 +1179,7 @@ message TripModifications {
   }
 
   message SelectedTrips {
-    // A list of trips affected with this replacement that all have the same new `shape_id`.
+    // A list of trips affected with this replacement that all have the same new `shape_id`. A `TripUpdate` with `schedule_relationship=REPLACEMENT` must not already exist for the trip.
     repeated string trip_ids = 1;
     // The ID of the new shape for the modified trips in this SelectedTrips.
     // May refer to a new shape added using a GTFS-RT Shape message, or to an existing shape defined in the GTFS-Static feed’s shapes.txt.

gtfs-realtime/spec/en/examples/migration-duplicated.md

@@ -1,13 +1,76 @@
-## Migration Guide - Transition from ADDED to DUPLICATED trips 
+## Migration Guide - Transition from ADDED to NEW or DUPLICATED trips 
+
+The GTFS-realtime `trip.schedule_relationship` of `NEW` represents a new trip that runs on a schedule unrelated to any existing scheduled trip.
 
 The GTFS-realtime `trip.schedule_relationship` of `DUPLICATED` represents a new trip that is the same as an existing scheduled trip except for service start date and time. 
 
-This migration guide defines how existing producers and consumers that were using the `ADDED` enumeration to represent duplicated trips should transition to the `DUPLICATED` enumeration. The goal is to minimize disruption to producers and consumers during the transition. 
+This migration guide defines how existing producers and consumers that were using the `ADDED` enumeration should transition to either the `NEW` and the `DUPLICATED` enumeration. The goal is to minimize disruption to producers and consumers during the transition. 
+
+*If you are a producer or consumer that has **not** used the `ADDED` enumeration, no action is required - you can produce/consume `NEW` and/or `DUPLICATED` trips without producing/consuming any `ADDED` entities.* 
 
-*If you are a producer or consumer that has **not** used the `ADDED` enumeration to describe duplicated trips, no action is required - you can produce/consume `DUPLICATED` trips without producing/consuming any `ADDED` entities.* 
+For a full history of the `NEW` enumeration, see the [`NEW` and `REPLACEMENT` proposal on GitHub](https://github.com/google/transit/pull/504).
 
 For a full history of the `DUPLICATED` enumeration, see the [`DUPLICATED` proposal on GitHub](https://github.com/google/transit/pull/221).
 
+### Which one to migrate to
+
+Both `NEW` and `DUPLICATED` enumeration are used to specify a trip which was not originally scheduled to run in the GTFS static.
+
+Use `NEW` if your trip cannot be described using any scheduled trips as a template. For example, if the trip calls at different stops from the regular trips of the route, or if the extra trip is pickup only at the beginning of the route despite that the regular trips allow both pickup and drop off at all stops.
+
+Use `DUPLICATED` if your trip is a copy of a scheduled trip, which may run at the same, or at different times of the original scheduled trip.
+
+### Using ADDED and NEW entities in the same feed
+
+If you are a producer who has been using the `ADDED` enumeration to specify trips which are unrelated to the schedule, to avoid disruption to existing consumers it is recommended that you continue to produce `ADDED` entitles for these trips but also add `NEW` entitles for the same trip.
+
+However, to prevent consumers from accidentally adding the same trip twice, the entities referencing the same trip **must** be linked using the same `trip_id`, `route_id` and `start_date`.
+In addition, the contents of the `stop_time_update` must also be the same as well.
+
+#### Producers
+
+~~~
+entity {
+  id: "ei0"
+  trip_update {
+    trip: {
+      trip_id: "1" // <-- a trip_id not found in the static GTFS
+      route_id: "A"
+      schedule_relationship: ADDED
+      start_date: "20200821" // <-- New trip date
+      start_time: "11:30:00" // <-- New trip time
+    }
+    stop_time_update {
+	... // The full list of the calling points of the trip
+    }
+  }
+}
+
+entity {
+  id: "ei10"
+  trip_update {
+    trip: {
+      trip_id: "1" // <-- The same trip_id as the above
+      route_id: "A" // <-- The same route_id as the above
+      schedule_relationship: NEW
+      start_date: "20200821" // <-- The same date as the above
+      start_time: "11:30:00" // <-- The same time as the above
+    }
+    stop_time_update {
+	... // <-- The same content as the above
+    }
+  }
+}
+~~~
+
+It is suggested that you notify existing consumers (e.g., via a developer mailing list) that the use of `ADDED` is being deprecated by a set deadline and that consumers should start consuming the `NEW` trips instead. The above strategy being used to match `ADDED` and `NEW` trip entities should also be mentioned and a link to this migration guide should be included. After the deadline passes, you can remove the `NEW` entities from your feed and publish only the `NEW` entities for newly-added trips.
+
+#### Consumers
+
+As mentioned above, producers will transition from `ADDED` to `NEW` enumerations by initially publishing two entities for each new trip using the same `trip_id`.
+
+Therefore, when a consumer implements support for `NEW` trips, it is important that consumers ignore any `ADDED` trips that have the same `trip_id` as a `NEW` trip `trip_id`.
+
 ### Using ADDED and DUPLICATED entities in same feed
 
 #### Producers
@@ -93,7 +156,7 @@ entity {
 }
 ~~~
 
-It is suggested that you notify existing consumers (e.g., via a developer mailing list) that the use of `ADDED` for duplicated trips is being deprecated by a set deadline and that consumers should start consuming the `DUPLICATED` trips instead. The above strategy being used to match `ADDED` and `DUPLICATED` trip entities should also be mentioned and a link to this migration guide should be included. After the deadline passes, you can remove the `ADDED` entities from your feed and publish only the `DUPLICATED` entities for duplicated trips.
+It is suggested that you notify existing consumers (e.g., via a developer mailing list) that the use of `ADDED` is being deprecated by a set deadline and that consumers should start consuming the `DUPLICATED` trips instead. The above strategy being used to match `ADDED` and `DUPLICATED` trip entities should also be mentioned and a link to this migration guide should be included. After the deadline passes, you can remove the `ADDED` entities from your feed and publish only the `DUPLICATED` entities for duplicated trips.
 
 #### Consumers