g9_chernyshev_miron_s2 - Details for Query 31

Submitted Time: 2026/08/18 13:40:34
Duration: 19 s
Succeeded Jobs: 60 61 62

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" CollectLimit "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n31.0 s (1.7 s, 4.3 s, 4.5 s (stage 92.0: task 259))"; 2 [labelType="html" label=" Project "]; 3 [labelType="html" label="Filter number of output rows: 0"]; } 4 [labelType="html" label="Scan csv number of output rows: 4,988,053 number of files read: 5 metadata time: 0 ms size of files read: 928.8 MiB"]; 2->0; 3->2; 4->3; }

CollectLimit 21

Project [toprettystring(ride_id#17, Some(Etc/UTC)) AS toprettystring(ride_id)#1409, toprettystring(rideable_type#18, Some(Etc/UTC)) AS toprettystring(rideable_type)#1410, toprettystring(started_at#19, Some(Etc/UTC)) AS toprettystring(started_at)#1411, toprettystring(ended_at#20, Some(Etc/UTC)) AS toprettystring(ended_at)#1412, toprettystring(start_station_name#21, Some(Etc/UTC)) AS toprettystring(start_station_name)#1413, toprettystring(cast(start_station_id#22 as float), Some(Etc/UTC)) AS toprettystring(start_station_id)#1414, toprettystring(end_station_name#23, Some(Etc/UTC)) AS toprettystring(end_station_name)#1415, toprettystring(cast(end_station_id#24 as float), Some(Etc/UTC)) AS toprettystring(end_station_id)#1416, toprettystring(start_lat#25, Some(Etc/UTC)) AS toprettystring(start_lat)#1417, toprettystring(start_lng#26, Some(Etc/UTC)) AS toprettystring(start_lng)#1418, toprettystring(end_lat#27, Some(Etc/UTC)) AS toprettystring(end_lat)#1419, toprettystring(end_lng#28, Some(Etc/UTC)) AS toprettystring(end_lng)#1420, toprettystring(member_casual#29, Some(Etc/UTC)) AS toprettystring(member_casual)#1421, toprettystring((ended_at#20 - started_at#19), Some(Etc/UTC)) AS toprettystring(duration)#1422, toprettystring((unix_timestamp(ended_at#20, yyyy-MM-dd HH:mm:ss, Some(Etc/UTC), false) - unix_timestamp(started_at#19, yyyy-MM-dd HH:mm:ss, Some(Etc/UTC), false)), Some(Etc/UTC)) AS toprettystring(duration_sec)#1423]

Filter ((isnotnull(started_at#19) AND isnotnull(ended_at#20)) AND (started_at#19 > ended_at#20))

WholeStageCodegen (1)

FileScan csv [ride_id#17,rideable_type#18,started_at#19,ended_at#20,start_station_name#21,start_station_id#22,end_station_name#23,end_station_id#24,start_lat#25,start_lng#26,end_lat#27,end_lng#28,member_casual#29] Batched: false, DataFilters: [isnotnull(started_at#19), isnotnull(ended_at#20), (started_at#19 > ended_at#20)], Format: CSV, Location: InMemoryFileIndex(5 paths)[s3a://rzvde-g9-chernyshev-miron/raw/citibike_data/202507/202507-citibi..., PartitionFilters: [], PushedFilters: [IsNotNull(started_at), IsNotNull(ended_at)], ReadSchema: struct<ride_id:string,rideable_type:string,started_at:timestamp,ended_at:timestamp,start_station_...

Details

== Physical Plan ==
CollectLimit (4)
+- * Project (3)
   +- * Filter (2)
      +- Scan csv  (1)


(1) Scan csv 
Output [13]: [ride_id#17, rideable_type#18, started_at#19, ended_at#20, start_station_name#21, start_station_id#22, end_station_name#23, end_station_id#24, start_lat#25, start_lng#26, end_lat#27, end_lng#28, member_casual#29]
Batched: false
Location: InMemoryFileIndex [s3a://rzvde-g9-chernyshev-miron/raw/citibike_data/202507/202507-citibike-tripdata-part00.csv, ... 4 entries]
PushedFilters: [IsNotNull(started_at), IsNotNull(ended_at)]
ReadSchema: struct<ride_id:string,rideable_type:string,started_at:timestamp,ended_at:timestamp,start_station_name:string,start_station_id:string,end_station_name:string,end_station_id:string,start_lat:double,start_lng:double,end_lat:double,end_lng:double,member_casual:string>

(2) Filter [codegen id : 1]
Input [13]: [ride_id#17, rideable_type#18, started_at#19, ended_at#20, start_station_name#21, start_station_id#22, end_station_name#23, end_station_id#24, start_lat#25, start_lng#26, end_lat#27, end_lng#28, member_casual#29]
Condition : ((isnotnull(started_at#19) AND isnotnull(ended_at#20)) AND (started_at#19 > ended_at#20))

(3) Project [codegen id : 1]
Output [15]: [toprettystring(ride_id#17, Some(Etc/UTC)) AS toprettystring(ride_id)#1409, toprettystring(rideable_type#18, Some(Etc/UTC)) AS toprettystring(rideable_type)#1410, toprettystring(started_at#19, Some(Etc/UTC)) AS toprettystring(started_at)#1411, toprettystring(ended_at#20, Some(Etc/UTC)) AS toprettystring(ended_at)#1412, toprettystring(start_station_name#21, Some(Etc/UTC)) AS toprettystring(start_station_name)#1413, toprettystring(cast(start_station_id#22 as float), Some(Etc/UTC)) AS toprettystring(start_station_id)#1414, toprettystring(end_station_name#23, Some(Etc/UTC)) AS toprettystring(end_station_name)#1415, toprettystring(cast(end_station_id#24 as float), Some(Etc/UTC)) AS toprettystring(end_station_id)#1416, toprettystring(start_lat#25, Some(Etc/UTC)) AS toprettystring(start_lat)#1417, toprettystring(start_lng#26, Some(Etc/UTC)) AS toprettystring(start_lng)#1418, toprettystring(end_lat#27, Some(Etc/UTC)) AS toprettystring(end_lat)#1419, toprettystring(end_lng#28, Some(Etc/UTC)) AS toprettystring(end_lng)#1420, toprettystring(member_casual#29, Some(Etc/UTC)) AS toprettystring(member_casual)#1421, toprettystring((ended_at#20 - started_at#19), Some(Etc/UTC)) AS toprettystring(duration)#1422, toprettystring((unix_timestamp(ended_at#20, yyyy-MM-dd HH:mm:ss, Some(Etc/UTC), false) - unix_timestamp(started_at#19, yyyy-MM-dd HH:mm:ss, Some(Etc/UTC), false)), Some(Etc/UTC)) AS toprettystring(duration_sec)#1423]
Input [13]: [ride_id#17, rideable_type#18, started_at#19, ended_at#20, start_station_name#21, start_station_id#22, end_station_name#23, end_station_id#24, start_lat#25, start_lng#26, end_lat#27, end_lng#28, member_casual#29]

(4) CollectLimit
Input [15]: [toprettystring(ride_id)#1409, toprettystring(rideable_type)#1410, toprettystring(started_at)#1411, toprettystring(ended_at)#1412, toprettystring(start_station_name)#1413, toprettystring(start_station_id)#1414, toprettystring(end_station_name)#1415, toprettystring(end_station_id)#1416, toprettystring(start_lat)#1417, toprettystring(start_lng)#1418, toprettystring(end_lat)#1419, toprettystring(end_lng)#1420, toprettystring(member_casual)#1421, toprettystring(duration)#1422, toprettystring(duration_sec)#1423]
Arguments: 21