CitiBike-External-S3 - Details for Query 12

Submitted Time: 2026/11/04 19:03:12
Duration: 1.3 min
Succeeded Jobs: 20 21

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; 1 [labelType="html" label="Execute InsertIntoHadoopFsRelationCommand task commit time total (min, med, max (stageId: taskId)) 24.5 s (1.5 s, 1.6 s, 1.8 s (stage 30.0: task 150)) number of written files: 30 job commit time: 51.0 s number of output rows: 4,632,234 number of dynamic part: 2 written output: 191.0 MiB"]; 2 [labelType="html" label=" WriteFiles "]; subgraph cluster3 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: total (min, med, max (stageId: taskId))\n2.4 m (2.2 s, 10.1 s, 11.1 s (stage 30.0: task 153))"; 4 [labelType="html" label="Sort sort time total (min, med, max (stageId: taskId)) 1.9 s (24 ms, 91 ms, 340 ms (stage 30.0: task 157)) peak memory total (min, med, max (stageId: taskId)) 1652.0 MiB (84.0 MiB, 108.0 MiB, 128.0 MiB (stage 30.0: task 157)) spill size total (min, med, max (stageId: taskId)) 1476.8 MiB (0.0 B, 112.0 MiB, 120.0 MiB (stage 30.0: task 158))"]; 5 [labelType="html" label=" Project "]; 6 [labelType="html" label="Filter number of output rows: 4,632,234"]; } 7 [labelType="html" label="Window spill size total (min, med, max (stageId: taskId)) 25.5 MiB (0.0 B, 0.0 B, 8.0 MiB (stage 30.0: task 150))"]; subgraph cluster8 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n56.3 s (1.3 s, 3.8 s, 5.5 s (stage 30.0: task 156))"; 9 [labelType="html" label="Sort sort time total (min, med, max (stageId: taskId)) 15.9 s (612 ms, 1.0 s, 1.6 s (stage 30.0: task 157)) peak memory total (min, med, max (stageId: taskId)) 1684.0 MiB (84.0 MiB, 108.0 MiB, 128.0 MiB (stage 30.0: task 157)) spill size total (min, med, max (stageId: taskId)) 700.0 MiB (0.0 B, 0.0 B, 100.0 MiB (stage 30.0: task 147))"]; } 10 [labelType="html" label="AQEShuffleRead number of partitions: 15 partition data size total (min, med, max (stageId: taskId)) 340.8 MiB (15.7 MiB, 23.4 MiB, 24.3 MiB (driver)) number of coalesced partitions: 15"]; 11 [labelType="html" label="Exchange shuffle records written: 4,731,900 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 3.3 s (0 ms, 46 ms, 360 ms (stage 28.0: task 144)) remote merged bytes read total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 30.0: task 148)) local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 30.0: task 148)) remote merged blocks fetched: 0 records read: 4,731,900 local bytes read total (min, med, max (stageId: taskId)) 85.7 MiB (3.4 MiB, 6.1 MiB, 6.9 MiB (stage 30.0: task 157)) fetch wait time total (min, med, max (stageId: taskId)) 21 ms (0 ms, 0 ms, 17 ms (stage 30.0: task 148)) remote bytes read total (min, med, max (stageId: taskId)) 239.0 MiB (10.9 MiB, 16.1 MiB, 19.5 MiB (stage 30.0: task 158)) merged fetch fallback count: 0 local blocks read: 58 remote merged chunks fetched: 0 remote blocks read: 167 data size total (min, med, max (stageId: taskId)) 1612.1 MiB (0.0 B, 13.7 MiB, 118.3 MiB (stage 28.0: task 136)) local merged bytes read total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 30.0: task 148)) number of partitions: 200 remote reqs duration total (min, med, max (stageId: taskId)) 5.6 s (78 ms, 378 ms, 734 ms (stage 30.0: task 159)) remote bytes read to disk total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 30.0: task 148)) shuffle bytes written total (min, med, max (stageId: taskId)) 324.7 MiB (0.0 B, 2.7 MiB, 24.5 MiB (stage 28.0: task 140))"]; subgraph cluster12 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n1.2 m (0 ms, 1.0 s, 6.7 s (stage 28.0: task 140))"; 13 [labelType="html" label=" Project "]; 14 [labelType="html" label=" Project "]; } 15 [labelType="html" label="Scan csv number of output rows: 4,731,900 number of files read: 5 metadata time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 30.0: task 148)) size of files read total (min, med, max (stageId: taskId)) 881.3 MiB (0.0 B, 0.0 B, 881.3 MiB (driver))"]; 1->0; 2->1; 4->2; 5->4; 6->5; 7->6; 9->7; 10->9; 11->10; 13->11; 14->13; 15->14; }

AdaptiveSparkPlan isFinalPlan=true

Execute InsertIntoHadoopFsRelationCommand s3a://rzvde-g11-terentyeva-kristina/raw_valid/citibike_data, false, [year#313, month#314], Parquet, [partitionOverwriteMode=dynamic, __partition_columns=["year","month"], path=s3a://rzvde-g11-terentyeva-kristina/raw_valid/citibike_data/], Overwrite, [ride_id, rideable_type, started_at, ended_at, start_station_name, start_station_id, end_station_name, end_station_id, start_lat, start_lng, end_lat, end_lng, member_casual, _source_file, _processed_dttm, year, month, _start_station_ride_num]

WriteFiles

Sort [year#313 ASC NULLS FIRST, month#314 ASC NULLS FIRST], false, 0

Filter (((isnotnull(flag_ride_id_valid#308) AND isnotnull(flag_time_valid#309)) AND isnotnull(flag_station_valid#310)) AND ((flag_ride_id_valid#308 AND flag_time_valid#309) AND flag_station_valid#310))

WholeStageCodegen (3)

Window [row_number() windowspecdefinition(start_station_id#176, started_at#208 ASC NULLS FIRST, ride_id#171 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS _start_station_ride_num#336], [start_station_id#176], [started_at#208 ASC NULLS FIRST, ride_id#171 ASC NULLS FIRST]

Sort [start_station_id#176 ASC NULLS FIRST, started_at#208 ASC NULLS FIRST, ride_id#171 ASC NULLS FIRST], false, 0

WholeStageCodegen (2)

AQEShuffleRead coalesced

Exchange hashpartitioning(start_station_id#176, 200), ENSURE_REQUIREMENTS, [plan_id=504]

Project [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, (isnotnull(ride_id#171) AND NOT (trim(ride_id#171, None) = )) AS flag_ride_id_valid#308, ((isnotnull(ended_at#209) AND isnotnull(started_at#208)) AND (ended_at#209 > started_at#208)) AS flag_time_valid#309, ((isnotnull(end_station_id#178) AND isnotnull(start_station_id#176)) AND NOT (end_station_id#178 = start_station_id#176)) AS flag_station_valid#310, input_file_name() AS _source_file#311, year(cast(started_at#208 as date)) AS year#313, month(cast(started_at#208 as date)) AS month#314]

Project [ride_id#171, rideable_type#172, cast(started_at#173 as timestamp) AS started_at#208, cast(ended_at#174 as timestamp) AS ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, cast(start_lat#179 as double) AS start_lat#203, cast(start_lng#180 as double) AS start_lng#204, cast(end_lat#181 as double) AS end_lat#205, cast(end_lng#182 as double) AS end_lng#206, member_casual#183]

WholeStageCodegen (1)

FileScan csv [ride_id#171,rideable_type#172,started_at#173,ended_at#174,start_station_name#175,start_station_id#176,end_station_name#177,end_station_id#178,start_lat#179,start_lng#180,end_lat#181,end_lng#182,member_casual#183] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(5 paths)[s3a://rzvde-g11-terentyeva-kristina/raw/citibike_data/202510/202510-ci..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<ride_id:string,rideable_type:string,started_at:string,ended_at:string,start_station_name:s...

Details

== Physical Plan ==
AdaptiveSparkPlan (24)
+- == Final Plan ==
   Execute InsertIntoHadoopFsRelationCommand (13)
   +- WriteFiles (12)
      +- * Sort (11)
         +- * Project (10)
            +- * Filter (9)
               +- Window (8)
                  +- * Sort (7)
                     +- AQEShuffleRead (6)
                        +- ShuffleQueryStage (5), Statistics(sizeInBytes=1612.1 MiB, rowCount=4.73E+6)
                           +- Exchange (4)
                              +- * Project (3)
                                 +- * Project (2)
                                    +- Scan csv  (1)
+- == Initial Plan ==
   Execute InsertIntoHadoopFsRelationCommand (23)
   +- WriteFiles (22)
      +- Sort (21)
         +- Project (20)
            +- Filter (19)
               +- Window (18)
                  +- Sort (17)
                     +- Exchange (16)
                        +- Project (15)
                           +- Project (14)
                              +- Scan csv  (1)


(1) Scan csv 
Output [13]: [ride_id#171, rideable_type#172, started_at#173, ended_at#174, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#179, start_lng#180, end_lat#181, end_lng#182, member_casual#183]
Batched: false
Location: InMemoryFileIndex [s3a://rzvde-g11-terentyeva-kristina/raw/citibike_data/202510/202510-citibike-tripdata-part00.csv, ... 4 entries]
ReadSchema: struct<ride_id:string,rideable_type:string,started_at:string,ended_at:string,start_station_name:string,start_station_id:string,end_station_name:string,end_station_id:string,start_lat:string,start_lng:string,end_lat:string,end_lng:string,member_casual:string>

(2) Project [codegen id : 1]
Output [13]: [ride_id#171, rideable_type#172, cast(started_at#173 as timestamp) AS started_at#208, cast(ended_at#174 as timestamp) AS ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, cast(start_lat#179 as double) AS start_lat#203, cast(start_lng#180 as double) AS start_lng#204, cast(end_lat#181 as double) AS end_lat#205, cast(end_lng#182 as double) AS end_lng#206, member_casual#183]
Input [13]: [ride_id#171, rideable_type#172, started_at#173, ended_at#174, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#179, start_lng#180, end_lat#181, end_lng#182, member_casual#183]

(3) Project [codegen id : 1]
Output [19]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, (isnotnull(ride_id#171) AND NOT (trim(ride_id#171, None) = )) AS flag_ride_id_valid#308, ((isnotnull(ended_at#209) AND isnotnull(started_at#208)) AND (ended_at#209 > started_at#208)) AS flag_time_valid#309, ((isnotnull(end_station_id#178) AND isnotnull(start_station_id#176)) AND NOT (end_station_id#178 = start_station_id#176)) AS flag_station_valid#310, input_file_name() AS _source_file#311, year(cast(started_at#208 as date)) AS year#313, month(cast(started_at#208 as date)) AS month#314]
Input [13]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183]

(4) Exchange
Input [19]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314]
Arguments: hashpartitioning(start_station_id#176, 200), ENSURE_REQUIREMENTS, [plan_id=504]

(5) ShuffleQueryStage
Output [19]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314]
Arguments: 0

(6) AQEShuffleRead
Input [19]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314]
Arguments: coalesced

(7) Sort [codegen id : 2]
Input [19]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314]
Arguments: [start_station_id#176 ASC NULLS FIRST, started_at#208 ASC NULLS FIRST, ride_id#171 ASC NULLS FIRST], false, 0

(8) Window
Input [19]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314]
Arguments: [row_number() windowspecdefinition(start_station_id#176, started_at#208 ASC NULLS FIRST, ride_id#171 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS _start_station_ride_num#336], [start_station_id#176], [started_at#208 ASC NULLS FIRST, ride_id#171 ASC NULLS FIRST]

(9) Filter [codegen id : 3]
Input [20]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314, _start_station_ride_num#336]
Condition : (((isnotnull(flag_ride_id_valid#308) AND isnotnull(flag_time_valid#309)) AND isnotnull(flag_station_valid#310)) AND ((flag_ride_id_valid#308 AND flag_time_valid#309) AND flag_station_valid#310))

(10) Project [codegen id : 3]
Output [18]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, _source_file#311, 2026-11-04 19:03:12.908131 AS _processed_dttm#312, year#313, month#314, _start_station_ride_num#336]
Input [20]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314, _start_station_ride_num#336]

(11) Sort [codegen id : 3]
Input [18]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, _source_file#311, _processed_dttm#312, year#313, month#314, _start_station_ride_num#336]
Arguments: [year#313 ASC NULLS FIRST, month#314 ASC NULLS FIRST], false, 0

(12) WriteFiles
Input [18]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, _source_file#311, _processed_dttm#312, year#313, month#314, _start_station_ride_num#336]

(13) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: s3a://rzvde-g11-terentyeva-kristina/raw_valid/citibike_data, false, [year#313, month#314], Parquet, [partitionOverwriteMode=dynamic, __partition_columns=["year","month"], path=s3a://rzvde-g11-terentyeva-kristina/raw_valid/citibike_data/], Overwrite, [ride_id, rideable_type, started_at, ended_at, start_station_name, start_station_id, end_station_name, end_station_id, start_lat, start_lng, end_lat, end_lng, member_casual, _source_file, _processed_dttm, year, month, _start_station_ride_num]

(14) Project
Output [13]: [ride_id#171, rideable_type#172, cast(started_at#173 as timestamp) AS started_at#208, cast(ended_at#174 as timestamp) AS ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, cast(start_lat#179 as double) AS start_lat#203, cast(start_lng#180 as double) AS start_lng#204, cast(end_lat#181 as double) AS end_lat#205, cast(end_lng#182 as double) AS end_lng#206, member_casual#183]
Input [13]: [ride_id#171, rideable_type#172, started_at#173, ended_at#174, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#179, start_lng#180, end_lat#181, end_lng#182, member_casual#183]

(15) Project
Output [19]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, (isnotnull(ride_id#171) AND NOT (trim(ride_id#171, None) = )) AS flag_ride_id_valid#308, ((isnotnull(ended_at#209) AND isnotnull(started_at#208)) AND (ended_at#209 > started_at#208)) AS flag_time_valid#309, ((isnotnull(end_station_id#178) AND isnotnull(start_station_id#176)) AND NOT (end_station_id#178 = start_station_id#176)) AS flag_station_valid#310, input_file_name() AS _source_file#311, year(cast(started_at#208 as date)) AS year#313, month(cast(started_at#208 as date)) AS month#314]
Input [13]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183]

(16) Exchange
Input [19]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314]
Arguments: hashpartitioning(start_station_id#176, 200), ENSURE_REQUIREMENTS, [plan_id=485]

(17) Sort
Input [19]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314]
Arguments: [start_station_id#176 ASC NULLS FIRST, started_at#208 ASC NULLS FIRST, ride_id#171 ASC NULLS FIRST], false, 0

(18) Window
Input [19]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314]
Arguments: [row_number() windowspecdefinition(start_station_id#176, started_at#208 ASC NULLS FIRST, ride_id#171 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS _start_station_ride_num#336], [start_station_id#176], [started_at#208 ASC NULLS FIRST, ride_id#171 ASC NULLS FIRST]

(19) Filter
Input [20]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314, _start_station_ride_num#336]
Condition : (((isnotnull(flag_ride_id_valid#308) AND isnotnull(flag_time_valid#309)) AND isnotnull(flag_station_valid#310)) AND ((flag_ride_id_valid#308 AND flag_time_valid#309) AND flag_station_valid#310))

(20) Project
Output [18]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, _source_file#311, 2026-11-04 19:03:12.908131 AS _processed_dttm#312, year#313, month#314, _start_station_ride_num#336]
Input [20]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, flag_ride_id_valid#308, flag_time_valid#309, flag_station_valid#310, _source_file#311, year#313, month#314, _start_station_ride_num#336]

(21) Sort
Input [18]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, _source_file#311, _processed_dttm#312, year#313, month#314, _start_station_ride_num#336]
Arguments: [year#313 ASC NULLS FIRST, month#314 ASC NULLS FIRST], false, 0

(22) WriteFiles
Input [18]: [ride_id#171, rideable_type#172, started_at#208, ended_at#209, start_station_name#175, start_station_id#176, end_station_name#177, end_station_id#178, start_lat#203, start_lng#204, end_lat#205, end_lng#206, member_casual#183, _source_file#311, _processed_dttm#312, year#313, month#314, _start_station_ride_num#336]

(23) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: s3a://rzvde-g11-terentyeva-kristina/raw_valid/citibike_data, false, [year#313, month#314], Parquet, [partitionOverwriteMode=dynamic, __partition_columns=["year","month"], path=s3a://rzvde-g11-terentyeva-kristina/raw_valid/citibike_data/], Overwrite, [ride_id, rideable_type, started_at, ended_at, start_station_name, start_station_id, end_station_name, end_station_id, start_lat, start_lng, end_lat, end_lng, member_casual, _source_file, _processed_dttm, year, month, _start_station_ride_num]

(24) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true