RDD Storage Info for AdaptiveSparkPlan isFinalPlan=false +- Project [ride_id#56, rideable_type#57, started_at#93, ended_at#94, start_station_name#60, start_station_id#124, end_station_name#62, end_station_id#155, start_lat#88, start_lng#89, end_lat#90, end_lng#91, member_casual#68, _start_station_id_fixed#109, _end_station_id_fixed#139, _ride_id_ok#171, _duration_ok#172, _station_id_ok#173, _source_file#174, 2026-11-06 11:04:13.790835 AS _processed_dttm#175, year#176, month#177, _start_station_ride_num#179] +- Window [row_number() windowspecdefinition(start_station_id#124, _w0#180, _w1#181, started_at#93 ASC NULLS FIRST, ride_id#56 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS _start_station_ride_num#179], [start_station_id#124, _w0#180, _w1#181], [started_at#93 ASC NULLS FIRST, ride_id#56 ASC NULLS FIRST] +- Sort [start_station_id#124 ASC NULLS FIRST, _w0#180 ASC NULLS FIRST, _w1#181 ASC NULLS FIRST, started_at#93 ASC NULLS FIRST, ride_id#56 ASC NULLS FIRST], false, 0 ...

  • Storage Level: Disk Memory Deserialized 1x Replicated
  • Cached Partitions: 3
  • Total Partitions: 3
  • Memory Size: 142.5 MiB
  • Disk Size: 0.0 B

Data Distribution on 1 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
172.19.0.20:33023 142.5 MiB (1520.7 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

3 Partitions

Page:
Block Name ▾ Storage Level Size in Memory Size on Disk Executors
rdd_28_2 Memory Deserialized 1x Replicated 17.9 MiB 0.0 B 172.19.0.20:33023
rdd_28_1 Memory Deserialized 1x Replicated 61.9 MiB 0.0 B 172.19.0.20:33023
rdd_28_0 Memory Deserialized 1x Replicated 62.7 MiB 0.0 B 172.19.0.20:33023
Page: