RDD Storage Info for AdaptiveSparkPlan isFinalPlan=false +- Project [ride_id#3716, rideable_type#3717, started_at#3718, ended_at#3719, start_station_name#3720, start_station_id#3721, end_station_name#3722, end_station_id#3723, start_lat#3724, start_lng#3725, end_lat#3726, end_lng#3727, member_casual#3728, _source_file#3294, 2026-11-01 20:50:07.599499 AS _processed_dttm#13637, _start_station_ride_num#13639, year#13640, month#13641] +- Window [row_number() windowspecdefinition(start_station_id#3721, _w0#13642, _w1#13643, started_at#3718 ASC NULLS FIRST, ride_id#3716 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS _start_station_ride_num#13639], [start_station_id#3721, _w0#13642, _w1#13643], [started_at#3718 ASC NULLS FIRST, ride_id#3716 ASC NULLS FIRST] +- Sort [start_station_id#3721 ASC NULLS FIRST, _w0#13642 ASC NULLS FIRST, _w1#13643 ASC NULLS FIRST, started_at#3718 ASC NULLS FIRST, ride_id#3716 ASC NULLS FIRST], false, 0 +- Exchange hashpartitioning(start_station_id...

  • Storage Level: Disk Memory Deserialized 1x Replicated
  • Cached Partitions: 2
  • Total Partitions: 2
  • Memory Size: 93.2 MiB
  • Disk Size: 0.0 B

Data Distribution on 1 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
172.19.0.22:35015 93.2 MiB (1336.4 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

2 Partitions

Page:
Block Name ▾ Storage Level Size in Memory Size on Disk Executors
rdd_366_1 Memory Deserialized 1x Replicated 46.7 MiB 0.0 B 172.19.0.22:35015
rdd_366_0 Memory Deserialized 1x Replicated 46.4 MiB 0.0 B 172.19.0.22:35015
Page: