RDD Storage Info for AdaptiveSparkPlan isFinalPlan=false +- Project [ride_id#128, rideable_type#129, started_at#669, ended_at#670, start_station_name#132, start_station_id#133, end_station_name#134, end_station_id#135, start_lat#675, start_lng#676, end_lat#677, end_lng#678, member_casual#140, _corrupt_record#141, _source_file#156, (isnotnull(ride_id#128) AND NOT (trim(ride_id#128, None) = )) AS _is_ride_id_not_empty#695, coalesce((ended_at#670 > started_at#669), false) AS _is_ended_after_started#696, coalesce(NOT (end_station_id#135 = start_station_id#133), false) AS _is_different_stations#697, ((((isnotnull(ride_id#128) AND NOT (trim(ride_id#128, None) = )) AND coalesce((ended_at#670 > started_at#669), false)) AND coalesce(NOT (end_station_id#135 = start_station_id#133), false)) AND isnull(_corrupt_record#141)) AS _is_valid#698] +- Project [ride_id#128, rideable_type#129, cast(started_at#130 as timestamp) AS started_at#669, cast(ended_at#131 as timestamp) AS ended_at#670, start_station_name#132, start_station_id#133, end_s...

  • Storage Level: Disk Memory Deserialized 1x Replicated
  • Cached Partitions: 2
  • Total Partitions: 2
  • Memory Size: 104.6 MiB
  • Disk Size: 0.0 B

Data Distribution on 1 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
172.19.0.22:35015 104.6 MiB (1336.4 MiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

2 Partitions

Page:
Block Name ▾ Storage Level Size in Memory Size on Disk Executors
rdd_40_1 Memory Deserialized 1x Replicated 50.5 MiB 0.0 B 172.19.0.22:35015
rdd_40_0 Memory Deserialized 1x Replicated 54.0 MiB 0.0 B 172.19.0.22:35015
Page: